2024-03-12T04:48:25.312 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-12T04:48:25.312 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383 --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/orig.config.yaml 2024-03-12T04:48:25.373 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:48:25.482 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7591383' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 94a539707733325322964c6a187c6a1b41be1daf subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 94a539707733325322964c6a187c6a1b41be1daf kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad owner: scheduled_teuthology@teuthology priority: 250 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8179 sha1: 94a539707733325322964c6a187c6a1b41be1daf sleep_before_teardown: 0 subset: 297/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLsrrTkXrYAX4VTMAJ3DmX0gfV+s6kMUH4n/1d/EVHX0VIPgg0rH8USo1F0jtAVzpQLNsr66vtssHnozDRJ6oY4= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLs44ux6wfQ88fyomfMFzV/FoEmQZ9Sa62ENQ0Wu1a+vQ0Adi3v2FBtdMSjMZZJHjcFwrYwvcZ1wbRaTbtqnTdI= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAz6qeUcD/wLadGJnm4gDvX8rE2iA8OXFctEfTtlzyqT7KqOqCQKaNPrb71+0DjeCs2Not8Hy+UU2UDoxI1ndB0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_14:02:35 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-12T04:48:25.482 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa; will attempt to use it 2024-03-12T04:48:25.483 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks 2024-03-12T04:48:25.483 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-12T04:48:25.484 INFO:teuthology.task.internal:Checking packages... 2024-03-12T04:48:25.508 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '94a539707733325322964c6a187c6a1b41be1daf' 2024-03-12T04:48:25.508 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-12T04:48:25.508 INFO:teuthology.packaging:ref: None 2024-03-12T04:48:25.508 INFO:teuthology.packaging:tag: None 2024-03-12T04:48:25.508 INFO:teuthology.packaging:branch: squid 2024-03-12T04:48:25.508 INFO:teuthology.packaging:sha1: 94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:48:25.509 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-12T04:48:25.736 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1607.g94a53970 2024-03-12T04:48:25.738 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-12T04:48:25.745 INFO:teuthology.task.internal:no buildpackages task found 2024-03-12T04:48:25.745 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-12T04:48:25.757 INFO:teuthology.task.internal:Saving configuration 2024-03-12T04:48:25.780 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-12T04:48:25.786 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T04:48:25.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 04:43:07.406793', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLsrrTkXrYAX4VTMAJ3DmX0gfV+s6kMUH4n/1d/EVHX0VIPgg0rH8USo1F0jtAVzpQLNsr66vtssHnozDRJ6oY4='} 2024-03-12T04:48:25.836 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 04:43:07.407838', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLs44ux6wfQ88fyomfMFzV/FoEmQZ9Sa62ENQ0Wu1a+vQ0Adi3v2FBtdMSjMZZJHjcFwrYwvcZ1wbRaTbtqnTdI='} 2024-03-12T04:48:25.860 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 04:43:07.408518', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAz6qeUcD/wLadGJnm4gDvX8rE2iA8OXFctEfTtlzyqT7KqOqCQKaNPrb71+0DjeCs2Not8Hy+UU2UDoxI1ndB0='} 2024-03-12T04:48:25.860 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-12T04:48:25.867 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-12T04:48:25.867 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-12T04:48:25.867 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-12T04:48:25.867 INFO:teuthology.run_tasks:Running task console_log... 2024-03-12T04:48:25.964 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f359eb40160>, signals=[15]) 2024-03-12T04:48:25.964 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-12T04:48:25.971 INFO:teuthology.task.internal:Opening connections... 2024-03-12T04:48:25.971 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T04:48:25.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:48:26.064 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-03-12T04:48:26.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:48:26.152 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-03-12T04:48:26.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:48:26.241 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-12T04:48:26.248 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-03-12T04:48:26.266 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-03-12T04:48:26.267 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-12T04:48:26.324 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-03-12T04:48:26.324 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-03-12T04:48:26.324 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-03-12T04:48:26.324 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T04:48:26.325 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T04:48:26.326 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-12T04:48:26.349 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-03-12T04:48:26.367 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-03-12T04:48:26.367 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2024-03-12T04:48:26.424 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T04:48:26.425 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T04:48:26.425 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T04:48:26.425 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-03-12T04:48:26.449 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-03-12T04:48:26.467 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-03-12T04:48:26.467 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-03-12T04:48:26.524 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-03-12T04:48:26.524 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-03-12T04:48:26.524 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-03-12T04:48:26.524 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T04:48:26.525 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T04:48:26.525 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-03-12T04:48:26.540 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-12T04:48:26.551 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-12T04:48:26.557 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-12T04:48:26.558 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-03-12T04:48:26.560 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-03-12T04:48:26.562 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-03-12T04:48:26.580 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-12T04:48:26.587 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-12T04:48:26.587 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-03-12T04:48:26.616 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-03-12T04:48:26.619 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-03-12T04:48:26.730 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-12T04:48:26.805 INFO:teuthology.run_tasks:Running task kernel... 2024-03-12T04:48:26.819 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-12T04:48:26.819 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-12T04:48:26.819 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-12T04:48:26.820 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T04:48:26.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:48:26.836 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-03-12T04:48:26.893 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-361.el9.x86_64 2024-03-12T04:48:26.893 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-361.el9.x86_64 2024-03-12T04:48:26.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-03-12T04:48:29.049 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-12T04:48:30.683 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-12T04:48:31.982 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-03-12T04:48:34.008 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-12T04:48:38.933 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-12T04:48:40.301 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 15 kB 00:00 2024-03-12T04:49:18.607 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 552 kB/s | 21 MB 00:38 2024-03-12T04:49:23.444 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-03-12T04:49:25.629 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-12T04:49:25.629 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-12T04:49:25.725 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-12T04:49:25.729 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-12T04:49:25.729 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-03-12T04:49:25.729 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-12T04:49:25.729 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-03-12T04:49:25.730 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:49:25.731 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 100 M 2024-03-12T04:49:25.731 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 123 M 2024-03-12T04:49:25.731 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-12T04:49:34.914 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 678 kB/s | 6.0 MB 00:09 2024-03-12T04:49:59.161 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 641 kB/s | 21 MB 00:33 2024-03-12T04:50:28.374 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 640 kB/s | 33 MB 00:53 2024-03-12T04:50:36.614 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 569 kB/s | 39 MB 01:10 2024-03-12T04:50:36.615 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-12T04:50:36.615 INFO:teuthology.orchestra.run.smithi100.stdout:Total 1.4 MB/s | 100 MB 01:10 2024-03-12T04:50:37.496 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-12T04:50:37.866 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-12T04:50:37.866 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-12T04:50:39.205 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-12T04:50:39.206 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-12T04:50:42.125 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-12T04:50:42.779 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-12T04:50:42.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T04:50:45.688 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T04:50:45.887 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T04:50:50.608 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T04:50:50.765 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T04:50:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T04:51:24.623 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T04:51:24.623 INFO:teuthology.orchestra.run.smithi100.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-12T04:51:24.623 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:51:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-12T04:51:25.139 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-12T04:51:25.139 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-12T04:51:25.139 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-12T04:51:25.139 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-12T04:51:25.564 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-12T04:51:25.564 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:51:25.565 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-12T04:51:26.056 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T04:51:26.194 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T04:51:26.194 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T04:51:26.194 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-12T04:51:26.194 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-12T04:51:26.194 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-12T04:51:26.194 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-12T04:51:26.195 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-03-12T04:51:26.195 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-03-12T04:51:26.315 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-12T04:51:26.316 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-12T04:51:26.316 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-03-12T04:51:26.360 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-03-12T04:51:26.360 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-03-12T04:51:26.360 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-03-12T04:51:26.360 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-03-12T04:51:26.360 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-03-12T04:51:26.360 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-03-12T04:51:26.361 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:51:26.362 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-12T04:51:26.362 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-12T04:51:26.362 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-12T04:51:26.362 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-03-12T04:51:26.362 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:51:26.362 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-03-12T04:51:26.363 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-12T04:51:26.363 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-427.el9.x86_64 2024-03-12T04:51:26.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-12T04:51:26.921 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-03-12T04:51:28.297 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-12T04:51:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-03-12T04:51:28.332 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-03-12T04:51:28.347 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.NUfWRXhvSJ 2024-03-12T04:51:28.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NUfWRXhvSJ 2024-03-12T04:51:28.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.NUfWRXhvSJ 2024-03-12T04:51:28.563 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.NUfWRXhvSJ is 6KB 2024-03-12T04:51:28.611 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.NUfWRXhvSJ 2024-03-12T04:51:28.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-03-12T04:51:28.690 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-12T04:51:28.690 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-12T04:51:28.690 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-12T04:51:28.691 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-12T04:51:28.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-12T04:51:28.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-03-12T04:51:58.863 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-12T04:51:58.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-12T04:51:58.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:52:58.867 DEBUG:teuthology.orchestra.remote:timed out 2024-03-12T04:53:07.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-12T04:53:07.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:53:08.276 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-12T04:53:08.671 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-12T04:53:08.672 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-12T04:53:08.672 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-03-12T04:53:08.686 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-427.el9.x86_64 2024-03-12T04:53:08.686 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-12T04:53:08.686 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T04:53:08.687 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-12T04:53:09.688 DEBUG:teuthology.parallel:result is None 2024-03-12T04:53:09.688 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-12T04:53:09.696 INFO:teuthology.task.internal:Creating test directory... 2024-03-12T04:53:09.697 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T04:53:09.699 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T04:53:09.702 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T04:53:09.721 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-12T04:53:09.730 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-12T04:53:09.740 INFO:teuthology.task.internal:Creating archive directory... 2024-03-12T04:53:09.740 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T04:53:09.755 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T04:53:09.760 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T04:53:09.882 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-12T04:53:09.890 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-12T04:53:09.891 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T04:53:09.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:53:09.908 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T04:53:09.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:53:09.925 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T04:53:09.929 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T04:53:09.950 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T04:53:09.999 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T04:53:10.006 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T04:53:10.025 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T04:53:10.033 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T04:53:10.050 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T04:53:10.056 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T04:53:10.057 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-12T04:53:10.065 INFO:teuthology.task.internal:Configuring sudo... 2024-03-12T04:53:10.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T04:53:10.067 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T04:53:10.069 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T04:53:10.100 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-12T04:53:10.109 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-12T04:53:10.109 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T04:53:10.134 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T04:53:10.141 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T04:53:10.157 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T04:53:10.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T04:53:10.277 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T04:53:10.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T04:53:10.402 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T04:53:10.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T04:53:10.469 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T04:53:10.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T04:53:10.597 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T04:53:10.639 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T04:53:10.685 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T04:53:10.685 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T04:53:10.753 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T04:53:10.798 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T04:53:10.854 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T04:53:10.937 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T04:53:10.981 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T04:53:10.981 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T04:53:11.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-03-12T04:53:11.053 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-03-12T04:53:11.056 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-03-12T04:53:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T04:53:11.098 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T04:53:11.131 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T04:53:11.508 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-12T04:53:11.515 INFO:teuthology.task.internal:Starting timer... 2024-03-12T04:53:11.515 INFO:teuthology.run_tasks:Running task pcp... 2024-03-12T04:53:11.527 INFO:teuthology.run_tasks:Running task selinux... 2024-03-12T04:53:11.535 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-12T04:53:11.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-03-12T04:53:11.612 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-03-12T04:53:11.614 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2024-03-12T04:53:11.668 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2024-03-12T04:53:11.670 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-03-12T04:53:11.720 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-03-12T04:53:11.722 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-12T04:53:11.722 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-03-12T04:53:11.741 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-03-12T04:53:11.741 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2024-03-12T04:53:11.762 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2024-03-12T04:53:11.762 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-03-12T04:53:11.789 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-03-12T04:53:11.789 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-03-12T04:53:11.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T04:53:11.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:53:11.817 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T04:53:11.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:53:11.846 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T04:53:11.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T04:53:11.876 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-12T04:53:11.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-03-12T04:53:12.027 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2024-03-12T04:53:12.063 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-03-12T04:53:12.098 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-12T04:53:12.108 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-12T04:53:12.124 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-12T04:53:12.125 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi156.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-12T04:58:09.323 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-03-12T04:58:09.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-12T04:58:09.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:58:09.418 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-12T04:58:09.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-12T04:58:09.501 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-12T04:58:09.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:58:09.588 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-03-12T04:58:09.657 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-12T04:58:09.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-12T04:58:09.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T04:58:09.746 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-03-12T04:58:09.815 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-12T04:58:09.815 INFO:teuthology.run_tasks:Running task clock... 2024-03-12T04:58:09.827 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-12T04:58:09.827 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T04:58:09.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T04:58:09.830 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T04:58:09.830 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T04:58:09.832 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T04:58:09.832 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T04:58:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T04:58:09.862 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T04:58:09.874 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T04:58:09.879 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T04:58:09.888 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T04:58:09.903 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-03-12T04:58:09.905 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T04:58:09.910 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2024-03-12T04:58:09.917 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-03-12T04:58:09.924 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2024-03-12T04:58:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T04:58:09.936 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-03-12T04:58:09.940 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T04:58:09.949 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-03-12T04:58:09.950 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T04:58:09.957 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T04:58:09.966 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T04:58:09.982 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T04:58:10.000 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-03-12T04:58:10.003 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-12T04:58:10.003 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T04:58:10.004 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-03-12T04:58:10.004 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.004 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.004 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.004 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T04:58:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-12T04:58:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.026 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-03-12T04:58:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T04:58:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-03-12T04:58:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T04:58:10.030 INFO:teuthology.run_tasks:Running task install... 2024-03-12T04:58:10.039 DEBUG:teuthology.task.install:project ceph 2024-03-12T04:58:10.039 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}} 2024-03-12T04:58:10.039 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'} 2024-03-12T04:58:10.039 INFO:teuthology.task.install:Using flavor: default 2024-03-12T04:58:10.048 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-12T04:58:10.048 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-12T04:58:10.048 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-12T04:58:10.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:58:10.051 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-12T04:58:10.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:58:10.052 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-12T04:58:10.052 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T04:58:10.205 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-12T04:58:10.205 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-12T04:58:10.209 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-12T04:58:10.209 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-12T04:58:10.285 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-12T04:58:10.285 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-12T04:58:10.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T04:58:10.510 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T04:58:10.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T04:58:10.513 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T04:58:10.513 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T04:58:10.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T04:58:10.539 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T04:58:10.540 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T04:58:10.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T04:58:10.542 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T04:58:10.542 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T04:58:10.542 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T04:58:10.543 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T04:58:10.570 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T04:58:10.571 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T04:58:10.610 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T04:58:10.611 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T04:58:10.640 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T04:58:10.690 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T04:58:10.691 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T04:58:10.721 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T04:58:10.757 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2024-03-12T04:58:10.757 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-03-12T04:58:10.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2024-03-12T04:58:10.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-03-12T04:58:10.786 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-03-12T04:58:10.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-03-12T04:58:10.993 INFO:teuthology.orchestra.run.smithi156.stdout:74 files removed 2024-03-12T04:58:10.997 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-03-12T04:58:11.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T04:58:11.022 INFO:teuthology.orchestra.run.smithi161.stdout:74 files removed 2024-03-12T04:58:11.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T04:58:11.049 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T04:58:11.789 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 171 kB/s | 82 kB 00:00 2024-03-12T04:58:11.843 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 162 kB/s | 82 kB 00:00 2024-03-12T04:58:11.948 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 134 kB/s | 82 kB 00:00 2024-03-12T04:58:12.440 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 16 kB/s | 10 kB 00:00 2024-03-12T04:58:12.560 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-12T04:58:12.741 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-12T04:58:13.177 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-03-12T04:58:13.275 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-03-12T04:58:13.697 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-12T04:58:13.744 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-12T04:58:15.314 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-12T04:58:15.377 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-03-12T04:58:16.196 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-12T04:58:17.218 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-03-12T04:58:19.224 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 287 B/s | 1.9 kB 00:06 2024-03-12T04:58:19.505 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 19 MB 00:02 2024-03-12T04:58:19.667 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-12T04:58:21.068 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-12T04:58:21.377 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-03-12T04:58:24.139 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.1 MB 00:01 2024-03-12T04:58:24.925 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-03-12T04:58:25.532 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 15 kB 00:00 2024-03-12T04:58:26.144 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.1 MB 00:01 2024-03-12T04:58:27.945 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-12T04:58:30.808 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-03-12T04:58:35.184 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.1 MB 00:00 2024-03-12T04:58:36.743 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-12T04:58:38.486 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-12T04:58:43.394 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-12T04:58:43.436 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 21 MB 00:15 2024-03-12T04:58:45.824 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:58:45.825 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:58:45.838 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T04:58:45.838 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T04:58:45.839 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T04:58:45.840 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T04:58:45.842 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T04:58:45.845 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T04:58:45.846 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T04:58:45.928 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-12T04:58:45.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-12T04:58:45.949 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T04:58:45.950 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T04:58:45.951 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:58:45.952 INFO:teuthology.orchestra.run.smithi156.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-12T04:58:45.953 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-12T04:58:45.954 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-12T04:58:45.955 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-12T04:58:45.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T04:58:45.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T04:58:45.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T04:58:45.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T04:58:45.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T04:58:45.961 INFO:teuthology.orchestra.run.smithi156.stdout:Installing weak dependencies: 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout:Install 150 Packages 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 24 Packages 2024-03-12T04:58:45.962 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T04:58:45.963 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 223 M 2024-03-12T04:58:45.963 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-03-12T04:58:46.337 INFO:teuthology.orchestra.run.smithi156.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-12T04:58:46.538 INFO:teuthology.orchestra.run.smithi156.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 4.3 MB/s | 886 kB 00:00 2024-03-12T04:58:46.579 INFO:teuthology.orchestra.run.smithi156.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 3.4 MB/s | 145 kB 00:00 2024-03-12T04:58:46.813 INFO:teuthology.orchestra.run.smithi156.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 9.1 MB/s | 2.1 MB 00:00 2024-03-12T04:58:47.006 INFO:teuthology.orchestra.run.smithi156.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-03-12T04:58:47.580 INFO:teuthology.orchestra.run.smithi156.stdout:(6/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 8.1 MB/s | 4.7 MB 00:00 2024-03-12T04:58:47.797 INFO:teuthology.orchestra.run.smithi156.stdout:(7/174): ceph-common-19.0.0-1607.g94a53970.el9. 12 MB/s | 20 MB 00:01 2024-03-12T04:58:48.372 INFO:teuthology.orchestra.run.smithi156.stdout:(8/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 2.4 MB/s | 5.3 MB 00:02 2024-03-12T04:58:48.402 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-12T04:58:48.522 INFO:teuthology.orchestra.run.smithi156.stdout:(9/174): ceph-radosgw-19.0.0-1607.g94a53970.el9 13 MB/s | 9.8 MB 00:00 2024-03-12T04:58:48.547 INFO:teuthology.orchestra.run.smithi156.stdout:(10/174): ceph-selinux-19.0.0-1607.g94a53970.el 144 kB/s | 25 kB 00:00 2024-03-12T04:58:48.656 INFO:teuthology.orchestra.run.smithi156.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 2.0 MB/s | 217 kB 00:00 2024-03-12T04:58:48.748 INFO:teuthology.orchestra.run.smithi156.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 295 kB/s | 27 kB 00:00 2024-03-12T04:58:48.848 INFO:teuthology.orchestra.run.smithi156.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 312 kB/s | 31 kB 00:00 2024-03-12T04:58:49.057 INFO:teuthology.orchestra.run.smithi156.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 3.5 MB/s | 742 kB 00:00 2024-03-12T04:58:49.174 INFO:teuthology.orchestra.run.smithi156.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.4 MB/s | 163 kB 00:00 2024-03-12T04:58:49.349 INFO:teuthology.orchestra.run.smithi156.stdout:(16/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 9.5 MB/s | 17 MB 00:01 2024-03-12T04:58:49.374 INFO:teuthology.orchestra.run.smithi156.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 632 kB/s | 127 kB 00:00 2024-03-12T04:58:49.416 INFO:teuthology.orchestra.run.smithi156.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 7.3 MB/s | 498 kB 00:00 2024-03-12T04:58:49.492 INFO:teuthology.orchestra.run.smithi156.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 603 kB/s | 45 kB 00:00 2024-03-12T04:58:49.575 INFO:teuthology.orchestra.run.smithi156.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.5 MB/s | 131 kB 00:00 2024-03-12T04:58:49.667 INFO:teuthology.orchestra.run.smithi156.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.7 MB/s | 161 kB 00:00 2024-03-12T04:58:49.759 INFO:teuthology.orchestra.run.smithi156.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 3.4 MB/s | 320 kB 00:00 2024-03-12T04:58:49.851 INFO:teuthology.orchestra.run.smithi156.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 3.2 MB/s | 298 kB 00:00 2024-03-12T04:58:49.935 INFO:teuthology.orchestra.run.smithi156.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.2 MB/s | 99 kB 00:00 2024-03-12T04:58:50.043 INFO:teuthology.orchestra.run.smithi156.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 788 kB/s | 85 kB 00:00 2024-03-12T04:58:50.294 INFO:teuthology.orchestra.run.smithi156.stdout:(26/174): librgw2-19.0.0-1607.g94a53970.el9.x86 5.6 MB/s | 5.1 MB 00:00 2024-03-12T04:58:50.394 INFO:teuthology.orchestra.run.smithi156.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.7 MB/s | 171 kB 00:00 2024-03-12T04:58:50.453 INFO:teuthology.orchestra.run.smithi156.stdout:(28/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 7.6 MB/s | 3.1 MB 00:00 2024-03-12T04:58:50.469 INFO:teuthology.orchestra.run.smithi156.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1607.g 418 kB/s | 31 kB 00:00 2024-03-12T04:58:50.511 INFO:teuthology.orchestra.run.smithi156.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.3 MB/s | 138 kB 00:00 2024-03-12T04:58:50.845 INFO:teuthology.orchestra.run.smithi156.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 4.7 MB/s | 1.8 MB 00:00 2024-03-12T04:58:50.882 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:58:50.883 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:58:50.898 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T04:58:50.899 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T04:58:50.900 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T04:58:50.901 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T04:58:50.903 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T04:58:50.906 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T04:58:50.907 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T04:58:50.945 INFO:teuthology.orchestra.run.smithi156.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.4 MB/s | 247 kB 00:00 2024-03-12T04:58:50.987 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-12T04:58:51.007 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-12T04:58:51.008 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T04:58:51.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T04:58:51.010 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T04:58:51.011 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-12T04:58:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T04:58:51.013 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T04:58:51.014 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T04:58:51.015 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-12T04:58:51.016 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T04:58:51.017 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T04:58:51.018 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T04:58:51.019 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-12T04:58:51.020 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-12T04:58:51.021 INFO:teuthology.orchestra.run.smithi100.stdout:Install 150 Packages 2024-03-12T04:58:51.021 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 24 Packages 2024-03-12T04:58:51.021 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:58:51.021 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 223 M 2024-03-12T04:58:51.021 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-12T04:58:51.045 INFO:teuthology.orchestra.run.smithi156.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 495 kB/s | 49 kB 00:00 2024-03-12T04:58:51.138 INFO:teuthology.orchestra.run.smithi156.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 167 kB/s | 15 kB 00:00 2024-03-12T04:58:51.563 INFO:teuthology.orchestra.run.smithi156.stdout:(35/174): ceph-test-19.0.0-1607.g94a53970.el9.x 16 MB/s | 47 MB 00:03 2024-03-12T04:58:51.588 INFO:teuthology.orchestra.run.smithi156.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 586 kB/s | 264 kB 00:00 2024-03-12T04:58:51.688 INFO:teuthology.orchestra.run.smithi156.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-03-12T04:58:51.755 INFO:teuthology.orchestra.run.smithi156.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 109 kB/s | 18 kB 00:00 2024-03-12T04:58:51.797 INFO:teuthology.orchestra.run.smithi156.stdout:(39/174): cephadm-19.0.0-1607.g94a53970.el9.noa 3.1 MB/s | 748 kB 00:00 2024-03-12T04:58:51.845 INFO:teuthology.orchestra.run.smithi100.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-12T04:58:51.897 INFO:teuthology.orchestra.run.smithi156.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-12T04:58:51.923 INFO:teuthology.orchestra.run.smithi156.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 845 kB/s | 106 kB 00:00 2024-03-12T04:58:51.964 INFO:teuthology.orchestra.run.smithi156.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-03-12T04:58:51.989 INFO:teuthology.orchestra.run.smithi156.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-12T04:58:52.015 INFO:teuthology.orchestra.run.smithi156.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 347 kB/s | 32 kB 00:00 2024-03-12T04:58:52.057 INFO:teuthology.orchestra.run.smithi156.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 257 kB/s | 17 kB 00:00 2024-03-12T04:58:52.062 INFO:teuthology.orchestra.run.smithi100.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 4.0 MB/s | 886 kB 00:00 2024-03-12T04:58:52.116 INFO:teuthology.orchestra.run.smithi156.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-12T04:58:52.129 INFO:teuthology.orchestra.run.smithi100.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 2.1 MB/s | 145 kB 00:00 2024-03-12T04:58:52.132 INFO:teuthology.orchestra.run.smithi156.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 87 kB/s | 10 kB 00:00 2024-03-12T04:58:52.158 INFO:teuthology.orchestra.run.smithi156.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 159 kB 00:00 2024-03-12T04:58:52.174 INFO:teuthology.orchestra.run.smithi156.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-12T04:58:52.200 INFO:teuthology.orchestra.run.smithi156.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-12T04:58:52.225 INFO:teuthology.orchestra.run.smithi156.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-12T04:58:52.242 INFO:teuthology.orchestra.run.smithi156.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-12T04:58:52.267 INFO:teuthology.orchestra.run.smithi156.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-12T04:58:52.292 INFO:teuthology.orchestra.run.smithi156.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-03-12T04:58:52.317 INFO:teuthology.orchestra.run.smithi156.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-12T04:58:52.334 INFO:teuthology.orchestra.run.smithi156.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 514 kB/s | 34 kB 00:00 2024-03-12T04:58:52.427 INFO:teuthology.orchestra.run.smithi156.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-12T04:58:52.446 INFO:teuthology.orchestra.run.smithi100.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 6.7 MB/s | 2.1 MB 00:00 2024-03-12T04:58:52.468 INFO:teuthology.orchestra.run.smithi156.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-12T04:58:52.493 INFO:teuthology.orchestra.run.smithi156.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.1 MB/s | 811 kB 00:00 2024-03-12T04:58:52.519 INFO:teuthology.orchestra.run.smithi156.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-12T04:58:52.577 INFO:teuthology.orchestra.run.smithi156.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-12T04:58:52.613 INFO:teuthology.orchestra.run.smithi100.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 8.9 MB/s | 1.5 MB 00:00 2024-03-12T04:58:52.652 INFO:teuthology.orchestra.run.smithi156.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-12T04:58:52.678 INFO:teuthology.orchestra.run.smithi156.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 499 kB/s | 78 kB 00:00 2024-03-12T04:58:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:(6/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 5.0 MB/s | 5.3 MB 00:01 2024-03-12T04:58:52.778 INFO:teuthology.orchestra.run.smithi156.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-12T04:58:52.803 INFO:teuthology.orchestra.run.smithi156.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 200 kB/s | 24 kB 00:00 2024-03-12T04:58:52.820 INFO:teuthology.orchestra.run.smithi156.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-03-12T04:58:52.854 INFO:teuthology.orchestra.run.smithi156.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 4.3 MB/s | 144 kB 00:00 2024-03-12T04:58:52.870 INFO:teuthology.orchestra.run.smithi156.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-03-12T04:58:52.954 INFO:teuthology.orchestra.run.smithi156.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-12T04:58:52.979 INFO:teuthology.orchestra.run.smithi156.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 217 kB 00:00 2024-03-12T04:58:53.004 INFO:teuthology.orchestra.run.smithi156.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-12T04:58:53.032 INFO:teuthology.orchestra.run.smithi156.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 893 kB/s | 69 kB 00:00 2024-03-12T04:58:53.058 INFO:teuthology.orchestra.run.smithi156.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 789 kB/s | 41 kB 00:00 2024-03-12T04:58:53.064 INFO:teuthology.orchestra.run.smithi100.stdout:(7/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 10 MB/s | 4.7 MB 00:00 2024-03-12T04:58:53.082 INFO:teuthology.orchestra.run.smithi156.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.0 MB/s | 238 kB 00:00 2024-03-12T04:58:53.108 INFO:teuthology.orchestra.run.smithi156.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-12T04:58:53.133 INFO:teuthology.orchestra.run.smithi156.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-12T04:58:53.159 INFO:teuthology.orchestra.run.smithi156.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 628 kB/s | 47 kB 00:00 2024-03-12T04:58:53.184 INFO:teuthology.orchestra.run.smithi156.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 570 kB/s | 43 kB 00:00 2024-03-12T04:58:53.226 INFO:teuthology.orchestra.run.smithi156.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 580 kB/s | 39 kB 00:00 2024-03-12T04:58:53.276 INFO:teuthology.orchestra.run.smithi156.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 461 kB/s | 41 kB 00:00 2024-03-12T04:58:53.460 INFO:teuthology.orchestra.run.smithi156.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 52 MB/s | 16 MB 00:00 2024-03-12T04:58:53.485 INFO:teuthology.orchestra.run.smithi156.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-12T04:58:53.502 INFO:teuthology.orchestra.run.smithi156.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 253 kB/s | 56 kB 00:00 2024-03-12T04:58:53.519 INFO:teuthology.orchestra.run.smithi156.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-12T04:58:53.569 INFO:teuthology.orchestra.run.smithi156.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 661 kB/s | 44 kB 00:00 2024-03-12T04:58:53.594 INFO:teuthology.orchestra.run.smithi156.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 858 kB/s | 79 kB 00:00 2024-03-12T04:58:53.620 INFO:teuthology.orchestra.run.smithi156.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2024-03-12T04:58:53.661 INFO:teuthology.orchestra.run.smithi156.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-12T04:58:53.686 INFO:teuthology.orchestra.run.smithi156.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-12T04:58:53.712 INFO:teuthology.orchestra.run.smithi156.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-12T04:58:53.737 INFO:teuthology.orchestra.run.smithi156.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-12T04:58:53.846 INFO:teuthology.orchestra.run.smithi156.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-12T04:58:53.871 INFO:teuthology.orchestra.run.smithi156.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-03-12T04:58:53.888 INFO:teuthology.orchestra.run.smithi156.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-03-12T04:58:53.922 INFO:teuthology.orchestra.run.smithi156.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-12T04:58:53.964 INFO:teuthology.orchestra.run.smithi100.stdout:(8/174): ceph-radosgw-19.0.0-1607.g94a53970.el9 11 MB/s | 9.8 MB 00:00 2024-03-12T04:58:54.030 INFO:teuthology.orchestra.run.smithi156.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-12T04:58:54.031 INFO:teuthology.orchestra.run.smithi100.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 379 kB/s | 25 kB 00:00 2024-03-12T04:58:54.047 INFO:teuthology.orchestra.run.smithi156.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 122 kB/s | 21 kB 00:00 2024-03-12T04:58:54.197 INFO:teuthology.orchestra.run.smithi156.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-12T04:58:54.222 INFO:teuthology.orchestra.run.smithi156.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-12T04:58:54.281 INFO:teuthology.orchestra.run.smithi100.stdout:(10/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 10 MB/s | 17 MB 00:01 2024-03-12T04:58:54.298 INFO:teuthology.orchestra.run.smithi156.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 465 kB/s | 35 kB 00:00 2024-03-12T04:58:54.332 INFO:teuthology.orchestra.run.smithi156.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.6 MB/s | 216 kB 00:00 2024-03-12T04:58:54.381 INFO:teuthology.orchestra.run.smithi100.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 2.1 MB/s | 217 kB 00:00 2024-03-12T04:58:54.465 INFO:teuthology.orchestra.run.smithi100.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 325 kB/s | 27 kB 00:00 2024-03-12T04:58:54.532 INFO:teuthology.orchestra.run.smithi100.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 468 kB/s | 31 kB 00:00 2024-03-12T04:58:54.648 INFO:teuthology.orchestra.run.smithi100.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 6.2 MB/s | 742 kB 00:00 2024-03-12T04:58:54.707 INFO:teuthology.orchestra.run.smithi156.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 250 kB/s | 94 kB 00:00 2024-03-12T04:58:54.732 INFO:teuthology.orchestra.run.smithi156.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 102 kB/s | 44 kB 00:00 2024-03-12T04:58:54.741 INFO:teuthology.orchestra.run.smithi100.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.7 MB/s | 163 kB 00:00 2024-03-12T04:58:54.757 INFO:teuthology.orchestra.run.smithi156.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-03-12T04:58:54.816 INFO:teuthology.orchestra.run.smithi100.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.7 MB/s | 127 kB 00:00 2024-03-12T04:58:54.899 INFO:teuthology.orchestra.run.smithi156.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.2 MB/s | 4.1 MB 00:00 2024-03-12T04:58:54.924 INFO:teuthology.orchestra.run.smithi100.stdout:(17/174): libradosstriper1-19.0.0-1607.g94a5397 4.5 MB/s | 498 kB 00:00 2024-03-12T04:58:54.941 INFO:teuthology.orchestra.run.smithi156.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-03-12T04:58:55.200 INFO:teuthology.orchestra.run.smithi100.stdout:(18/174): ceph-common-19.0.0-1607.g94a53970.el9 5.6 MB/s | 20 MB 00:03 2024-03-12T04:58:55.292 INFO:teuthology.orchestra.run.smithi100.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 494 kB/s | 45 kB 00:00 2024-03-12T04:58:55.325 INFO:teuthology.orchestra.run.smithi156.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-03-12T04:58:55.400 INFO:teuthology.orchestra.run.smithi100.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.2 MB/s | 131 kB 00:00 2024-03-12T04:58:55.476 INFO:teuthology.orchestra.run.smithi100.stdout:(21/174): librgw2-19.0.0-1607.g94a53970.el9.x86 9.3 MB/s | 5.1 MB 00:00 2024-03-12T04:58:55.501 INFO:teuthology.orchestra.run.smithi100.stdout:(22/174): python3-cephfs-19.0.0-1607.g94a53970. 1.6 MB/s | 161 kB 00:00 2024-03-12T04:58:55.551 INFO:teuthology.orchestra.run.smithi100.stdout:(23/174): python3-rados-19.0.0-1607.g94a53970.e 4.2 MB/s | 320 kB 00:00 2024-03-12T04:58:55.593 INFO:teuthology.orchestra.run.smithi100.stdout:(24/174): python3-rbd-19.0.0-1607.g94a53970.el9 3.2 MB/s | 298 kB 00:00 2024-03-12T04:58:55.626 INFO:teuthology.orchestra.run.smithi100.stdout:(25/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.3 MB/s | 99 kB 00:00 2024-03-12T04:58:55.669 INFO:teuthology.orchestra.run.smithi100.stdout:(26/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-12T04:58:55.769 INFO:teuthology.orchestra.run.smithi100.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.7 MB/s | 171 kB 00:00 2024-03-12T04:58:55.861 INFO:teuthology.orchestra.run.smithi100.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 343 kB/s | 31 kB 00:00 2024-03-12T04:58:55.944 INFO:teuthology.orchestra.run.smithi100.stdout:(29/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 9.7 MB/s | 3.1 MB 00:00 2024-03-12T04:58:55.970 INFO:teuthology.orchestra.run.smithi100.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 1.2 MB/s | 138 kB 00:00 2024-03-12T04:58:56.145 INFO:teuthology.orchestra.run.smithi100.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 8.9 MB/s | 1.8 MB 00:00 2024-03-12T04:58:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.6 MB/s | 247 kB 00:00 2024-03-12T04:58:56.329 INFO:teuthology.orchestra.run.smithi100.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 538 kB/s | 49 kB 00:00 2024-03-12T04:58:56.413 INFO:teuthology.orchestra.run.smithi100.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 184 kB/s | 15 kB 00:00 2024-03-12T04:58:56.496 INFO:teuthology.orchestra.run.smithi100.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 3.1 MB/s | 264 kB 00:00 2024-03-12T04:58:56.613 INFO:teuthology.orchestra.run.smithi100.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 6.3 MB/s | 748 kB 00:00 2024-03-12T04:58:57.255 INFO:teuthology.orchestra.run.smithi100.stdout:(37/174): ceph-test-19.0.0-1607.g94a53970.el9.x 15 MB/s | 47 MB 00:03 2024-03-12T04:58:57.281 INFO:teuthology.orchestra.run.smithi100.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 27 kB/s | 18 kB 00:00 2024-03-12T04:58:57.291 INFO:teuthology.orchestra.run.smithi156.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 129 kB/s | 309 kB 00:02 2024-03-12T04:58:57.381 INFO:teuthology.orchestra.run.smithi100.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-03-12T04:58:57.492 INFO:teuthology.orchestra.run.smithi156.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-03-12T04:58:57.565 INFO:teuthology.orchestra.run.smithi100.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 805 kB/s | 106 kB 00:00 2024-03-12T04:58:57.614 INFO:teuthology.orchestra.run.smithi100.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2024-03-12T04:58:57.648 INFO:teuthology.orchestra.run.smithi100.stdout:(42/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-12T04:58:57.682 INFO:teuthology.orchestra.run.smithi100.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 307 kB 00:00 2024-03-12T04:58:57.699 INFO:teuthology.orchestra.run.smithi100.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 378 kB/s | 32 kB 00:00 2024-03-12T04:58:57.816 INFO:teuthology.orchestra.run.smithi100.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 17 kB 00:00 2024-03-12T04:58:57.817 INFO:teuthology.orchestra.run.smithi156.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 149 kB/s | 49 kB 00:00 2024-03-12T04:58:57.858 INFO:teuthology.orchestra.run.smithi100.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 63 kB/s | 10 kB 00:00 2024-03-12T04:58:57.925 INFO:teuthology.orchestra.run.smithi100.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-03-12T04:58:57.950 INFO:teuthology.orchestra.run.smithi100.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-03-12T04:58:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 379 kB/s | 44 kB 00:00 2024-03-12T04:58:58.000 INFO:teuthology.orchestra.run.smithi100.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-12T04:58:58.051 INFO:teuthology.orchestra.run.smithi100.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.7 MB/s | 578 kB 00:00 2024-03-12T04:58:58.076 INFO:teuthology.orchestra.run.smithi100.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 469 kB/s | 46 kB 00:00 2024-03-12T04:58:58.101 INFO:teuthology.orchestra.run.smithi100.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 282 kB 00:00 2024-03-12T04:58:58.126 INFO:teuthology.orchestra.run.smithi100.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-12T04:58:58.151 INFO:teuthology.orchestra.run.smithi100.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 693 kB/s | 34 kB 00:00 2024-03-12T04:58:58.176 INFO:teuthology.orchestra.run.smithi100.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-12T04:58:58.402 INFO:teuthology.orchestra.run.smithi100.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-03-12T04:58:58.510 INFO:teuthology.orchestra.run.smithi100.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 2.4 MB/s | 811 kB 00:00 2024-03-12T04:58:58.552 INFO:teuthology.orchestra.run.smithi100.stdout:(59/174): postgresql-13.11-1.el9.x86_64.rpm 3.8 MB/s | 1.5 MB 00:00 2024-03-12T04:58:58.586 INFO:teuthology.orchestra.run.smithi100.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 737 kB/s | 134 kB 00:00 2024-03-12T04:58:58.650 INFO:teuthology.orchestra.run.smithi156.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 81 kB/s | 67 kB 00:00 2024-03-12T04:58:58.720 INFO:teuthology.orchestra.run.smithi100.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-03-12T04:58:58.745 INFO:teuthology.orchestra.run.smithi100.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 501 kB/s | 78 kB 00:00 2024-03-12T04:58:58.778 INFO:teuthology.orchestra.run.smithi100.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-12T04:58:58.895 INFO:teuthology.orchestra.run.smithi100.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-12T04:58:58.921 INFO:teuthology.orchestra.run.smithi100.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-03-12T04:58:58.946 INFO:teuthology.orchestra.run.smithi100.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2024-03-12T04:58:58.971 INFO:teuthology.orchestra.run.smithi100.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-12T04:58:58.997 INFO:teuthology.orchestra.run.smithi100.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-03-12T04:58:59.130 INFO:teuthology.orchestra.run.smithi100.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-12T04:58:59.164 INFO:teuthology.orchestra.run.smithi100.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-12T04:58:59.189 INFO:teuthology.orchestra.run.smithi100.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 927 kB/s | 174 kB 00:00 2024-03-12T04:58:59.214 INFO:teuthology.orchestra.run.smithi100.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 833 kB/s | 69 kB 00:00 2024-03-12T04:58:59.240 INFO:teuthology.orchestra.run.smithi100.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 831 kB/s | 41 kB 00:00 2024-03-12T04:58:59.265 INFO:teuthology.orchestra.run.smithi100.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-12T04:58:59.290 INFO:teuthology.orchestra.run.smithi100.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-12T04:58:59.315 INFO:teuthology.orchestra.run.smithi100.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 954 kB/s | 47 kB 00:00 2024-03-12T04:58:59.340 INFO:teuthology.orchestra.run.smithi100.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-12T04:58:59.365 INFO:teuthology.orchestra.run.smithi100.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-12T04:58:59.699 INFO:teuthology.orchestra.run.smithi100.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-12T04:58:59.716 INFO:teuthology.orchestra.run.smithi100.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 111 kB/s | 39 kB 00:00 2024-03-12T04:58:59.733 INFO:teuthology.orchestra.run.smithi100.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 113 kB/s | 41 kB 00:00 2024-03-12T04:58:59.758 INFO:teuthology.orchestra.run.smithi100.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-12T04:58:59.775 INFO:teuthology.orchestra.run.smithi100.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-12T04:58:59.809 INFO:teuthology.orchestra.run.smithi100.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 835 kB/s | 63 kB 00:00 2024-03-12T04:58:59.834 INFO:teuthology.orchestra.run.smithi100.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 583 kB/s | 44 kB 00:00 2024-03-12T04:58:59.859 INFO:teuthology.orchestra.run.smithi100.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-12T04:58:59.884 INFO:teuthology.orchestra.run.smithi100.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-12T04:58:59.918 INFO:teuthology.orchestra.run.smithi100.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-12T04:58:59.943 INFO:teuthology.orchestra.run.smithi100.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-12T04:58:59.968 INFO:teuthology.orchestra.run.smithi100.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-12T04:58:59.994 INFO:teuthology.orchestra.run.smithi100.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-12T04:59:00.927 INFO:teuthology.orchestra.run.smithi100.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 190 kB/s | 187 kB 00:00 2024-03-12T04:59:00.977 INFO:teuthology.orchestra.run.smithi100.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 199 kB/s | 195 kB 00:00 2024-03-12T04:59:01.078 INFO:teuthology.orchestra.run.smithi100.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-03-12T04:59:01.128 INFO:teuthology.orchestra.run.smithi100.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-03-12T04:59:01.142 INFO:teuthology.orchestra.run.smithi156.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 61 kB/s | 151 kB 00:02 2024-03-12T04:59:01.761 INFO:teuthology.orchestra.run.smithi100.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-03-12T04:59:01.845 INFO:teuthology.orchestra.run.smithi100.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 434 kB/s | 803 kB 00:01 2024-03-12T04:59:01.962 INFO:teuthology.orchestra.run.smithi100.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-12T04:59:02.062 INFO:teuthology.orchestra.run.smithi100.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-03-12T04:59:02.087 INFO:teuthology.orchestra.run.smithi100.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-03-12T04:59:02.121 INFO:teuthology.orchestra.run.smithi100.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 597 kB/s | 35 kB 00:00 2024-03-12T04:59:02.146 INFO:teuthology.orchestra.run.smithi100.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-03-12T04:59:02.150 INFO:teuthology.orchestra.run.smithi156.stdout:(113/174): abseil-cpp-20211102.0-3.el9.x86_64.r 74 kB/s | 548 kB 00:07 2024-03-12T04:59:02.180 INFO:teuthology.orchestra.run.smithi100.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-12T04:59:02.256 INFO:teuthology.orchestra.run.smithi100.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-12T04:59:02.447 INFO:teuthology.orchestra.run.smithi100.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.1 MB/s | 4.1 MB 00:01 2024-03-12T04:59:02.490 INFO:teuthology.orchestra.run.smithi100.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-03-12T04:59:02.823 INFO:teuthology.orchestra.run.smithi100.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 58 kB/s | 19 kB 00:00 2024-03-12T04:59:02.981 INFO:teuthology.orchestra.run.smithi100.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 755 kB/s | 548 kB 00:00 2024-03-12T04:59:03.107 INFO:teuthology.orchestra.run.smithi100.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-03-12T04:59:03.132 INFO:teuthology.orchestra.run.smithi100.stdout:(110/174): gperftools-libs-2.9.1-2.el9.x86_64.r 451 kB/s | 309 kB 00:00 2024-03-12T04:59:03.224 INFO:teuthology.orchestra.run.smithi100.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-03-12T04:59:03.249 INFO:teuthology.orchestra.run.smithi100.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-03-12T04:59:03.333 INFO:teuthology.orchestra.run.smithi100.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-12T04:59:03.450 INFO:teuthology.orchestra.run.smithi100.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-03-12T04:59:03.491 INFO:teuthology.orchestra.run.smithi100.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-03-12T04:59:03.558 INFO:teuthology.orchestra.run.smithi100.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-03-12T04:59:03.650 INFO:teuthology.orchestra.run.smithi100.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-03-12T04:59:03.675 INFO:teuthology.orchestra.run.smithi100.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-03-12T04:59:03.701 INFO:teuthology.orchestra.run.smithi100.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 154 kB/s | 32 kB 00:00 2024-03-12T04:59:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-03-12T04:59:03.826 INFO:teuthology.orchestra.run.smithi100.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-12T04:59:03.852 INFO:teuthology.orchestra.run.smithi100.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-12T04:59:03.902 INFO:teuthology.orchestra.run.smithi100.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-03-12T04:59:03.936 INFO:teuthology.orchestra.run.smithi100.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-12T04:59:04.011 INFO:teuthology.orchestra.run.smithi100.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-12T04:59:04.071 INFO:teuthology.orchestra.run.smithi100.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 79 kB/s | 11 kB 00:00 2024-03-12T04:59:04.120 INFO:teuthology.orchestra.run.smithi100.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-03-12T04:59:04.153 INFO:teuthology.orchestra.run.smithi100.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 124 kB/s | 18 kB 00:00 2024-03-12T04:59:04.178 INFO:teuthology.orchestra.run.smithi100.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 216 kB/s | 23 kB 00:00 2024-03-12T04:59:04.221 INFO:teuthology.orchestra.run.smithi100.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-03-12T04:59:04.254 INFO:teuthology.orchestra.run.smithi100.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-03-12T04:59:04.296 INFO:teuthology.orchestra.run.smithi100.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-03-12T04:59:04.338 INFO:teuthology.orchestra.run.smithi100.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 394 kB/s | 46 kB 00:00 2024-03-12T04:59:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 627 kB/s | 79 kB 00:00 2024-03-12T04:59:04.439 INFO:teuthology.orchestra.run.smithi100.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 405 kB/s | 58 kB 00:00 2024-03-12T04:59:04.522 INFO:teuthology.orchestra.run.smithi100.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-03-12T04:59:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 673 kB/s | 90 kB 00:00 2024-03-12T04:59:04.664 INFO:teuthology.orchestra.run.smithi100.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-03-12T04:59:04.706 INFO:teuthology.orchestra.run.smithi100.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-03-12T04:59:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-03-12T04:59:04.823 INFO:teuthology.orchestra.run.smithi100.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-03-12T04:59:04.899 INFO:teuthology.orchestra.run.smithi100.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-03-12T04:59:04.941 INFO:teuthology.orchestra.run.smithi100.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-03-12T04:59:05.024 INFO:teuthology.orchestra.run.smithi100.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-12T04:59:05.058 INFO:teuthology.orchestra.run.smithi100.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 768 kB/s | 90 kB 00:00 2024-03-12T04:59:05.142 INFO:teuthology.orchestra.run.smithi100.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-12T04:59:05.167 INFO:teuthology.orchestra.run.smithi100.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-03-12T04:59:05.200 INFO:teuthology.orchestra.run.smithi100.stdout:(148/174): python3-pecan-1.4.2-3.el9.noarch.rpm 316 kB/s | 272 kB 00:00 2024-03-12T04:59:05.242 INFO:teuthology.orchestra.run.smithi100.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-12T04:59:05.651 INFO:teuthology.orchestra.run.smithi100.stdout:(150/174): librbd1-19.0.0-1607.g94a53970.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-03-12T04:59:05.693 INFO:teuthology.orchestra.run.smithi100.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-12T04:59:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:(152/174): librados2-19.0.0-1607.g94a53970.el9. 5.8 MB/s | 3.4 MB 00:00 2024-03-12T04:59:05.960 INFO:teuthology.orchestra.run.smithi100.stdout:(153/174): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-03-12T04:59:06.769 INFO:teuthology.orchestra.run.smithi100.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 54 kB/s | 42 kB 00:00 2024-03-12T04:59:07.919 INFO:teuthology.orchestra.run.smithi100.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 216 kB/s | 248 kB 00:01 2024-03-12T04:59:08.668 INFO:teuthology.orchestra.run.smithi100.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 95 kB/s | 71 kB 00:00 2024-03-12T04:59:09.008 INFO:teuthology.orchestra.run.smithi156.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 80 kB/s | 548 kB 00:06 2024-03-12T04:59:09.286 INFO:teuthology.orchestra.run.smithi100.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 532 kB/s | 1.8 MB 00:03 2024-03-12T04:59:09.344 INFO:teuthology.orchestra.run.smithi100.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 54 kB/s | 36 kB 00:00 2024-03-12T04:59:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 73 kB/s | 43 kB 00:00 2024-03-12T04:59:09.903 INFO:teuthology.orchestra.run.smithi100.stdout:(159/174): NetworkManager-1.46.0-1.el9.x86_64.r 557 kB/s | 2.3 MB 00:04 2024-03-12T04:59:10.237 INFO:teuthology.orchestra.run.smithi100.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 202 kB/s | 192 kB 00:00 2024-03-12T04:59:10.242 INFO:teuthology.orchestra.run.smithi156.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 50 kB/s | 32 kB 00:00 2024-03-12T04:59:10.428 INFO:teuthology.orchestra.run.smithi100.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 236 kB/s | 256 kB 00:01 2024-03-12T04:59:10.526 INFO:teuthology.orchestra.run.smithi156.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 53 kB/s | 15 kB 00:00 2024-03-12T04:59:10.537 INFO:teuthology.orchestra.run.smithi100.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 36 kB/s | 23 kB 00:00 2024-03-12T04:59:11.095 INFO:teuthology.orchestra.run.smithi100.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 126 kB/s | 108 kB 00:00 2024-03-12T04:59:11.168 INFO:teuthology.orchestra.run.smithi156.stdout:(118/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 84 kB/s | 838 kB 00:10 2024-03-12T04:59:11.371 INFO:teuthology.orchestra.run.smithi100.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 164 kB/s | 155 kB 00:00 2024-03-12T04:59:11.421 INFO:teuthology.orchestra.run.smithi100.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 153 kB/s | 136 kB 00:00 2024-03-12T04:59:11.880 INFO:teuthology.orchestra.run.smithi100.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 81 kB/s | 64 kB 00:00 2024-03-12T04:59:12.046 INFO:teuthology.orchestra.run.smithi100.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 44 kB/s | 29 kB 00:00 2024-03-12T04:59:12.518 INFO:teuthology.orchestra.run.smithi156.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 86 kB/s | 172 kB 00:01 2024-03-12T04:59:13.855 INFO:teuthology.orchestra.run.smithi100.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 509 kB/s | 1.2 MB 00:02 2024-03-12T04:59:14.534 INFO:teuthology.orchestra.run.smithi156.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 106 kB/s | 358 kB 00:03 2024-03-12T04:59:14.785 INFO:teuthology.orchestra.run.smithi156.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 141 kB/s | 35 kB 00:00 2024-03-12T04:59:15.180 INFO:teuthology.orchestra.run.smithi100.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 351 kB/s | 465 kB 00:01 2024-03-12T04:59:15.639 INFO:teuthology.orchestra.run.smithi100.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 587 kB/s | 2.2 MB 00:03 2024-03-12T04:59:15.727 INFO:teuthology.orchestra.run.smithi156.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 94 kB/s | 303 kB 00:03 2024-03-12T04:59:15.830 INFO:teuthology.orchestra.run.smithi100.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 618 kB/s | 2.3 MB 00:03 2024-03-12T04:59:15.855 INFO:teuthology.orchestra.run.smithi100.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 46 kB/s | 31 kB 00:00 2024-03-12T04:59:15.914 INFO:teuthology.orchestra.run.smithi100.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 897 kB/s | 74 kB 00:00 2024-03-12T04:59:16.711 INFO:teuthology.orchestra.run.smithi156.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 105 kB/s | 203 kB 00:01 2024-03-12T04:59:17.981 INFO:teuthology.orchestra.run.smithi100.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 469 kB/s | 1.1 MB 00:02 2024-03-12T04:59:17.986 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-12T04:59:17.986 INFO:teuthology.orchestra.run.smithi100.stdout:Total 8.3 MB/s | 223 MB 00:26 2024-03-12T04:59:18.352 INFO:teuthology.orchestra.run.smithi156.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 88 kB/s | 144 kB 00:01 2024-03-12T04:59:18.468 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-12T04:59:18.494 INFO:teuthology.orchestra.run.smithi156.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-03-12T04:59:18.769 INFO:teuthology.orchestra.run.smithi156.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 64 kB/s | 18 kB 00:00 2024-03-12T04:59:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-12T04:59:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-12T04:59:18.944 INFO:teuthology.orchestra.run.smithi156.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 132 kB/s | 23 kB 00:00 2024-03-12T04:59:19.103 INFO:teuthology.orchestra.run.smithi156.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 123 kB/s | 19 kB 00:00 2024-03-12T04:59:19.287 INFO:teuthology.orchestra.run.smithi156.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 107 kB/s | 20 kB 00:00 2024-03-12T04:59:20.801 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-12T04:59:20.801 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-12T04:59:27.725 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-12T04:59:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-12T04:59:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-12T04:59:27.907 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:59:27.917 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T04:59:27.957 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-12T04:59:28.016 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-12T04:59:28.052 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-12T04:59:28.078 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-12T04:59:28.126 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-12T04:59:28.447 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-12T04:59:28.504 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:59:28.608 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T04:59:28.663 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:59:28.746 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T04:59:28.782 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T04:59:28.839 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-12T04:59:29.027 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-12T04:59:29.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-12T04:59:29.105 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:59:29.341 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T04:59:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:59:29.513 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T04:59:29.560 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-12T04:59:29.583 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:59:29.599 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T04:59:29.759 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:59:29.789 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T04:59:29.821 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-12T04:59:29.999 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-12T04:59:30.161 INFO:teuthology.orchestra.run.smithi156.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 96 kB/s | 1.0 MB 00:10 2024-03-12T04:59:30.189 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:59:30.250 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T04:59:30.412 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:59:30.451 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:59:30.670 INFO:teuthology.orchestra.run.smithi156.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 91 kB/s | 46 kB 00:00 2024-03-12T04:59:30.685 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T04:59:30.798 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-12T04:59:30.892 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-12T04:59:30.923 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-12T04:59:30.950 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-12T04:59:30.985 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-12T04:59:31.058 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-12T04:59:31.105 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-12T04:59:31.143 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-12T04:59:31.208 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-12T04:59:31.243 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-12T04:59:31.286 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-12T04:59:31.319 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-12T04:59:31.347 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-12T04:59:31.415 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-12T04:59:31.451 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-12T04:59:31.494 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-12T04:59:31.524 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-12T04:59:31.591 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-12T04:59:31.638 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-12T04:59:31.711 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-12T04:59:31.736 INFO:teuthology.orchestra.run.smithi156.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 74 kB/s | 79 kB 00:01 2024-03-12T04:59:31.851 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-12T04:59:31.907 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-12T04:59:32.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-12T04:59:32.502 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-12T04:59:32.545 INFO:teuthology.orchestra.run.smithi156.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 71 kB/s | 58 kB 00:00 2024-03-12T04:59:32.580 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-12T04:59:32.639 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:59:32.648 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T04:59:32.680 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-12T04:59:32.737 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-12T04:59:32.775 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-12T04:59:32.805 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-12T04:59:33.071 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-12T04:59:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-12T04:59:33.351 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-12T04:59:33.394 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-12T04:59:33.530 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-12T04:59:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-12T04:59:33.644 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-12T04:59:33.717 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-12T04:59:33.748 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-12T04:59:33.784 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-12T04:59:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-12T04:59:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-12T04:59:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-12T04:59:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-12T04:59:34.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-12T04:59:34.141 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-12T04:59:34.172 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-12T04:59:34.213 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-12T04:59:34.306 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-12T04:59:34.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-12T04:59:34.378 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-12T04:59:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-12T04:59:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-12T04:59:34.525 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-12T04:59:34.653 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-12T04:59:34.761 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-12T04:59:34.803 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-12T04:59:34.851 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-12T04:59:34.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-12T04:59:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-12T04:59:35.149 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-12T04:59:35.245 INFO:teuthology.orchestra.run.smithi156.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 101 kB/s | 272 kB 00:02 2024-03-12T04:59:35.277 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-12T04:59:35.321 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-12T04:59:35.362 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-12T04:59:35.393 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-12T04:59:35.422 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-12T04:59:35.428 INFO:teuthology.orchestra.run.smithi156.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-03-12T04:59:35.456 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-12T04:59:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-12T04:59:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-12T04:59:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-12T04:59:35.633 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-12T04:59:35.667 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-12T04:59:35.700 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-12T04:59:35.742 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-12T04:59:35.776 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-12T04:59:35.872 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-12T04:59:35.891 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 300 kB/s | 21 MB 01:10 2024-03-12T04:59:35.962 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-12T04:59:36.003 INFO:teuthology.orchestra.run.smithi156.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 156 kB/s | 90 kB 00:00 2024-03-12T04:59:36.007 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-12T04:59:36.039 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-12T04:59:36.094 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-12T04:59:36.190 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-12T04:59:36.248 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-12T04:59:36.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-12T04:59:37.021 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-12T04:59:37.054 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-12T04:59:37.102 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-12T04:59:37.181 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-12T04:59:37.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-12T04:59:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-12T04:59:37.429 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-12T04:59:37.506 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-12T04:59:37.854 INFO:teuthology.orchestra.run.smithi156.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 15 kB/s | 28 kB 00:01 2024-03-12T04:59:37.895 INFO:teuthology.orchestra.run.smithi156.stdout:(138/174): python3-grpcio-1.46.7-10.el9.x86_64. 94 kB/s | 2.0 MB 00:22 2024-03-12T04:59:38.054 INFO:teuthology.orchestra.run.smithi156.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 154 kB/s | 31 kB 00:00 2024-03-12T04:59:38.146 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-12T04:59:38.262 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-12T04:59:38.521 INFO:teuthology.orchestra.run.smithi156.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 127 kB/s | 59 kB 00:00 2024-03-12T04:59:39.038 INFO:teuthology.orchestra.run.smithi156.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 69 kB/s | 36 kB 00:00 2024-03-12T04:59:39.478 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-12T04:59:39.557 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-12T04:59:39.588 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-12T04:59:39.627 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-12T04:59:39.654 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-12T04:59:39.855 INFO:teuthology.orchestra.run.smithi156.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 62 kB/s | 51 kB 00:00 2024-03-12T04:59:40.022 INFO:teuthology.orchestra.run.smithi156.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 89 kB/s | 188 kB 00:02 2024-03-12T04:59:40.027 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-12T04:59:40.118 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:59:40.412 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T04:59:40.556 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-12T04:59:40.593 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-12T04:59:40.774 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-03-12T04:59:40.847 INFO:teuthology.orchestra.run.smithi156.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 109 kB/s | 90 kB 00:00 2024-03-12T04:59:41.052 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-12T04:59:41.079 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:59:41.150 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T04:59:41.172 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-12T04:59:43.186 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:59:43.186 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T04:59:43.198 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T04:59:43.199 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T04:59:43.200 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T04:59:43.201 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T04:59:43.203 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T04:59:43.205 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T04:59:43.206 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T04:59:43.286 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================ 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================ 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-12T04:59:43.304 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-12T04:59:43.305 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-12T04:59:43.306 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T04:59:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T04:59:43.308 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T04:59:43.309 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-12T04:59:43.310 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-12T04:59:43.311 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-12T04:59:43.312 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T04:59:43.313 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-12T04:59:43.314 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T04:59:43.315 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T04:59:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================ 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout:Install 150 Packages 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 24 Packages 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T04:59:43.317 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:59:43.318 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 223 M 2024-03-12T04:59:43.318 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-03-12T04:59:43.361 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:59:43.488 INFO:teuthology.orchestra.run.smithi156.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 63 kB/s | 230 kB 00:03 2024-03-12T04:59:43.664 INFO:teuthology.orchestra.run.smithi156.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2024-03-12T04:59:43.895 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T04:59:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:59:44.024 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T04:59:44.024 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T04:59:44.024 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T04:59:44.024 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:59:44.065 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:59:44.067 INFO:teuthology.orchestra.run.smithi161.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-12T04:59:44.300 INFO:teuthology.orchestra.run.smithi161.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 3.7 MB/s | 886 kB 00:00 2024-03-12T04:59:44.368 INFO:teuthology.orchestra.run.smithi161.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 2.1 MB/s | 145 kB 00:00 2024-03-12T04:59:44.693 INFO:teuthology.orchestra.run.smithi161.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 6.5 MB/s | 2.1 MB 00:00 2024-03-12T04:59:44.835 INFO:teuthology.orchestra.run.smithi161.stdout:(5/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 5.7 MB/s | 5.3 MB 00:00 2024-03-12T04:59:44.926 INFO:teuthology.orchestra.run.smithi161.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-03-12T04:59:45.305 INFO:teuthology.orchestra.run.smithi156.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 96 kB/s | 427 kB 00:04 2024-03-12T04:59:45.393 INFO:teuthology.orchestra.run.smithi161.stdout:(7/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-03-12T04:59:46.122 INFO:teuthology.orchestra.run.smithi156.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 78 kB/s | 192 kB 00:02 2024-03-12T04:59:46.160 INFO:teuthology.orchestra.run.smithi161.stdout:(8/174): ceph-common-19.0.0-1607.g94a53970.el9. 8.8 MB/s | 20 MB 00:02 2024-03-12T04:59:46.210 INFO:teuthology.orchestra.run.smithi161.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 508 kB/s | 25 kB 00:00 2024-03-12T04:59:46.494 INFO:teuthology.orchestra.run.smithi161.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 8.9 MB/s | 9.8 MB 00:01 2024-03-12T04:59:46.561 INFO:teuthology.orchestra.run.smithi161.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 3.2 MB/s | 217 kB 00:00 2024-03-12T04:59:46.619 INFO:teuthology.orchestra.run.smithi161.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 465 kB/s | 27 kB 00:00 2024-03-12T04:59:46.678 INFO:teuthology.orchestra.run.smithi161.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 535 kB/s | 31 kB 00:00 2024-03-12T04:59:46.820 INFO:teuthology.orchestra.run.smithi161.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 5.1 MB/s | 742 kB 00:00 2024-03-12T04:59:46.856 INFO:teuthology.orchestra.run.smithi156.stdout:(149/174): librados2-19.0.0-1607.g94a53970.el9. 4.6 MB/s | 3.4 MB 00:00 2024-03-12T04:59:46.987 INFO:teuthology.orchestra.run.smithi161.stdout:(15/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 8.1 MB/s | 17 MB 00:02 2024-03-12T04:59:47.012 INFO:teuthology.orchestra.run.smithi161.stdout:(16/174): libcephsqlite-19.0.0-1607.g94a53970.e 851 kB/s | 163 kB 00:00 2024-03-12T04:59:47.046 INFO:teuthology.orchestra.run.smithi161.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 2.1 MB/s | 127 kB 00:00 2024-03-12T04:59:47.104 INFO:teuthology.orchestra.run.smithi161.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 5.3 MB/s | 498 kB 00:00 2024-03-12T04:59:47.180 INFO:teuthology.orchestra.run.smithi161.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 600 kB/s | 45 kB 00:00 2024-03-12T04:59:47.206 INFO:teuthology.orchestra.run.smithi156.stdout:(150/174): librbd1-19.0.0-1607.g94a53970.el9.x8 9.0 MB/s | 3.1 MB 00:00 2024-03-12T04:59:47.247 INFO:teuthology.orchestra.run.smithi161.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.9 MB/s | 131 kB 00:00 2024-03-12T04:59:47.248 INFO:teuthology.orchestra.run.smithi156.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-12T04:59:47.314 INFO:teuthology.orchestra.run.smithi161.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 2.4 MB/s | 161 kB 00:00 2024-03-12T04:59:47.389 INFO:teuthology.orchestra.run.smithi161.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 4.2 MB/s | 320 kB 00:00 2024-03-12T04:59:47.464 INFO:teuthology.orchestra.run.smithi161.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 3.9 MB/s | 298 kB 00:00 2024-03-12T04:59:47.531 INFO:teuthology.orchestra.run.smithi161.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.5 MB/s | 99 kB 00:00 2024-03-12T04:59:47.565 INFO:teuthology.orchestra.run.smithi156.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 7.2 MB/s | 2.3 MB 00:00 2024-03-12T04:59:47.598 INFO:teuthology.orchestra.run.smithi161.stdout:(25/174): librgw2-19.0.0-1607.g94a53970.el9.x86 9.3 MB/s | 5.1 MB 00:00 2024-03-12T04:59:47.623 INFO:teuthology.orchestra.run.smithi161.stdout:(26/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 930 kB/s | 85 kB 00:00 2024-03-12T04:59:47.691 INFO:teuthology.orchestra.run.smithi161.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-12T04:59:47.749 INFO:teuthology.orchestra.run.smithi161.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 540 kB/s | 31 kB 00:00 2024-03-12T04:59:47.765 INFO:teuthology.orchestra.run.smithi156.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 9.1 MB/s | 1.8 MB 00:00 2024-03-12T04:59:47.798 INFO:teuthology.orchestra.run.smithi156.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.2 MB/s | 42 kB 00:00 2024-03-12T04:59:47.816 INFO:teuthology.orchestra.run.smithi161.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.0 MB/s | 138 kB 00:00 2024-03-12T04:59:47.857 INFO:teuthology.orchestra.run.smithi156.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-12T04:59:47.891 INFO:teuthology.orchestra.run.smithi156.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2024-03-12T04:59:47.899 INFO:teuthology.orchestra.run.smithi161.stdout:(30/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 10 MB/s | 3.1 MB 00:00 2024-03-12T04:59:47.933 INFO:teuthology.orchestra.run.smithi156.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 874 kB/s | 36 kB 00:00 2024-03-12T04:59:47.975 INFO:teuthology.orchestra.run.smithi156.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-12T04:59:48.017 INFO:teuthology.orchestra.run.smithi156.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-12T04:59:48.066 INFO:teuthology.orchestra.run.smithi161.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 7.1 MB/s | 1.8 MB 00:00 2024-03-12T04:59:48.067 INFO:teuthology.orchestra.run.smithi156.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 461 kB/s | 23 kB 00:00 2024-03-12T04:59:48.109 INFO:teuthology.orchestra.run.smithi156.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-12T04:59:48.142 INFO:teuthology.orchestra.run.smithi161.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 3.2 MB/s | 247 kB 00:00 2024-03-12T04:59:48.159 INFO:teuthology.orchestra.run.smithi156.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-12T04:59:48.200 INFO:teuthology.orchestra.run.smithi161.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 849 kB/s | 49 kB 00:00 2024-03-12T04:59:48.234 INFO:teuthology.orchestra.run.smithi156.stdout:(163/174): libarrow-9.0.0-10.el9.x86_64.rpm 85 kB/s | 4.4 MB 00:52 2024-03-12T04:59:48.259 INFO:teuthology.orchestra.run.smithi161.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 263 kB/s | 15 kB 00:00 2024-03-12T04:59:48.260 INFO:teuthology.orchestra.run.smithi156.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-12T04:59:48.319 INFO:teuthology.orchestra.run.smithi156.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 492 kB/s | 29 kB 00:00 2024-03-12T04:59:48.334 INFO:teuthology.orchestra.run.smithi161.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 3.4 MB/s | 264 kB 00:00 2024-03-12T04:59:48.352 INFO:teuthology.orchestra.run.smithi156.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 546 kB/s | 64 kB 00:00 2024-03-12T04:59:48.493 INFO:teuthology.orchestra.run.smithi161.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-12T04:59:48.502 INFO:teuthology.orchestra.run.smithi156.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 1.2 MB 00:00 2024-03-12T04:59:48.560 INFO:teuthology.orchestra.run.smithi161.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 274 kB/s | 18 kB 00:00 2024-03-12T04:59:48.643 INFO:teuthology.orchestra.run.smithi161.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 9.9 MB/s | 7.4 MB 00:00 2024-03-12T04:59:48.794 INFO:teuthology.orchestra.run.smithi156.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.8 MB/s | 2.3 MB 00:00 2024-03-12T04:59:48.818 INFO:teuthology.orchestra.run.smithi161.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-03-12T04:59:48.860 INFO:teuthology.orchestra.run.smithi161.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 920 kB 00:00 2024-03-12T04:59:48.861 INFO:teuthology.orchestra.run.smithi156.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.2 MB/s | 2.2 MB 00:00 2024-03-12T04:59:48.886 INFO:teuthology.orchestra.run.smithi161.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-12T04:59:48.903 INFO:teuthology.orchestra.run.smithi156.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-03-12T04:59:48.911 INFO:teuthology.orchestra.run.smithi161.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-12T04:59:48.928 INFO:teuthology.orchestra.run.smithi161.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 770 kB/s | 32 kB 00:00 2024-03-12T04:59:48.928 INFO:teuthology.orchestra.run.smithi156.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 466 kB/s | 31 kB 00:00 2024-03-12T04:59:49.021 INFO:teuthology.orchestra.run.smithi156.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 811 kB/s | 74 kB 00:00 2024-03-12T04:59:49.036 INFO:teuthology.orchestra.run.smithi161.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-12T04:59:49.053 INFO:teuthology.orchestra.run.smithi161.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 17 kB 00:00 2024-03-12T04:59:49.054 INFO:teuthology.orchestra.run.smithi156.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-03-12T04:59:49.087 INFO:teuthology.orchestra.run.smithi161.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 304 kB/s | 10 kB 00:00 2024-03-12T04:59:49.112 INFO:teuthology.orchestra.run.smithi161.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-12T04:59:49.145 INFO:teuthology.orchestra.run.smithi161.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-12T04:59:49.171 INFO:teuthology.orchestra.run.smithi161.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-12T04:59:49.196 INFO:teuthology.orchestra.run.smithi161.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-12T04:59:49.221 INFO:teuthology.orchestra.run.smithi161.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 936 kB/s | 46 kB 00:00 2024-03-12T04:59:49.247 INFO:teuthology.orchestra.run.smithi161.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-12T04:59:49.272 INFO:teuthology.orchestra.run.smithi161.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-12T04:59:49.297 INFO:teuthology.orchestra.run.smithi161.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-12T04:59:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-03-12T04:59:49.398 INFO:teuthology.orchestra.run.smithi161.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 47 MB/s | 4.7 MB 00:00 2024-03-12T04:59:49.431 INFO:teuthology.orchestra.run.smithi161.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 14 MB/s | 1.5 MB 00:00 2024-03-12T04:59:49.457 INFO:teuthology.orchestra.run.smithi161.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-12T04:59:49.482 INFO:teuthology.orchestra.run.smithi161.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-12T04:59:49.583 INFO:teuthology.orchestra.run.smithi161.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 45 MB/s | 5.7 MB 00:00 2024-03-12T04:59:49.616 INFO:teuthology.orchestra.run.smithi161.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.7 MB/s | 1.0 MB 00:00 2024-03-12T04:59:49.641 INFO:teuthology.orchestra.run.smithi161.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-12T04:59:50.083 INFO:teuthology.orchestra.run.smithi161.stdout:(63/174): ceph-test-19.0.0-1607.g94a53970.el9.x 12 MB/s | 47 MB 00:03 2024-03-12T04:59:50.159 INFO:teuthology.orchestra.run.smithi161.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2024-03-12T04:59:50.174 INFO:teuthology.orchestra.run.smithi161.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 46 kB/s | 24 kB 00:00 2024-03-12T04:59:50.200 INFO:teuthology.orchestra.run.smithi161.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 159 kB/s | 18 kB 00:00 2024-03-12T04:59:50.225 INFO:teuthology.orchestra.run.smithi161.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-12T04:59:50.242 INFO:teuthology.orchestra.run.smithi161.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 473 kB/s | 32 kB 00:00 2024-03-12T04:59:50.326 INFO:teuthology.orchestra.run.smithi161.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-12T04:59:50.353 INFO:teuthology.orchestra.run.smithi161.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 217 kB 00:00 2024-03-12T04:59:50.376 INFO:teuthology.orchestra.run.smithi161.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-12T04:59:50.401 INFO:teuthology.orchestra.run.smithi161.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 917 kB/s | 69 kB 00:00 2024-03-12T04:59:50.418 INFO:teuthology.orchestra.run.smithi161.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 994 kB/s | 41 kB 00:00 2024-03-12T04:59:50.443 INFO:teuthology.orchestra.run.smithi161.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-12T04:59:50.469 INFO:teuthology.orchestra.run.smithi161.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-12T04:59:50.486 INFO:teuthology.orchestra.run.smithi161.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-12T04:59:50.511 INFO:teuthology.orchestra.run.smithi161.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-12T04:59:50.528 INFO:teuthology.orchestra.run.smithi161.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 733 kB/s | 43 kB 00:00 2024-03-12T04:59:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 52 MB/s | 16 MB 00:00 2024-03-12T04:59:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 150 kB/s | 41 kB 00:00 2024-03-12T04:59:50.829 INFO:teuthology.orchestra.run.smithi161.stdout:(81/174): python3-toml-0.10.2-6.el9.noarch.rpm 130 kB/s | 39 kB 00:00 2024-03-12T04:59:50.854 INFO:teuthology.orchestra.run.smithi161.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-12T04:59:50.879 INFO:teuthology.orchestra.run.smithi161.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 755 kB/s | 56 kB 00:00 2024-03-12T04:59:50.904 INFO:teuthology.orchestra.run.smithi161.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-12T04:59:50.929 INFO:teuthology.orchestra.run.smithi161.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-03-12T04:59:50.955 INFO:teuthology.orchestra.run.smithi161.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-12T04:59:50.980 INFO:teuthology.orchestra.run.smithi161.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-12T04:59:51.014 INFO:teuthology.orchestra.run.smithi161.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-12T04:59:51.030 INFO:teuthology.orchestra.run.smithi161.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-12T04:59:51.056 INFO:teuthology.orchestra.run.smithi161.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-12T04:59:51.081 INFO:teuthology.orchestra.run.smithi161.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-03-12T04:59:51.989 INFO:teuthology.orchestra.run.smithi161.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 195 kB/s | 187 kB 00:00 2024-03-12T04:59:52.048 INFO:teuthology.orchestra.run.smithi161.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 202 kB/s | 195 kB 00:00 2024-03-12T04:59:52.215 INFO:teuthology.orchestra.run.smithi161.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 129 kB/s | 21 kB 00:00 2024-03-12T04:59:52.282 INFO:teuthology.orchestra.run.smithi161.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-03-12T04:59:52.607 INFO:teuthology.orchestra.run.smithi161.stdout:(96/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 13 MB/s | 4.1 MB 00:00 2024-03-12T04:59:52.674 INFO:teuthology.orchestra.run.smithi161.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-12T04:59:52.741 INFO:teuthology.orchestra.run.smithi161.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-03-12T04:59:52.808 INFO:teuthology.orchestra.run.smithi161.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-12T04:59:52.866 INFO:teuthology.orchestra.run.smithi161.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 598 kB/s | 35 kB 00:00 2024-03-12T04:59:52.892 INFO:teuthology.orchestra.run.smithi161.stdout:(101/174): python3-requests-2.25.1-8.el9.noarch 138 kB/s | 125 kB 00:00 2024-03-12T04:59:52.934 INFO:teuthology.orchestra.run.smithi161.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 676 kB/s | 44 kB 00:00 2024-03-12T04:59:52.992 INFO:teuthology.orchestra.run.smithi161.stdout:(103/174): libgfortran-11.4.1-3.el9.x86_64.rpm 420 kB/s | 803 kB 00:01 2024-03-12T04:59:53.076 INFO:teuthology.orchestra.run.smithi161.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 193 kB/s | 22 kB 00:00 2024-03-12T04:59:53.101 INFO:teuthology.orchestra.run.smithi161.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 449 kB/s | 94 kB 00:00 2024-03-12T04:59:53.801 INFO:teuthology.orchestra.run.smithi161.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 441 kB/s | 309 kB 00:00 2024-03-12T04:59:53.868 INFO:teuthology.orchestra.run.smithi161.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 692 kB/s | 548 kB 00:00 2024-03-12T04:59:54.077 INFO:teuthology.orchestra.run.smithi161.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-03-12T04:59:54.235 INFO:teuthology.orchestra.run.smithi161.stdout:(109/174): protobuf-compiler-3.14.0-13.el9.x86_ 695 kB/s | 863 kB 00:01 2024-03-12T04:59:54.369 INFO:teuthology.orchestra.run.smithi161.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 365 kB/s | 49 kB 00:00 2024-03-12T04:59:54.436 INFO:teuthology.orchestra.run.smithi161.stdout:(111/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 72 kB/s | 26 kB 00:00 2024-03-12T04:59:54.486 INFO:teuthology.orchestra.run.smithi161.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-03-12T04:59:54.628 INFO:teuthology.orchestra.run.smithi161.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 787 kB/s | 151 kB 00:00 2024-03-12T04:59:54.811 INFO:teuthology.orchestra.run.smithi161.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-03-12T04:59:55.053 INFO:teuthology.orchestra.run.smithi161.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 180 kB/s | 43 kB 00:00 2024-03-12T04:59:55.078 INFO:teuthology.orchestra.run.smithi161.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-03-12T04:59:55.146 INFO:teuthology.orchestra.run.smithi161.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-03-12T04:59:55.171 INFO:teuthology.orchestra.run.smithi161.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-12T04:59:55.196 INFO:teuthology.orchestra.run.smithi161.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2024-03-12T04:59:55.371 INFO:teuthology.orchestra.run.smithi161.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 763 kB/s | 172 kB 00:00 2024-03-12T04:59:55.413 INFO:teuthology.orchestra.run.smithi161.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-03-12T04:59:55.555 INFO:teuthology.orchestra.run.smithi161.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 1.4 MB/s | 203 kB 00:00 2024-03-12T04:59:55.638 INFO:teuthology.orchestra.run.smithi161.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 768 kB/s | 358 kB 00:00 2024-03-12T04:59:55.821 INFO:teuthology.orchestra.run.smithi161.stdout:(124/174): python3-colorama-0.4.4-12.el9.noarch 78 kB/s | 35 kB 00:00 2024-03-12T04:59:55.930 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T04:59:55.930 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-03-12T04:59:55.930 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-03-12T04:59:55.930 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-03-12T04:59:55.930 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-03-12T04:59:55.931 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-03-12T04:59:55.931 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-03-12T04:59:55.931 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-03-12T04:59:55.931 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:59:55.938 INFO:teuthology.orchestra.run.smithi161.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 481 kB/s | 144 kB 00:00 2024-03-12T04:59:56.022 INFO:teuthology.orchestra.run.smithi161.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-03-12T04:59:56.047 INFO:teuthology.orchestra.run.smithi161.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 47 kB/s | 11 kB 00:00 2024-03-12T04:59:56.096 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:59:56.106 INFO:teuthology.orchestra.run.smithi161.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 106 kB/s | 18 kB 00:00 2024-03-12T04:59:56.114 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T04:59:56.114 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:59:56.114 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T04:59:56.114 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:59:56.131 INFO:teuthology.orchestra.run.smithi161.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-03-12T04:59:56.156 INFO:teuthology.orchestra.run.smithi161.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-03-12T04:59:56.315 INFO:teuthology.orchestra.run.smithi161.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 94 kB/s | 20 kB 00:00 2024-03-12T04:59:56.340 INFO:teuthology.orchestra.run.smithi161.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-03-12T04:59:56.523 INFO:teuthology.orchestra.run.smithi161.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-03-12T04:59:56.549 INFO:teuthology.orchestra.run.smithi161.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 277 kB/s | 58 kB 00:00 2024-03-12T04:59:56.591 INFO:teuthology.orchestra.run.smithi161.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 394 kB/s | 16 kB 00:00 2024-03-12T04:59:56.641 INFO:teuthology.orchestra.run.smithi161.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-12T04:59:56.708 INFO:teuthology.orchestra.run.smithi161.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-03-12T04:59:56.733 INFO:teuthology.orchestra.run.smithi161.stdout:(138/174): python3-more-itertools-8.12.0-2.el9. 188 kB/s | 79 kB 00:00 2024-03-12T04:59:56.758 INFO:teuthology.orchestra.run.smithi161.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-03-12T04:59:56.892 INFO:teuthology.orchestra.run.smithi161.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-03-12T04:59:56.942 INFO:teuthology.orchestra.run.smithi161.stdout:(141/174): python3-pecan-1.4.2-3.el9.noarch.rpm 650 kB/s | 272 kB 00:00 2024-03-12T04:59:56.976 INFO:teuthology.orchestra.run.smithi161.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-03-12T04:59:57.001 INFO:teuthology.orchestra.run.smithi161.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 704 kB/s | 188 kB 00:00 2024-03-12T04:59:57.068 INFO:teuthology.orchestra.run.smithi161.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-03-12T04:59:57.177 INFO:teuthology.orchestra.run.smithi161.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-03-12T04:59:57.202 INFO:teuthology.orchestra.run.smithi161.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 447 kB/s | 90 kB 00:00 2024-03-12T04:59:57.336 INFO:teuthology.orchestra.run.smithi161.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-03-12T04:59:57.436 INFO:teuthology.orchestra.run.smithi161.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 85 kB/s | 20 kB 00:00 2024-03-12T04:59:57.503 INFO:teuthology.orchestra.run.smithi161.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 638 kB/s | 192 kB 00:00 2024-03-12T04:59:57.582 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:59:57.608 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T04:59:57.608 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:59:57.608 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T04:59:57.608 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:59:57.608 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T04:59:57.608 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:59:57.770 INFO:teuthology.orchestra.run.smithi161.stdout:(150/174): librados2-19.0.0-1607.g94a53970.el9. 10 MB/s | 3.4 MB 00:00 2024-03-12T04:59:57.812 INFO:teuthology.orchestra.run.smithi161.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-12T04:59:57.816 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:59:57.839 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T04:59:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:59:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T04:59:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:59:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T04:59:57.839 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:59:57.887 INFO:teuthology.orchestra.run.smithi161.stdout:(152/174): librbd1-19.0.0-1607.g94a53970.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-03-12T04:59:57.937 INFO:teuthology.orchestra.run.smithi161.stdout:(153/174): thrift-0.15.0-2.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-03-12T04:59:58.275 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:59:58.298 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T04:59:58.298 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T04:59:58.298 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T04:59:58.299 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:59:58.299 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T04:59:58.299 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T04:59:58.402 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-12T04:59:58.612 INFO:teuthology.orchestra.run.smithi161.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 63 kB/s | 42 kB 00:00 2024-03-12T04:59:58.819 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:59:58.859 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T04:59:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-12T04:59:58.950 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-12T04:59:58.973 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:59:59.203 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T04:59:59.233 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T04:59:59.471 INFO:teuthology.orchestra.run.smithi161.stdout:(155/174): NetworkManager-1.46.0-1.el9.x86_64.r 1.4 MB/s | 2.3 MB 00:01 2024-03-12T04:59:59.637 INFO:teuthology.orchestra.run.smithi161.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 242 kB/s | 248 kB 00:01 2024-03-12T05:00:00.196 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T05:00:00.233 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T05:00:00.246 INFO:teuthology.orchestra.run.smithi161.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 92 kB/s | 71 kB 00:00 2024-03-12T05:00:00.313 INFO:teuthology.orchestra.run.smithi161.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 54 kB/s | 36 kB 00:00 2024-03-12T05:00:00.350 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T05:00:00.520 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-12T05:00:00.543 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T05:00:00.567 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T05:00:00.567 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:00.568 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T05:00:00.568 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T05:00:00.568 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T05:00:00.568 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:00.619 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T05:00:00.631 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T05:00:00.690 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-12T05:00:01.088 INFO:teuthology.orchestra.run.smithi161.stdout:(159/174): NetworkManager-libnm-1.46.0-1.el9.x8 581 kB/s | 1.8 MB 00:03 2024-03-12T05:00:01.163 INFO:teuthology.orchestra.run.smithi161.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 210 kB/s | 192 kB 00:00 2024-03-12T05:00:01.314 INFO:teuthology.orchestra.run.smithi161.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 256 kB/s | 256 kB 00:01 2024-03-12T05:00:01.547 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-12T05:00:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T05:00:01.594 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T05:00:01.595 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:01.595 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T05:00:01.595 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T05:00:01.595 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T05:00:01.595 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:01.631 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T05:00:01.652 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T05:00:01.653 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:01.653 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T05:00:01.653 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:01.698 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:01.721 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:01.721 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:01.721 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T05:00:01.721 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T05:00:01.721 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T05:00:01.721 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:01.747 INFO:teuthology.orchestra.run.smithi161.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 35 kB/s | 23 kB 00:00 2024-03-12T05:00:01.964 INFO:teuthology.orchestra.run.smithi161.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 135 kB/s | 108 kB 00:00 2024-03-12T05:00:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T05:00:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T05:00:02.047 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:02.047 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T05:00:02.047 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T05:00:02.047 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T05:00:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:02.214 INFO:teuthology.orchestra.run.smithi161.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 172 kB/s | 155 kB 00:00 2024-03-12T05:00:02.648 INFO:teuthology.orchestra.run.smithi161.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 151 kB/s | 136 kB 00:00 2024-03-12T05:00:02.723 INFO:teuthology.orchestra.run.smithi161.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 84 kB/s | 64 kB 00:00 2024-03-12T05:00:02.907 INFO:teuthology.orchestra.run.smithi161.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 43 kB/s | 29 kB 00:00 2024-03-12T05:00:03.294 INFO:teuthology.orchestra.run.smithi156.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 90 kB/s | 1.6 MB 00:17 2024-03-12T05:00:03.297 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-03-12T05:00:03.297 INFO:teuthology.orchestra.run.smithi156.stdout:Total 2.9 MB/s | 223 MB 01:17 2024-03-12T05:00:03.749 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-03-12T05:00:04.018 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-03-12T05:00:04.019 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-03-12T05:00:05.057 INFO:teuthology.orchestra.run.smithi161.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 515 kB/s | 1.2 MB 00:02 2024-03-12T05:00:05.834 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-03-12T05:00:05.835 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-03-12T05:00:06.398 INFO:teuthology.orchestra.run.smithi161.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 347 kB/s | 465 kB 00:01 2024-03-12T05:00:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T05:00:06.440 INFO:teuthology.orchestra.run.smithi161.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 594 kB/s | 2.2 MB 00:03 2024-03-12T05:00:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-12T05:00:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-12T05:00:06.552 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-12T05:00:06.726 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-12T05:00:06.824 INFO:teuthology.orchestra.run.smithi161.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 597 kB/s | 2.3 MB 00:03 2024-03-12T05:00:06.842 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-12T05:00:06.891 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-12T05:00:06.944 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-12T05:00:07.001 INFO:teuthology.orchestra.run.smithi161.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 422 kB/s | 74 kB 00:00 2024-03-12T05:00:07.017 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-12T05:00:07.043 INFO:teuthology.orchestra.run.smithi161.stdout:(173/174): util-linux-user-2.37.4-18.el9.x86_64 48 kB/s | 31 kB 00:00 2024-03-12T05:00:07.059 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-12T05:00:07.084 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-12T05:00:07.125 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-12T05:00:07.154 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-12T05:00:07.210 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-12T05:00:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-12T05:00:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-12T05:00:07.891 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:07.926 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:07.946 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:08.759 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-12T05:00:08.835 INFO:teuthology.orchestra.run.smithi161.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 459 kB/s | 1.1 MB 00:02 2024-03-12T05:00:08.840 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-03-12T05:00:08.840 INFO:teuthology.orchestra.run.smithi161.stdout:Total 8.7 MB/s | 223 MB 00:25 2024-03-12T05:00:08.854 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-12T05:00:08.896 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-12T05:00:08.991 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-12T05:00:09.025 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-12T05:00:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-12T05:00:09.075 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-12T05:00:09.101 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-12T05:00:09.102 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T05:00:09.189 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T05:00:09.219 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-12T05:00:09.219 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T05:00:09.302 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-03-12T05:00:09.304 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T05:00:09.330 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-12T05:00:09.363 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-12T05:00:09.389 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-12T05:00:09.414 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-12T05:00:09.447 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-12T05:00:09.486 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-12T05:00:09.519 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-12T05:00:09.543 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-12T05:00:09.569 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-12T05:00:09.577 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-03-12T05:00:09.577 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-03-12T05:00:09.594 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-12T05:00:09.619 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-12T05:00:09.626 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:09.626 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:09.816 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:09.816 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T05:00:09.899 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T05:00:09.924 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-12T05:00:09.949 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-12T05:00:09.982 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-12T05:00:10.008 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-12T05:00:10.074 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T05:00:11.294 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-03-12T05:00:11.295 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-03-12T05:00:12.654 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-03-12T05:00:12.789 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-12T05:00:12.823 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-12T05:00:12.854 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T05:00:12.863 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T05:00:12.906 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-12T05:00:12.961 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-12T05:00:12.993 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-12T05:00:13.019 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-12T05:00:13.061 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-12T05:00:13.402 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-12T05:00:13.461 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:00:13.568 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:00:13.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:00:13.670 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:00:13.700 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T05:00:13.734 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-12T05:00:13.911 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-12T05:00:13.965 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-12T05:00:13.993 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:00:14.079 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:00:14.102 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T05:00:14.150 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T05:00:14.198 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-12T05:00:14.221 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T05:00:14.236 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T05:00:14.404 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T05:00:14.435 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T05:00:14.467 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-12T05:00:14.570 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-12T05:00:14.678 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T05:00:14.721 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T05:00:14.799 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T05:00:14.856 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T05:00:14.988 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T05:00:14.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-12T05:00:14.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-12T05:00:14.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-12T05:00:14.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-12T05:00:14.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-12T05:00:14.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-12T05:00:14.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-12T05:00:14.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-12T05:00:14.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-12T05:00:14.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:00:14.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:00:14.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T05:00:14.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-12T05:00:14.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-12T05:00:14.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-12T05:00:14.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:00:14.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-12T05:00:14.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-12T05:00:14.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-12T05:00:14.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-12T05:00:14.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-12T05:00:14.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-12T05:00:14.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-12T05:00:14.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-12T05:00:14.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-12T05:00:14.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-12T05:00:14.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-12T05:00:14.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-12T05:00:14.993 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-12T05:00:14.993 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-12T05:00:14.993 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-12T05:00:14.993 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-12T05:00:14.993 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-12T05:00:14.993 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-12T05:00:14.994 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-12T05:00:14.994 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-12T05:00:14.994 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-12T05:00:14.994 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-12T05:00:14.994 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-12T05:00:14.995 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-12T05:00:14.995 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-12T05:00:14.995 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-12T05:00:14.995 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-12T05:00:14.995 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-12T05:00:14.995 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-12T05:00:14.996 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-12T05:00:14.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-12T05:00:14.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-12T05:00:14.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-12T05:00:15.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-12T05:00:15.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-12T05:00:15.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-12T05:00:15.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-12T05:00:15.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-12T05:00:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-12T05:00:15.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-12T05:00:15.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-12T05:00:15.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-12T05:00:15.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-12T05:00:15.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-12T05:00:15.068 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T05:00:15.177 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-12T05:00:15.272 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-12T05:00:15.303 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-12T05:00:15.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-12T05:00:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-12T05:00:15.439 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-12T05:00:15.493 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-12T05:00:15.531 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-12T05:00:15.595 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-12T05:00:15.630 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-12T05:00:15.674 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-12T05:00:15.706 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-12T05:00:15.735 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-12T05:00:15.803 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-12T05:00:15.840 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-12T05:00:15.883 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-12T05:00:15.912 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-12T05:00:15.980 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-12T05:00:16.026 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-12T05:00:16.099 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-12T05:00:16.232 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-12T05:00:16.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-12T05:00:16.808 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-12T05:00:16.882 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-12T05:00:16.959 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-12T05:00:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T05:00:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T05:00:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-12T05:00:17.083 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-12T05:00:17.121 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-12T05:00:17.153 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-12T05:00:17.378 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-12T05:00:17.579 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-12T05:00:17.646 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-12T05:00:17.690 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-12T05:00:17.835 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-12T05:00:17.876 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-12T05:00:17.931 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-12T05:00:18.012 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-12T05:00:18.044 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-12T05:00:18.081 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-12T05:00:18.114 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-03-12T05:00:18.114 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-12T05:00:18.114 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T05:00:18.115 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T05:00:18.116 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.117 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.118 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T05:00:18.119 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T05:00:18.120 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-03-12T05:00:18.121 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T05:00:18.122 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T05:00:18.123 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T05:00:18.124 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T05:00:18.125 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T05:00:18.126 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-12T05:00:18.127 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T05:00:18.128 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T05:00:18.129 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-12T05:00:18.129 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T05:00:18.129 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:00:18.129 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-12T05:00:18.163 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-12T05:00:18.206 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-12T05:00:18.249 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-12T05:00:18.270 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-12T05:00:18.323 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-12T05:00:18.353 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T05:00:18.363 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-12T05:00:18.382 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-12T05:00:18.420 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-12T05:00:18.434 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-12T05:00:18.464 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-12T05:00:18.474 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-12T05:00:18.505 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-12T05:00:18.506 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-12T05:00:18.541 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-12T05:00:18.580 DEBUG:teuthology.parallel:result is None 2024-03-12T05:00:18.583 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-12T05:00:18.600 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-12T05:00:18.640 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-12T05:00:18.671 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-12T05:00:18.697 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-12T05:00:18.721 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-12T05:00:18.794 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-12T05:00:18.905 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-12T05:00:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-12T05:00:18.967 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:00:19.024 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-12T05:00:19.088 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:00:19.126 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:00:19.199 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:00:19.230 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T05:00:19.276 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-12T05:00:19.279 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-12T05:00:19.379 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-12T05:00:19.442 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-12T05:00:19.450 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-12T05:00:19.503 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-12T05:00:19.531 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:00:19.617 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:00:19.626 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-12T05:00:19.640 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T05:00:19.668 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-12T05:00:19.703 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-12T05:00:19.744 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-12T05:00:19.767 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T05:00:19.782 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-12T05:00:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-12T05:00:19.841 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-12T05:00:19.873 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-12T05:00:19.903 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-12T05:00:19.932 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-12T05:00:19.933 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T05:00:19.964 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-12T05:00:19.966 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-12T05:00:19.996 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-12T05:00:20.013 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-12T05:00:20.045 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-12T05:00:20.075 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-12T05:00:20.103 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-12T05:00:20.151 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-12T05:00:20.186 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-12T05:00:20.206 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T05:00:20.219 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-12T05:00:20.258 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-12T05:00:20.260 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-12T05:00:20.295 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-12T05:00:20.321 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T05:00:20.376 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T05:00:20.382 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-12T05:00:20.465 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-12T05:00:20.509 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-12T05:00:20.541 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-12T05:00:20.582 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-12T05:00:20.588 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-12T05:00:20.684 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-12T05:00:20.691 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-12T05:00:20.734 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-12T05:00:20.785 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-12T05:00:20.822 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-12T05:00:20.849 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-12T05:00:20.884 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-12T05:00:20.958 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-12T05:00:21.013 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-12T05:00:21.051 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-12T05:00:21.116 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-12T05:00:21.158 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-12T05:00:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-12T05:00:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-12T05:00:21.262 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-12T05:00:21.331 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-12T05:00:21.367 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-12T05:00:21.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-12T05:00:21.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-12T05:00:21.453 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-12T05:00:21.506 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-12T05:00:21.510 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-12T05:00:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-12T05:00:21.554 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-12T05:00:21.588 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-12T05:00:21.628 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-12T05:00:21.667 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-12T05:00:21.717 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-12T05:00:21.778 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-12T05:00:21.832 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-12T05:00:21.894 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-12T05:00:21.949 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-12T05:00:22.033 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-12T05:00:22.354 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-12T05:00:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-12T05:00:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-12T05:00:22.554 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T05:00:22.562 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-12T05:00:22.595 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-12T05:00:22.643 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-12T05:00:22.673 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-12T05:00:22.681 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-12T05:00:22.711 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-12T05:00:22.786 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-12T05:00:22.944 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-12T05:00:23.141 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-12T05:00:23.188 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-12T05:00:23.240 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-12T05:00:23.379 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-12T05:00:23.427 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-12T05:00:23.481 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-12T05:00:23.545 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-12T05:00:23.577 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-12T05:00:23.614 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-12T05:00:23.660 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-12T05:00:23.696 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-12T05:00:23.739 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-12T05:00:23.781 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-12T05:00:23.915 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-12T05:00:23.962 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-12T05:00:23.999 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-12T05:00:24.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-12T05:00:24.042 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-12T05:00:24.124 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-12T05:00:24.134 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-12T05:00:24.155 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-12T05:00:24.186 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-12T05:00:24.191 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-12T05:00:24.214 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-12T05:00:24.231 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-12T05:00:24.257 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-12T05:00:24.281 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-12T05:00:24.354 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-12T05:00:24.473 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-12T05:00:24.520 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-12T05:00:24.553 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T05:00:24.573 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-12T05:00:24.614 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-12T05:00:24.662 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-12T05:00:24.718 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-12T05:00:24.734 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T05:00:24.766 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-12T05:00:24.802 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-12T05:00:24.902 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-12T05:00:24.952 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-12T05:00:25.079 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-12T05:00:25.124 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-12T05:00:25.157 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-12T05:00:25.188 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-12T05:00:25.225 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-12T05:00:25.255 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-12T05:00:25.258 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-12T05:00:25.289 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T05:00:25.298 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-12T05:00:25.329 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-12T05:00:25.359 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-12T05:00:25.360 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T05:00:25.382 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-12T05:00:25.434 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-12T05:00:25.469 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-12T05:00:25.502 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-12T05:00:25.544 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-12T05:00:25.578 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-12T05:00:25.666 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-12T05:00:25.758 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-12T05:00:25.801 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-12T05:00:25.833 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-12T05:00:25.879 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-12T05:00:25.984 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-12T05:00:26.042 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-12T05:00:26.773 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-12T05:00:26.814 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-12T05:00:26.847 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-12T05:00:26.895 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-12T05:00:26.974 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-12T05:00:27.015 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-12T05:00:27.194 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-12T05:00:27.230 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-12T05:00:27.308 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-12T05:00:27.511 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T05:00:27.546 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T05:00:27.950 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-12T05:00:28.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-12T05:00:28.077 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T05:00:28.108 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T05:00:28.189 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T05:00:28.189 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T05:00:28.189 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T05:00:28.190 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:28.217 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T05:00:29.291 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-12T05:00:29.379 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-12T05:00:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-12T05:00:29.441 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-12T05:00:29.469 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-12T05:00:29.777 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-12T05:00:29.807 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T05:00:29.897 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-12T05:00:29.929 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-12T05:00:29.965 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-12T05:00:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-12T05:00:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T05:00:30.523 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-12T05:00:30.553 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-12T05:00:32.710 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T05:00:32.741 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T05:00:33.280 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-12T05:00:33.319 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T05:00:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-12T05:00:33.385 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T05:00:33.385 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T05:00:33.385 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:33.412 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T05:00:39.729 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T05:00:39.729 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2024-03-12T05:00:39.729 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2024-03-12T05:00:39.729 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2024-03-12T05:00:39.729 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2024-03-12T05:00:39.729 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2024-03-12T05:00:39.730 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2024-03-12T05:00:39.730 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2024-03-12T05:00:39.730 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:39.890 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T05:00:39.905 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T05:00:39.905 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:39.905 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T05:00:39.905 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:41.384 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T05:00:41.409 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T05:00:41.409 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:41.409 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T05:00:41.409 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T05:00:41.410 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T05:00:41.410 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:41.627 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T05:00:41.650 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T05:00:41.650 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:41.650 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T05:00:41.650 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T05:00:41.650 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T05:00:41.650 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:42.104 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T05:00:42.126 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T05:00:42.127 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:42.127 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T05:00:42.127 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T05:00:42.127 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T05:00:42.127 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:42.214 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-12T05:00:42.583 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T05:00:42.613 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T05:00:42.645 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-12T05:00:42.696 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-12T05:00:42.720 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T05:00:42.951 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T05:00:42.979 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T05:00:43.936 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T05:00:43.963 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T05:00:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T05:00:44.216 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-12T05:00:44.248 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T05:00:44.271 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T05:00:44.271 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:44.271 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T05:00:44.271 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T05:00:44.271 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T05:00:44.272 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:44.315 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T05:00:44.326 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T05:00:44.371 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-12T05:00:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-12T05:00:45.099 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-03-12T05:00:45.099 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-03-12T05:00:45.100 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-03-12T05:00:45.100 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-03-12T05:00:45.100 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-03-12T05:00:45.100 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-03-12T05:00:45.100 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-03-12T05:00:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:45.222 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-12T05:00:45.259 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T05:00:45.261 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T05:00:45.274 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-12T05:00:45.274 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:45.274 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T05:00:45.274 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:45.284 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T05:00:45.284 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:45.284 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T05:00:45.284 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T05:00:45.284 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T05:00:45.284 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:45.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T05:00:45.350 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T05:00:45.350 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:45.350 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T05:00:45.350 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:45.404 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:45.426 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:45.426 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:45.426 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T05:00:45.426 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T05:00:45.427 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T05:00:45.427 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:45.739 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T05:00:45.761 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T05:00:45.761 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:45.761 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T05:00:45.761 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T05:00:45.761 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T05:00:45.762 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:00:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T05:00:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-12T05:00:46.754 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:46.754 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T05:00:46.754 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T05:00:46.754 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T05:00:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:46.962 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T05:00:46.984 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-12T05:00:46.984 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:46.985 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T05:00:46.985 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T05:00:46.985 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T05:00:46.985 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:47.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T05:00:47.442 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-12T05:00:47.443 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:47.443 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T05:00:47.443 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T05:00:47.443 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T05:00:47.443 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:47.539 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-12T05:00:47.965 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T05:00:47.995 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-12T05:00:48.027 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-12T05:00:48.087 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-12T05:00:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T05:00:48.342 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-12T05:00:48.378 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T05:00:49.335 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-12T05:00:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T05:00:49.461 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-12T05:00:49.623 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-12T05:00:49.655 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T05:00:49.678 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-12T05:00:49.678 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:49.678 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T05:00:49.678 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T05:00:49.678 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T05:00:49.678 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:49.731 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T05:00:49.741 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-12T05:00:49.787 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-12T05:00:50.176 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T05:00:50.234 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-12T05:00:50.265 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-12T05:00:50.292 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-12T05:00:50.472 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-12T05:00:50.591 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-12T05:00:50.631 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-12T05:00:50.648 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-12T05:00:50.670 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-12T05:00:50.674 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T05:00:50.697 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-12T05:00:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T05:00:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T05:00:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T05:00:50.698 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:50.742 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T05:00:50.745 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-12T05:00:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-12T05:00:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T05:00:50.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:50.775 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-12T05:00:50.799 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-12T05:00:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:50.832 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-12T05:00:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:50.839 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:50.839 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T05:00:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T05:00:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T05:00:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:50.861 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-12T05:00:50.915 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-12T05:00:50.977 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-12T05:00:51.010 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-12T05:00:51.151 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T05:00:51.173 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-12T05:00:51.173 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T05:00:51.174 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T05:00:51.174 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T05:00:51.174 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T05:00:51.174 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:00:51.543 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:51.583 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:51.602 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:52.438 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-12T05:00:52.521 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-12T05:00:52.561 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-12T05:00:52.650 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-12T05:00:52.683 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-12T05:00:52.708 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-12T05:00:52.733 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-12T05:00:52.760 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-12T05:00:52.760 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T05:00:52.848 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T05:00:52.886 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-12T05:00:52.886 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T05:00:52.954 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T05:00:52.980 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-12T05:00:53.013 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-12T05:00:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-12T05:00:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-12T05:00:53.096 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-12T05:00:53.136 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-12T05:00:53.169 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-12T05:00:53.194 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-12T05:00:53.219 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-12T05:00:53.244 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-12T05:00:53.270 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-12T05:00:53.276 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:53.276 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:53.465 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:53.465 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T05:00:53.531 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T05:00:53.558 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-12T05:00:53.581 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-12T05:00:53.639 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-12T05:00:53.665 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-12T05:00:53.729 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T05:00:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T05:00:55.648 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-12T05:00:55.678 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-12T05:00:55.706 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-12T05:00:55.891 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-12T05:00:56.013 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-12T05:00:56.054 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-12T05:00:56.093 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-12T05:00:56.145 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-12T05:00:56.189 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-12T05:00:56.213 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-12T05:00:56.245 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-12T05:00:56.274 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-12T05:00:56.321 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-12T05:00:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-12T05:00:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-12T05:00:57.040 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:57.079 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:57.099 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-12T05:00:57.939 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-12T05:00:58.024 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-12T05:00:58.065 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-12T05:00:58.147 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-12T05:00:58.186 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-12T05:00:58.211 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-12T05:00:58.236 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-12T05:00:58.262 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-12T05:00:58.263 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T05:00:58.351 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-12T05:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-12T05:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T05:00:58.461 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-12T05:00:58.468 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-12T05:00:58.469 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-12T05:00:58.470 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-12T05:00:58.471 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-12T05:00:58.472 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-12T05:00:58.473 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-12T05:00:58.474 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-12T05:00:58.475 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-12T05:00:58.476 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-12T05:00:58.477 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-12T05:00:58.478 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-12T05:00:58.479 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-12T05:00:58.480 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-12T05:00:58.481 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-12T05:00:58.482 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-12T05:00:58.483 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-12T05:00:58.483 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-12T05:00:58.483 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-12T05:00:58.483 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-12T05:00:58.515 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-12T05:00:58.541 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-12T05:00:58.566 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-12T05:00:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-12T05:00:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-12T05:00:58.679 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-12T05:00:58.704 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-12T05:00:58.729 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-12T05:00:58.755 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-12T05:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-12T05:00:58.787 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:58.787 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:58.966 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-12T05:00:58.967 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T05:00:59.049 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-12T05:00:59.074 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-12T05:00:59.099 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-12T05:00:59.132 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-12T05:00:59.158 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-12T05:00:59.223 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T05:01:01.647 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T05:01:01.648 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.650 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.651 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T05:01:01.652 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-16.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T05:01:01.653 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: lua-5.4.4-4.el9.x86_64 2024-03-12T05:01:01.654 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-12T05:01:01.655 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T05:01:01.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T05:01:01.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T05:01:01.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T05:01:01.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T05:01:01.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T05:01:01.661 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T05:01:01.662 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T05:01:01.662 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-12T05:01:01.662 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-03-12T05:01:02.120 DEBUG:teuthology.parallel:result is None 2024-03-12T05:01:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-12T05:01:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-12T05:01:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-12T05:01:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-12T05:01:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-12T05:01:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-12T05:01:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-12T05:01:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-12T05:01:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-12T05:01:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-12T05:01:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-12T05:01:03.961 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-12T05:01:03.962 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-12T05:01:03.963 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-12T05:01:03.964 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-12T05:01:03.965 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-12T05:01:03.966 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-12T05:01:03.966 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-12T05:01:06.867 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-12T05:01:06.867 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T05:01:06.868 INFO:teuthology.orchestra.run.smithi161.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-03-12T05:01:06.869 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.870 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-12T05:01:06.871 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-16.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.872 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-03-12T05:01:06.873 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-12T05:01:06.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T05:01:06.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T05:01:06.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T05:01:06.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-12T05:01:06.878 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T05:01:06.879 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T05:01:06.880 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T05:01:06.880 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T05:01:06.880 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T05:01:06.880 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T05:01:06.880 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:01:06.880 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-12T05:01:07.317 DEBUG:teuthology.parallel:result is None 2024-03-12T05:01:07.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:01:07.465 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T05:01:07.492 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-1607.g94a53970.el9 2024-03-12T05:01:07.493 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-12T05:01:07.493 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-12T05:01:07.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:01:07.718 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T05:01:07.744 INFO:teuthology.orchestra.run.smithi156.stdout:19.0.0-1607.g94a53970.el9 2024-03-12T05:01:07.744 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-12T05:01:07.744 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-12T05:01:07.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:01:07.885 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T05:01:07.906 INFO:teuthology.orchestra.run.smithi161.stdout:19.0.0-1607.g94a53970.el9 2024-03-12T05:01:07.906 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-12T05:01:07.906 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-12T05:01:07.908 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-12T05:01:07.908 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:01:07.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T05:01:07.943 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:01:07.943 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T05:01:07.978 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:01:07.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T05:01:08.007 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-12T05:01:08.007 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:01:08.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T05:01:08.037 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T05:01:08.107 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:01:08.107 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T05:01:08.138 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T05:01:08.208 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:01:08.208 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T05:01:08.234 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T05:01:08.301 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-12T05:01:08.302 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:01:08.302 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T05:01:08.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T05:01:08.400 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:01:08.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T05:01:08.430 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T05:01:08.500 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:01:08.500 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T05:01:08.527 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T05:01:08.593 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-12T05:01:08.593 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:01:08.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T05:01:08.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T05:01:08.697 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:01:08.697 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T05:01:08.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T05:01:08.799 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:01:08.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T05:01:08.825 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T05:01:08.892 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-12T05:01:08.993 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-12T05:01:08.993 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:01:08.994 INFO:tasks.cephadm:Cluster fsid is 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:01:08.994 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-12T05:01:08.994 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.100', 'mon.b': '172.21.15.156', 'mon.c': '172.21.15.161'} 2024-03-12T05:01:08.995 INFO:tasks.cephadm:First mon is mon.a on smithi100 2024-03-12T05:01:08.995 INFO:tasks.cephadm:First mgr is x 2024-03-12T05:01:08.995 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-12T05:01:08.995 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-03-12T05:01:09.026 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2024-03-12T05:01:09.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-03-12T05:01:09.084 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-12T05:01:09.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:01:09.239 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77431/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:08:19.449750', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77429/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:06:59.696700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-12T05:01:09.380 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 94a539707733325322964c6a187c6a1b41be1daf from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:01:09.382 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm 2024-03-12T05:01:09.382 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:10.264 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 12 05:01 /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:10.264 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:11.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 12 05:01 /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:11.322 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:12.399 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 12 05:01 /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:12.399 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:12.418 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:12.436 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T05:01:12.470 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf on all hosts... 2024-03-12T05:01:12.470 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-12T05:01:12.473 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-12T05:01:12.479 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-12T05:01:13.357 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T05:01:13.403 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T05:01:13.735 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-03-12T05:01:45.624 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2024-03-12T05:01:48.059 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:01:48.956 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:01:48.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-03-12T05:01:49.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2024-03-12T05:01:49.069 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-03-12T05:01:49.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-03-12T05:01:49.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2024-03-12T05:01:49.160 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-03-12T05:01:49.190 INFO:tasks.cephadm:Writing seed config... 2024-03-12T05:01:49.191 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-12T05:01:49.192 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-12T05:01:49.193 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-12T05:01:49.194 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-12T05:01:49.195 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-12T05:01:49.195 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-12T05:01:49.195 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-12T05:01:49.195 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:01:49.195 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-12T05:01:49.210 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8a1680ae-e02d-11ee-95c8-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-12T05:01:49.210 DEBUG:teuthology.orchestra.run.smithi100:mon.a> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service 2024-03-12T05:01:49.253 DEBUG:teuthology.orchestra.run.smithi100:mgr.x> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x.service 2024-03-12T05:01:49.296 INFO:tasks.cephadm:Bootstrapping... 2024-03-12T05:01:49.296 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf -v bootstrap --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.100 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:01:49.487 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-12T05:01:49.487 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf', '-v', 'bootstrap', '--fsid', '8a1680ae-e02d-11ee-95c8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.100', '--skip-admin-label'] 2024-03-12T05:01:49.511 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-12T05:01:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-12T05:01:49.511 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying podman|docker is present... 2024-03-12T05:01:49.535 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-12T05:01:49.535 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying lvm2 is present... 2024-03-12T05:01:49.536 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying time synchronization is in place... 2024-03-12T05:01:49.543 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-12T05:01:49.543 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-12T05:01:49.549 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-12T05:01:49.549 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-03-12T05:01:49.555 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-03-12T05:01:49.561 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-03-12T05:01:49.561 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-03-12T05:01:49.561 INFO:teuthology.orchestra.run.smithi100.stdout:Repeating the final host check... 2024-03-12T05:01:49.584 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-12T05:01:49.584 INFO:teuthology.orchestra.run.smithi100.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-12T05:01:49.584 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl is present 2024-03-12T05:01:49.584 INFO:teuthology.orchestra.run.smithi100.stdout:lvcreate is present 2024-03-12T05:01:49.590 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-12T05:01:49.590 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-12T05:01:49.595 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-12T05:01:49.596 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-03-12T05:01:49.602 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-03-12T05:01:49.608 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-03-12T05:01:49.608 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-03-12T05:01:49.608 INFO:teuthology.orchestra.run.smithi100.stdout:Host looks OK 2024-03-12T05:01:49.608 INFO:teuthology.orchestra.run.smithi100.stdout:Cluster fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:01:49.608 INFO:teuthology.orchestra.run.smithi100.stdout:Acquiring lock 140455740233328 on /run/cephadm/8a1680ae-e02d-11ee-95c8-87774f69a715.lock 2024-03-12T05:01:49.608 INFO:teuthology.orchestra.run.smithi100.stdout:Lock 140455740233328 acquired on /run/cephadm/8a1680ae-e02d-11ee-95c8-87774f69a715.lock 2024-03-12T05:01:49.609 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 3300 ... 2024-03-12T05:01:49.609 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 6789 ... 2024-03-12T05:01:49.609 INFO:teuthology.orchestra.run.smithi100.stdout:Base mon IP(s) is [172.21.15.100:3300, 172.21.15.100:6789], mon addrv is [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-03-12T05:01:49.612 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-03-12T05:01:49.612 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.100 metric 100 2024-03-12T05:01:49.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-12T05:01:49.615 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-12T05:01:49.615 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2024-03-12T05:01:49.617 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-12T05:01:49.617 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-12T05:01:49.617 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-12T05:01:49.617 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-12T05:01:49.617 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-03-12T05:01:49.617 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-12T05:01:49.618 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-03-12T05:01:49.618 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-03-12T05:01:49.618 INFO:teuthology.orchestra.run.smithi100.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-12T05:01:49.618 INFO:teuthology.orchestra.run.smithi100.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-12T05:01:49.619 INFO:teuthology.orchestra.run.smithi100.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T05:01:50.067 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-12T05:01:50.067 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-12T05:01:50.067 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Getting image source signatures 2024-03-12T05:01:50.067 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:0b3c7bee4e66dab4b5376aaf0baf8ba5fc1c3e66c27dc601157782d617bd9dc6 2024-03-12T05:01:50.067 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-12T05:01:50.068 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying config sha256:a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-12T05:01:50.068 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-12T05:01:51.055 INFO:teuthology.orchestra.run.smithi100.stdout:ceph: stdout ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-12T05:01:51.056 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph version: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-12T05:01:51.056 INFO:teuthology.orchestra.run.smithi100.stdout:Extracting ceph user uid/gid from container image... 2024-03-12T05:01:51.927 INFO:teuthology.orchestra.run.smithi100.stdout:stat: stdout 167 167 2024-03-12T05:01:51.927 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial keys... 2024-03-12T05:01:52.757 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQDA4e9lTsdhDBAAiXbLr9pDS3k0A+hgvmwQnA== 2024-03-12T05:02:00.199 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQDB4e9ly1VvMRAASLSQhKpyGjxVKmsK1hSndA== 2024-03-12T05:02:05.181 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQDK4e9lKV2xNBAA69+bgqYzzybjomurm+xUfA== 2024-03-12T05:02:05.182 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial monmap... 2024-03-12T05:02:06.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-12T05:02:06.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-12T05:02:06.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:06.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-12T05:02:06.171 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool for a [v2:172.21.15.100:3300,v1:172.21.15.100:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-12T05:02:06.172 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-03-12T05:02:06.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: set fsid to 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:06.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-12T05:02:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:02:06.172 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mon... 2024-03-12T05:02:07.111 INFO:teuthology.orchestra.run.smithi100.stdout:create mon.a on 2024-03-12T05:02:07.318 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-12T05:02:07.514 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-12T05:02:08.052 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715.target. 2024-03-12T05:02:08.052 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715.target. 2024-03-12T05:02:08.304 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a 2024-03-12T05:02:08.304 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service: Unit ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service not loaded. 2024-03-12T05:02:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715.target.wants/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@.service. 2024-03-12T05:02:08.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:08 smithi100 ceph-mon[31837]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-12T05:02:08.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:08 smithi100 bash[31816]: bcbb1ccc3dc03b0f2d1df10013be6236512f6570e72cb0de7702e948f8e79e88 2024-03-12T05:02:08.991 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T05:02:08.992 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-03-12T05:02:08.999 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T05:02:08.999 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-03-12T05:02:08.999 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-03-12T05:02:08.999 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-03-12T05:02:08.999 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon to start... 2024-03-12T05:02:08.999 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon... 2024-03-12T05:02:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:08 smithi100 systemd[1]: Started Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:02:10.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:09 smithi100 ceph-mon[31837]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-12T05:02:10.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:09 smithi100 ceph-mon[31837]: monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:02:10.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:09 smithi100 ceph-mon[31837]: fsmap 2024-03-12T05:02:10.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:09 smithi100 ceph-mon[31837]: osdmap e1: 0 total, 0 up, 0 in 2024-03-12T05:02:10.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:09 smithi100 ceph-mon[31837]: mgrmap e1: no daemons active 2024-03-12T05:02:10.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:09 smithi100 ceph-mon[31837]: from='client.? 172.21.15.100:0/780722422' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-12T05:02:11.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout cluster: 2024-03-12T05:02:11.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout id: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:11.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout services: 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.594831s) 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout data: 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pgs: 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.444+0000 7fadfce23700 1 Processor -- start 2024-03-12T05:02:11.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.444+0000 7fadfce23700 1 -- start start 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadfce23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf8106450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadfce23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf807ae30 con 0x7fadf8106030 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf8106450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf8106450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59544/0 (socket says 172.21.15.100:59544) 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadf659c700 1 -- 172.21.15.100:0/2000677759 learned_addr learned my addr 172.21.15.100:0/2000677759 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadf659c700 1 -- 172.21.15.100:0/2000677759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf807af70 con 0x7fadf8106030 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.445+0000 7fadf659c700 1 --2- 172.21.15.100:0/2000677759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf8106450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fade80098a0 tx=0x7fade8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2e4857fe684dcc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadf559a700 1 -- 172.21.15.100:0/2000677759 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade8004160 con 0x7fadf8106030 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadf559a700 1 -- 172.21.15.100:0/2000677759 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade80042c0 con 0x7fadf8106030 2024-03-12T05:02:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadf559a700 1 -- 172.21.15.100:0/2000677759 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade80045a0 con 0x7fadf8106030 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 -- 172.21.15.100:0/2000677759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 msgr2=0x7fadf8106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 --2- 172.21.15.100:0/2000677759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf8106450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fade80098a0 tx=0x7fade8009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 -- 172.21.15.100:0/2000677759 shutdown_connections 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 --2- 172.21.15.100:0/2000677759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf8106450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 -- 172.21.15.100:0/2000677759 >> 172.21.15.100:0/2000677759 conn(0x7fadf8101c70 msgr2=0x7fadf8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 -- 172.21.15.100:0/2000677759 shutdown_connections 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.446+0000 7fadfce23700 1 -- 172.21.15.100:0/2000677759 wait complete. 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadfce23700 1 Processor -- start 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadfce23700 1 -- start start 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadfce23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:11.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadfce23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf819d340 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59548/0 (socket says 172.21.15.100:59548) 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.447+0000 7fadf659c700 1 -- 172.21.15.100:0/780722422 learned_addr learned my addr 172.21.15.100:0/780722422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fadf659c700 1 -- 172.21.15.100:0/780722422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8009550 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fadf659c700 1 --2- 172.21.15.100:0/780722422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf819ce00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fadf807a270 tx=0x7fade8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade8011630 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade8011790 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf819d540 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade8011a30 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf819d9e0 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fade8022070 con 0x7fadf8106030 2024-03-12T05:02:11.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.448+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fade801c450 con 0x7fadf8106030 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.449+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8106b70 con 0x7fadf8106030 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.451+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fade8011b90 con 0x7fadf8106030 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.520+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fadf81a04d0 con 0x7fadf8106030 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.520+0000 7fade77fe700 1 -- 172.21.15.100:0/780722422 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fade8011d30 con 0x7fadf8106030 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 msgr2=0x7fadf819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 --2- 172.21.15.100:0/780722422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf819ce00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fadf807a270 tx=0x7fade8004750 comp rx=0 tx=0).stop 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 shutdown_connections 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 --2- 172.21.15.100:0/780722422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8106030 0x7fadf819ce00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 >> 172.21.15.100:0/780722422 conn(0x7fadf8101c70 msgr2=0x7fadf81935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 shutdown_connections 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:09.521+0000 7fadfce23700 1 -- 172.21.15.100:0/780722422 wait complete. 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:mon is available 2024-03-12T05:02:11.575 INFO:teuthology.orchestra.run.smithi100.stdout:Assimilating anything we can from ceph.conf... 2024-03-12T05:02:12.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:12 smithi100 ceph-mon[31837]: from='client.? 172.21.15.100:0/3775507662' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-12T05:02:12.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:12 smithi100 ceph-mon[31837]: from='client.? 172.21.15.100:0/3775507662' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-12T05:02:12.633 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:12.633 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-03-12T05:02:12.633 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:12.633 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-03-12T05:02:12.633 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-12T05:02:12.633 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-12T05:02:12.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.974+0000 7f47564b2700 1 Processor -- start 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.974+0000 7f47564b2700 1 -- start start 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.974+0000 7f47564b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f4750106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.974+0000 7f47564b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750107020 con 0x7f4750106630 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f4750106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f4750106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40634/0 (socket says 172.21.15.100:40634) 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474ffff700 1 -- 172.21.15.100:0/3537387572 learned_addr learned my addr 172.21.15.100:0/3537387572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474ffff700 1 -- 172.21.15.100:0/3537387572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750107830 con 0x7f4750106630 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474ffff700 1 --2- 172.21.15.100:0/3537387572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f4750106a50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47400098a0 tx=0x7f4740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e082dc7ddd577e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474effd700 1 -- 172.21.15.100:0/3537387572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4740004160 con 0x7f4750106630 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474effd700 1 -- 172.21.15.100:0/3537387572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47400042c0 con 0x7f4750106630 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.975+0000 7f474effd700 1 -- 172.21.15.100:0/3537387572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47400045a0 con 0x7f4750106630 2024-03-12T05:02:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 -- 172.21.15.100:0/3537387572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 msgr2=0x7f4750106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 --2- 172.21.15.100:0/3537387572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f4750106a50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47400098a0 tx=0x7f4740009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 -- 172.21.15.100:0/3537387572 shutdown_connections 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 --2- 172.21.15.100:0/3537387572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f4750106a50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 -- 172.21.15.100:0/3537387572 >> 172.21.15.100:0/3537387572 conn(0x7f4750101c70 msgr2=0x7f4750104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 -- 172.21.15.100:0/3537387572 shutdown_connections 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 -- 172.21.15.100:0/3537387572 wait complete. 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 Processor -- start 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.976+0000 7f47564b2700 1 -- start start 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f47564b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f475018f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f47564b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475018f730 con 0x7f4750106630 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f475018f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f475018f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40642/0 (socket says 172.21.15.100:40642) 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474ffff700 1 -- 172.21.15.100:0/3775507662 learned_addr learned my addr 172.21.15.100:0/3775507662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:12.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474ffff700 1 -- 172.21.15.100:0/3775507662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009550 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474ffff700 1 --2- 172.21.15.100:0/3775507662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f475018f1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4740004000 tx=0x7f4740004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4740011630 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740011790 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475018d920 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4740011a30 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.977+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475018ddc0 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.978+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4740022070 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.978+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f474001c450 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.978+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475018e7d0 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:11.981+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4740011b90 con 0x7f4750106630 2024-03-12T05:02:12.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.047+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4750107ba0 con 0x7f4750106630 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.050+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f4740011de0 con 0x7f4750106630 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.051+0000 7f474d7fa700 1 -- 172.21.15.100:0/3775507662 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f474001c6b0 con 0x7f4750106630 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.051+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 msgr2=0x7f475018f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.051+0000 7f47564b2700 1 --2- 172.21.15.100:0/3775507662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f475018f1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4740004000 tx=0x7f4740004750 comp rx=0 tx=0).stop 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.052+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 shutdown_connections 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.052+0000 7f47564b2700 1 --2- 172.21.15.100:0/3775507662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750106630 0x7f475018f1f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.052+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 >> 172.21.15.100:0/3775507662 conn(0x7f4750101c70 msgr2=0x7f4750191a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.052+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 shutdown_connections 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:12.052+0000 7f47564b2700 1 -- 172.21.15.100:0/3775507662 wait complete. 2024-03-12T05:02:12.638 INFO:teuthology.orchestra.run.smithi100.stdout:Generating new minimal ceph.conf... 2024-03-12T05:02:13.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:13 smithi100 ceph-mon[31837]: from='client.? 172.21.15.100:0/1224168010' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:02:13.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f4624b6a700 1 Processor -- start 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f4624b6a700 1 -- start start 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f4624b6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f4624b6a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620107020 con 0x7f4620106630 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f461e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f461e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40654/0 (socket says 172.21.15.100:40654) 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.071+0000 7f461e59c700 1 -- 172.21.15.100:0/1268677162 learned_addr learned my addr 172.21.15.100:0/1268677162 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f461e59c700 1 -- 172.21.15.100:0/1268677162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620107830 con 0x7f4620106630 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f461e59c700 1 --2- 172.21.15.100:0/1268677162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620106a50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4618009e80 tx=0x7f461800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a047f5992829d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f461d59a700 1 -- 172.21.15.100:0/1268677162 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461800e070 con 0x7f4620106630 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f461d59a700 1 -- 172.21.15.100:0/1268677162 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f461800b8c0 con 0x7f4620106630 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f461d59a700 1 -- 172.21.15.100:0/1268677162 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4618004550 con 0x7f4620106630 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f4624b6a700 1 -- 172.21.15.100:0/1268677162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 msgr2=0x7f4620106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:13.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.072+0000 7f4624b6a700 1 --2- 172.21.15.100:0/1268677162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620106a50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4618009e80 tx=0x7f461800b330 comp rx=0 tx=0).stop 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 -- 172.21.15.100:0/1268677162 shutdown_connections 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 --2- 172.21.15.100:0/1268677162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620106a50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 -- 172.21.15.100:0/1268677162 >> 172.21.15.100:0/1268677162 conn(0x7f4620101c70 msgr2=0x7f4620104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 -- 172.21.15.100:0/1268677162 shutdown_connections 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 -- 172.21.15.100:0/1268677162 wait complete. 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 Processor -- start 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 -- start start 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620191750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.073+0000 7f4624b6a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620191c90 con 0x7f4620106630 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f461e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f461e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40662/0 (socket says 172.21.15.100:40662) 2024-03-12T05:02:13.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f461e59c700 1 -- 172.21.15.100:0/1224168010 learned_addr learned my addr 172.21.15.100:0/1224168010 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f461e59c700 1 -- 172.21.15.100:0/1224168010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618009b30 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f461e59c700 1 --2- 172.21.15.100:0/1224168010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620191750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4618004070 tx=0x7f4618003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461800e040 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f461800bdf0 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462018fed0 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461801c8c0 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4620190350 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f461801cab0 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.074+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f461801b850 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.075+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4620068d80 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.077+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f461802a060 con 0x7f4620106630 2024-03-12T05:02:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.144+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f462010ab20 con 0x7f4620106630 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.144+0000 7f460f7fe700 1 -- 172.21.15.100:0/1224168010 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f4618020070 con 0x7f4620106630 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 msgr2=0x7f4620191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 --2- 172.21.15.100:0/1224168010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620191750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4618004070 tx=0x7f4618003d00 comp rx=0 tx=0).stop 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 shutdown_connections 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 --2- 172.21.15.100:0/1224168010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4620106630 0x7f4620191750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 >> 172.21.15.100:0/1224168010 conn(0x7f4620101c70 msgr2=0x7f4620102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 shutdown_connections 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:13.145+0000 7f4624b6a700 1 -- 172.21.15.100:0/1224168010 wait complete. 2024-03-12T05:02:13.675 INFO:teuthology.orchestra.run.smithi100.stdout:Restarting the monitor... 2024-03-12T05:02:13.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:13 smithi100 systemd[1]: Stopping Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:02:14.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a[31833]: 2024-03-12T05:02:13.931+0000 7ff64fb95700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:02:14.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a[31833]: 2024-03-12T05:02:13.931+0000 7ff64fb95700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-12T05:02:14.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 podman[32080]: 2024-03-12 05:02:14.116301047 +0000 UTC m=+0.289827044 container died bcbb1ccc3dc03b0f2d1df10013be6236512f6570e72cb0de7702e948f8e79e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-12T05:02:14.626 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 podman[32080]: 2024-03-12 05:02:14.440388367 +0000 UTC m=+0.613914363 container cleanup bcbb1ccc3dc03b0f2d1df10013be6236512f6570e72cb0de7702e948f8e79e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-12T05:02:14.626 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 bash[32080]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a 2024-03-12T05:02:15.018 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 podman[32094]: 2024-03-12 05:02:14.624429265 +0000 UTC m=+0.505187167 container remove bcbb1ccc3dc03b0f2d1df10013be6236512f6570e72cb0de7702e948f8e79e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-12T05:02:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-12T05:02:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 systemd[1]: Stopped Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:02:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:14 smithi100 systemd[1]: Starting Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:02:15.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 podman[32191]: 2024-03-12 05:02:15.016958353 +0000 UTC m=+0.086715190 container create 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-94a5397) 2024-03-12T05:02:15.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 podman[32191]: 2024-03-12 05:02:14.94918876 +0000 UTC m=+0.018945580 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:02:15.650 INFO:teuthology.orchestra.run.smithi100.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 podman[32191]: 2024-03-12 05:02:15.561377776 +0000 UTC m=+0.631134587 container init 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 podman[32191]: 2024-03-12 05:02:15.564394961 +0000 UTC m=+0.634151772 container start 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: set uid:gid to 167:167 (ceph:ceph) 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: pidfile_write: ignore empty --pid-file 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: load: jerasure load: lrc 2024-03-12T05:02:15.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: RocksDB version: 7.9.2 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Git sha 0 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: DB SUMMARY 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: DB Session ID: MTQYSD8FEIU2K8FJ87VH 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: CURRENT file: CURRENT 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: IDENTITY file: IDENTITY 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101158 ; 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.error_if_exists: 0 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.create_if_missing: 0 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.paranoid_checks: 1 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-12T05:02:15.773 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-12T05:02:15.774 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.env: 0x55f038096ae0 2024-03-12T05:02:15.774 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.fs: PosixFileSystem 2024-03-12T05:02:15.774 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.info_log: 0x55f03a8258c0 2024-03-12T05:02:15.774 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_file_opening_threads: 16 2024-03-12T05:02:15.774 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.statistics: (nil) 2024-03-12T05:02:15.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.use_fsync: 0 2024-03-12T05:02:15.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_log_file_size: 0 2024-03-12T05:02:15.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-12T05:02:15.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-12T05:02:15.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.keep_log_file_num: 1000 2024-03-12T05:02:15.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.recycle_log_file_num: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.allow_fallocate: 1 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.allow_mmap_reads: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.allow_mmap_writes: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.use_direct_reads: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.create_missing_column_families: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.db_log_dir: 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.wal_dir: 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-12T05:02:15.776 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.advise_random_on_open: 1 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.db_write_buffer_size: 0 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.write_buffer_manager: 0x55f03a7f9400 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.rate_limiter: (nil) 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.wal_recovery_mode: 2 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.enable_thread_tracking: 0 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.enable_pipelined_write: 0 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.unordered_write: 0 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-12T05:02:15.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.row_cache: None 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.wal_filter: None 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.allow_ingest_behind: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.two_write_queues: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.manual_wal_flush: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.wal_compression: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.atomic_flush: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.log_readahead_size: 0 2024-03-12T05:02:15.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.best_efforts_recovery: 0 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.allow_data_in_errors: 0 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.db_host_id: __hostname__ 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_background_jobs: 2 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_background_compactions: -1 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_subcompactions: 1 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_total_wal_size: 0 2024-03-12T05:02:15.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_open_files: -1 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bytes_per_sync: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_readahead_size: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_background_flushes: -1 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Compression algorithms supported: 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kZSTD supported: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kXpressCompression supported: 0 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kLZ4HCCompression supported: 1 2024-03-12T05:02:15.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kZlibCompression supported: 1 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kSnappyCompression supported: 1 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kLZ4Compression supported: 1 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: kBZip2Compression supported: 0 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.merge_operator: 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_filter: None 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_filter_factory: None 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.sst_partitioner_factory: None 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-12T05:02:15.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f03a8259c0) 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_type: 0 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_index_type: 0 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_shortening: 1 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: checksum: 4 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: no_block_cache: 0 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache: 0x55f03a829090 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_options: 2024-03-12T05:02:15.782 INFO:journalctl@ceph.mon.a.smithi100.stdout: capacity : 536870912 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_shard_bits : 4 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: strict_capacity_limit : 0 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_compressed: (nil) 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: persistent_cache: (nil) 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size: 4096 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size_deviation: 10 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_restart_interval: 16 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_block_restart_interval: 1 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: metadata_block_size: 4096 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: partition_filters: 0 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: use_delta_encoding: 1 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: filter_policy: bloomfilter 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: whole_key_filtering: 1 2024-03-12T05:02:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout: verify_compression: 0 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: format_version: 5 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_index_compression: 1 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_align: 0 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: max_auto_readahead_size: 262144 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: prepopulate_block_cache: 0 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: initial_auto_readahead_size: 8192 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.write_buffer_size: 33554432 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_write_buffer_number: 2 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression: NoCompression 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression: Disabled 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.prefix_extractor: nullptr 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-12T05:02:15.784 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.num_levels: 7 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.level: 32767 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.strategy: 0 2024-03-12T05:02:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.enabled: false 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.target_file_size_base: 67108864 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-12T05:02:15.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.arena_block_size: 1048576 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.disable_auto_compactions: 0 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-12T05:02:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.table_properties_collectors: 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.inplace_update_support: 0 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.bloom_locality: 0 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.max_successive_merges: 0 2024-03-12T05:02:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.paranoid_file_checks: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.force_consistency_checks: 1 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.report_bg_io_stats: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.ttl: 2592000 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.enable_blob_files: false 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.min_blob_size: 0 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.blob_file_size: 268435456 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-12T05:02:15.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.blob_file_starting_level: 0 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf017db4-e8a4-4fd6-b482-bf4d8d18a86d 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710219735594621, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710219735595657, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96164, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 94162, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14130, "raw_average_key_size": 50, "raw_value_size": 86519, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710219735, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf017db4-e8a4-4fd6-b482-bf4d8d18a86d", "db_session_id": "MTQYSD8FEIU2K8FJ87VH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710219735595719, "job": 1, "event": "recovery_finished"} 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f03a91a000 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: DB pointer 0x55f03a904000 2024-03-12T05:02:15.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** DB Stats ** 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: L0 2/0 95.75 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T05:02:15.791 INFO:journalctl@ceph.mon.a.smithi100.stdout: Sum 2/0 95.75 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-12T05:02:15.792 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative compaction: 0.00 GB write, 29.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval compaction: 0.00 GB write, 29.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache BinnedLRUCache@0x55f03a829090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???) e1 preinit fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).mds e1 new map 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).mds e1 print_map 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: e1 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T05:02:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout: legacy client fscid: -1 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout: No filesystems configured 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 bash[32191]: 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 systemd[1]: Started Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: fsmap 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: osdmap e1: 0 total, 0 up, 0 in 2024-03-12T05:02:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:15 smithi100 ceph-mon[32210]: mgrmap e1: no daemons active 2024-03-12T05:02:16.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7b4b1700 1 Processor -- start 2024-03-12T05:02:16.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7b4b1700 1 -- start start 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7b4b1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b74106450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7b4b1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7407ae30 con 0x7f5b74106030 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7924d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b74106450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7924d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b74106450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40676/0 (socket says 172.21.15.100:40676) 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.133+0000 7f5b7924d700 1 -- 172.21.15.100:0/2951030499 learned_addr learned my addr 172.21.15.100:0/2951030499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b7924d700 1 -- 172.21.15.100:0/2951030499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7407af70 con 0x7f5b74106030 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b7924d700 1 --2- 172.21.15.100:0/2951030499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b74106450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5b700098a0 tx=0x7f5b70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6044b9f47f07892b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b6bfff700 1 -- 172.21.15.100:0/2951030499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b70004160 con 0x7f5b74106030 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b6bfff700 1 -- 172.21.15.100:0/2951030499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b700042c0 con 0x7f5b74106030 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b6bfff700 1 -- 172.21.15.100:0/2951030499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b70011ac0 con 0x7f5b74106030 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/2951030499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 msgr2=0x7f5b74106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.134+0000 7f5b7b4b1700 1 --2- 172.21.15.100:0/2951030499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b74106450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5b700098a0 tx=0x7f5b70009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:16.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/2951030499 shutdown_connections 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 --2- 172.21.15.100:0/2951030499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b74106450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/2951030499 >> 172.21.15.100:0/2951030499 conn(0x7f5b74101c70 msgr2=0x7f5b74104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/2951030499 shutdown_connections 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/2951030499 wait complete. 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 Processor -- start 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 -- start start 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b7419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7b4b1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7419dc00 con 0x7f5b74106030 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.135+0000 7f5b7924d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b7419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b7924d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b7419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40686/0 (socket says 172.21.15.100:40686) 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b7924d700 1 -- 172.21.15.100:0/1583659054 learned_addr learned my addr 172.21.15.100:0/1583659054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b7924d700 1 -- 172.21.15.100:0/1583659054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70009550 con 0x7f5b74106030 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b7924d700 1 --2- 172.21.15.100:0/1583659054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b7419d6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009130 tx=0x7f5b70011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b70003aa0 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b70003c00 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7419de00 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b70024800 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7419e2a0 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5b7002d3f0 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.136+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5b70019760 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.137+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b74197350 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.139+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5b70024e20 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.206+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5b741076d0 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.208+0000 7f5b6a7fc700 1 -- 172.21.15.100:0/1583659054 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5b7001abd0 con 0x7f5b74106030 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 msgr2=0x7f5b7419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 --2- 172.21.15.100:0/1583659054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b7419d6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009130 tx=0x7f5b70011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 shutdown_connections 2024-03-12T05:02:16.771 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 --2- 172.21.15.100:0/1583659054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b74106030 0x7f5b7419d6c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:16.772 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 >> 172.21.15.100:0/1583659054 conn(0x7f5b74101c70 msgr2=0x7f5b74194490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:16.772 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 shutdown_connections 2024-03-12T05:02:16.772 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:16.209+0000 7f5b7b4b1700 1 -- 172.21.15.100:0/1583659054 wait complete. 2024-03-12T05:02:16.772 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-12T05:02:16.803 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:02:16.803 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mgr... 2024-03-12T05:02:16.803 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-12T05:02:16.804 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-12T05:02:17.050 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x 2024-03-12T05:02:17.050 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x.service: Unit ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x.service not loaded. 2024-03-12T05:02:17.204 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715.target.wants/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@.service. 2024-03-12T05:02:17.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:17 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1583659054' entity='client.admin' 2024-03-12T05:02:17.323 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 systemd[1]: Starting Ceph mgr.x for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:02:17.691 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 podman[32426]: 2024-03-12 05:02:17.428697638 +0000 UTC m=+0.105287783 container create 481c1b521344bdcbaeaf670c27a9fee1dfc50dd0da062a20d80f3400edb5ad8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:02:17.691 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 podman[32426]: 2024-03-12 05:02:17.341026809 +0000 UTC m=+0.017616968 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-12T05:02:17.691 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 podman[32426]: 2024-03-12 05:02:17.614295658 +0000 UTC m=+0.290885804 container init 481c1b521344bdcbaeaf670c27a9fee1dfc50dd0da062a20d80f3400edb5ad8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T05:02:17.691 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 podman[32426]: 2024-03-12 05:02:17.616737182 +0000 UTC m=+0.293327337 container start 481c1b521344bdcbaeaf670c27a9fee1dfc50dd0da062a20d80f3400edb5ad8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-94a5397) 2024-03-12T05:02:17.691 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 bash[32426]: 481c1b521344bdcbaeaf670c27a9fee1dfc50dd0da062a20d80f3400edb5ad8d 2024-03-12T05:02:17.704 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T05:02:17.704 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-03-12T05:02:17.711 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T05:02:17.711 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-03-12T05:02:17.711 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-03-12T05:02:17.711 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-03-12T05:02:17.719 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T05:02:17.719 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-03-12T05:02:17.727 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T05:02:17.727 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-03-12T05:02:17.727 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-03-12T05:02:17.727 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-12T05:02:17.727 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr to start... 2024-03-12T05:02:17.727 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr... 2024-03-12T05:02:17.949 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:17 smithi100 systemd[1]: Started Ceph mgr.x for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:02:18.247 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:18.177+0000 7fccd79be200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:02:18.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:18 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1286719493' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T05:02:18.539 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:18.275+0000 7fccd79be200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:02:18.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:18.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:18.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:02:18.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T05:02:18.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:18.778 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T05:02:18.779 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T05:02:18.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T05:02:18.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T05:02:08.927829+0000", 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.165+0000 7f3c73375700 1 Processor -- start 2024-03-12T05:02:18.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.165+0000 7f3c73375700 1 -- start start 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.166+0000 7f3c73375700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.166+0000 7f3c73375700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c07aed0 con 0x7f3c6c108160 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.166+0000 7f3c71111700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.166+0000 7f3c71111700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40712/0 (socket says 172.21.15.100:40712) 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.166+0000 7f3c71111700 1 -- 172.21.15.100:0/1826384556 learned_addr learned my addr 172.21.15.100:0/1826384556 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.166+0000 7f3c71111700 1 -- 172.21.15.100:0/1826384556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c10aa90 con 0x7f3c6c108160 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c71111700 1 --2- 172.21.15.100:0/1826384556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c10a550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098a0 tx=0x7f3c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcdda4588334c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c63fff700 1 -- 172.21.15.100:0/1826384556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68004160 con 0x7f3c6c108160 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c63fff700 1 -- 172.21.15.100:0/1826384556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c680042c0 con 0x7f3c6c108160 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c63fff700 1 -- 172.21.15.100:0/1826384556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68011ac0 con 0x7f3c6c108160 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c73375700 1 -- 172.21.15.100:0/1826384556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 msgr2=0x7f3c6c10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:18.790 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c73375700 1 --2- 172.21.15.100:0/1826384556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c10a550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098a0 tx=0x7f3c68009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c73375700 1 -- 172.21.15.100:0/1826384556 shutdown_connections 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c73375700 1 --2- 172.21.15.100:0/1826384556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c10a550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.167+0000 7f3c73375700 1 -- 172.21.15.100:0/1826384556 >> 172.21.15.100:0/1826384556 conn(0x7f3c6c101c70 msgr2=0x7f3c6c104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c73375700 1 -- 172.21.15.100:0/1826384556 shutdown_connections 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c73375700 1 -- 172.21.15.100:0/1826384556 wait complete. 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c73375700 1 Processor -- start 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c73375700 1 -- start start 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c73375700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c73375700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c19b9d0 con 0x7f3c6c108160 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.168+0000 7f3c71111700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c71111700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40716/0 (socket says 172.21.15.100:40716) 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c71111700 1 -- 172.21.15.100:0/1286719493 learned_addr learned my addr 172.21.15.100:0/1286719493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c71111700 1 -- 172.21.15.100:0/1286719493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009550 con 0x7f3c6c108160 2024-03-12T05:02:18.791 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c71111700 1 --2- 172.21.15.100:0/1286719493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c19b490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009130 tx=0x7f3c68011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68003aa0 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c19bbd0 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c19c070 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c68003c00 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68024800 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3c6802d3f0 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.169+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3c680197d0 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.170+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c50005360 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.172+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3c6801abd0 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.244+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3c500058e0 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.245+0000 7f3c627fc700 1 -- 172.21.15.100:0/1286719493 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f3c6801ad70 con 0x7f3c6c108160 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 msgr2=0x7f3c6c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:18.792 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 --2- 172.21.15.100:0/1286719493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c19b490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009130 tx=0x7f3c68011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:18.793 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 shutdown_connections 2024-03-12T05:02:18.793 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 --2- 172.21.15.100:0/1286719493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c6c108160 0x7f3c6c19b490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:18.793 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 >> 172.21.15.100:0/1286719493 conn(0x7f3c6c101c70 msgr2=0x7f3c6c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:18.793 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 shutdown_connections 2024-03-12T05:02:18.793 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:18.247+0000 7f3c73375700 1 -- 172.21.15.100:0/1286719493 wait complete. 2024-03-12T05:02:18.793 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (1/15)... 2024-03-12T05:02:19.073 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:18.930+0000 7fccd79be200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:02:19.073 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.024+0000 7fccd79be200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:02:19.573 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.163+0000 7fccd79be200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:02:19.573 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.259+0000 7fccd79be200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:02:19.573 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.342+0000 7fccd79be200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:02:19.932 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.624+0000 7fccd79be200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:02:19.932 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.739+0000 7fccd79be200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:02:19.932 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.830+0000 7fccd79be200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:02:19.932 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:19.929+0000 7fccd79be200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:02:20.323 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:20.109+0000 7fccd79be200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:02:21.183 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:21 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:21.106+0000 7fccd79be200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:02:21.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1818938977' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T05:02:21.573 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:21 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:21.243+0000 7fccd79be200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:02:21.871 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:21 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:21.582+0000 7fccd79be200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:02:21.871 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:21 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:21.673+0000 7fccd79be200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:02:21.871 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:21 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:21.868+0000 7fccd79be200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:02:21.890 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:21.892 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T05:02:21.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T05:02:21.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T05:02:21.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T05:02:21.904 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T05:02:21.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T05:02:08.927829+0000", 2024-03-12T05:02:21.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5f12e700 1 Processor -- start 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5f12e700 1 -- start start 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5f12e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a58106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5f12e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58107030 con 0x7f9a58106640 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5ceca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a58106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5ceca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a58106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55794/0 (socket says 172.21.15.100:55794) 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.197+0000 7f9a5ceca700 1 -- 172.21.15.100:0/1834957077 learned_addr learned my addr 172.21.15.100:0/1834957077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.198+0000 7f9a5ceca700 1 -- 172.21.15.100:0/1834957077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58107840 con 0x7f9a58106640 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.198+0000 7f9a5ceca700 1 --2- 172.21.15.100:0/1834957077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a58106a60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0098a0 tx=0x7f9a4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145b5828a0848bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:21.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.198+0000 7f9a577fe700 1 -- 172.21.15.100:0/1834957077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a4c004160 con 0x7f9a58106640 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.198+0000 7f9a577fe700 1 -- 172.21.15.100:0/1834957077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a4c0042c0 con 0x7f9a58106640 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.198+0000 7f9a577fe700 1 -- 172.21.15.100:0/1834957077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a4c011ac0 con 0x7f9a58106640 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1834957077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 msgr2=0x7f9a58106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 --2- 172.21.15.100:0/1834957077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a58106a60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0098a0 tx=0x7f9a4c009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1834957077 shutdown_connections 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 --2- 172.21.15.100:0/1834957077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a58106a60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1834957077 >> 172.21.15.100:0/1834957077 conn(0x7f9a58101c80 msgr2=0x7f9a581040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1834957077 shutdown_connections 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1834957077 wait complete. 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 Processor -- start 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.199+0000 7f9a5f12e700 1 -- start start 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5f12e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a581917b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:21.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5f12e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58191cf0 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5ceca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a581917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5ceca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a581917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55810/0 (socket says 172.21.15.100:55810) 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5ceca700 1 -- 172.21.15.100:0/1818938977 learned_addr learned my addr 172.21.15.100:0/1818938977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5ceca700 1 -- 172.21.15.100:0/1818938977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c009550 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5ceca700 1 --2- 172.21.15.100:0/1818938977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a581917b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a4c003aa0 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a4c003c00 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5818fee0 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a4c0248a0 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.200+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a58190380 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.201+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9a4c02d3f0 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.201+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a4c019770 con 0x7f9a58106640 2024-03-12T05:02:21.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.201+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a58190d90 con 0x7f9a58106640 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.203+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9a4c01ac60 con 0x7f9a58106640 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.275+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9a5810aa70 con 0x7f9a58106640 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.276+0000 7f9a55ffb700 1 -- 172.21.15.100:0/1818938977 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f9a4c035090 con 0x7f9a58106640 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 msgr2=0x7f9a581917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 --2- 172.21.15.100:0/1818938977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a581917b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 shutdown_connections 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 --2- 172.21.15.100:0/1818938977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a58106640 0x7f9a581917b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 >> 172.21.15.100:0/1818938977 conn(0x7f9a58101c80 msgr2=0x7f9a58102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 shutdown_connections 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:21.277+0000 7f9a5f12e700 1 -- 172.21.15.100:0/1818938977 wait complete. 2024-03-12T05:02:21.910 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (2/15)... 2024-03-12T05:02:22.323 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:22 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:22.054+0000 7fccd79be200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:02:22.593 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:22 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:22.334+0000 7fccd79be200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:02:22.593 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:22 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:22.590+0000 7fccd79be200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:02:23.073 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:22 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:22.823+0000 7fccd79be200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:02:24.012 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:23 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:23.612+0000 7fccd79be200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:02:24.281 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:24 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:24.185+0000 7fccd79be200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:02:24.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:24 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1017553531' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T05:02:24.573 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:24 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:24.287+0000 7fccd79be200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:02:24.573 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:24 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:24.511+0000 7fccd79be200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:02:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T05:02:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T05:02:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T05:02:24.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T05:02:08.927829+0000", 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:24.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.301+0000 7f363fa66700 1 Processor -- start 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.302+0000 7f363fa66700 1 -- start start 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.302+0000 7f363fa66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f3638106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.302+0000 7f363fa66700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638106fe0 con 0x7f36381065f0 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.302+0000 7f363d802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f3638106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.302+0000 7f363d802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f3638106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55822/0 (socket says 172.21.15.100:55822) 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.302+0000 7f363d802700 1 -- 172.21.15.100:0/327009031 learned_addr learned my addr 172.21.15.100:0/327009031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.303+0000 7f363d802700 1 -- 172.21.15.100:0/327009031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36381077f0 con 0x7f36381065f0 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.303+0000 7f363d802700 1 --2- 172.21.15.100:0/327009031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f3638106a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36340098a0 tx=0x7f3634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41826269a80db8ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.303+0000 7f363c800700 1 -- 172.21.15.100:0/327009031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3634004160 con 0x7f36381065f0 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.303+0000 7f363c800700 1 -- 172.21.15.100:0/327009031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36340042c0 con 0x7f36381065f0 2024-03-12T05:02:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.303+0000 7f363c800700 1 -- 172.21.15.100:0/327009031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3634011ac0 con 0x7f36381065f0 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 -- 172.21.15.100:0/327009031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 msgr2=0x7f3638106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 --2- 172.21.15.100:0/327009031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f3638106a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36340098a0 tx=0x7f3634009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 -- 172.21.15.100:0/327009031 shutdown_connections 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 --2- 172.21.15.100:0/327009031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f3638106a10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 -- 172.21.15.100:0/327009031 >> 172.21.15.100:0/327009031 conn(0x7f3638101c70 msgr2=0x7f3638104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 -- 172.21.15.100:0/327009031 shutdown_connections 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.304+0000 7f363fa66700 1 -- 172.21.15.100:0/327009031 wait complete. 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363fa66700 1 Processor -- start 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363fa66700 1 -- start start 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363fa66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f363819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363fa66700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363819a940 con 0x7f36381065f0 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363d802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f363819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:24.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363d802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f363819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55832/0 (socket says 172.21.15.100:55832) 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363d802700 1 -- 172.21.15.100:0/1017553531 learned_addr learned my addr 172.21.15.100:0/1017553531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363d802700 1 -- 172.21.15.100:0/1017553531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009550 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.305+0000 7f363d802700 1 --2- 172.21.15.100:0/1017553531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f363819a400 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3634003f70 tx=0x7f3634011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.306+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36340039f0 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.306+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363819ab40 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.306+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3634003b50 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.306+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f363401a690 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.306+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363819afe0 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.307+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f363401a920 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.307+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3634019700 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.307+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363810aa20 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.309+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f36340247c0 con 0x7f36381065f0 2024-03-12T05:02:24.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.379+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f363819db40 con 0x7f36381065f0 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.380+0000 7f362effd700 1 -- 172.21.15.100:0/1017553531 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f36340249f0 con 0x7f36381065f0 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 msgr2=0x7f363819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 --2- 172.21.15.100:0/1017553531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f363819a400 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3634003f70 tx=0x7f3634011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 shutdown_connections 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 --2- 172.21.15.100:0/1017553531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36381065f0 0x7f363819a400 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 >> 172.21.15.100:0/1017553531 conn(0x7f3638101c70 msgr2=0x7f36381027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 shutdown_connections 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:24.381+0000 7f363fa66700 1 -- 172.21.15.100:0/1017553531 wait complete. 2024-03-12T05:02:24.971 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (3/15)... 2024-03-12T05:02:25.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: Activating manager daemon x 2024-03-12T05:02:25.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: mgrmap e2: x(active, starting, since 0.00318968s) 2024-03-12T05:02:25.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: Manager daemon x is now available 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' 2024-03-12T05:02:25.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:25 smithi100 ceph-mon[32210]: from='mgr.14100 172.21.15.100:0/1325251730' entity='mgr.x' 2024-03-12T05:02:26.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:26 smithi100 ceph-mon[32210]: mgrmap e3: x(active, since 1.00685s) 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T05:02:28.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T05:02:28.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:28.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T05:02:28.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T05:02:28.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T05:02:28.331 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T05:02:28.332 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T05:02:28.333 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T05:02:08.927829+0000", 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:28.334 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.360+0000 7fef0465a700 1 Processor -- start 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.360+0000 7fef0465a700 1 -- start start 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef0465a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef0465a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc107030 con 0x7feefc106640 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef023f6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef023f6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55904/0 (socket says 172.21.15.100:55904) 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef023f6700 1 -- 172.21.15.100:0/209852501 learned_addr learned my addr 172.21.15.100:0/209852501 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef023f6700 1 -- 172.21.15.100:0/209852501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc107840 con 0x7feefc106640 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.361+0000 7fef023f6700 1 --2- 172.21.15.100:0/209852501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc106a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098a0 tx=0x7feeec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08f453299d63e84 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef013f4700 1 -- 172.21.15.100:0/209852501 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feeec004160 con 0x7feefc106640 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef013f4700 1 -- 172.21.15.100:0/209852501 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeec0042c0 con 0x7feefc106640 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef013f4700 1 -- 172.21.15.100:0/209852501 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feeec011ac0 con 0x7feefc106640 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 -- 172.21.15.100:0/209852501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 msgr2=0x7feefc106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 --2- 172.21.15.100:0/209852501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc106a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098a0 tx=0x7feeec009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:28.335 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 -- 172.21.15.100:0/209852501 shutdown_connections 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 --2- 172.21.15.100:0/209852501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc106a60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 -- 172.21.15.100:0/209852501 >> 172.21.15.100:0/209852501 conn(0x7feefc101c80 msgr2=0x7feefc1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 -- 172.21.15.100:0/209852501 shutdown_connections 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.362+0000 7fef0465a700 1 -- 172.21.15.100:0/209852501 wait complete. 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef0465a700 1 Processor -- start 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef0465a700 1 -- start start 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef0465a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef0465a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc19ba80 con 0x7feefc106640 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef023f6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef023f6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc19b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55916/0 (socket says 172.21.15.100:55916) 2024-03-12T05:02:28.336 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef023f6700 1 -- 172.21.15.100:0/162103484 learned_addr learned my addr 172.21.15.100:0/162103484 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.363+0000 7fef023f6700 1 -- 172.21.15.100:0/162103484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009550 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7fef023f6700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc19b540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feeec003f70 tx=0x7feeec011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feeec0039f0 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeec003b50 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc19bc80 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feeec01a690 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc19c120 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.364+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 3) v1 ==== 45652+0+0 (secure 0 0 0) 0x7feeec01a900 con 0x7feefc106640 2024-03-12T05:02:28.337 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.365+0000 7feef37fe700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7feee8038b50 0x7feee803b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.365+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feeec04ccb0 con 0x7feefc106640 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.365+0000 7fef01bf5700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7feee8038b50 0x7feee803b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.365+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc1956e0 con 0x7feefc106640 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.365+0000 7fef01bf5700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7feee8038b50 0x7feee803b010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feef8006fd0 tx=0x7feef8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.370+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feeec018c20 con 0x7feefc106640 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.529+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7feeec02d910 con 0x7feefc106640 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.640+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7feefc10aa70 con 0x7feefc106640 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.640+0000 7feef37fe700 1 -- 172.21.15.100:0/162103484 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7feeec018c20 con 0x7feefc106640 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.642+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7feee8038b50 msgr2=0x7feee803b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.642+0000 7fef0465a700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7feee8038b50 0x7feee803b010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feef8006fd0 tx=0x7feef8006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.642+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 msgr2=0x7feefc19b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.642+0000 7fef0465a700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc19b540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feeec003f70 tx=0x7feeec011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.642+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 shutdown_connections 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.642+0000 7fef0465a700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7feee8038b50 0x7feee803b010 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.643+0000 7fef0465a700 1 --2- 172.21.15.100:0/162103484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feefc106640 0x7feefc19b540 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.643+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 >> 172.21.15.100:0/162103484 conn(0x7feefc101c80 msgr2=0x7feefc102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.643+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 shutdown_connections 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:27.643+0000 7fef0465a700 1 -- 172.21.15.100:0/162103484 wait complete. 2024-03-12T05:02:28.339 INFO:teuthology.orchestra.run.smithi100.stdout:mgr is available 2024-03-12T05:02:28.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:28 smithi100 ceph-mon[32210]: mgrmap e4: x(active, since 3s) 2024-03-12T05:02:28.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:28 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/162103484' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-12T05:02:29.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa19a974700 1 Processor -- start 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa19a974700 1 -- start start 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa19a974700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19410a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa19a974700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19407ae70 con 0x7fa194108200 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa193fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19410a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:29.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa193fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19410a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55920/0 (socket says 172.21.15.100:55920) 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa193fff700 1 -- 172.21.15.100:0/1454082436 learned_addr learned my addr 172.21.15.100:0/1454082436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa193fff700 1 -- 172.21.15.100:0/1454082436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19410ab30 con 0x7fa194108200 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.705+0000 7fa193fff700 1 --2- 172.21.15.100:0/1454082436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19410a5f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa1880098a0 tx=0x7fa188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c1d354472e166d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa192ffd700 1 -- 172.21.15.100:0/1454082436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188004160 con 0x7fa194108200 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa192ffd700 1 -- 172.21.15.100:0/1454082436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1880042c0 con 0x7fa194108200 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa192ffd700 1 -- 172.21.15.100:0/1454082436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188011ac0 con 0x7fa194108200 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 -- 172.21.15.100:0/1454082436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 msgr2=0x7fa19410a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 --2- 172.21.15.100:0/1454082436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19410a5f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa1880098a0 tx=0x7fa188009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 -- 172.21.15.100:0/1454082436 shutdown_connections 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 --2- 172.21.15.100:0/1454082436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19410a5f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 -- 172.21.15.100:0/1454082436 >> 172.21.15.100:0/1454082436 conn(0x7fa194101c70 msgr2=0x7fa194104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 -- 172.21.15.100:0/1454082436 shutdown_connections 2024-03-12T05:02:29.399 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.706+0000 7fa19a974700 1 -- 172.21.15.100:0/1454082436 wait complete. 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa19a974700 1 Processor -- start 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa19a974700 1 -- start start 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa19a974700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa19a974700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19419ba10 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa193fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa193fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19419b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55930/0 (socket says 172.21.15.100:55930) 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.707+0000 7fa193fff700 1 -- 172.21.15.100:0/2186754371 learned_addr learned my addr 172.21.15.100:0/2186754371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa193fff700 1 -- 172.21.15.100:0/2186754371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188009550 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa193fff700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19419b4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa188009130 tx=0x7fa188011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188003aa0 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa188003c00 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188024800 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19419bc10 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.708+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19419c0b0 con 0x7fa194108200 2024-03-12T05:02:29.400 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.709+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7fa188024a70 con 0x7fa194108200 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.709+0000 7fa1917fa700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fa17c038cb0 0x7fa17c03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.709+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa18804ca10 con 0x7fa194108200 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.709+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19410b930 con 0x7fa194108200 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.709+0000 7fa1937fe700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fa17c038cb0 0x7fa17c03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.709+0000 7fa1937fe700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fa17c038cb0 0x7fa17c03b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa184006fd0 tx=0x7fa184006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.714+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa188024d20 con 0x7fa194108200 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.907+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa19419e990 con 0x7fa194108200 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.908+0000 7fa1917fa700 1 -- 172.21.15.100:0/2186754371 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fa18802c350 con 0x7fa194108200 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.909+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fa17c038cb0 msgr2=0x7fa17c03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.909+0000 7fa19a974700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fa17c038cb0 0x7fa17c03b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa184006fd0 tx=0x7fa184006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.909+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 msgr2=0x7fa19419b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.909+0000 7fa19a974700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19419b4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa188009130 tx=0x7fa188011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:29.401 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.910+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 shutdown_connections 2024-03-12T05:02:29.402 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.910+0000 7fa19a974700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fa17c038cb0 0x7fa17c03b170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:29.402 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.910+0000 7fa19a974700 1 --2- 172.21.15.100:0/2186754371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa194108200 0x7fa19419b4d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:29.402 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.910+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 >> 172.21.15.100:0/2186754371 conn(0x7fa194101c70 msgr2=0x7fa194102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:29.402 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.910+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 shutdown_connections 2024-03-12T05:02:29.402 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:28.910+0000 7fa19a974700 1 -- 172.21.15.100:0/2186754371 wait complete. 2024-03-12T05:02:29.402 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling cephadm module... 2024-03-12T05:02:29.782 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:29 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2186754371' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-12T05:02:30.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:30 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2956065920' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-12T05:02:30.823 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:30 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: ignoring --setuser ceph since I am not root 2024-03-12T05:02:30.823 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:30 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: ignoring --setgroup ceph since I am not root 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.803+0000 7f1cc57f7700 1 Processor -- start 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.803+0000 7f1cc57f7700 1 -- start start 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.803+0000 7f1cc57f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc0105e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.803+0000 7f1cc57f7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc01063e0 con 0x7f1cc01059f0 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc0105e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc0105e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55942/0 (socket says 172.21.15.100:55942) 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbeffd700 1 -- 172.21.15.100:0/738432508 learned_addr learned my addr 172.21.15.100:0/738432508 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbeffd700 1 -- 172.21.15.100:0/738432508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0106520 con 0x7f1cc01059f0 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbeffd700 1 --2- 172.21.15.100:0/738432508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc0105e10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00098a0 tx=0x7f1cb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aef6a8aacf785 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbe7fc700 1 -- 172.21.15.100:0/738432508 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cb0004160 con 0x7f1cc01059f0 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbe7fc700 1 -- 172.21.15.100:0/738432508 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cb00042c0 con 0x7f1cc01059f0 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.804+0000 7f1cbe7fc700 1 -- 172.21.15.100:0/738432508 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cb0011ac0 con 0x7f1cc01059f0 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 -- 172.21.15.100:0/738432508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 msgr2=0x7f1cc0105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 --2- 172.21.15.100:0/738432508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc0105e10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00098a0 tx=0x7f1cb0009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 -- 172.21.15.100:0/738432508 shutdown_connections 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 --2- 172.21.15.100:0/738432508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc0105e10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 -- 172.21.15.100:0/738432508 >> 172.21.15.100:0/738432508 conn(0x7f1cc01013d0 msgr2=0x7f1cc01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 -- 172.21.15.100:0/738432508 shutdown_connections 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 -- 172.21.15.100:0/738432508 wait complete. 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 Processor -- start 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.805+0000 7f1cc57f7700 1 -- start start 2024-03-12T05:02:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cc57f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc01082f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cc57f7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0108830 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc01082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc01082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55956/0 (socket says 172.21.15.100:55956) 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbeffd700 1 -- 172.21.15.100:0/2956065920 learned_addr learned my addr 172.21.15.100:0/2956065920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbeffd700 1 -- 172.21.15.100:0/2956065920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0009550 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbeffd700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc01082f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0003f70 tx=0x7f1cb0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cb00039f0 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cb0003b50 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cb001a690 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc0106a20 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.806+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc0106ec0 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.807+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f1cb001a900 con 0x7f1cc01059f0 2024-03-12T05:02:31.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.807+0000 7f1cbcff9700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f1cac038c60 0x7f1cac03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.807+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc00632a0 con 0x7f1cc01059f0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.807+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1cb004cfc0 con 0x7f1cc01059f0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.807+0000 7f1cb67fc700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f1cac038c60 0x7f1cac03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.808+0000 7f1cb67fc700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f1cac038c60 0x7f1cac03b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006fd0 tx=0x7f1ca8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:29.812+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1cb001abb0 con 0x7f1cc01059f0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.056+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1cc0109750 con 0x7f1cc01059f0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.532+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1cb00186d0 con 0x7f1cc01059f0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.533+0000 7f1cbcff9700 1 -- 172.21.15.100:0/2956065920 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f1cb002d960 con 0x7f1cc01059f0 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f1cac038c60 msgr2=0x7f1cac03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f1cac038c60 0x7f1cac03b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006fd0 tx=0x7f1ca8006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 msgr2=0x7f1cc01082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc01082f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0003f70 tx=0x7f1cb0011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:31.105 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 shutdown_connections 2024-03-12T05:02:31.106 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f1cac038c60 0x7f1cac03b120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:31.106 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 --2- 172.21.15.100:0/2956065920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc01059f0 0x7f1cc01082f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:31.106 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 >> 172.21.15.100:0/2956065920 conn(0x7f1cc01013d0 msgr2=0x7f1cc0101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:31.106 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 shutdown_connections 2024-03-12T05:02:31.106 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:30.535+0000 7f1cc57f7700 1 -- 172.21.15.100:0/2956065920 wait complete. 2024-03-12T05:02:31.430 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:31.097+0000 7f0e75077200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:02:31.430 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:31.195+0000 7f0e75077200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:02:31.734 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:31 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2956065920' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-12T05:02:31.734 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:31 smithi100 ceph-mon[32210]: mgrmap e5: x(active, since 6s) 2024-03-12T05:02:32.073 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:31.861+0000 7f0e75077200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:02:32.073 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:31.954+0000 7f0e75077200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:02:32.384 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.098+0000 7f0e75077200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:02:32.384 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.194+0000 7f0e75077200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:02:32.384 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.278+0000 7f0e75077200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:02:32.392 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.526+0000 7fc1439dd700 1 Processor -- start 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc1439dd700 1 -- start start 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc1439dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c10a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc1439dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c0757b0 con 0x7fc13c108210 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc141779700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c10a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc141779700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c10a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59844/0 (socket says 172.21.15.100:59844) 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc141779700 1 -- 172.21.15.100:0/650565942 learned_addr learned my addr 172.21.15.100:0/650565942 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc141779700 1 -- 172.21.15.100:0/650565942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c10ab40 con 0x7fc13c108210 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.527+0000 7fc141779700 1 --2- 172.21.15.100:0/650565942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c10a600 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098a0 tx=0x7fc138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19e0344535c14bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc133fff700 1 -- 172.21.15.100:0/650565942 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc138004160 con 0x7fc13c108210 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc133fff700 1 -- 172.21.15.100:0/650565942 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1380042c0 con 0x7fc13c108210 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc133fff700 1 -- 172.21.15.100:0/650565942 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc138011ac0 con 0x7fc13c108210 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 -- 172.21.15.100:0/650565942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 msgr2=0x7fc13c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 --2- 172.21.15.100:0/650565942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c10a600 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098a0 tx=0x7fc138009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 -- 172.21.15.100:0/650565942 shutdown_connections 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 --2- 172.21.15.100:0/650565942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c10a600 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 -- 172.21.15.100:0/650565942 >> 172.21.15.100:0/650565942 conn(0x7fc13c101c80 msgr2=0x7fc13c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 -- 172.21.15.100:0/650565942 shutdown_connections 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.528+0000 7fc1439dd700 1 -- 172.21.15.100:0/650565942 wait complete. 2024-03-12T05:02:32.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc1439dd700 1 Processor -- start 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc1439dd700 1 -- start start 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc1439dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc1439dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c19ba30 con 0x7fc13c108210 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc141779700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc141779700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c19b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59846/0 (socket says 172.21.15.100:59846) 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc141779700 1 -- 172.21.15.100:0/3512318575 learned_addr learned my addr 172.21.15.100:0/3512318575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.529+0000 7fc141779700 1 -- 172.21.15.100:0/3512318575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138009550 con 0x7fc13c108210 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.530+0000 7fc141779700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c19b4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc138003f70 tx=0x7fc138011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.530+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1380039f0 con 0x7fc13c108210 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.530+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc138003b50 con 0x7fc13c108210 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.530+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c19bc30 con 0x7fc13c108210 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.530+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c19c0d0 con 0x7fc13c108210 2024-03-12T05:02:32.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.530+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc13801a760 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.531+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fc138024430 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.531+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c10b940 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.531+0000 7fc1327fc700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 0x7fc12803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.531+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc13804cee0 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.531+0000 7fc140f78700 1 -- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 msgr2=0x7fc12803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.531+0000 7fc140f78700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 0x7fc12803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.536+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1380249f0 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.732+0000 7fc140f78700 1 -- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 msgr2=0x7fc12803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.732+0000 7fc140f78700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 0x7fc12803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.801+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc13c19eea0 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.801+0000 7fc1327fc700 1 -- 172.21.15.100:0/3512318575 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc138018350 con 0x7fc13c108210 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.803+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 msgr2=0x7fc12803b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 0x7fc12803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 msgr2=0x7fc13c19b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c19b4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc138003f70 tx=0x7fc138011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 shutdown_connections 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7fc128038c60 0x7fc12803b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 --2- 172.21.15.100:0/3512318575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c108210 0x7fc13c19b4f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 >> 172.21.15.100:0/3512318575 conn(0x7fc13c101c80 msgr2=0x7fc13c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 shutdown_connections 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:31.804+0000 7fc1439dd700 1 -- 172.21.15.100:0/3512318575 wait complete. 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-03-12T05:02:32.397 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 5... 2024-03-12T05:02:32.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3512318575' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-12T05:02:32.674 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.557+0000 7f0e75077200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:02:32.983 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.671+0000 7f0e75077200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:02:32.983 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.762+0000 7f0e75077200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:02:32.983 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:32.859+0000 7f0e75077200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:02:33.322 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:33 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:33.035+0000 7f0e75077200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:02:34.323 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:33 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:33.985+0000 7f0e75077200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:02:34.323 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:34.116+0000 7f0e75077200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:02:34.735 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:34.446+0000 7f0e75077200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:02:34.735 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:34.536+0000 7f0e75077200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:02:34.736 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:34.733+0000 7f0e75077200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:02:35.073 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:34.919+0000 7f0e75077200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:02:35.489 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:35 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:35.205+0000 7f0e75077200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:02:35.489 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:35 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:35.486+0000 7f0e75077200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:02:35.786 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:35 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:35.722+0000 7f0e75077200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:02:36.822 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:36 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:36.520+0000 7f0e75077200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:02:37.437 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:37.116+0000 7f0e75077200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:02:37.437 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:37.217+0000 7f0e75077200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:02:37.437 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:37.434+0000 7f0e75077200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: Active manager daemon x restarted 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: Activating manager daemon x 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: osdmap e2: 0 total, 0 up, 0 in 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: mgrmap e6: x(active, starting, since 0.00314489s) 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T05:02:37.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: Manager daemon x is now available 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:02:37.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:37 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: Found migration_current of "None". Setting to last migration. 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:38.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:38 smithi100 ceph-mon[32210]: mgrmap e7: x(active, since 1.00536s) 2024-03-12T05:02:39.127 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:39.127 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f077590f700 1 Processor -- start 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f077590f700 1 -- start start 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f077590f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f077010a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f077590f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07700757d0 con 0x7f07701081f0 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f076effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f077010a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f076effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f077010a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59858/0 (socket says 172.21.15.100:59858) 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.776+0000 7f076effd700 1 -- 172.21.15.100:0/441768586 learned_addr learned my addr 172.21.15.100:0/441768586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:39.128 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f076effd700 1 -- 172.21.15.100:0/441768586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077010ab20 con 0x7f07701081f0 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f076effd700 1 --2- 172.21.15.100:0/441768586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f077010a5e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07640098a0 tx=0x7f0764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f038626e4ac7664 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f076dffb700 1 -- 172.21.15.100:0/441768586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0764004160 con 0x7f07701081f0 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f076dffb700 1 -- 172.21.15.100:0/441768586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07640042c0 con 0x7f07701081f0 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f076dffb700 1 -- 172.21.15.100:0/441768586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0764011ac0 con 0x7f07701081f0 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f077590f700 1 -- 172.21.15.100:0/441768586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 msgr2=0x7f077010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.777+0000 7f077590f700 1 --2- 172.21.15.100:0/441768586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f077010a5e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07640098a0 tx=0x7f0764009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 -- 172.21.15.100:0/441768586 shutdown_connections 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 --2- 172.21.15.100:0/441768586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f077010a5e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 -- 172.21.15.100:0/441768586 >> 172.21.15.100:0/441768586 conn(0x7f0770101c80 msgr2=0x7f07701040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 -- 172.21.15.100:0/441768586 shutdown_connections 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 -- 172.21.15.100:0/441768586 wait complete. 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 Processor -- start 2024-03-12T05:02:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 -- start start 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.778+0000 7f077590f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f0770107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f077590f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770107fb0 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f076effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f0770107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f076effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f0770107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59866/0 (socket says 172.21.15.100:59866) 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f076effd700 1 -- 172.21.15.100:0/2537680869 learned_addr learned my addr 172.21.15.100:0/2537680869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f076effd700 1 -- 172.21.15.100:0/2537680869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009550 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f076effd700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f0770107a70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0764003f70 tx=0x7f0764011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07640039f0 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0764003b50 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07701061a0 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f076401a760 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.779+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0770106640 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.780+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f076401a9d0 con 0x7f07701081f0 2024-03-12T05:02:39.130 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.780+0000 7f077490d700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.780+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 --> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f075803b830 con 0x7f0758038c60 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.780+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f076404ceb0 con 0x7f07701081f0 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.780+0000 7f076e7fc700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 msgr2=0x7f075803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.780+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.981+0000 7f076e7fc700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 msgr2=0x7f075803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:32.981+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:33.382+0000 7f076e7fc700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 msgr2=0x7f075803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:33.382+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:34.183+0000 7f076e7fc700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 msgr2=0x7f075803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:34.183+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:35.784+0000 7f076e7fc700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 msgr2=0x7f075803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/981252995 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:35.784+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-12T05:02:39.131 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:37.450+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7f076402da00 con 0x7f07701081f0 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:37.450+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 msgr2=0x7f075803b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:37.450+0000 7f077490d700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.451+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7f076404ece0 con 0x7f07701081f0 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.451+0000 7f077490d700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f075803c550 0x7f075803e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.451+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f075803b830 con 0x7f075803c550 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.451+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f075803c550 0x7f075803e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.452+0000 7f076e7fc700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f075803c550 0x7f075803e940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0760003a10 tx=0x7f0760009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.453+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f075803b830 con 0x7f075803c550 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0770068d80 con 0x7f075803c550 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077490d700 1 -- 172.21.15.100:0/2537680869 <== mgr.14120 v2:172.21.15.100:6800/16620000 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0770068d80 con 0x7f075803c550 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f075803c550 msgr2=0x7f075803e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f075803c550 0x7f075803e940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0760003a10 tx=0x7f0760009290 comp rx=0 tx=0).stop 2024-03-12T05:02:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 msgr2=0x7f0770107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f0770107a70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0764003f70 tx=0x7f0764011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 shutdown_connections 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f075803c550 0x7f075803e940 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07701081f0 0x7f0770107a70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 --2- 172.21.15.100:0/2537680869 >> [v2:172.21.15.100:6800/981252995,v1:172.21.15.100:6801/981252995] conn(0x7f0758038c60 0x7f075803b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.459+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 >> 172.21.15.100:0/2537680869 conn(0x7f0770101c80 msgr2=0x7f0770102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.460+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 shutdown_connections 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:38.460+0000 7f077590f700 1 -- 172.21.15.100:0/2537680869 wait complete. 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 5 is available 2024-03-12T05:02:39.133 INFO:teuthology.orchestra.run.smithi100.stdout:Setting orchestrator backend to cephadm... 2024-03-12T05:02:39.751 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:39 smithi100 ceph-mon[32210]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-12T05:02:39.751 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:39 smithi100 ceph-mon[32210]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-12T05:02:39.752 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:39 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:02:40.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.533+0000 7f62f7e47700 1 Processor -- start 2024-03-12T05:02:40.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f7e47700 1 -- start start 2024-03-12T05:02:40.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f7e47700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f0108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:40.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f7e47700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f0108610 con 0x7f62f0107c20 2024-03-12T05:02:40.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f5be3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f0108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:40.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f5be3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f0108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59918/0 (socket says 172.21.15.100:59918) 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f5be3700 1 -- 172.21.15.100:0/3624772780 learned_addr learned my addr 172.21.15.100:0/3624772780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f5be3700 1 -- 172.21.15.100:0/3624772780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f0108750 con 0x7f62f0107c20 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.534+0000 7f62f5be3700 1 --2- 172.21.15.100:0/3624772780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f0108040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009e80 tx=0x7f62e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3665db7bd977ad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f4be1700 1 -- 172.21.15.100:0/3624772780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e800e070 con 0x7f62f0107c20 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f4be1700 1 -- 172.21.15.100:0/3624772780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62e800b8c0 con 0x7f62f0107c20 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f4be1700 1 -- 172.21.15.100:0/3624772780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e8004550 con 0x7f62f0107c20 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 -- 172.21.15.100:0/3624772780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 msgr2=0x7f62f0108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 --2- 172.21.15.100:0/3624772780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f0108040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009e80 tx=0x7f62e800b330 comp rx=0 tx=0).stop 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 -- 172.21.15.100:0/3624772780 shutdown_connections 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 --2- 172.21.15.100:0/3624772780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f0108040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 -- 172.21.15.100:0/3624772780 >> 172.21.15.100:0/3624772780 conn(0x7f62f0101450 msgr2=0x7f62f0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 -- 172.21.15.100:0/3624772780 shutdown_connections 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.535+0000 7f62f7e47700 1 -- 172.21.15.100:0/3624772780 wait complete. 2024-03-12T05:02:40.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f7e47700 1 Processor -- start 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f7e47700 1 -- start start 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f7e47700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f7e47700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f019db90 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f5be3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f5be3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59924/0 (socket says 172.21.15.100:59924) 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f5be3700 1 -- 172.21.15.100:0/2729116101 learned_addr learned my addr 172.21.15.100:0/2729116101 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.536+0000 7f62f5be3700 1 -- 172.21.15.100:0/2729116101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8009b30 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.537+0000 7f62f5be3700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f019d650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fd0 tx=0x7f62e8003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.537+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e800e040 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.537+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62e80128f0 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.537+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f019dd90 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.537+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e801c890 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.537+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f019e210 con 0x7f62f0107c20 2024-03-12T05:02:40.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.538+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f62e8012a60 con 0x7f62f0107c20 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.538+0000 7f62e6ffd700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f62dc038d00 0x7f62dc03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.538+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f0106420 con 0x7f62f0107c20 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.538+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f62e804cd20 con 0x7f62f0107c20 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.538+0000 7f62f53e2700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f62dc038d00 0x7f62dc03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.538+0000 7f62f53e2700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f62dc038d00 0x7f62dc03b1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.543+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62e8020080 con 0x7f62f0107c20 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.749+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f62f00620b0 con 0x7f62dc038d00 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.754+0000 7f62e6ffd700 1 -- 172.21.15.100:0/2729116101 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f62f00620b0 con 0x7f62dc038d00 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f62dc038d00 msgr2=0x7f62dc03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f62dc038d00 0x7f62dc03b1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 msgr2=0x7f62f019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f019d650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fd0 tx=0x7f62e8003d70 comp rx=0 tx=0).stop 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 shutdown_connections 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f62dc038d00 0x7f62dc03b1c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:40.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 --2- 172.21.15.100:0/2729116101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f0107c20 0x7f62f019d650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:40.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 >> 172.21.15.100:0/2729116101 conn(0x7f62f0101450 msgr2=0x7f62f0193e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:40.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 shutdown_connections 2024-03-12T05:02:40.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:39.757+0000 7f62f7e47700 1 -- 172.21.15.100:0/2729116101 wait complete. 2024-03-12T05:02:40.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: [12/Mar/2024:05:02:38] ENGINE Bus STARTING 2024-03-12T05:02:40.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: [12/Mar/2024:05:02:39] ENGINE Serving on https://172.21.15.100:7150 2024-03-12T05:02:40.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: [12/Mar/2024:05:02:39] ENGINE Serving on http://172.21.15.100:8765 2024-03-12T05:02:40.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: [12/Mar/2024:05:02:39] ENGINE Bus STARTED 2024-03-12T05:02:40.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: mgrmap e8: x(active, since 2s) 2024-03-12T05:02:40.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:40.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:40 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a33fe2700 1 Processor -- start 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a33fe2700 1 -- start start 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a33fe2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a33fe2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c07b910 con 0x7f3a2c07ce90 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a31d7e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:41.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a31d7e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c07b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33566/0 (socket says 172.21.15.100:33566) 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a31d7e700 1 -- 172.21.15.100:0/3064103303 learned_addr learned my addr 172.21.15.100:0/3064103303 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.687+0000 7f3a31d7e700 1 -- 172.21.15.100:0/3064103303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c07ba50 con 0x7f3a2c07ce90 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a31d7e700 1 --2- 172.21.15.100:0/3064103303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c07b340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a280098a0 tx=0x7f3a28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3c5c4376aaee4e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a30d7c700 1 -- 172.21.15.100:0/3064103303 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a28004160 con 0x7f3a2c07ce90 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a30d7c700 1 -- 172.21.15.100:0/3064103303 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a280042c0 con 0x7f3a2c07ce90 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a30d7c700 1 -- 172.21.15.100:0/3064103303 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a28011ac0 con 0x7f3a2c07ce90 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3064103303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 msgr2=0x7f3a2c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 --2- 172.21.15.100:0/3064103303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c07b340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a280098a0 tx=0x7f3a28009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3064103303 shutdown_connections 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 --2- 172.21.15.100:0/3064103303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c07b340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3064103303 >> 172.21.15.100:0/3064103303 conn(0x7f3a2c102da0 msgr2=0x7f3a2c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:41.462 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3064103303 shutdown_connections 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.688+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3064103303 wait complete. 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a33fe2700 1 Processor -- start 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a33fe2700 1 -- start start 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a33fe2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a33fe2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c19dc20 con 0x7f3a2c07ce90 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a31d7e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a31d7e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33578/0 (socket says 172.21.15.100:33578) 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a31d7e700 1 -- 172.21.15.100:0/3081460060 learned_addr learned my addr 172.21.15.100:0/3081460060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.689+0000 7f3a31d7e700 1 -- 172.21.15.100:0/3081460060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a28009550 con 0x7f3a2c07ce90 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.690+0000 7f3a31d7e700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c19d6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a28003f70 tx=0x7f3a28011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.690+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a280039f0 con 0x7f3a2c07ce90 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.690+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a28003b50 con 0x7f3a2c07ce90 2024-03-12T05:02:41.463 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.690+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c19de20 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.690+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a2801a760 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.690+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c19e2c0 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.691+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f3a2801a9d0 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.691+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c10db10 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.691+0000 7f3a22ffd700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3a18038c60 0x7f3a1803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.691+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a2804cea0 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.691+0000 7f3a3157d700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3a18038c60 0x7f3a1803b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.691+0000 7f3a3157d700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3a18038c60 0x7f3a1803b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006fd0 tx=0x7f3a1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.696+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a28018c20 con 0x7f3a2c07ce90 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.889+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a2c103fe0 con 0x7f3a18038c60 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.890+0000 7f3a22ffd700 1 -- 172.21.15.100:0/3081460060 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f3a2c103fe0 con 0x7f3a18038c60 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3a18038c60 msgr2=0x7f3a1803b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:41.464 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3a18038c60 0x7f3a1803b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006fd0 tx=0x7f3a1c006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 msgr2=0x7f3a2c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c19d6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a28003f70 tx=0x7f3a28011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 shutdown_connections 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3a18038c60 0x7f3a1803b120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 --2- 172.21.15.100:0/3081460060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2c07ce90 0x7f3a2c19d6e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 >> 172.21.15.100:0/3081460060 conn(0x7f3a2c102da0 msgr2=0x7f3a2c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 shutdown_connections 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:40.892+0000 7f3a33fe2700 1 -- 172.21.15.100:0/3081460060 wait complete. 2024-03-12T05:02:41.465 INFO:teuthology.orchestra.run.smithi100.stdout:Generating ssh key... 2024-03-12T05:02:41.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:41 smithi100 ceph-mon[32210]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: Generating public/private rsa key pair. 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: Your identification has been saved in /tmp/tmpam5km1sg/key. 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: Your public key has been saved in /tmp/tmpam5km1sg/key.pub. 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: The key fingerprint is: 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: SHA256:YxBDwF+SGt5aeNTQs5f/RAuExbvuB9HM4ds8+s2Oimo ceph-8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: The key's randomart image is: 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: +---[RSA 3072]----+ 2024-03-12T05:02:42.501 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | ..o== +. | 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | o +o= . o . | 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | . B.o o o * . | 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | + =.. o + * | 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | + S. . = = | 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | . . . + +.o| 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | . +. .| 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | E ...oo.| 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: | .....oooo+| 2024-03-12T05:02:42.502 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: +----[SHA256]-----+ 2024-03-12T05:02:42.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-mon[32210]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:42.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:42.777 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:42 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6ddd41a700 1 Processor -- start 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6ddd41a700 1 -- start start 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6ddd41a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd8106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6ddd41a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd807ae70 con 0x7f6dd8105ff0 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd8106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd8106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33580/0 (socket says 172.21.15.100:33580) 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.876+0000 7f6dd6ffd700 1 -- 172.21.15.100:0/2180083140 learned_addr learned my addr 172.21.15.100:0/2180083140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:42.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6dd6ffd700 1 -- 172.21.15.100:0/2180083140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8106950 con 0x7f6dd8105ff0 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6dd6ffd700 1 --2- 172.21.15.100:0/2180083140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd8106410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00098a0 tx=0x7f6dd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d873eef564ce21fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6dd5ffb700 1 -- 172.21.15.100:0/2180083140 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dd0004160 con 0x7f6dd8105ff0 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6dd5ffb700 1 -- 172.21.15.100:0/2180083140 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd00042c0 con 0x7f6dd8105ff0 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6dd5ffb700 1 -- 172.21.15.100:0/2180083140 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dd0011ac0 con 0x7f6dd8105ff0 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6ddd41a700 1 -- 172.21.15.100:0/2180083140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 msgr2=0x7f6dd8106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6ddd41a700 1 --2- 172.21.15.100:0/2180083140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd8106410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00098a0 tx=0x7f6dd0009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6ddd41a700 1 -- 172.21.15.100:0/2180083140 shutdown_connections 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6ddd41a700 1 --2- 172.21.15.100:0/2180083140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd8106410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6ddd41a700 1 -- 172.21.15.100:0/2180083140 >> 172.21.15.100:0/2180083140 conn(0x7f6dd8101c70 msgr2=0x7f6dd8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.877+0000 7f6ddd41a700 1 -- 172.21.15.100:0/2180083140 shutdown_connections 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6ddd41a700 1 -- 172.21.15.100:0/2180083140 wait complete. 2024-03-12T05:02:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6ddd41a700 1 Processor -- start 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6ddd41a700 1 -- start start 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6ddd41a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6ddd41a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819dbf0 con 0x7f6dd8105ff0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33584/0 (socket says 172.21.15.100:33584) 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.878+0000 7f6dd6ffd700 1 -- 172.21.15.100:0/3936735292 learned_addr learned my addr 172.21.15.100:0/3936735292 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6dd6ffd700 1 -- 172.21.15.100:0/3936735292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0009550 con 0x7f6dd8105ff0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6dd6ffd700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd819d6b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0003f70 tx=0x7f6dd0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dd00039f0 con 0x7f6dd8105ff0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd0003b50 con 0x7f6dd8105ff0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd819ddf0 con 0x7f6dd8105ff0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dd001a760 con 0x7f6dd8105ff0 2024-03-12T05:02:42.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.879+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd819e290 con 0x7f6dd8105ff0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.880+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f6dd001a8e0 con 0x7f6dd8105ff0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.880+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd8197320 con 0x7f6dd8105ff0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.880+0000 7f6dbffff700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f6dc0038cb0 0x7f6dc003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.880+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6dd004d000 con 0x7f6dd8105ff0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.880+0000 7f6dd67fc700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f6dc0038cb0 0x7f6dc003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.881+0000 7f6dd67fc700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f6dc0038cb0 0x7f6dc003b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006fd0 tx=0x7f6dcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:41.885+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dd0080d20 con 0x7f6dd8105ff0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.075+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dd802d9d0 con 0x7f6dc0038cb0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.204+0000 7f6dbffff700 1 -- 172.21.15.100:0/3936735292 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6dd802d9d0 con 0x7f6dc0038cb0 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f6dc0038cb0 msgr2=0x7f6dc003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f6dc0038cb0 0x7f6dc003b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006fd0 tx=0x7f6dcc006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 msgr2=0x7f6dd819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd819d6b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0003f70 tx=0x7f6dd0011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 shutdown_connections 2024-03-12T05:02:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f6dc0038cb0 0x7f6dc003b170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 --2- 172.21.15.100:0/3936735292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dd8105ff0 0x7f6dd819d6b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 >> 172.21.15.100:0/3936735292 conn(0x7f6dd8101c70 msgr2=0x7f6dd8194460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 shutdown_connections 2024-03-12T05:02:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:42.206+0000 7f6ddd41a700 1 -- 172.21.15.100:0/3936735292 wait complete. 2024-03-12T05:02:43.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:43 smithi100 ceph-mon[32210]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:43.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:43 smithi100 ceph-mon[32210]: Generating ssh key... 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.168+0000 7f1fd7c9c700 1 Processor -- start 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd7c9c700 1 -- start start 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd7c9c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd7c9c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd0108110 con 0x7f1fd0107720 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd5a38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd5a38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33590/0 (socket says 172.21.15.100:33590) 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd5a38700 1 -- 172.21.15.100:0/4004513424 learned_addr learned my addr 172.21.15.100:0/4004513424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd5a38700 1 -- 172.21.15.100:0/4004513424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd01088d0 con 0x7f1fd0107720 2024-03-12T05:02:43.874 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.169+0000 7f1fd5a38700 1 --2- 172.21.15.100:0/4004513424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd0107b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00098a0 tx=0x7f1fc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6897992bcf677fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd4a36700 1 -- 172.21.15.100:0/4004513424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fc0004160 con 0x7f1fd0107720 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd4a36700 1 -- 172.21.15.100:0/4004513424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fc00042c0 con 0x7f1fd0107720 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd4a36700 1 -- 172.21.15.100:0/4004513424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fc0011ac0 con 0x7f1fd0107720 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/4004513424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 msgr2=0x7f1fd0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 --2- 172.21.15.100:0/4004513424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd0107b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00098a0 tx=0x7f1fc0009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/4004513424 shutdown_connections 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 --2- 172.21.15.100:0/4004513424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd0107b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/4004513424 >> 172.21.15.100:0/4004513424 conn(0x7f1fd0102da0 msgr2=0x7f1fd01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/4004513424 shutdown_connections 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.170+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/4004513424 wait complete. 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd7c9c700 1 Processor -- start 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd7c9c700 1 -- start start 2024-03-12T05:02:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd7c9c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd7c9c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd019baa0 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd5a38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd5a38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33598/0 (socket says 172.21.15.100:33598) 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd5a38700 1 -- 172.21.15.100:0/3049002623 learned_addr learned my addr 172.21.15.100:0/3049002623 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.171+0000 7f1fd5a38700 1 -- 172.21.15.100:0/3049002623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc0009550 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.172+0000 7f1fd5a38700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd019b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009fb0 tx=0x7f1fc0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.172+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fc00038b0 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.172+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fc0003a10 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.172+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd019bca0 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.172+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fc00195e0 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.172+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd019c140 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.173+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f1fc00198a0 con 0x7f1fd0107720 2024-03-12T05:02:43.876 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.173+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd010bb00 con 0x7f1fd0107720 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.173+0000 7f1fc6ffd700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f1fbc038c60 0x7f1fbc03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.173+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1fc004ce70 con 0x7f1fd0107720 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.173+0000 7f1fd5237700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f1fbc038c60 0x7f1fbc03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.173+0000 7f1fd5237700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f1fbc038c60 0x7f1fbc03b120 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006fd0 tx=0x7f1fcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.178+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fc002b340 con 0x7f1fd0107720 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.371+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fd0104060 con 0x7f1fbc038c60 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.371+0000 7f1fc6ffd700 1 -- 172.21.15.100:0/3049002623 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1fd0104060 con 0x7f1fbc038c60 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.373+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f1fbc038c60 msgr2=0x7f1fbc03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.373+0000 7f1fd7c9c700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f1fbc038c60 0x7f1fbc03b120 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006fd0 tx=0x7f1fcc006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.373+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 msgr2=0x7f1fd019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.373+0000 7f1fd7c9c700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd019b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009fb0 tx=0x7f1fc0011b20 comp rx=0 tx=0).stop 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.374+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 shutdown_connections 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.374+0000 7f1fd7c9c700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f1fbc038c60 0x7f1fbc03b120 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.374+0000 7f1fd7c9c700 1 --2- 172.21.15.100:0/3049002623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd0107720 0x7f1fd019b560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.374+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 >> 172.21.15.100:0/3049002623 conn(0x7f1fd0102da0 msgr2=0x7f1fd0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.374+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 shutdown_connections 2024-03-12T05:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:43.374+0000 7f1fd7c9c700 1 -- 172.21.15.100:0/3049002623 wait complete. 2024-03-12T05:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-12T05:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:Adding key to root@localhost authorized_keys... 2024-03-12T05:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:Adding host smithi100... 2024-03-12T05:02:44.711 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:44 smithi100 ceph-mon[32210]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:45.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:45 smithi100 ceph-mon[32210]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:46.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:46 smithi100 ceph-mon[32210]: Deploying cephadm binary to smithi100 2024-03-12T05:02:49.373 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Added host 'smithi100' with addr '172.21.15.100' 2024-03-12T05:02:49.373 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28f5e28700 1 Processor -- start 2024-03-12T05:02:49.373 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28f5e28700 1 -- start start 2024-03-12T05:02:49.373 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28f5e28700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f0107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28f5e28700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0108150 con 0x7f28f0107760 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f0107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f0107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33610/0 (socket says 172.21.15.100:33610) 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.250+0000 7f28ef7fe700 1 -- 172.21.15.100:0/4017715180 learned_addr learned my addr 172.21.15.100:0/4017715180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.251+0000 7f28ef7fe700 1 -- 172.21.15.100:0/4017715180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0108910 con 0x7f28f0107760 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.251+0000 7f28ef7fe700 1 --2- 172.21.15.100:0/4017715180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f0107b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28e00098a0 tx=0x7f28e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ba6a5f4e5506c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.251+0000 7f28eeffd700 1 -- 172.21.15.100:0/4017715180 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28e0004160 con 0x7f28f0107760 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.251+0000 7f28eeffd700 1 -- 172.21.15.100:0/4017715180 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28e00042c0 con 0x7f28f0107760 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.251+0000 7f28eeffd700 1 -- 172.21.15.100:0/4017715180 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28e0011ac0 con 0x7f28f0107760 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- 172.21.15.100:0/4017715180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 msgr2=0x7f28f0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 --2- 172.21.15.100:0/4017715180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f0107b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28e00098a0 tx=0x7f28e0009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:49.374 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- 172.21.15.100:0/4017715180 shutdown_connections 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 --2- 172.21.15.100:0/4017715180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f0107b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- 172.21.15.100:0/4017715180 >> 172.21.15.100:0/4017715180 conn(0x7f28f0102da0 msgr2=0x7f28f01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- 172.21.15.100:0/4017715180 shutdown_connections 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- 172.21.15.100:0/4017715180 wait complete. 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 Processor -- start 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- start start 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.252+0000 7f28f5e28700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019ba50 con 0x7f28f0107760 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33616/0 (socket says 172.21.15.100:33616) 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ef7fe700 1 -- 172.21.15.100:0/638010055 learned_addr learned my addr 172.21.15.100:0/638010055 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ef7fe700 1 -- 172.21.15.100:0/638010055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009550 con 0x7f28f0107760 2024-03-12T05:02:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ef7fe700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f019b510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28e0003f70 tx=0x7f28e0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28e00039f0 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28e0003b50 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f019bc50 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28e001a690 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.253+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f019c0f0 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.254+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f28e001a900 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.254+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f010bb40 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.254+0000 7f28ed7fa700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f28dc038cb0 0x7f28dc03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.254+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f28e004d030 con 0x7f28f0107760 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.254+0000 7f28e7fff700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f28dc038cb0 0x7f28dc03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.255+0000 7f28e7fff700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f28dc038cb0 0x7f28dc03b170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f28d8006fd0 tx=0x7f28d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:49.376 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.259+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28e0080d20 con 0x7f28f0107760 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:44.455+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}) v1 -- 0x7f28f0104010 con 0x7f28dc038cb0 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.758+0000 7f28ed7fa700 1 -- 172.21.15.100:0/638010055 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f28f0104010 con 0x7f28dc038cb0 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f28dc038cb0 msgr2=0x7f28dc03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f28dc038cb0 0x7f28dc03b170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f28d8006fd0 tx=0x7f28d8006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 msgr2=0x7f28f019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f019b510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28e0003f70 tx=0x7f28e0011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 shutdown_connections 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f28dc038cb0 0x7f28dc03b170 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 --2- 172.21.15.100:0/638010055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0107760 0x7f28f019b510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 >> 172.21.15.100:0/638010055 conn(0x7f28f0102da0 msgr2=0x7f28f0103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:49.377 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 shutdown_connections 2024-03-12T05:02:49.378 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:48.761+0000 7f28f5e28700 1 -- 172.21.15.100:0/638010055 wait complete. 2024-03-12T05:02:49.378 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mon service... 2024-03-12T05:02:50.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:49 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:50.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:49 smithi100 ceph-mon[32210]: Added host smithi100 2024-03-12T05:02:50.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:49 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:02:50.723 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.792+0000 7f3ce9481700 1 Processor -- start 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.792+0000 7f3ce9481700 1 -- start start 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.792+0000 7f3ce9481700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce4106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.792+0000 7f3ce9481700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4106fe0 con 0x7f3ce41065f0 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce4106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce4106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33632/0 (socket says 172.21.15.100:33632) 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce2ffd700 1 -- 172.21.15.100:0/188357079 learned_addr learned my addr 172.21.15.100:0/188357079 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce2ffd700 1 -- 172.21.15.100:0/188357079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce41077f0 con 0x7f3ce41065f0 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce2ffd700 1 --2- 172.21.15.100:0/188357079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce4106a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098a0 tx=0x7f3cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d946b2e67bff197d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:50.724 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce1ffb700 1 -- 172.21.15.100:0/188357079 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4004160 con 0x7f3ce41065f0 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce1ffb700 1 -- 172.21.15.100:0/188357079 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cd40042c0 con 0x7f3ce41065f0 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.793+0000 7f3ce1ffb700 1 -- 172.21.15.100:0/188357079 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4011ac0 con 0x7f3ce41065f0 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 -- 172.21.15.100:0/188357079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 msgr2=0x7f3ce4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 --2- 172.21.15.100:0/188357079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce4106a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098a0 tx=0x7f3cd4009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 -- 172.21.15.100:0/188357079 shutdown_connections 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 --2- 172.21.15.100:0/188357079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce4106a10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 -- 172.21.15.100:0/188357079 >> 172.21.15.100:0/188357079 conn(0x7f3ce4101c70 msgr2=0x7f3ce4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 -- 172.21.15.100:0/188357079 shutdown_connections 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 -- 172.21.15.100:0/188357079 wait complete. 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.794+0000 7f3ce9481700 1 Processor -- start 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce9481700 1 -- start start 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce9481700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce9481700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419a940 con 0x7f3ce41065f0 2024-03-12T05:02:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33640/0 (socket says 172.21.15.100:33640) 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce2ffd700 1 -- 172.21.15.100:0/1213462324 learned_addr learned my addr 172.21.15.100:0/1213462324 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce2ffd700 1 -- 172.21.15.100:0/1213462324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009550 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce2ffd700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce419a400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000c00 tx=0x7f3cd40036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cd40039b0 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cd4020070 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.795+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce419ab40 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.796+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cd401c6c0 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.796+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce419afe0 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.796+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f3cd4022030 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.796+0000 7f3ccbfff700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3ccc038d00 0x7f3ccc03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.796+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce410aa20 con 0x7f3ce41065f0 2024-03-12T05:02:50.726 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.797+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3cd404f0c0 con 0x7f3ce41065f0 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.797+0000 7f3ce27fc700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3ccc038d00 0x7f3ccc03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.797+0000 7f3ce27fc700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3ccc038d00 0x7f3ccc03b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8006fd0 tx=0x7f3cd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:49.802+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3cd400b3e0 con 0x7f3ce41065f0 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.010+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3ce4102eb0 con 0x7f3ccc038d00 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.015+0000 7f3ccbfff700 1 -- 172.21.15.100:0/1213462324 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3ce4102eb0 con 0x7f3ccc038d00 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3ccc038d00 msgr2=0x7f3ccc03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3ccc038d00 0x7f3ccc03b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8006fd0 tx=0x7f3cd8006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 msgr2=0x7f3ce419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce419a400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000c00 tx=0x7f3cd40036a0 comp rx=0 tx=0).stop 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 shutdown_connections 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f3ccc038d00 0x7f3ccc03b1c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 --2- 172.21.15.100:0/1213462324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ce41065f0 0x7f3ce419a400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 >> 172.21.15.100:0/1213462324 conn(0x7f3ce4101c70 msgr2=0x7f3ce41027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:50.727 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 shutdown_connections 2024-03-12T05:02:50.728 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:50.017+0000 7f3ce9481700 1 -- 172.21.15.100:0/1213462324 wait complete. 2024-03-12T05:02:50.728 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mgr service... 2024-03-12T05:02:51.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:51 smithi100 ceph-mon[32210]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:51.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:51 smithi100 ceph-mon[32210]: Saving service mon spec with placement count:5 2024-03-12T05:02:51.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:51 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:52.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:52 smithi100 ceph-mon[32210]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:02:52.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:52 smithi100 ceph-mon[32210]: Saving service mgr spec with placement count:2 2024-03-12T05:02:52.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:52 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:52.677 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc09343700 1 Processor -- start 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc09343700 1 -- start start 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc09343700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0410a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc09343700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0407aed0 con 0x7efc04108160 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc02ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0410a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc02ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0410a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51220/0 (socket says 172.21.15.100:51220) 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc02ffd700 1 -- 172.21.15.100:0/2925067247 learned_addr learned my addr 172.21.15.100:0/2925067247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.150+0000 7efc02ffd700 1 -- 172.21.15.100:0/2925067247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0410aa90 con 0x7efc04108160 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc02ffd700 1 --2- 172.21.15.100:0/2925067247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0410a550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efbf40098a0 tx=0x7efbf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9219bec1e810cb7c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc01ffb700 1 -- 172.21.15.100:0/2925067247 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf4004160 con 0x7efc04108160 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc01ffb700 1 -- 172.21.15.100:0/2925067247 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbf40042c0 con 0x7efc04108160 2024-03-12T05:02:52.678 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc01ffb700 1 -- 172.21.15.100:0/2925067247 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf4011ac0 con 0x7efc04108160 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 -- 172.21.15.100:0/2925067247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 msgr2=0x7efc0410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 --2- 172.21.15.100:0/2925067247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0410a550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efbf40098a0 tx=0x7efbf4009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 -- 172.21.15.100:0/2925067247 shutdown_connections 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 --2- 172.21.15.100:0/2925067247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0410a550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 -- 172.21.15.100:0/2925067247 >> 172.21.15.100:0/2925067247 conn(0x7efc04101c70 msgr2=0x7efc04104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 -- 172.21.15.100:0/2925067247 shutdown_connections 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.151+0000 7efc09343700 1 -- 172.21.15.100:0/2925067247 wait complete. 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc09343700 1 Processor -- start 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc09343700 1 -- start start 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc09343700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc09343700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419b9d0 con 0x7efc04108160 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc02ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:52.679 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc02ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51234/0 (socket says 172.21.15.100:51234) 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.152+0000 7efc02ffd700 1 -- 172.21.15.100:0/3316380415 learned_addr learned my addr 172.21.15.100:0/3316380415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efc02ffd700 1 -- 172.21.15.100:0/3316380415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4009550 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efc02ffd700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0419b490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efbf4003f70 tx=0x7efbf4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf40039f0 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbf4003b50 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0419bbd0 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf401a760 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.153+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0419c070 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.154+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7efbf401a9d0 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.154+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0410b8f0 con 0x7efc04108160 2024-03-12T05:02:52.680 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.154+0000 7efbebfff700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7efbec038c60 0x7efbec03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.154+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efbf404cf10 con 0x7efc04108160 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.154+0000 7efc027fc700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7efbec038c60 0x7efbec03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.154+0000 7efc027fc700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7efbec038c60 0x7efbec03b120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efbf8006fd0 tx=0x7efbf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.159+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efbf401ac80 con 0x7efc04108160 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.374+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efc04102eb0 con 0x7efbec038c60 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.381+0000 7efbebfff700 1 -- 172.21.15.100:0/3316380415 <== mgr.14120 v2:172.21.15.100:6800/16620000 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efc04102eb0 con 0x7efbec038c60 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7efbec038c60 msgr2=0x7efbec03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7efbec038c60 0x7efbec03b120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efbf8006fd0 tx=0x7efbf8006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 msgr2=0x7efc0419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0419b490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efbf4003f70 tx=0x7efbf4011b40 comp rx=0 tx=0).stop 2024-03-12T05:02:52.681 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 shutdown_connections 2024-03-12T05:02:52.682 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7efbec038c60 0x7efbec03b120 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:52.682 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 --2- 172.21.15.100:0/3316380415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc04108160 0x7efc0419b490 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:52.682 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 >> 172.21.15.100:0/3316380415 conn(0x7efc04101c70 msgr2=0x7efc041027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:52.682 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 shutdown_connections 2024-03-12T05:02:52.682 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:51.384+0000 7efc09343700 1 -- 172.21.15.100:0/3316380415 wait complete. 2024-03-12T05:02:53.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:53 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:53.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:53 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:53.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2106622978' entity='client.admin' 2024-03-12T05:02:53.842 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.092+0000 7f228aedf700 1 Processor -- start 2024-03-12T05:02:53.842 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.092+0000 7f228aedf700 1 -- start start 2024-03-12T05:02:53.842 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.092+0000 7f228aedf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f2284107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:53.842 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.092+0000 7f228aedf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284108150 con 0x7f2284107760 2024-03-12T05:02:53.842 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f2288c7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f2284107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:53.842 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f2288c7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f2284107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51242/0 (socket says 172.21.15.100:51242) 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f2288c7b700 1 -- 172.21.15.100:0/1357553137 learned_addr learned my addr 172.21.15.100:0/1357553137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f2288c7b700 1 -- 172.21.15.100:0/1357553137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2284108910 con 0x7f2284107760 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f2288c7b700 1 --2- 172.21.15.100:0/1357553137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f2284107b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22780098a0 tx=0x7f2278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abee35fe98a4406 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f22837fe700 1 -- 172.21.15.100:0/1357553137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2278004160 con 0x7f2284107760 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f22837fe700 1 -- 172.21.15.100:0/1357553137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22780042c0 con 0x7f2284107760 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.093+0000 7f22837fe700 1 -- 172.21.15.100:0/1357553137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2278011a80 con 0x7f2284107760 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 -- 172.21.15.100:0/1357553137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 msgr2=0x7f2284107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 --2- 172.21.15.100:0/1357553137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f2284107b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22780098a0 tx=0x7f2278009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 -- 172.21.15.100:0/1357553137 shutdown_connections 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 --2- 172.21.15.100:0/1357553137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f2284107b80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 -- 172.21.15.100:0/1357553137 >> 172.21.15.100:0/1357553137 conn(0x7f2284102da0 msgr2=0x7f22841051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 -- 172.21.15.100:0/1357553137 shutdown_connections 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.094+0000 7f228aedf700 1 -- 172.21.15.100:0/1357553137 wait complete. 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f228aedf700 1 Processor -- start 2024-03-12T05:02:53.843 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f228aedf700 1 -- start start 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f228aedf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f228407cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f228aedf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228407d0d0 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2288c7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f228407cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2288c7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f228407cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51254/0 (socket says 172.21.15.100:51254) 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2288c7b700 1 -- 172.21.15.100:0/2106622978 learned_addr learned my addr 172.21.15.100:0/2106622978 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2288c7b700 1 -- 172.21.15.100:0/2106622978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278009550 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2288c7b700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f228407cb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2278011740 tx=0x7f2278011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22780039e0 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2278003b40 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.095+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228407b2c0 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.096+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2278024740 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.096+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228407b760 con 0x7f2284107760 2024-03-12T05:02:53.844 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.096+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f227802d3f0 con 0x7f2284107760 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.097+0000 7f2281ffb700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f2274038cb0 0x7f227403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.097+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f227804dfd0 con 0x7f2284107760 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.097+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228410bb40 con 0x7f2284107760 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.097+0000 7f2283fff700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f2274038cb0 0x7f227403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.097+0000 7f2283fff700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f2274038cb0 0x7f227403b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2270006fd0 tx=0x7f2270006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.102+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f227802d6f0 con 0x7f2284107760 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.301+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f228410bd50 con 0x7f2284107760 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.306+0000 7f2281ffb700 1 -- 172.21.15.100:0/2106622978 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f227804d940 con 0x7f2284107760 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f2274038cb0 msgr2=0x7f227403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f2274038cb0 0x7f227403b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2270006fd0 tx=0x7f2270006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 msgr2=0x7f228407cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f228407cb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2278011740 tx=0x7f2278011b00 comp rx=0 tx=0).stop 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 shutdown_connections 2024-03-12T05:02:53.845 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f2274038cb0 0x7f227403b170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:53.846 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 --2- 172.21.15.100:0/2106622978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284107760 0x7f228407cb90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:53.846 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.308+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 >> 172.21.15.100:0/2106622978 conn(0x7f2284102da0 msgr2=0x7f2284103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:53.846 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.309+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 shutdown_connections 2024-03-12T05:02:53.846 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:53.309+0000 7f228aedf700 1 -- 172.21.15.100:0/2106622978 wait complete. 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.235+0000 7f53f14ae700 1 Processor -- start 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.235+0000 7f53f14ae700 1 -- start start 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53f14ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec105e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53f14ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec106410 con 0x7f53ec105a20 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51264/0 (socket says 172.21.15.100:51264) 2024-03-12T05:02:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53eaffd700 1 -- 172.21.15.100:0/4156196280 learned_addr learned my addr 172.21.15.100:0/4156196280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53eaffd700 1 -- 172.21.15.100:0/4156196280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec106550 con 0x7f53ec105a20 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53eaffd700 1 --2- 172.21.15.100:0/4156196280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec105e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0098a0 tx=0x7f53dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb65fada9b08216 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53e9ffb700 1 -- 172.21.15.100:0/4156196280 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53dc004160 con 0x7f53ec105a20 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53e9ffb700 1 -- 172.21.15.100:0/4156196280 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc0042c0 con 0x7f53ec105a20 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.236+0000 7f53e9ffb700 1 -- 172.21.15.100:0/4156196280 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53dc011a80 con 0x7f53ec105a20 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 -- 172.21.15.100:0/4156196280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 msgr2=0x7f53ec105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 --2- 172.21.15.100:0/4156196280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec105e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0098a0 tx=0x7f53dc009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 -- 172.21.15.100:0/4156196280 shutdown_connections 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 --2- 172.21.15.100:0/4156196280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec105e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 -- 172.21.15.100:0/4156196280 >> 172.21.15.100:0/4156196280 conn(0x7f53ec101420 msgr2=0x7f53ec103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 -- 172.21.15.100:0/4156196280 shutdown_connections 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.237+0000 7f53f14ae700 1 -- 172.21.15.100:0/4156196280 wait complete. 2024-03-12T05:02:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53f14ae700 1 Processor -- start 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53f14ae700 1 -- start start 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53f14ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53f14ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec19ba80 con 0x7f53ec105a20 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec19b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51278/0 (socket says 172.21.15.100:51278) 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.238+0000 7f53eaffd700 1 -- 172.21.15.100:0/2647368976 learned_addr learned my addr 172.21.15.100:0/2647368976 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53eaffd700 1 -- 172.21.15.100:0/2647368976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc009550 con 0x7f53ec105a20 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53eaffd700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec19b540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53dc011740 tx=0x7f53dc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53dc003b20 con 0x7f53ec105a20 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec19bc80 con 0x7f53ec105a20 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc01a910 con 0x7f53ec105a20 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53dc024850 con 0x7f53ec105a20 2024-03-12T05:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.239+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec19c120 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.240+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f53dc02d3f0 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.240+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d8005360 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.240+0000 7f53d3fff700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f53d4038d00 0x7f53d403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.240+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f53dc04dfd0 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.240+0000 7f53ea7fc700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f53d4038d00 0x7f53d403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.241+0000 7f53ea7fc700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f53d4038d00 0x7f53d403b1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.246+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53dc010970 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.448+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f53d8005610 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.450+0000 7f53d3fff700 1 -- 172.21.15.100:0/2647368976 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f53dc00b3e0 con 0x7f53ec105a20 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f53d4038d00 msgr2=0x7f53d403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f53d4038d00 0x7f53d403b1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 msgr2=0x7f53ec19b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec19b540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53dc011740 tx=0x7f53dc011b00 comp rx=0 tx=0).stop 2024-03-12T05:02:55.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 shutdown_connections 2024-03-12T05:02:55.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f53d4038d00 0x7f53d403b1c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:55.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 --2- 172.21.15.100:0/2647368976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ec105a20 0x7f53ec19b540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:55.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.452+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 >> 172.21.15.100:0/2647368976 conn(0x7f53ec101420 msgr2=0x7f53ec101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:55.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.453+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 shutdown_connections 2024-03-12T05:02:55.104 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:54.453+0000 7f53f14ae700 1 -- 172.21.15.100:0/2647368976 wait complete. 2024-03-12T05:02:55.104 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling the dashboard module... 2024-03-12T05:02:55.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2647368976' entity='client.admin' 2024-03-12T05:02:57.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:56 smithi100 ceph-mon[32210]: from='mgr.14120 172.21.15.100:0/1647995372' entity='mgr.x' 2024-03-12T05:02:57.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1847850699' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-12T05:02:57.007 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:56 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: ignoring --setuser ceph since I am not root 2024-03-12T05:02:57.007 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:56 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: ignoring --setgroup ceph since I am not root 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f8054692700 1 Processor -- start 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f8054692700 1 -- start start 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f8054692700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c108030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f8054692700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c108600 con 0x7f804c107c10 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f805242e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f805242e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51294/0 (socket says 172.21.15.100:51294) 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.923+0000 7f805242e700 1 -- 172.21.15.100:0/629071896 learned_addr learned my addr 172.21.15.100:0/629071896 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f805242e700 1 -- 172.21.15.100:0/629071896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c106820 con 0x7f804c107c10 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f805242e700 1 --2- 172.21.15.100:0/629071896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c108030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80480098a0 tx=0x7f8048009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5527ac34ea984444 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:57.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f805142c700 1 -- 172.21.15.100:0/629071896 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8048004160 con 0x7f804c107c10 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f805142c700 1 -- 172.21.15.100:0/629071896 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80480042c0 con 0x7f804c107c10 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f805142c700 1 -- 172.21.15.100:0/629071896 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8048011a80 con 0x7f804c107c10 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f8054692700 1 -- 172.21.15.100:0/629071896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 msgr2=0x7f804c108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.924+0000 7f8054692700 1 --2- 172.21.15.100:0/629071896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c108030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80480098a0 tx=0x7f8048009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 -- 172.21.15.100:0/629071896 shutdown_connections 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 --2- 172.21.15.100:0/629071896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c108030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 -- 172.21.15.100:0/629071896 >> 172.21.15.100:0/629071896 conn(0x7f804c1013d0 msgr2=0x7f804c1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 -- 172.21.15.100:0/629071896 shutdown_connections 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 -- 172.21.15.100:0/629071896 wait complete. 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 Processor -- start 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.925+0000 7f8054692700 1 -- start start 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f8054692700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f8054692700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c19a950 con 0x7f804c107c10 2024-03-12T05:02:57.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f805242e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f805242e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51310/0 (socket says 172.21.15.100:51310) 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f805242e700 1 -- 172.21.15.100:0/1847850699 learned_addr learned my addr 172.21.15.100:0/1847850699 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f805242e700 1 -- 172.21.15.100:0/1847850699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048009550 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f805242e700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c19a410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8048011740 tx=0x7f8048011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80480039e0 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c19ab50 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8048003b40 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8048024740 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.926+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c19aff0 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.927+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f804802d3f0 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.927+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c10aa60 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.928+0000 7f80437fe700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f8038038cb0 0x7f803803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.928+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f804804d090 con 0x7f804c107c10 2024-03-12T05:02:57.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.928+0000 7f8051c2d700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f8038038cb0 0x7f803803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.928+0000 7f8051c2d700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f8038038cb0 0x7f803803b170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f803c006fd0 tx=0x7f803c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:55.933+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8048010970 con 0x7f804c107c10 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.193+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f804c10ac70 con 0x7f804c107c10 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.747+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f804804ca00 con 0x7f804c107c10 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.749+0000 7f80437fe700 1 -- 172.21.15.100:0/1847850699 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f8048019460 con 0x7f804c107c10 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f8038038cb0 msgr2=0x7f803803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f8038038cb0 0x7f803803b170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f803c006fd0 tx=0x7f803c006e40 comp rx=0 tx=0).stop 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 msgr2=0x7f804c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c19a410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8048011740 tx=0x7f8048011b00 comp rx=0 tx=0).stop 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 shutdown_connections 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f8038038cb0 0x7f803803b170 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:57.329 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 --2- 172.21.15.100:0/1847850699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804c107c10 0x7f804c19a410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:57.330 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.750+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 >> 172.21.15.100:0/1847850699 conn(0x7f804c1013d0 msgr2=0x7f804c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:57.330 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.751+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 shutdown_connections 2024-03-12T05:02:57.330 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:56.751+0000 7f8054692700 1 -- 172.21.15.100:0/1847850699 wait complete. 2024-03-12T05:02:57.572 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:57 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:57.326+0000 7fbca054c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:02:57.572 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:57 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:57.428+0000 7fbca054c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:02:57.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1847850699' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-12T05:02:57.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:57 smithi100 ceph-mon[32210]: mgrmap e9: x(active, since 19s) 2024-03-12T05:02:58.322 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.086+0000 7fbca054c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:02:58.323 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.180+0000 7fbca054c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:02:58.607 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.324+0000 7fbca054c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:02:58.607 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.420+0000 7fbca054c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:02:58.607 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.504+0000 7fbca054c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:02:58.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:02:58.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-12T05:02:58.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T05:02:58.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-12T05:02:58.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57dd830700 1 Processor -- start 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57dd830700 1 -- start start 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57dd830700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d810a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57dd830700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d807ae10 con 0x7f57d8108260 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d810a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d810a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51326/0 (socket says 172.21.15.100:51326) 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57d6ffd700 1 -- 172.21.15.100:0/3886863289 learned_addr learned my addr 172.21.15.100:0/3886863289 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.786+0000 7f57d6ffd700 1 -- 172.21.15.100:0/3886863289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d807af50 con 0x7f57d8108260 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57d6ffd700 1 --2- 172.21.15.100:0/3886863289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d810a650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57c80098a0 tx=0x7f57c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d9a7e8ffd8805a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:58.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57d5ffb700 1 -- 172.21.15.100:0/3886863289 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c8004160 con 0x7f57d8108260 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57d5ffb700 1 -- 172.21.15.100:0/3886863289 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c80042c0 con 0x7f57d8108260 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57d5ffb700 1 -- 172.21.15.100:0/3886863289 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c8011a80 con 0x7f57d8108260 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57dd830700 1 -- 172.21.15.100:0/3886863289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 msgr2=0x7f57d810a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57dd830700 1 --2- 172.21.15.100:0/3886863289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d810a650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57c80098a0 tx=0x7f57c8009bb0 comp rx=0 tx=0).stop 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57dd830700 1 -- 172.21.15.100:0/3886863289 shutdown_connections 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57dd830700 1 --2- 172.21.15.100:0/3886863289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d810a650 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57dd830700 1 -- 172.21.15.100:0/3886863289 >> 172.21.15.100:0/3886863289 conn(0x7f57d8101c70 msgr2=0x7f57d8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.787+0000 7f57dd830700 1 -- 172.21.15.100:0/3886863289 shutdown_connections 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.788+0000 7f57dd830700 1 -- 172.21.15.100:0/3886863289 wait complete. 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.788+0000 7f57dd830700 1 Processor -- start 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.788+0000 7f57dd830700 1 -- start start 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.788+0000 7f57dd830700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:58.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.788+0000 7f57dd830700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d819b980 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51336/0 (socket says 172.21.15.100:51336) 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57d6ffd700 1 -- 172.21.15.100:0/3272022693 learned_addr learned my addr 172.21.15.100:0/3272022693 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57d6ffd700 1 -- 172.21.15.100:0/3272022693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c8009550 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57d6ffd700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d819b440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57c8011740 tx=0x7f57c8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c8003b20 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d819bb80 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c801a910 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c8024900 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.789+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d819c020 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.790+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f57c8024b70 con 0x7f57d8108260 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.790+0000 7f57dc82e700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 0x7f57c003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:02:58.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.790+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f57c804cfa0 con 0x7f57d8108260 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.790+0000 7f57d67fc700 1 -- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 msgr2=0x7f57c003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.790+0000 7f57d67fc700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 0x7f57c003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.790+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d810b960 con 0x7f57d8108260 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.795+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f57c802d3f0 con 0x7f57d8108260 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.991+0000 7f57d67fc700 1 -- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 msgr2=0x7f57c003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:57.991+0000 7f57d67fc700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 0x7f57c003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.046+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f57d819eb10 con 0x7f57d8108260 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.046+0000 7f57dc82e700 1 -- 172.21.15.100:0/3272022693 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f57c8022310 con 0x7f57d8108260 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.049+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 msgr2=0x7f57c003b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:02:58.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.049+0000 7f57dd830700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 0x7f57c003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.049+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 msgr2=0x7f57d819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.049+0000 7f57dd830700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d819b440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57c8011740 tx=0x7f57c8011a50 comp rx=0 tx=0).stop 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.050+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 shutdown_connections 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.050+0000 7f57dd830700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7f57c0038cb0 0x7f57c003b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.050+0000 7f57dd830700 1 --2- 172.21.15.100:0/3272022693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d8108260 0x7f57d819b440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.050+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 >> 172.21.15.100:0/3272022693 conn(0x7f57d8101c70 msgr2=0x7f57d81026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.050+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 shutdown_connections 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:58.050+0000 7f57dd830700 1 -- 172.21.15.100:0/3272022693 wait complete. 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-03-12T05:02:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 9... 2024-03-12T05:02:58.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3272022693' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-12T05:02:58.859 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.779+0000 7fbca054c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:02:59.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.891+0000 7fbca054c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:02:59.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:58.978+0000 7fbca054c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:02:59.572 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:59.080+0000 7fbca054c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:02:59.572 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:02:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:02:59.256+0000 7fbca054c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:03:00.470 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:00.210+0000 7fbca054c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:03:00.470 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:00.340+0000 7fbca054c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:03:00.763 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:00.671+0000 7fbca054c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:03:00.763 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:00.761+0000 7fbca054c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:03:01.072 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:00.957+0000 7fbca054c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:03:01.430 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:01.142+0000 7fbca054c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:03:01.688 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:01.428+0000 7fbca054c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:03:01.688 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:01.686+0000 7fbca054c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:03:02.071 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:01.924+0000 7fbca054c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:03:03.072 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:02.723+0000 7fbca054c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:03:03.572 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:03.300+0000 7fbca054c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:03:03.572 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:03.399+0000 7fbca054c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:03:04.072 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:03:03.619+0000 7fbca054c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: Active manager daemon x restarted 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: Activating manager daemon x 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: osdmap e3: 0 total, 0 up, 0 in 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: mgrmap e10: x(active, starting, since 0.00314543s) 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: Manager daemon x is now available 2024-03-12T05:03:04.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:04.074 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:04.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T05:03:04.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T05:03:04.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:04 smithi100 ceph-mon[32210]: mgrmap e11: x(active, since 1.00619s) 2024-03-12T05:03:05.245 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-03-12T05:03:05.245 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-12T05:03:05.245 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-12T05:03:05.245 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-03-12T05:03:05.245 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.060+0000 7fdd4ca6f700 1 Processor -- start 2024-03-12T05:03:05.245 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4ca6f700 1 -- start start 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4ca6f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4ca6f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48075810 con 0x7fdd481081b0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51352/0 (socket says 172.21.15.100:51352) 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4659c700 1 -- 172.21.15.100:0/3099737765 learned_addr learned my addr 172.21.15.100:0/3099737765 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4659c700 1 -- 172.21.15.100:0/3099737765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4810aae0 con 0x7fdd481081b0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.061+0000 7fdd4659c700 1 --2- 172.21.15.100:0/3099737765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4810a5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098a0 tx=0x7fdd38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3e0f82abe145f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4559a700 1 -- 172.21.15.100:0/3099737765 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd38004160 con 0x7fdd481081b0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4559a700 1 -- 172.21.15.100:0/3099737765 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd380042c0 con 0x7fdd481081b0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4559a700 1 -- 172.21.15.100:0/3099737765 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd38011a80 con 0x7fdd481081b0 2024-03-12T05:03:05.246 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/3099737765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 msgr2=0x7fdd4810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/3099737765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4810a5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098a0 tx=0x7fdd38009bb0 comp rx=0 tx=0).stop 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/3099737765 shutdown_connections 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/3099737765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4810a5a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/3099737765 >> 172.21.15.100:0/3099737765 conn(0x7fdd48101c80 msgr2=0x7fdd481040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/3099737765 shutdown_connections 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.062+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/3099737765 wait complete. 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4ca6f700 1 Processor -- start 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4ca6f700 1 -- start start 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4ca6f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4ca6f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4819b9a0 con 0x7fdd481081b0 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51366/0 (socket says 172.21.15.100:51366) 2024-03-12T05:03:05.247 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.063+0000 7fdd4659c700 1 -- 172.21.15.100:0/663266133 learned_addr learned my addr 172.21.15.100:0/663266133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd4659c700 1 -- 172.21.15.100:0/663266133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38009550 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd4659c700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4819b460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd38011740 tx=0x7fdd38011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd38003b20 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd3801a910 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4819bba0 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd380248c0 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.064+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4819c040 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.065+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fdd3802d3f0 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.065+0000 7fdd377fe700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.065+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 --> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd3003b930 con 0x7fdd30038d60 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.065+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd3801a430 con 0x7fdd481081b0 2024-03-12T05:03:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.065+0000 7fdd45d9b700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 msgr2=0x7fdd3003b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.065+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.266+0000 7fdd45d9b700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 msgr2=0x7fdd3003b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.266+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.667+0000 7fdd45d9b700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 msgr2=0x7fdd3003b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:02:59.667+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:00.468+0000 7fdd45d9b700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 msgr2=0x7fdd3003b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:00.468+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:02.069+0000 7fdd45d9b700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 msgr2=0x7fdd3003b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/16620000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:02.069+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:03.634+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fdd3802d920 con 0x7fdd481081b0 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:03.634+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 msgr2=0x7fdd3003b220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:03.634+0000 7fdd377fe700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.635+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7fdd3804e660 con 0x7fdd481081b0 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.636+0000 7fdd377fe700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd3003c810 0x7fdd3003ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.636+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd3003b930 con 0x7fdd3003c810 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.636+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd3003c810 0x7fdd3003ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.636+0000 7fdd45d9b700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd3003c810 0x7fdd3003ec00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c003a10 tx=0x7fdd3c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.637+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdd3003b930 con 0x7fdd3003c810 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.644+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdd4810b980 con 0x7fdd3003c810 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.644+0000 7fdd377fe700 1 -- 172.21.15.100:0/663266133 <== mgr.14152 v2:172.21.15.100:6800/327651532 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fdd4810b980 con 0x7fdd3003c810 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd3003c810 msgr2=0x7fdd3003ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd3003c810 0x7fdd3003ec00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c003a10 tx=0x7fdd3c009270 comp rx=0 tx=0).stop 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 msgr2=0x7fdd4819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4819b460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd38011740 tx=0x7fdd38011b00 comp rx=0 tx=0).stop 2024-03-12T05:03:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 shutdown_connections 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd3003c810 0x7fdd3003ec00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481081b0 0x7fdd4819b460 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 --2- 172.21.15.100:0/663266133 >> [v2:172.21.15.100:6800/16620000,v1:172.21.15.100:6801/16620000] conn(0x7fdd30038d60 0x7fdd3003b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 >> 172.21.15.100:0/663266133 conn(0x7fdd48101c80 msgr2=0x7fdd481026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 shutdown_connections 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:04.645+0000 7fdd4ca6f700 1 -- 172.21.15.100:0/663266133 wait complete. 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 9 is available 2024-03-12T05:03:05.251 INFO:teuthology.orchestra.run.smithi100.stdout:Generating a dashboard self-signed certificate... 2024-03-12T05:03:05.910 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:05 smithi100 ceph-mon[32210]: [12/Mar/2024:05:03:03] ENGINE Bus STARTING 2024-03-12T05:03:05.910 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:05 smithi100 ceph-mon[32210]: [12/Mar/2024:05:03:04] ENGINE Serving on https://172.21.15.100:7150 2024-03-12T05:03:05.911 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:05 smithi100 ceph-mon[32210]: [12/Mar/2024:05:03:04] ENGINE Serving on http://172.21.15.100:8765 2024-03-12T05:03:05.911 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:05 smithi100 ceph-mon[32210]: [12/Mar/2024:05:03:04] ENGINE Bus STARTED 2024-03-12T05:03:05.911 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:05 smithi100 ceph-mon[32210]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-12T05:03:05.911 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:05 smithi100 ceph-mon[32210]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-12T05:03:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:06 smithi100 ceph-mon[32210]: mgrmap e12: x(active, since 2s) 2024-03-12T05:03:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:06.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-12T05:03:06.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfe13a1700 1 Processor -- start 2024-03-12T05:03:06.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfe13a1700 1 -- start start 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfe13a1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc10a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfe13a1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc07ae70 con 0x7fbfdc1081f0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc10a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc10a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33916/0 (socket says 172.21.15.100:33916) 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.907+0000 7fbfdaffd700 1 -- 172.21.15.100:0/1431646240 learned_addr learned my addr 172.21.15.100:0/1431646240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.908+0000 7fbfdaffd700 1 -- 172.21.15.100:0/1431646240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc10ab20 con 0x7fbfdc1081f0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.908+0000 7fbfdaffd700 1 --2- 172.21.15.100:0/1431646240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc10a5e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00098a0 tx=0x7fbfd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca069aa7c73890c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.908+0000 7fbfd9ffb700 1 -- 172.21.15.100:0/1431646240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd0004160 con 0x7fbfdc1081f0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.908+0000 7fbfd9ffb700 1 -- 172.21.15.100:0/1431646240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd00042c0 con 0x7fbfdc1081f0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.908+0000 7fbfd9ffb700 1 -- 172.21.15.100:0/1431646240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd0011a80 con 0x7fbfdc1081f0 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1431646240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 msgr2=0x7fbfdc10a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 --2- 172.21.15.100:0/1431646240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc10a5e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00098a0 tx=0x7fbfd0009bb0 comp rx=0 tx=0).stop 2024-03-12T05:03:06.964 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1431646240 shutdown_connections 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 --2- 172.21.15.100:0/1431646240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc10a5e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1431646240 >> 172.21.15.100:0/1431646240 conn(0x7fbfdc101ca0 msgr2=0x7fbfdc1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1431646240 shutdown_connections 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.909+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1431646240 wait complete. 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfe13a1700 1 Processor -- start 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfe13a1700 1 -- start start 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfe13a1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfe13a1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc19db80 con 0x7fbfdc1081f0 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33932/0 (socket says 172.21.15.100:33932) 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfdaffd700 1 -- 172.21.15.100:0/1370132550 learned_addr learned my addr 172.21.15.100:0/1370132550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.910+0000 7fbfdaffd700 1 -- 172.21.15.100:0/1370132550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0009550 con 0x7fbfdc1081f0 2024-03-12T05:03:06.965 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.911+0000 7fbfdaffd700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc19d640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0011740 tx=0x7fbfd0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.911+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd0003b20 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.911+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc19dd80 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.911+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc19e220 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.911+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd001a910 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.911+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd00248c0 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.912+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc10b960 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.912+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fbfd002d3f0 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.912+0000 7fbfc3fff700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbfc4038d60 0x7fbfc403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.912+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbfd004d130 con 0x7fbfdc1081f0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.912+0000 7fbfda7fc700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbfc4038d60 0x7fbfc403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.913+0000 7fbfda7fc700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbfc4038d60 0x7fbfc403b220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:06.966 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:05.922+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbfd0010970 con 0x7fbfdc1081f0 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.126+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfdc102d80 con 0x7fbfc4038d60 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.371+0000 7fbfc3fff700 1 -- 172.21.15.100:0/1370132550 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbfdc102d80 con 0x7fbfc4038d60 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbfc4038d60 msgr2=0x7fbfc403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbfc4038d60 0x7fbfc403b220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 msgr2=0x7fbfdc19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc19d640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0011740 tx=0x7fbfd0011b00 comp rx=0 tx=0).stop 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 shutdown_connections 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbfc4038d60 0x7fbfc403b220 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 --2- 172.21.15.100:0/1370132550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc1081f0 0x7fbfdc19d640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.373+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 >> 172.21.15.100:0/1370132550 conn(0x7fbfdc101ca0 msgr2=0x7fbfdc102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.374+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 shutdown_connections 2024-03-12T05:03:06.967 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:06.374+0000 7fbfe13a1700 1 -- 172.21.15.100:0/1370132550 wait complete. 2024-03-12T05:03:06.968 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial admin user... 2024-03-12T05:03:07.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:07 smithi100 ceph-mon[32210]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:07.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:07 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vWyhcKgKZfX73UYUew.EXeu9RC52u89Qe2reZcrpZmZH89HKvz4o6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710219787, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.419+0000 7fac2ccac700 1 Processor -- start 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.419+0000 7fac2ccac700 1 -- start start 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.419+0000 7fac2ccac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.419+0000 7fac2ccac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac28075810 con 0x7fac281081b0 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33944/0 (socket says 172.21.15.100:33944) 2024-03-12T05:03:08.617 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2659c700 1 -- 172.21.15.100:0/1841945370 learned_addr learned my addr 172.21.15.100:0/1841945370 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2659c700 1 -- 172.21.15.100:0/1841945370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2810aae0 con 0x7fac281081b0 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2659c700 1 --2- 172.21.15.100:0/1841945370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2810a5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0098a0 tx=0x7fac1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b262174e424aac5c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2559a700 1 -- 172.21.15.100:0/1841945370 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac1c004160 con 0x7fac281081b0 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2559a700 1 -- 172.21.15.100:0/1841945370 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1c0042c0 con 0x7fac281081b0 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.420+0000 7fac2559a700 1 -- 172.21.15.100:0/1841945370 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac1c011a80 con 0x7fac281081b0 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 -- 172.21.15.100:0/1841945370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 msgr2=0x7fac2810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 --2- 172.21.15.100:0/1841945370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2810a5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0098a0 tx=0x7fac1c009bb0 comp rx=0 tx=0).stop 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 -- 172.21.15.100:0/1841945370 shutdown_connections 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 --2- 172.21.15.100:0/1841945370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2810a5a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 -- 172.21.15.100:0/1841945370 >> 172.21.15.100:0/1841945370 conn(0x7fac28101c80 msgr2=0x7fac281040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 -- 172.21.15.100:0/1841945370 shutdown_connections 2024-03-12T05:03:08.618 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.421+0000 7fac2ccac700 1 -- 172.21.15.100:0/1841945370 wait complete. 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2ccac700 1 Processor -- start 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2ccac700 1 -- start start 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2ccac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2ccac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2819b9c0 con 0x7fac281081b0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33952/0 (socket says 172.21.15.100:33952) 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.422+0000 7fac2659c700 1 -- 172.21.15.100:0/2022136117 learned_addr learned my addr 172.21.15.100:0/2022136117 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac2659c700 1 -- 172.21.15.100:0/2022136117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009550 con 0x7fac281081b0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac2659c700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2819b480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac1c011740 tx=0x7fac1c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac1c003b20 con 0x7fac281081b0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1c01a910 con 0x7fac281081b0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac1c0248c0 con 0x7fac281081b0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2819bbc0 con 0x7fac281081b0 2024-03-12T05:03:08.619 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.423+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2819c060 con 0x7fac281081b0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.424+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fac1c02d3f0 con 0x7fac281081b0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.424+0000 7fac177fe700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fac10038d00 0x7fac1003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.424+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac1c04d230 con 0x7fac281081b0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.424+0000 7fac25d9b700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fac10038d00 0x7fac1003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.425+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2810b980 con 0x7fac281081b0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.425+0000 7fac25d9b700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fac10038d00 0x7fac1003b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac18006fd0 tx=0x7fac18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.430+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac1c02d740 con 0x7fac281081b0 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.690+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fac2810bb60 con 0x7fac10038d00 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.914+0000 7fac177fe700 1 -- 172.21.15.100:0/2022136117 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fac2810bb60 con 0x7fac10038d00 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fac10038d00 msgr2=0x7fac1003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fac10038d00 0x7fac1003b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac18006fd0 tx=0x7fac18006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 msgr2=0x7fac2819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2819b480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac1c011740 tx=0x7fac1c011b00 comp rx=0 tx=0).stop 2024-03-12T05:03:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 shutdown_connections 2024-03-12T05:03:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fac10038d00 0x7fac1003b1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 --2- 172.21.15.100:0/2022136117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281081b0 0x7fac2819b480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 >> 172.21.15.100:0/2022136117 conn(0x7fac28101c80 msgr2=0x7fac28102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 shutdown_connections 2024-03-12T05:03:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:07.916+0000 7fac2ccac700 1 -- 172.21.15.100:0/2022136117 wait complete. 2024-03-12T05:03:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:Fetching dashboard port number... 2024-03-12T05:03:08.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:08 smithi100 ceph-mon[32210]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:08.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:10.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:09 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1365226538' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-12T05:03:10.066 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 8443 2024-03-12T05:03:10.066 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.041+0000 7f18cbd21700 1 Processor -- start 2024-03-12T05:03:10.066 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18cbd21700 1 -- start start 2024-03-12T05:03:10.066 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18cbd21700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c4107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18cbd21700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4108150 con 0x7f18c4107760 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c9abd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c4107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c9abd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c4107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33966/0 (socket says 172.21.15.100:33966) 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c9abd700 1 -- 172.21.15.100:0/660258285 learned_addr learned my addr 172.21.15.100:0/660258285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c9abd700 1 -- 172.21.15.100:0/660258285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4108910 con 0x7f18c4107760 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c9abd700 1 --2- 172.21.15.100:0/660258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c4107b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0098a0 tx=0x7f18bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd6576e82b592ff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c8abb700 1 -- 172.21.15.100:0/660258285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18bc004160 con 0x7f18c4107760 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c8abb700 1 -- 172.21.15.100:0/660258285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18bc0042c0 con 0x7f18c4107760 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.042+0000 7f18c8abb700 1 -- 172.21.15.100:0/660258285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18bc011a80 con 0x7f18c4107760 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 -- 172.21.15.100:0/660258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 msgr2=0x7f18c4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 --2- 172.21.15.100:0/660258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c4107b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0098a0 tx=0x7f18bc009bb0 comp rx=0 tx=0).stop 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 -- 172.21.15.100:0/660258285 shutdown_connections 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 --2- 172.21.15.100:0/660258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c4107b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:10.067 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 -- 172.21.15.100:0/660258285 >> 172.21.15.100:0/660258285 conn(0x7f18c4102da0 msgr2=0x7f18c41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 -- 172.21.15.100:0/660258285 shutdown_connections 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.043+0000 7f18cbd21700 1 -- 172.21.15.100:0/660258285 wait complete. 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18cbd21700 1 Processor -- start 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18cbd21700 1 -- start start 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18cbd21700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18cbd21700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419ba30 con 0x7f18c4107760 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18c9abd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18c9abd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33982/0 (socket says 172.21.15.100:33982) 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18c9abd700 1 -- 172.21.15.100:0/1365226538 learned_addr learned my addr 172.21.15.100:0/1365226538 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18c9abd700 1 -- 172.21.15.100:0/1365226538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc009550 con 0x7f18c4107760 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18c9abd700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c419b4f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18bc011740 tx=0x7f18bc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18bc0039e0 con 0x7f18c4107760 2024-03-12T05:03:10.068 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c419bc30 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18bc003b40 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18bc024710 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.044+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c419c0d0 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.045+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f18bc0249d0 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.045+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c410bb40 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.045+0000 7f18baffd700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f18b0038cb0 0x7f18b003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.045+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f18bc04cfe0 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.046+0000 7f18c92bc700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f18b0038cb0 0x7f18b003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.046+0000 7f18c92bc700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f18b0038cb0 0x7f18b003b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18b4006fd0 tx=0x7f18b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.050+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18bc01a910 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.245+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f18c419e9b0 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.246+0000 7f18baffd700 1 -- 172.21.15.100:0/1365226538 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18bc04c870 con 0x7f18c4107760 2024-03-12T05:03:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f18b0038cb0 msgr2=0x7f18b003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f18b0038cb0 0x7f18b003b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18b4006fd0 tx=0x7f18b4006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 msgr2=0x7f18c419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c419b4f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18bc011740 tx=0x7f18bc011b00 comp rx=0 tx=0).stop 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 shutdown_connections 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f18b0038cb0 0x7f18b003b170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 --2- 172.21.15.100:0/1365226538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18c4107760 0x7f18c419b4f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 >> 172.21.15.100:0/1365226538 conn(0x7f18c4102da0 msgr2=0x7f18c4103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 shutdown_connections 2024-03-12T05:03:10.070 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:09.248+0000 7f18cbd21700 1 -- 172.21.15.100:0/1365226538 wait complete. 2024-03-12T05:03:10.075 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-12T05:03:10.076 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-03-12T05:03:10.084 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-12T05:03:10.084 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-03-12T05:03:10.084 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-03-12T05:03:10.084 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph Dashboard is now available at: 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout: URL: https://smithi100.front.sepia.ceph.com:8443/ 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout: User: admin 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout: Password: 9o0c9xwjyn 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:10.086 INFO:teuthology.orchestra.run.smithi100.stdout:Saving cluster configuration to /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config directory 2024-03-12T05:03:10.087 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling autotune for osd_memory_target 2024-03-12T05:03:11.022 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:10 smithi100 ceph-mon[32210]: mgrmap e13: x(active, since 6s) 2024-03-12T05:03:11.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.517+0000 7feeb0434700 1 Processor -- start 2024-03-12T05:03:11.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.517+0000 7feeb0434700 1 -- start start 2024-03-12T05:03:11.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeb0434700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8105e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:11.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeb0434700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea8106440 con 0x7feea8105a50 2024-03-12T05:03:11.276 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeae1d0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:11.276 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeae1d0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47622/0 (socket says 172.21.15.100:47622) 2024-03-12T05:03:11.276 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeae1d0700 1 -- 172.21.15.100:0/3525325460 learned_addr learned my addr 172.21.15.100:0/3525325460 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:11.276 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeae1d0700 1 -- 172.21.15.100:0/3525325460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea81089e0 con 0x7feea8105a50 2024-03-12T05:03:11.276 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feeae1d0700 1 --2- 172.21.15.100:0/3525325460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8105e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee980098a0 tx=0x7fee98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589b6dd9a625b081 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:11.276 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.518+0000 7feead1ce700 1 -- 172.21.15.100:0/3525325460 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee98004160 con 0x7feea8105a50 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feead1ce700 1 -- 172.21.15.100:0/3525325460 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee980042c0 con 0x7feea8105a50 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feead1ce700 1 -- 172.21.15.100:0/3525325460 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee98011a80 con 0x7feea8105a50 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 -- 172.21.15.100:0/3525325460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 msgr2=0x7feea8105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 --2- 172.21.15.100:0/3525325460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8105e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee980098a0 tx=0x7fee98009bb0 comp rx=0 tx=0).stop 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 -- 172.21.15.100:0/3525325460 shutdown_connections 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 --2- 172.21.15.100:0/3525325460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8105e70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 -- 172.21.15.100:0/3525325460 >> 172.21.15.100:0/3525325460 conn(0x7feea8101420 msgr2=0x7feea8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 -- 172.21.15.100:0/3525325460 shutdown_connections 2024-03-12T05:03:11.277 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.519+0000 7feeb0434700 1 -- 172.21.15.100:0/3525325460 wait complete. 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeb0434700 1 Processor -- start 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeb0434700 1 -- start start 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeb0434700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeb0434700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea81087a0 con 0x7feea8105a50 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeae1d0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeae1d0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8108260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47636/0 (socket says 172.21.15.100:47636) 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeae1d0700 1 -- 172.21.15.100:0/2197374923 learned_addr learned my addr 172.21.15.100:0/2197374923 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:11.278 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.520+0000 7feeae1d0700 1 -- 172.21.15.100:0/2197374923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98009550 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.521+0000 7feeae1d0700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8108260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee98011740 tx=0x7fee98011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.521+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee98003b20 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.521+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee9801a910 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.521+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea8106990 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.521+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee98024850 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.521+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea8106e30 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.522+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fee9802d3f0 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.522+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea810bc10 con 0x7feea8105a50 2024-03-12T05:03:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.522+0000 7fee9f7fe700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee94038da0 0x7fee9403b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.522+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee9804d200 con 0x7feea8105a50 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.522+0000 7feead9cf700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee94038da0 0x7fee9403b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.522+0000 7feead9cf700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee94038da0 0x7fee9403b260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feea4006fd0 tx=0x7feea4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.527+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee9801a430 con 0x7feea8105a50 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.724+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7feea810be20 con 0x7feea8105a50 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.725+0000 7fee9f7fe700 1 -- 172.21.15.100:0/2197374923 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fee98018350 con 0x7feea8105a50 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee94038da0 msgr2=0x7fee9403b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:11.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee94038da0 0x7fee9403b260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feea4006fd0 tx=0x7feea4006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 msgr2=0x7feea8108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8108260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee98011740 tx=0x7fee98011b00 comp rx=0 tx=0).stop 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 shutdown_connections 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee94038da0 0x7fee9403b260 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 --2- 172.21.15.100:0/2197374923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea8105a50 0x7feea8108260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 >> 172.21.15.100:0/2197374923 conn(0x7feea8101420 msgr2=0x7feea8101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 shutdown_connections 2024-03-12T05:03:11.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:10.727+0000 7feeb0434700 1 -- 172.21.15.100:0/2197374923 wait complete. 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.697+0000 7f2ce576d700 1 Processor -- start 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.697+0000 7f2ce576d700 1 -- start start 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2ce576d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce0106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2ce576d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0107030 con 0x7f2ce0106640 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cdeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce0106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cdeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce0106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47644/0 (socket says 172.21.15.100:47644) 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cdeffd700 1 -- 172.21.15.100:0/1269669646 learned_addr learned my addr 172.21.15.100:0/1269669646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cdeffd700 1 -- 172.21.15.100:0/1269669646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0107840 con 0x7f2ce0106640 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cdeffd700 1 --2- 172.21.15.100:0/1269669646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce0106a60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40098a0 tx=0x7f2cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9920519fcd526c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:12.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cddffb700 1 -- 172.21.15.100:0/1269669646 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4004160 con 0x7f2ce0106640 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cddffb700 1 -- 172.21.15.100:0/1269669646 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cd40042c0 con 0x7f2ce0106640 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.698+0000 7f2cddffb700 1 -- 172.21.15.100:0/1269669646 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4011a80 con 0x7f2ce0106640 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 -- 172.21.15.100:0/1269669646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 msgr2=0x7f2ce0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 --2- 172.21.15.100:0/1269669646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce0106a60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40098a0 tx=0x7f2cd4009bb0 comp rx=0 tx=0).stop 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 -- 172.21.15.100:0/1269669646 shutdown_connections 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 --2- 172.21.15.100:0/1269669646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce0106a60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 -- 172.21.15.100:0/1269669646 >> 172.21.15.100:0/1269669646 conn(0x7f2ce0101c80 msgr2=0x7f2ce01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 -- 172.21.15.100:0/1269669646 shutdown_connections 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.699+0000 7f2ce576d700 1 -- 172.21.15.100:0/1269669646 wait complete. 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2ce576d700 1 Processor -- start 2024-03-12T05:03:12.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2ce576d700 1 -- start start 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2ce576d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce01917b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2ce576d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0191cf0 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2cdeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce01917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2cdeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce01917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47654/0 (socket says 172.21.15.100:47654) 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2cdeffd700 1 -- 172.21.15.100:0/3560198312 learned_addr learned my addr 172.21.15.100:0/3560198312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.700+0000 7f2cdeffd700 1 -- 172.21.15.100:0/3560198312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4009550 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.701+0000 7f2cdeffd700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce01917b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4011740 tx=0x7f2cd4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.701+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd40039e0 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.701+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cd4003b40 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.701+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce018fee0 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.701+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4024740 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.701+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce0190380 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.702+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f2cd402d3f0 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.702+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00632a0 con 0x7f2ce0106640 2024-03-12T05:03:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.702+0000 7f2cc7fff700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2cc8038da0 0x7f2cc803b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.702+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2cd404d200 con 0x7f2ce0106640 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.702+0000 7f2cde7fc700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2cc8038da0 0x7f2cc803b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.702+0000 7f2cde7fc700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2cc8038da0 0x7f2cc803b260 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:11.707+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cd4018ba0 con 0x7f2ce0106640 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.003+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2ce010aa70 con 0x7f2ce0106640 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.005+0000 7f2cc7fff700 1 -- 172.21.15.100:0/3560198312 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f2cd404ca90 con 0x7f2ce0106640 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2cc8038da0 msgr2=0x7f2cc803b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2cc8038da0 0x7f2cc803b260 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 msgr2=0x7f2ce01917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce01917b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4011740 tx=0x7f2cd4011b00 comp rx=0 tx=0).stop 2024-03-12T05:03:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 shutdown_connections 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2cc8038da0 0x7f2cc803b260 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 --2- 172.21.15.100:0/3560198312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0106640 0x7f2ce01917b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 >> 172.21.15.100:0/3560198312 conn(0x7f2ce0101c80 msgr2=0x7f2ce0102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.008+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 shutdown_connections 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-03-12T05:03:12.009+0000 7f2ce576d700 1 -- 172.21.15.100:0/3560198312 wait complete. 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:Or, if you are only running a single cluster on this host: 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.567 INFO:teuthology.orchestra.run.smithi100.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout: ceph telemetry on 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout:For more information see: 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:12.568 INFO:teuthology.orchestra.run.smithi100.stdout:Bootstrap complete. 2024-03-12T05:03:12.601 INFO:tasks.cephadm:Fetching config... 2024-03-12T05:03:12.602 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:03:12.602 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-12T05:03:12.618 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-12T05:03:12.618 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:03:12.618 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-12T05:03:12.672 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-12T05:03:12.672 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:03:12.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-12T05:03:12.739 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-12T05:03:12.739 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:03:12.740 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-12T05:03:12.794 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-12T05:03:12.794 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T05:03:12.886 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:03:12.902 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T05:03:12.950 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:03:12.965 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T05:03:13.013 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY6Db6ogQ6HXF+Y8ifXaFgQS+0rzmv9FI3FjsOlcvLkmMk6tGN3Bb8woSkEE4Hfn9O3wNM1OLAJ2vwpgdX4KL76VdRgeEs2dt35FgN/xgNCbyACoQSMxROaX94V24KHJmrlgga1qhcpAFzxZR5z1FBnVbNRNhF0yOVqYXqnYzceSlJh5/dZkFOP22zQ786tbOPcBMtBBb09d7ursasZo4rmonlSbQJdlTJqr+OCmOwXcoQbCwa8GhtqdSXdKnUdEPQDe7wg/ntsiVDtmByW5tVt5ZzlPoSHKXdwkgjsyneSHnQi19/Y0ayEroorZOq07h8HvcOlTo4Sfvk+vBJFiFsoUok6t0uJLQfywTRWeKNoByum8hI8ABNyM/PHEoRW82WzTj0JtNXWUunhOyue5nNBYWJLm3wCOY30MR0b+lFdKYItcurhm5RDAXyUgfv0PIK/EpeSG8WIWl5DAz4nyRpVh5wZj/+lzV4c4bfuMmG1R2Qn4x3WXO/Mtw4v65iAvM= ceph-8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:03:13.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-12T05:03:13.272 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:13.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:13 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3560198312' entity='client.admin' 2024-03-12T05:03:14.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/887903522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 msgr2=0x7f1cf81009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:14.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 --2- 172.21.15.100:0/887903522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf81009b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009910 tx=0x7f1cec009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/887903522 shutdown_connections 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 --2- 172.21.15.100:0/887903522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf81009b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/887903522 >> 172.21.15.100:0/887903522 conn(0x7f1cf80fbb70 msgr2=0x7f1cf80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/887903522 shutdown_connections 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.950+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/887903522 wait complete. 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cfcfc5700 1 Processor -- start 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cfcfc5700 1 -- start start 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cfcfc5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf806a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cfcfc5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf806a8d0 con 0x7f1cf8100590 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf806a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47662/0 (socket says 172.21.15.100:47662) 2024-03-12T05:03:14.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cf659c700 1 -- 172.21.15.100:0/564490892 learned_addr learned my addr 172.21.15.100:0/564490892 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cf659c700 1 -- 172.21.15.100:0/564490892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0095f0 con 0x7f1cf8100590 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.951+0000 7f1cf659c700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf806a390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cec005850 tx=0x7f1cec004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.952+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cec01d070 con 0x7f1cf8100590 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.952+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf81998d0 con 0x7f1cf8100590 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.952+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cec00be40 con 0x7f1cf8100590 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.952+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cec022ea0 con 0x7f1cf8100590 2024-03-12T05:03:14.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.952+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf8199d70 con 0x7f1cf8100590 2024-03-12T05:03:14.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.952+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f1cec00f500 con 0x7f1cf8100590 2024-03-12T05:03:14.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.953+0000 7f1ce77fe700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ce0038d50 0x7f1ce003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:14.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.953+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1cec04de80 con 0x7f1cf8100590 2024-03-12T05:03:14.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.953+0000 7f1cf5d9b700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ce0038d50 0x7f1ce003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:14.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.953+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf818f3f0 con 0x7f1cf8100590 2024-03-12T05:03:14.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.953+0000 7f1cf5d9b700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ce0038d50 0x7f1ce003b210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006fd0 tx=0x7f1ce8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:14.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:14.958+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1cec027020 con 0x7f1cf8100590 2024-03-12T05:03:15.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.162+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f1cf80632a0 con 0x7f1cf8100590 2024-03-12T05:03:15.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.164+0000 7f1ce77fe700 1 -- 172.21.15.100:0/564490892 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1cec017440 con 0x7f1cf8100590 2024-03-12T05:03:15.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ce0038d50 msgr2=0x7f1ce003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:15.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ce0038d50 0x7f1ce003b210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006fd0 tx=0x7f1ce8006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:15.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 msgr2=0x7f1cf806a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:15.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf806a390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cec005850 tx=0x7f1cec004ff0 comp rx=0 tx=0).stop 2024-03-12T05:03:15.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 shutdown_connections 2024-03-12T05:03:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ce0038d50 0x7f1ce003b210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 --2- 172.21.15.100:0/564490892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf8100590 0x7f1cf806a390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 >> 172.21.15.100:0/564490892 conn(0x7f1cf80fbb70 msgr2=0x7f1cf80fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 shutdown_connections 2024-03-12T05:03:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:15.167+0000 7f1cfcfc5700 1 -- 172.21.15.100:0/564490892 wait complete. 2024-03-12T05:03:15.962 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-12T05:03:15.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-12T05:03:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:16.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/564490892' entity='client.admin' 2024-03-12T05:03:16.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:16.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:16.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:03:16.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:16.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:16.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:17.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 -- 172.21.15.100:0/1727125455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 msgr2=0x7f7d0c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:17.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 --2- 172.21.15.100:0/1727125455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c1003b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009910 tx=0x7f7cfc009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:17.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 -- 172.21.15.100:0/1727125455 shutdown_connections 2024-03-12T05:03:17.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 --2- 172.21.15.100:0/1727125455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c1003b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:17.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 -- 172.21.15.100:0/1727125455 >> 172.21.15.100:0/1727125455 conn(0x7f7d0c0fbb30 msgr2=0x7f7d0c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:17.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 -- 172.21.15.100:0/1727125455 shutdown_connections 2024-03-12T05:03:17.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.693+0000 7f7d10f9c700 1 -- 172.21.15.100:0/1727125455 wait complete. 2024-03-12T05:03:17.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d10f9c700 1 Processor -- start 2024-03-12T05:03:17.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d10f9c700 1 -- start start 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d10f9c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d10f9c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c199390 con 0x7f7d0c0fff90 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d0a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d0a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47666/0 (socket says 172.21.15.100:47666) 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d0a59c700 1 -- 172.21.15.100:0/274840033 learned_addr learned my addr 172.21.15.100:0/274840033 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.694+0000 7f7d0a59c700 1 -- 172.21.15.100:0/274840033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0095f0 con 0x7f7d0c0fff90 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7d0a59c700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c198e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc004f40 tx=0x7f7cfc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cfc01d070 con 0x7f7d0c0fff90 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc00be40 con 0x7f7d0c0fff90 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c199590 con 0x7f7d0c0fff90 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cfc00f460 con 0x7f7d0c0fff90 2024-03-12T05:03:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c199a30 con 0x7f7d0c0fff90 2024-03-12T05:03:17.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.695+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f7cfc00f650 con 0x7f7d0c0fff90 2024-03-12T05:03:17.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.696+0000 7f7cfb7fe700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7cf4038db0 0x7f7cf403b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:17.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.696+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7cfc04cf30 con 0x7f7d0c0fff90 2024-03-12T05:03:17.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.696+0000 7f7d09d9b700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7cf4038db0 0x7f7cf403b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:17.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.696+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c1926e0 con 0x7f7d0c0fff90 2024-03-12T05:03:17.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.696+0000 7f7d09d9b700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7cf4038db0 0x7f7cf403b270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.701+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cfc027020 con 0x7f7d0c0fff90 2024-03-12T05:03:17.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.897+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d0c1099d0 con 0x7f7cf4038db0 2024-03-12T05:03:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.900+0000 7f7cfb7fe700 1 -- 172.21.15.100:0/274840033 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7d0c1099d0 con 0x7f7cf4038db0 2024-03-12T05:03:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7cf4038db0 msgr2=0x7f7cf403b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7cf4038db0 0x7f7cf403b270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 msgr2=0x7f7d0c198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c198e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc004f40 tx=0x7f7cfc005020 comp rx=0 tx=0).stop 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 shutdown_connections 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7cf4038db0 0x7f7cf403b270 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 --2- 172.21.15.100:0/274840033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d0c0fff90 0x7f7d0c198e50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 >> 172.21.15.100:0/274840033 conn(0x7f7d0c0fbb30 msgr2=0x7f7d0c0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 shutdown_connections 2024-03-12T05:03:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:17.902+0000 7f7d10f9c700 1 -- 172.21.15.100:0/274840033 wait complete. 2024-03-12T05:03:18.633 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2024-03-12T05:03:18.633 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:03:18.633 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2024-03-12T05:03:18.651 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:03:18.652 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:18.709 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2024-03-12T05:03:18.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch host add smithi156 2024-03-12T05:03:18.950 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:19.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:03:19.169 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:19.169 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:19.169 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:20.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 -- 172.21.15.100:0/335971118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 msgr2=0x7f30bc102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:20.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 --2- 172.21.15.100:0/335971118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc102bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009910 tx=0x7f30ac009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:20.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 -- 172.21.15.100:0/335971118 shutdown_connections 2024-03-12T05:03:20.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 --2- 172.21.15.100:0/335971118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc102bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 -- 172.21.15.100:0/335971118 >> 172.21.15.100:0/335971118 conn(0x7f30bc0fc040 msgr2=0x7f30bc0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 -- 172.21.15.100:0/335971118 shutdown_connections 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 -- 172.21.15.100:0/335971118 wait complete. 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.370+0000 7f30c226a700 1 Processor -- start 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30c226a700 1 -- start start 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30c226a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30c226a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc197550 con 0x7f30bc102810 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30bbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30bbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47676/0 (socket says 172.21.15.100:47676) 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30bbfff700 1 -- 172.21.15.100:0/2858265517 learned_addr learned my addr 172.21.15.100:0/2858265517 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30bbfff700 1 -- 172.21.15.100:0/2858265517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0095f0 con 0x7f30bc102810 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30bbfff700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc197010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30ac005850 tx=0x7f30ac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:20.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.371+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30ac01d070 con 0x7f30bc102810 2024-03-12T05:03:20.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.372+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc197750 con 0x7f30bc102810 2024-03-12T05:03:20.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.372+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30ac00bc30 con 0x7f30bc102810 2024-03-12T05:03:20.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.372+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30ac003a20 con 0x7f30bc102810 2024-03-12T05:03:20.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.372+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc197b70 con 0x7f30bc102810 2024-03-12T05:03:20.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.372+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f30ac003b80 con 0x7f30bc102810 2024-03-12T05:03:20.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.372+0000 7f30b97fa700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f30a4038db0 0x7f30a403b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:20.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.373+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f30ac04cb50 con 0x7f30bc102810 2024-03-12T05:03:20.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.373+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc0632a0 con 0x7f30bc102810 2024-03-12T05:03:20.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.373+0000 7f30bb7fe700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f30a4038db0 0x7f30a403b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:20.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.373+0000 7f30bb7fe700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f30a4038db0 0x7f30a403b270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f30b0006fd0 tx=0x7f30b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:20.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.377+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30ac026080 con 0x7f30bc102810 2024-03-12T05:03:20.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:20.573+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7f30bc10baf0 con 0x7f30a4038db0 2024-03-12T05:03:21.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:20 smithi100 ceph-mon[32210]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:22.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:21 smithi100 ceph-mon[32210]: Deploying cephadm binary to smithi156 2024-03-12T05:03:24.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.810+0000 7f30b97fa700 1 -- 172.21.15.100:0/2858265517 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f30bc10baf0 con 0x7f30a4038db0 2024-03-12T05:03:24.812 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi156' with addr '172.21.15.156' 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f30a4038db0 msgr2=0x7f30a403b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f30a4038db0 0x7f30a403b270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f30b0006fd0 tx=0x7f30b0006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 msgr2=0x7f30bc197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc197010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30ac005850 tx=0x7f30ac004ff0 comp rx=0 tx=0).stop 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 shutdown_connections 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f30a4038db0 0x7f30a403b270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 --2- 172.21.15.100:0/2858265517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30bc102810 0x7f30bc197010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 >> 172.21.15.100:0/2858265517 conn(0x7f30bc0fc040 msgr2=0x7f30bc1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:24.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 shutdown_connections 2024-03-12T05:03:24.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:24.812+0000 7f30c226a700 1 -- 172.21.15.100:0/2858265517 wait complete. 2024-03-12T05:03:25.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:24 smithi100 ceph-mon[32210]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:25.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-12T05:03:25.659 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:26.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:25 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:26.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:25 smithi100 ceph-mon[32210]: Added host smithi156 2024-03-12T05:03:26.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:25 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:26.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:25 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:27.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:26 smithi100 ceph-mon[32210]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:27.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 -- 172.21.15.100:0/1078144547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 msgr2=0x7f7aec107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 --2- 172.21.15.100:0/1078144547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec107a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009910 tx=0x7f7adc009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 -- 172.21.15.100:0/1078144547 shutdown_connections 2024-03-12T05:03:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 --2- 172.21.15.100:0/1078144547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec107a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 -- 172.21.15.100:0/1078144547 >> 172.21.15.100:0/1078144547 conn(0x7f7aec0fd190 msgr2=0x7f7aec0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 -- 172.21.15.100:0/1078144547 shutdown_connections 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 -- 172.21.15.100:0/1078144547 wait complete. 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.214+0000 7f7af14f4700 1 Processor -- start 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7af14f4700 1 -- start start 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7af14f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7af14f4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec1974c0 con 0x7f7aec107680 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7aeaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7aeaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53234/0 (socket says 172.21.15.100:53234) 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7aeaffd700 1 -- 172.21.15.100:0/2117169680 learned_addr learned my addr 172.21.15.100:0/2117169680 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7aeaffd700 1 -- 172.21.15.100:0/2117169680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0095f0 con 0x7f7aec107680 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7aeaffd700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec196f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7adc004f40 tx=0x7f7adc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7adc01d070 con 0x7f7aec107680 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec197720 con 0x7f7aec107680 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7adc00bbd0 con 0x7f7aec107680 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.215+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7adc003900 con 0x7f7aec107680 2024-03-12T05:03:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.216+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec197ba0 con 0x7f7aec107680 2024-03-12T05:03:27.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.216+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f7adc003bc0 con 0x7f7aec107680 2024-03-12T05:03:27.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.216+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec0632a0 con 0x7f7aec107680 2024-03-12T05:03:27.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.217+0000 7f7ae8ff9700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7ad8038d50 0x7f7ad803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:27.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.217+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7adc04cb70 con 0x7f7aec107680 2024-03-12T05:03:27.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.217+0000 7f7ae27fc700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7ad8038d50 0x7f7ad803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:27.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.217+0000 7f7ae27fc700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7ad8038d50 0x7f7ad803b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4006fd0 tx=0x7f7ad4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.221+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7adc015350 con 0x7f7aec107680 2024-03-12T05:03:27.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.420+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7aec10bc30 con 0x7f7ad8038d50 2024-03-12T05:03:27.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.422+0000 7f7ae8ff9700 1 -- 172.21.15.100:0/2117169680 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f7aec10bc30 con 0x7f7ad8038d50 2024-03-12T05:03:27.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:27.423 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2024-03-12T05:03:27.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7ad8038d50 msgr2=0x7f7ad803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:27.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7ad8038d50 0x7f7ad803b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4006fd0 tx=0x7f7ad4006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 msgr2=0x7f7aec196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec196f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7adc004f40 tx=0x7f7adc005020 comp rx=0 tx=0).stop 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 shutdown_connections 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7ad8038d50 0x7f7ad803b210 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 --2- 172.21.15.100:0/2117169680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec107680 0x7f7aec196f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 >> 172.21.15.100:0/2117169680 conn(0x7f7aec0fd190 msgr2=0x7f7aec1091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 shutdown_connections 2024-03-12T05:03:27.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:27.424+0000 7f7af14f4700 1 -- 172.21.15.100:0/2117169680 wait complete. 2024-03-12T05:03:28.006 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-03-12T05:03:28.006 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:03:28.006 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-03-12T05:03:28.024 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:03:28.025 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:28.081 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-03-12T05:03:28.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch host add smithi161 2024-03-12T05:03:28.315 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:28 smithi100 ceph-mon[32210]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:03:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:28 smithi100 ceph-mon[32210]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 -- 172.21.15.100:0/245522188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 msgr2=0x7fbe4c107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 --2- 172.21.15.100:0/245522188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c107a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009910 tx=0x7fbe3c009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 -- 172.21.15.100:0/245522188 shutdown_connections 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 --2- 172.21.15.100:0/245522188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c107a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 -- 172.21.15.100:0/245522188 >> 172.21.15.100:0/245522188 conn(0x7fbe4c0fd140 msgr2=0x7fbe4c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 -- 172.21.15.100:0/245522188 shutdown_connections 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.846+0000 7fbe51ee5700 1 -- 172.21.15.100:0/245522188 wait complete. 2024-03-12T05:03:29.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe51ee5700 1 Processor -- start 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe51ee5700 1 -- start start 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe51ee5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe51ee5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1974c0 con 0x7fbe4c107630 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe4b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe4b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53260/0 (socket says 172.21.15.100:53260) 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.847+0000 7fbe4b7fe700 1 -- 172.21.15.100:0/1066180263 learned_addr learned my addr 172.21.15.100:0/1066180263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:29.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe4b7fe700 1 -- 172.21.15.100:0/1066180263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0095f0 con 0x7fbe4c107630 2024-03-12T05:03:29.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe4b7fe700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c196f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c004f40 tx=0x7fbe3c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:29.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe3c01d070 con 0x7fbe4c107630 2024-03-12T05:03:29.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe3c00bbd0 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c197720 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe3c003900 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.848+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c197ba0 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.849+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fbe3c003b70 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.849+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c0632a0 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.849+0000 7fbe48ff9700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe34038db0 0x7fbe3403b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.849+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe3c04cb20 con 0x7fbe4c107630 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.849+0000 7fbe4affd700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe34038db0 0x7fbe3403b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.850+0000 7fbe4affd700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe34038db0 0x7fbe3403b270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:29.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:29.854+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe3c029360 con 0x7fbe4c107630 2024-03-12T05:03:30.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:30.047+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe4c10bc20 con 0x7fbe34038db0 2024-03-12T05:03:31.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:31.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:31.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:31.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:31.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:31.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:03:31.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:31.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:32.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:31 smithi100 ceph-mon[32210]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:32.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:31 smithi100 ceph-mon[32210]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:03:32.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:31 smithi100 ceph-mon[32210]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:32.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:31 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:32.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:31 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:32.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:31 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:33.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:32 smithi100 ceph-mon[32210]: Deploying cephadm binary to smithi161 2024-03-12T05:03:33.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:32 smithi100 ceph-mon[32210]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:33.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:32 smithi100 ceph-mon[32210]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:03:33.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:32 smithi100 ceph-mon[32210]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:34.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.419+0000 7fbe48ff9700 1 -- 172.21.15.100:0/1066180263 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbe4c10bc20 con 0x7fbe34038db0 2024-03-12T05:03:34.420 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-03-12T05:03:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe34038db0 msgr2=0x7fbe3403b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe34038db0 0x7fbe3403b270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 msgr2=0x7fbe4c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c196f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c004f40 tx=0x7fbe3c005020 comp rx=0 tx=0).stop 2024-03-12T05:03:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 shutdown_connections 2024-03-12T05:03:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe34038db0 0x7fbe3403b270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 --2- 172.21.15.100:0/1066180263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c107630 0x7fbe4c196f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.421+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 >> 172.21.15.100:0/1066180263 conn(0x7fbe4c0fd140 msgr2=0x7fbe4c1090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.422+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 shutdown_connections 2024-03-12T05:03:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:34.422+0000 7fbe51ee5700 1 -- 172.21.15.100:0/1066180263 wait complete. 2024-03-12T05:03:35.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:34 smithi100 ceph-mon[32210]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:35.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:35.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:35.216 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-12T05:03:35.460 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:35.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:35 smithi100 ceph-mon[32210]: Added host smithi161 2024-03-12T05:03:35.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:35.885 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:36.891 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:36 smithi100 ceph-mon[32210]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:36.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2606615781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 msgr2=0x7f9a98074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:36.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 --2- 172.21.15.100:0/2606615781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98074370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009910 tx=0x7f9a88009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:36.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2606615781 shutdown_connections 2024-03-12T05:03:36.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 --2- 172.21.15.100:0/2606615781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98074370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2606615781 >> 172.21.15.100:0/2606615781 conn(0x7f9a980fd140 msgr2=0x7f9a980ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2606615781 shutdown_connections 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.907+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2606615781 wait complete. 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a9da6d700 1 Processor -- start 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a9da6d700 1 -- start start 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a9da6d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a9da6d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98199780 con 0x7f9a98073f90 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55688/0 (socket says 172.21.15.100:55688) 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a977fe700 1 -- 172.21.15.100:0/2043283484 learned_addr learned my addr 172.21.15.100:0/2043283484 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:36.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.908+0000 7f9a977fe700 1 -- 172.21.15.100:0/2043283484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880095f0 con 0x7f9a98073f90 2024-03-12T05:03:36.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.909+0000 7f9a977fe700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98199240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006010 tx=0x7f9a88004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:36.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.909+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8801d070 con 0x7f9a98073f90 2024-03-12T05:03:36.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.909+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a8800bc30 con 0x7f9a98073f90 2024-03-12T05:03:36.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.909+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a880039a0 con 0x7f9a98073f90 2024-03-12T05:03:36.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.909+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a98199980 con 0x7f9a98073f90 2024-03-12T05:03:36.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.909+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a98199da0 con 0x7f9a98073f90 2024-03-12T05:03:36.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.910+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f9a88003c20 con 0x7f9a98073f90 2024-03-12T05:03:36.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.910+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a980756d0 con 0x7f9a98073f90 2024-03-12T05:03:36.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.910+0000 7f9a957fa700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9a84038d50 0x7f9a8403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:36.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.910+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a8804cb50 con 0x7f9a98073f90 2024-03-12T05:03:36.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.910+0000 7f9a8effd700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9a84038d50 0x7f9a8403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:36.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.910+0000 7f9a8effd700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9a84038d50 0x7f9a8403b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9a80006fd0 tx=0x7f9a80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:36.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:36.915+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a88029540 con 0x7f9a98073f90 2024-03-12T05:03:37.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.116+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a9810dd30 con 0x7f9a84038d50 2024-03-12T05:03:37.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.117+0000 7f9a957fa700 1 -- 172.21.15.100:0/2043283484 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f9a9810dd30 con 0x7f9a84038d50 2024-03-12T05:03:37.119 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:03:37.119 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-03-12T05:03:37.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.119+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9a84038d50 msgr2=0x7f9a8403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:37.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.119+0000 7f9a9da6d700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9a84038d50 0x7f9a8403b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9a80006fd0 tx=0x7f9a80006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:37.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.119+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 msgr2=0x7f9a98199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:37.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.119+0000 7f9a9da6d700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98199240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006010 tx=0x7f9a88004ff0 comp rx=0 tx=0).stop 2024-03-12T05:03:37.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.120+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 shutdown_connections 2024-03-12T05:03:37.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.120+0000 7f9a9da6d700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9a84038d50 0x7f9a8403b210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:37.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.120+0000 7f9a9da6d700 1 --2- 172.21.15.100:0/2043283484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a98073f90 0x7f9a98199240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:37.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.120+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 >> 172.21.15.100:0/2043283484 conn(0x7f9a980fd140 msgr2=0x7f9a98105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:37.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.120+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 shutdown_connections 2024-03-12T05:03:37.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:37.120+0000 7f9a9da6d700 1 -- 172.21.15.100:0/2043283484 wait complete. 2024-03-12T05:03:37.671 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-12T05:03:37.671 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd crush tunables default 2024-03-12T05:03:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:03:38.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:38.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:38 smithi100 ceph-mon[32210]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:03:38.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:38 smithi100 ceph-mon[32210]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.364+0000 7fa854c8f700 1 -- 172.21.15.100:0/2923142256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 msgr2=0x7fa850106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.364+0000 7fa854c8f700 1 --2- 172.21.15.100:0/2923142256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa850106a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa848009910 tx=0x7fa848009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.364+0000 7fa854c8f700 1 -- 172.21.15.100:0/2923142256 shutdown_connections 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.364+0000 7fa854c8f700 1 --2- 172.21.15.100:0/2923142256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa850106a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 -- 172.21.15.100:0/2923142256 >> 172.21.15.100:0/2923142256 conn(0x7fa8500fc080 msgr2=0x7fa8500fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 -- 172.21.15.100:0/2923142256 shutdown_connections 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 -- 172.21.15.100:0/2923142256 wait complete. 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 Processor -- start 2024-03-12T05:03:39.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 -- start start 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa8501990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.365+0000 7fa854c8f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501995f0 con 0x7fa850106620 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa84e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa8501990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa84e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa8501990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55706/0 (socket says 172.21.15.100:55706) 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa84e59c700 1 -- 172.21.15.100:0/4080176711 learned_addr learned my addr 172.21.15.100:0/4080176711 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa84e59c700 1 -- 172.21.15.100:0/4080176711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8480095f0 con 0x7fa850106620 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa84e59c700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa8501990b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa848004f40 tx=0x7fa848005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa84801d070 con 0x7fa850106620 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa84800bc30 con 0x7fa850106620 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa850199850 con 0x7fa850106620 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa848003970 con 0x7fa850106620 2024-03-12T05:03:39.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.366+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa850199d50 con 0x7fa850106620 2024-03-12T05:03:39.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.367+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fa848003be0 con 0x7fa850106620 2024-03-12T05:03:39.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.367+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa850101c90 con 0x7fa850106620 2024-03-12T05:03:39.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.367+0000 7fa8477fe700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa83c038db0 0x7fa83c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:39.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.367+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa84804c8f0 con 0x7fa850106620 2024-03-12T05:03:39.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.367+0000 7fa84dd9b700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa83c038db0 0x7fa83c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:39.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.368+0000 7fa84dd9b700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa83c038db0 0x7fa83c03b270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa838006fd0 tx=0x7fa838006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.372+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa84802ada0 con 0x7fa850106620 2024-03-12T05:03:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.568+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa8500632a0 con 0x7fa850106620 2024-03-12T05:03:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.619+0000 7fa8477fe700 1 -- 172.21.15.100:0/4080176711 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa848026070 con 0x7fa850106620 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa83c038db0 msgr2=0x7fa83c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa83c038db0 0x7fa83c03b270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa838006fd0 tx=0x7fa838006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 msgr2=0x7fa8501990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa8501990b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa848004f40 tx=0x7fa848005020 comp rx=0 tx=0).stop 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 shutdown_connections 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa83c038db0 0x7fa83c03b270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 --2- 172.21.15.100:0/4080176711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa850106620 0x7fa8501990b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 >> 172.21.15.100:0/4080176711 conn(0x7fa8500fc080 msgr2=0x7fa8500fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 shutdown_connections 2024-03-12T05:03:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:03:39.621+0000 7fa854c8f700 1 -- 172.21.15.100:0/4080176711 wait complete. 2024-03-12T05:03:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-03-12T05:03:39.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:39 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4080176711' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-12T05:03:40.226 INFO:tasks.cephadm:Adding mon.a on smithi100 2024-03-12T05:03:40.226 INFO:tasks.cephadm:Adding mon.b on smithi156 2024-03-12T05:03:40.226 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-03-12T05:03:40.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch apply mon '3;smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c' 2024-03-12T05:03:40.461 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-12T05:03:40.509 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4080176711' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:03:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 -- 172.21.15.161:0/1890355357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 msgr2=0x7fe3d0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 --2- 172.21.15.161:0/1890355357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0107a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009910 tx=0x7fe3c4009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 -- 172.21.15.161:0/1890355357 shutdown_connections 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 --2- 172.21.15.161:0/1890355357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0107a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 -- 172.21.15.161:0/1890355357 >> 172.21.15.161:0/1890355357 conn(0x7fe3d00fd160 msgr2=0x7fe3d00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 -- 172.21.15.161:0/1890355357 shutdown_connections 2024-03-12T05:03:41.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.965+0000 7fe3d789b700 1 -- 172.21.15.161:0/1890355357 wait complete. 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d789b700 1 Processor -- start 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d789b700 1 -- start start 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d789b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d789b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01974e0 con 0x7fe3d0107650 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d5637700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d5637700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.161:44754/0 (socket says 172.21.15.161:44754) 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.966+0000 7fe3d5637700 1 -- 172.21.15.161:0/3479864714 learned_addr learned my addr 172.21.15.161:0/3479864714 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3d5637700 1 -- 172.21.15.161:0/3479864714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c40095f0 con 0x7fe3d0107650 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3d5637700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0196fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006010 tx=0x7fe3c4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c401d070 con 0x7fe3d0107650 2024-03-12T05:03:41.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d0197740 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c400bbd0 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c4003880 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.967+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d0197bc0 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.968+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d00632a0 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.968+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fe3c400b6f0 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.968+0000 7fe3c27fc700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe3bc038d50 0x7fe3bc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.968+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe3c404ca80 con 0x7fe3d0107650 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.968+0000 7fe3d4e36700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe3bc038d50 0x7fe3bc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:41.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.969+0000 7fe3d4e36700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe3bc038d50 0x7fe3bc03b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006fd0 tx=0x7fe3cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:41.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:41.973+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3c4029540 con 0x7fe3d0107650 2024-03-12T05:03:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:03:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:03:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:42.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:42.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:42.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.176+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3d010bc40 con 0x7fe3bc038d50 2024-03-12T05:03:42.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.182+0000 7fe3c27fc700 1 -- 172.21.15.161:0/3479864714 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe3d010bc40 con 0x7fe3bc038d50 2024-03-12T05:03:42.184 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-03-12T05:03:42.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe3bc038d50 msgr2=0x7fe3bc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:42.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe3bc038d50 0x7fe3bc03b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006fd0 tx=0x7fe3cc006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:42.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 msgr2=0x7fe3d0196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0196fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006010 tx=0x7fe3c4004ff0 comp rx=0 tx=0).stop 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 shutdown_connections 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe3bc038d50 0x7fe3bc03b210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 --2- 172.21.15.161:0/3479864714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d0107650 0x7fe3d0196fa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.184+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 >> 172.21.15.161:0/3479864714 conn(0x7fe3d00fd160 msgr2=0x7fe3d01091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.185+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 shutdown_connections 2024-03-12T05:03:42.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:42.185+0000 7fe3d789b700 1 -- 172.21.15.161:0/3479864714 wait complete. 2024-03-12T05:03:42.802 DEBUG:teuthology.orchestra.run.smithi156:mon.b> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.b.service 2024-03-12T05:03:42.805 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.c.service 2024-03-12T05:03:42.807 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T05:03:42.807 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T05:03:43.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:43.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:43.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:43.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:43.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:43.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:43.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:03:43.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:43.897 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:44.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:43 smithi100 ceph-mon[32210]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:44.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:43 smithi100 ceph-mon[32210]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c;count:3 2024-03-12T05:03:44.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:43 smithi100 ceph-mon[32210]: Deploying daemon mon.c on smithi161 2024-03-12T05:03:45.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:44 smithi100 ceph-mon[32210]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:46.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 -- 172.21.15.161:0/325709220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 msgr2=0x7fcda8107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 --2- 172.21.15.161:0/325709220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8107a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009910 tx=0x7fcda4009c20 comp rx=0 tx=0).stop 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 -- 172.21.15.161:0/325709220 shutdown_connections 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 --2- 172.21.15.161:0/325709220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8107a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 -- 172.21.15.161:0/325709220 >> 172.21.15.161:0/325709220 conn(0x7fcda80fd140 msgr2=0x7fcda80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 -- 172.21.15.161:0/325709220 shutdown_connections 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.535+0000 7fcdb0482700 1 -- 172.21.15.161:0/325709220 wait complete. 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdb0482700 1 Processor -- start 2024-03-12T05:03:46.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdb0482700 1 -- start start 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdb0482700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdb0482700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81974d0 con 0x7fcda8107630 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdae21e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdae21e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.161:44778/0 (socket says 172.21.15.161:44778) 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.536+0000 7fcdae21e700 1 -- 172.21.15.161:0/1162902017 learned_addr learned my addr 172.21.15.161:0/1162902017 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcdae21e700 1 -- 172.21.15.161:0/1162902017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40095f0 con 0x7fcda8107630 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcdae21e700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8196f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcda4004f40 tx=0x7fcda4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcda401d070 con 0x7fcda8107630 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda8197730 con 0x7fcda8107630 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda400bbd0 con 0x7fcda8107630 2024-03-12T05:03:46.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcda4003900 con 0x7fcda8107630 2024-03-12T05:03:46.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.537+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda8197bb0 con 0x7fcda8107630 2024-03-12T05:03:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.538+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fcda400bd40 con 0x7fcda8107630 2024-03-12T05:03:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.538+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda80632a0 con 0x7fcda8107630 2024-03-12T05:03:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.538+0000 7fcd9f7fe700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcd94038d50 0x7fcd9403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.538+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcda404cb70 con 0x7fcda8107630 2024-03-12T05:03:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.538+0000 7fcdada1d700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcd94038d50 0x7fcd9403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.539+0000 7fcdada1d700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcd94038d50 0x7fcd9403b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd98006fd0 tx=0x7fcd98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:46.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.544+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcda4029360 con 0x7fcda8107630 2024-03-12T05:03:46.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.841+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcda810bbb0 con 0x7fcda8107630 2024-03-12T05:03:46.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.842+0000 7fcd9f7fe700 1 -- 172.21.15.161:0/1162902017 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7fcda4026070 con 0x7fcda8107630 2024-03-12T05:03:46.843 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:03:46.844 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","modified":"2024-03-12T05:02:05.586925Z","created":"2024-03-12T05:02:05.586925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-12T05:03:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcd94038d50 msgr2=0x7fcd9403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcd94038d50 0x7fcd9403b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd98006fd0 tx=0x7fcd98006e40 comp rx=0 tx=0).stop 2024-03-12T05:03:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 msgr2=0x7fcda8196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8196f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcda4004f40 tx=0x7fcda4005020 comp rx=0 tx=0).stop 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 shutdown_connections 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcd94038d50 0x7fcd9403b210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 --2- 172.21.15.161:0/1162902017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8107630 0x7fcda8196f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 >> 172.21.15.161:0/1162902017 conn(0x7fcda80fd140 msgr2=0x7fcda80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 shutdown_connections 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:46.845+0000 7fcdb0482700 1 -- 172.21.15.161:0/1162902017 wait complete. 2024-03-12T05:03:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-03-12T05:03:47.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:46 smithi100 ceph-mon[32210]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:47.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 podman[36024]: 2024-03-12 05:03:47.651252003 +0000 UTC m=+0.151382204 container create 72c9153137b7742899fc2ad622863835b350570d3347af46c109748d2d19804c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-12T05:03:48.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:47 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/1162902017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:03:48.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 podman[36024]: 2024-03-12 05:03:47.845408357 +0000 UTC m=+0.345538557 container init 72c9153137b7742899fc2ad622863835b350570d3347af46c109748d2d19804c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 podman[36024]: 2024-03-12 05:03:47.848411145 +0000 UTC m=+0.348541338 container start 72c9153137b7742899fc2ad622863835b350570d3347af46c109748d2d19804c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: set uid:gid to 167:167 (ceph:ceph) 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pidfile_write: ignore empty --pid-file 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: load: jerasure load: lrc 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: RocksDB version: 7.9.2 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Git sha 0 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-12T05:03:48.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: DB SUMMARY 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: DB Session ID: FJ9YM5M24RSPXVZMOH0Z 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: CURRENT file: CURRENT 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: IDENTITY file: IDENTITY 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.error_if_exists: 0 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.create_if_missing: 0 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.paranoid_checks: 1 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-12T05:03:48.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.env: 0x562db9304ae0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.fs: PosixFileSystem 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.info_log: 0x562db9e238c0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_file_opening_threads: 16 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.statistics: (nil) 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.use_fsync: 0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_log_file_size: 0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.keep_log_file_num: 1000 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.recycle_log_file_num: 0 2024-03-12T05:03:48.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.allow_fallocate: 1 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.allow_mmap_reads: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.allow_mmap_writes: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.use_direct_reads: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.create_missing_column_families: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.db_log_dir: 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.wal_dir: 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-12T05:03:48.231 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.advise_random_on_open: 1 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.db_write_buffer_size: 0 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.write_buffer_manager: 0x562db9df7400 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.rate_limiter: (nil) 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.wal_recovery_mode: 2 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.enable_thread_tracking: 0 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.enable_pipelined_write: 0 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.unordered_write: 0 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-12T05:03:48.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-12T05:03:48.233 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-12T05:03:48.233 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-12T05:03:48.233 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.row_cache: None 2024-03-12T05:03:48.233 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.wal_filter: None 2024-03-12T05:03:48.234 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-12T05:03:48.234 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.allow_ingest_behind: 0 2024-03-12T05:03:48.234 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.two_write_queues: 0 2024-03-12T05:03:48.234 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.manual_wal_flush: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.wal_compression: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.atomic_flush: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.log_readahead_size: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.best_efforts_recovery: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.allow_data_in_errors: 0 2024-03-12T05:03:48.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.db_host_id: __hostname__ 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_background_jobs: 2 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_background_compactions: -1 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_subcompactions: 1 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_total_wal_size: 0 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_open_files: -1 2024-03-12T05:03:48.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bytes_per_sync: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_readahead_size: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_background_flushes: -1 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Compression algorithms supported: 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kZSTD supported: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kXpressCompression supported: 0 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kLZ4HCCompression supported: 1 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kZlibCompression supported: 1 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kSnappyCompression supported: 1 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kLZ4Compression supported: 1 2024-03-12T05:03:48.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: kBZip2Compression supported: 0 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.merge_operator: 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_filter: None 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_filter_factory: None 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.sst_partitioner_factory: None 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562db9e239c0) 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-03-12T05:03:48.238 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_type: 0 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_index_type: 0 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_shortening: 1 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: checksum: 4 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: no_block_cache: 0 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache: 0x562db9e27090 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_options: 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: capacity : 536870912 2024-03-12T05:03:48.239 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_shard_bits : 4 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: strict_capacity_limit : 0 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_compressed: (nil) 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: persistent_cache: (nil) 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size: 4096 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size_deviation: 10 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_restart_interval: 16 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_block_restart_interval: 1 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: metadata_block_size: 4096 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: partition_filters: 0 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: use_delta_encoding: 1 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: filter_policy: bloomfilter 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: whole_key_filtering: 1 2024-03-12T05:03:48.240 INFO:journalctl@ceph.mon.c.smithi161.stdout: verify_compression: 0 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: format_version: 5 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_index_compression: 1 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_align: 0 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: max_auto_readahead_size: 262144 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: prepopulate_block_cache: 0 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: initial_auto_readahead_size: 8192 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.write_buffer_size: 33554432 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_write_buffer_number: 2 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression: NoCompression 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression: Disabled 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.prefix_extractor: nullptr 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-12T05:03:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.num_levels: 7 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-12T05:03:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.level: 32767 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.strategy: 0 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.enabled: false 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.target_file_size_base: 67108864 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-12T05:03:48.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.arena_block_size: 1048576 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-12T05:03:48.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.disable_auto_compactions: 0 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.table_properties_collectors: 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.inplace_update_support: 0 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-12T05:03:48.245 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.bloom_locality: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.max_successive_merges: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.paranoid_file_checks: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.force_consistency_checks: 1 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.report_bg_io_stats: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.ttl: 2592000 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.enable_blob_files: false 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.min_blob_size: 0 2024-03-12T05:03:48.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.blob_file_size: 268435456 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.blob_file_starting_level: 0 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3c9c83fe-0bc2-4577-8e50-8551d83abf30 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710219827877889, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710219827878322, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710219827, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3c9c83fe-0bc2-4577-8e50-8551d83abf30", "db_session_id": "FJ9YM5M24RSPXVZMOH0Z", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710219827878402, "job": 1, "event": "recovery_finished"} 2024-03-12T05:03:48.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562db9f18000 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: DB pointer 0x562db9f02000 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** DB Stats ** 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-12T05:03:48.248 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T05:03:48.249 INFO:journalctl@ceph.mon.c.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-12T05:03:48.250 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache BinnedLRUCache@0x562db9e27090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: starting mon.c rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(???) e0 preinit fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).mds e1 new map 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: e1 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T05:03:48.251 INFO:journalctl@ceph.mon.c.smithi161.stdout: legacy client fscid: -1 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout: No filesystems configured 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T05:03:48.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:48.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Deploying cephadm binary to smithi161 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.254 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Added host smithi161 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4080176711' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4080176711' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.256 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c;count:3 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: Deploying daemon mon.c on smithi161 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/1162902017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 ceph-mon[36043]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-12T05:03:48.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 bash[36024]: 72c9153137b7742899fc2ad622863835b350570d3347af46c109748d2d19804c 2024-03-12T05:03:48.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:47 smithi161 systemd[1]: Started Ceph mon.c for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:03:48.438 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T05:03:48.439 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T05:03:48.673 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 -- 172.21.15.161:0/2024898557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 msgr2=0x7efed0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2024898557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efed0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efee4100570 tx=0x7efed400feb0 comp rx=0 tx=0).stop 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 -- 172.21.15.161:0/2024898557 shutdown_connections 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2024898557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efed0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 -- 172.21.15.161:0/2024898557 >> 172.21.15.161:0/2024898557 conn(0x7efee40fc110 msgr2=0x7efee40fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 -- 172.21.15.161:0/2024898557 shutdown_connections 2024-03-12T05:03:52.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.997+0000 7efee8dbd700 1 -- 172.21.15.161:0/2024898557 wait complete. 2024-03-12T05:03:53.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.998+0000 7efee8dbd700 1 Processor -- start 2024-03-12T05:03:53.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.998+0000 7efee8dbd700 1 -- start start 2024-03-12T05:03:53.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee8dbd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efee4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:53.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee8dbd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:53.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee8dbd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee406ae00 con 0x7efed00035c0 2024-03-12T05:03:53.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee8dbd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee41a0fd0 con 0x7efee41984a0 2024-03-12T05:03:53.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:53.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57040/0 (socket says 172.21.15.161:57040) 2024-03-12T05:03:53.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 -- 172.21.15.161:0/2759528525 learned_addr learned my addr 172.21.15.161:0/2759528525 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:03:53.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee259c700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efee4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:53.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 -- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 msgr2=0x7efee4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 -- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 msgr2=0x7efee4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efed9d9b700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee259c700 1 -- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 msgr2=0x7efee4198920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee259c700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:52.999+0000 7efee259c700 1 -- 172.21.15.161:0/2759528525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed400f900 con 0x7efed00035c0 2024-03-12T05:03:53.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.000+0000 7efee259c700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efee4197f60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed40056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.001+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7efed4004aa0 con 0x7efed00035c0 2024-03-12T05:03:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.001+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee41a1200 con 0x7efed00035c0 2024-03-12T05:03:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.001+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efed4013e40 con 0x7efed00035c0 2024-03-12T05:03:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.001+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7efed4011b20 con 0x7efed00035c0 2024-03-12T05:03:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.001+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee41a1750 con 0x7efed00035c0 2024-03-12T05:03:53.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.002+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7efed4003680 con 0x7efed00035c0 2024-03-12T05:03:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.002+0000 7efedbfff700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efec4038af0 0x7efec403afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.002+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efed404e590 con 0x7efed00035c0 2024-03-12T05:03:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.002+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee418fee0 con 0x7efed00035c0 2024-03-12T05:03:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.002+0000 7efed9d9b700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efec4038af0 0x7efec403afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:53.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.003+0000 7efed9d9b700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efec4038af0 0x7efec403afb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efecc009180 tx=0x7efecc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:53.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.008+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efed40109c0 con 0x7efed00035c0 2024-03-12T05:03:53.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.284+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efee406a510 con 0x7efed00035c0 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: mon.a calling monitor election 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: mon.c calling monitor election 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:53.286 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: fsmap 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: mgrmap e13: x(active, since 49s) 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: overall HEALTH_OK 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:53.287 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:53.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.285+0000 7efedbfff700 1 -- 172.21.15.161:0/2759528525 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7efee406a510 con 0x7efed00035c0 2024-03-12T05:03:53.288 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:03:53.288 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","modified":"2024-03-12T05:03:47.902112Z","created":"2024-03-12T05:02:05.586925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-12T05:03:53.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efec4038af0 msgr2=0x7efec403afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efec4038af0 0x7efec403afb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efecc009180 tx=0x7efecc008040 comp rx=0 tx=0).stop 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 msgr2=0x7efee4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efee4197f60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed40056f0 comp rx=0 tx=0).stop 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 shutdown_connections 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efec4038af0 0x7efec403afb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed00035c0 0x7efee4197f60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 --2- 172.21.15.161:0/2759528525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efee41984a0 0x7efee4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 >> 172.21.15.161:0/2759528525 conn(0x7efee40fc110 msgr2=0x7efee40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 shutdown_connections 2024-03-12T05:03:53.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:53.288+0000 7efee8dbd700 1 -- 172.21.15.161:0/2759528525 wait complete. 2024-03-12T05:03:53.291 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-03-12T05:03:53.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:03:53.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: mon.a calling monitor election 2024-03-12T05:03:53.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: mon.c calling monitor election 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: fsmap 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:03:53.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: mgrmap e13: x(active, since 49s) 2024-03-12T05:03:53.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: overall HEALTH_OK 2024-03-12T05:03:53.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:53.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: Deploying daemon mon.b on smithi156 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/2759528525' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:54.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:54 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:54.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:03:54.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:03:54.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:03:54.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: Deploying daemon mon.b on smithi156 2024-03-12T05:03:54.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/2759528525' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:03:54.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:54.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:54 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:03:54.841 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T05:03:54.841 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T05:03:55.087 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:03:56.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 -- 172.21.15.161:0/3629840509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c30074010 msgr2=0x7f8c300743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:56.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 --2- 172.21.15.161:0/3629840509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c30074010 0x7f8c300743f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800b0a0 tx=0x7f8c2800b3b0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 -- 172.21.15.161:0/3629840509 shutdown_connections 2024-03-12T05:03:56.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 --2- 172.21.15.161:0/3629840509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c30074010 0x7f8c300743f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 -- 172.21.15.161:0/3629840509 >> 172.21.15.161:0/3629840509 conn(0x7f8c300fd240 msgr2=0x7f8c300ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:56.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 -- 172.21.15.161:0/3629840509 shutdown_connections 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.429+0000 7f8c3763f700 1 -- 172.21.15.161:0/3629840509 wait complete. 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c3763f700 1 Processor -- start 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c3763f700 1 -- start start 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c3763f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c3763f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c301a28f0 0x7f8c3019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c3763f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301a2d50 con 0x7f8c301a28f0 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c3763f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301a2ec0 con 0x7f8c30074010 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c353db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c353db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57046/0 (socket says 172.21.15.161:57046) 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c353db700 1 -- 172.21.15.161:0/638032692 learned_addr learned my addr 172.21.15.161:0/638032692 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:03:56.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c34bda700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c301a28f0 0x7f8c3019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c353db700 1 -- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 msgr2=0x7f8c301a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c353db700 1 -- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 msgr2=0x7f8c301a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.430+0000 7f8c353db700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c353db700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c34bda700 1 -- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 msgr2=0x7f8c301a23b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c34bda700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c34bda700 1 -- 172.21.15.161:0/638032692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c28009d00 con 0x7f8c301a28f0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c34bda700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c301a28f0 0x7f8c3019d710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b4d0 tx=0x7f8c2000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8c20011630 con 0x7f8c301a28f0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3019dd10 con 0x7f8c301a28f0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c20011c70 con 0x7f8c301a28f0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8c20010de0 con 0x7f8c301a28f0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.431+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3019e260 con 0x7f8c301a28f0 2024-03-12T05:03:56.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.432+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f8c2001a400 con 0x7f8c301a28f0 2024-03-12T05:03:56.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.432+0000 7f8c267fc700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8c1c038af0 0x7f8c1c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:03:56.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.432+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30075750 con 0x7f8c301a28f0 2024-03-12T05:03:56.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.432+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c2004b460 con 0x7f8c301a28f0 2024-03-12T05:03:56.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.433+0000 7f8c353db700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8c1c038af0 0x7f8c1c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:03:56.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.433+0000 7f8c353db700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8c1c038af0 0x7f8c1c03afb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c280076a0 tx=0x7f8c280096e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:03:56.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.438+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c200193b0 con 0x7f8c301a28f0 2024-03-12T05:03:56.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.712+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8c300632a0 con 0x7f8c301a28f0 2024-03-12T05:03:56.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.713+0000 7f8c267fc700 1 -- 172.21.15.161:0/638032692 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7f8c200172c0 con 0x7f8c301a28f0 2024-03-12T05:03:56.715 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:03:56.715 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","modified":"2024-03-12T05:03:47.902112Z","created":"2024-03-12T05:02:05.586925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8c1c038af0 msgr2=0x7f8c1c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8c1c038af0 0x7f8c1c03afb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c280076a0 tx=0x7f8c280096e0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c301a28f0 msgr2=0x7f8c3019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c301a28f0 0x7f8c3019d710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b4d0 tx=0x7f8c2000b890 comp rx=0 tx=0).stop 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 shutdown_connections 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8c1c038af0 0x7f8c1c03afb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c30074010 0x7f8c301a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 --2- 172.21.15.161:0/638032692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c301a28f0 0x7f8c3019d710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 >> 172.21.15.161:0/638032692 conn(0x7f8c300fd240 msgr2=0x7f8c301034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:03:56.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 shutdown_connections 2024-03-12T05:03:56.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:03:56.715+0000 7f8c3763f700 1 -- 172.21.15.161:0/638032692 wait complete. 2024-03-12T05:03:56.718 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-03-12T05:03:56.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:03:56 smithi161 ceph-mon[36043]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:57.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:03:56 smithi100 ceph-mon[32210]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:03:57.209 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:03:57 smithi156 systemd[1]: Started Ceph mon.b for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:03:58.296 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-12T05:03:58.296 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T05:03:58.539 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: mon.a calling monitor election 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: mon.c calling monitor election 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: mon.b calling monitor election 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: fsmap 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: mgrmap e13: x(active, since 58s) 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: overall HEALTH_OK 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:02.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: mon.a calling monitor election 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: mon.c calling monitor election 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: mon.b calling monitor election 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: fsmap 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: mgrmap e13: x(active, since 58s) 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: overall HEALTH_OK 2024-03-12T05:04:02.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:02.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: mon.a calling monitor election 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: mon.c calling monitor election 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: mon.b calling monitor election 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: fsmap 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: osdmap e4: 0 total, 0 up, 0 in 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: mgrmap e13: x(active, since 58s) 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: overall HEALTH_OK 2024-03-12T05:04:02.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:02.574 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 -- 172.21.15.161:0/2663590193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec1077b0 msgr2=0x7f8ad4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 --2- 172.21.15.161:0/2663590193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec1077b0 0x7f8ad4005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0005fa0 tx=0x7f8ae000ffc0 comp rx=0 tx=0).stop 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 -- 172.21.15.161:0/2663590193 shutdown_connections 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 --2- 172.21.15.161:0/2663590193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec1077b0 0x7f8ad4005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 -- 172.21.15.161:0/2663590193 >> 172.21.15.161:0/2663590193 conn(0x7f8aec0fbd80 msgr2=0x7f8aec0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 -- 172.21.15.161:0/2663590193 shutdown_connections 2024-03-12T05:04:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.816+0000 7f8af0e06700 1 -- 172.21.15.161:0/2663590193 wait complete. 2024-03-12T05:04:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 Processor -- start 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 -- start start 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 0x7f8aec197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec19bfa0 0x7f8aec19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec197dc0 con 0x7f8aec19bfa0 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1a3d10 con 0x7f8aec1077b0 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.817+0000 7f8af0e06700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1a3e50 con 0x7f8aec1976d0 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58232/0 (socket says 172.21.15.161:58232) 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 -- 172.21.15.161:0/1429444998 learned_addr learned my addr 172.21.15.161:0/1429444998 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec19bfa0 0x7f8aec19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 msgr2=0x7f8aec19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-12T05:04:02.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 msgr2=0x7f8aec19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8ae9d9b700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aea59c700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 0x7f8aec197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 msgr2=0x7f8aec19ba60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aea59c700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 msgr2=0x7f8aec197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-12T05:04:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 msgr2=0x7f8aec197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aea59c700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 msgr2=0x7f8aec197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aea59c700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 0x7f8aec197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aea59c700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 0x7f8aec197190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 0x7f8aec197190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 -- 172.21.15.161:0/1429444998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae000f9c0 con 0x7f8aec19bfa0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8aead9d700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec19bfa0 0x7f8aec19c450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400bd50 tx=0x7f8ae400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.818+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400cd10 con 0x7f8aec19bfa0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.819+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec1a4130 con 0x7f8aec19bfa0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.819+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ae400ce70 con 0x7f8aec19bfa0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.819+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40128e0 con 0x7f8aec19bfa0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.819+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec1a45e0 con 0x7f8aec19bfa0 2024-03-12T05:04:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.820+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f8ae4012b40 con 0x7f8aec19bfa0 2024-03-12T05:04:02.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.820+0000 7f8adb7fe700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8acc0391e0 0x7f8acc03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:02.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.820+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ae404b4a0 con 0x7f8aec19bfa0 2024-03-12T05:04:02.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.820+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec190f50 con 0x7f8aec19bfa0 2024-03-12T05:04:02.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.820+0000 7f8aea59c700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8acc0391e0 0x7f8acc03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:02.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.821+0000 7f8aea59c700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8acc0391e0 0x7f8acc03b6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00059d0 tx=0x7f8ae00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:02.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:02.826+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ae4019070 con 0x7f8aec19bfa0 2024-03-12T05:04:03.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.102+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8aec1984f0 con 0x7f8aec19bfa0 2024-03-12T05:04:03.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.103+0000 7f8adb7fe700 1 -- 172.21.15.161:0/1429444998 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1289 (secure 0 0 0) 0x7f8ae4011da0 con 0x7f8aec19bfa0 2024-03-12T05:04:03.105 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-12T05:04:03.105 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","modified":"2024-03-12T05:03:56.999880Z","created":"2024-03-12T05:02:05.586925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.105+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8acc0391e0 msgr2=0x7f8acc03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.105+0000 7f8af0e06700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8acc0391e0 0x7f8acc03b6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00059d0 tx=0x7f8ae00058e0 comp rx=0 tx=0).stop 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.105+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec19bfa0 msgr2=0x7f8aec19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.105+0000 7f8af0e06700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec19bfa0 0x7f8aec19c450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400bd50 tx=0x7f8ae400be30 comp rx=0 tx=0).stop 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 shutdown_connections 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8acc0391e0 0x7f8acc03b6a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aec1077b0 0x7f8aec197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:03.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aec19bfa0 0x7f8aec19c450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:03.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 --2- 172.21.15.161:0/1429444998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aec1976d0 0x7f8aec19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:03.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 >> 172.21.15.161:0/1429444998 conn(0x7f8aec0fbd80 msgr2=0x7f8aec104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:03.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 shutdown_connections 2024-03-12T05:04:03.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:03.106+0000 7f8af0e06700 1 -- 172.21.15.161:0/1429444998 wait complete. 2024-03-12T05:04:03.108 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-03-12T05:04:03.412 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:03.412 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:03.412 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:03.412 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:03.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:03.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:03.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:03.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:03.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:03.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:03.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:03.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:04:03.733 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-12T05:04:03.733 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph config generate-minimal-conf 2024-03-12T05:04:03.979 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:04:04.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:04 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/1429444998' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:04:04.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:04 smithi100 ceph-mon[32210]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:04.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.360 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:04 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/1429444998' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:04:04.360 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.360 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.360 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:04 smithi161 ceph-mon[36043]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:04.360 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:04 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/1429444998' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T05:04:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:04 smithi156 ceph-mon[36270]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Reconfiguring mon.a (unknown last config time)... 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:05.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:05 smithi100 ceph-mon[32210]: Reconfiguring daemon mon.a on smithi100 2024-03-12T05:04:05.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:05.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:05.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.460 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Reconfiguring mon.a (unknown last config time)... 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:05.461 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:05 smithi156 ceph-mon[36270]: Reconfiguring daemon mon.a on smithi100 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:04:05.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Reconfiguring mon.a (unknown last config time)... 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:05.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:05.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:05 smithi161 ceph-mon[36043]: Reconfiguring daemon mon.a on smithi100 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 -- 172.21.15.100:0/3459151406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 msgr2=0x7f0d041069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/3459151406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041069d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b0a0 tx=0x7f0cfc00b3b0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 -- 172.21.15.100:0/3459151406 shutdown_connections 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/3459151406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041069d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 -- 172.21.15.100:0/3459151406 >> 172.21.15.100:0/3459151406 conn(0x7f0d04077310 msgr2=0x7f0d040fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 -- 172.21.15.100:0/3459151406 shutdown_connections 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.557+0000 7f0d08d1e700 1 -- 172.21.15.100:0/3459151406 wait complete. 2024-03-12T05:04:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 Processor -- start 2024-03-12T05:04:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 -- start start 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d04197620 0x7f0d0419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0419bef0 0x7f0d0419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04197d10 con 0x7f0d041065f0 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d041a3c60 con 0x7f0d0419bef0 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d08d1e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d041a3da0 con 0x7f0d04197620 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:05.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57516/0 (socket says 172.21.15.100:57516) 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.558+0000 7f0d0259c700 1 -- 172.21.15.100:0/1303929785 learned_addr learned my addr 172.21.15.100:0/1303929785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d01d9b700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d04197620 0x7f0d0419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d0259c700 1 -- 172.21.15.100:0/1303929785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d04197620 msgr2=0x7f0d0419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d0259c700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d04197620 0x7f0d0419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d0259c700 1 -- 172.21.15.100:0/1303929785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0419bef0 msgr2=0x7f0d0419c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d02d9d700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0419bef0 0x7f0d0419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d0259c700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0419bef0 0x7f0d0419c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d0259c700 1 -- 172.21.15.100:0/1303929785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009d00 con 0x7f0d041065f0 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d01d9b700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d04197620 0x7f0d0419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d02d9d700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0419bef0 0x7f0d0419c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d0259c700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041970e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc015040 tx=0x7f0cfc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00e070 con 0x7f0d041065f0 2024-03-12T05:04:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d041a4080 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc0055a0 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc01db40 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.559+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d041a4600 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.560+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f0cfc012430 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.560+0000 7f0cf37fe700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0cec0391e0 0x7f0cec03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.560+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cfc04c880 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.560+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d040632a0 con 0x7f0d041065f0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.560+0000 7f0d01d9b700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0cec0391e0 0x7f0cec03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.561+0000 7f0d01d9b700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0cec0391e0 0x7f0cec03b6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80079b0 tx=0x7f0cf8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:05.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.565+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0cfc022070 con 0x7f0d041065f0 2024-03-12T05:04:05.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.763+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0d0410a8b0 con 0x7f0d041065f0 2024-03-12T05:04:05.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.763+0000 7f0cf37fe700 1 -- 172.21.15.100:0/1303929785 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f0cfc017030 con 0x7f0d041065f0 2024-03-12T05:04:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:# minimal ceph.conf for 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:04:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-03-12T05:04:05.765 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:04:05.765 INFO:teuthology.orchestra.run.smithi100.stdout: mon_host = [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-03-12T05:04:05.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0cec0391e0 msgr2=0x7f0cec03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0cec0391e0 0x7f0cec03b6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80079b0 tx=0x7f0cf8008040 comp rx=0 tx=0).stop 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 msgr2=0x7f0d041970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041970e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc015040 tx=0x7f0cfc009500 comp rx=0 tx=0).stop 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 shutdown_connections 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0cec0391e0 0x7f0cec03b6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0419bef0 0x7f0d0419c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d041065f0 0x7f0d041970e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 --2- 172.21.15.100:0/1303929785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d04197620 0x7f0d0419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 >> 172.21.15.100:0/1303929785 conn(0x7f0d04077310 msgr2=0x7f0d041040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 shutdown_connections 2024-03-12T05:04:05.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:05.765+0000 7f0d08d1e700 1 -- 172.21.15.100:0/1303929785 wait complete. 2024-03-12T05:04:06.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:06 smithi100 ceph-mon[32210]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:06.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:06 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1303929785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:06.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:06 smithi156 ceph-mon[36270]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:06.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:06 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1303929785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:06.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:06 smithi161 ceph-mon[36043]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:06.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:06 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1303929785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:06.735 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-12T05:04:06.735 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:04:06.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T05:04:06.764 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:04:06.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:04:06.831 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:04:06.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T05:04:06.867 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:04:06.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:04:06.939 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:04:06.939 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T05:04:06.973 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:04:06.973 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:04:07.044 INFO:tasks.cephadm:Adding mgr.x on smithi100 2024-03-12T05:04:07.044 INFO:tasks.cephadm:Adding mgr.y on smithi156 2024-03-12T05:04:07.044 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-03-12T05:04:07.045 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch apply mgr '3;smithi100=x;smithi156=y;smithi161=z' 2024-03-12T05:04:07.325 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:04:08.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: Reconfiguring mon.b (monmap changed)... 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: Reconfiguring daemon mon.b on smithi156 2024-03-12T05:04:08.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:08 smithi156 ceph-mon[36270]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: Reconfiguring mon.b (monmap changed)... 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: Reconfiguring daemon mon.b on smithi156 2024-03-12T05:04:08.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:08 smithi161 ceph-mon[36043]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: Reconfiguring mon.b (monmap changed)... 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: Reconfiguring daemon mon.b on smithi156 2024-03-12T05:04:08.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:08 smithi100 ceph-mon[32210]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:08.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 -- 172.21.15.161:0/3640996787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4100690 msgr2=0x7ff2f4100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:08.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/3640996787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4100690 0x7ff2f4100ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009910 tx=0x7ff2e8009c20 comp rx=0 tx=0).stop 2024-03-12T05:04:08.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 -- 172.21.15.161:0/3640996787 shutdown_connections 2024-03-12T05:04:08.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/3640996787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4100690 0x7ff2f4100ab0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 -- 172.21.15.161:0/3640996787 >> 172.21.15.161:0/3640996787 conn(0x7ff2f40fbc50 msgr2=0x7ff2f40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 -- 172.21.15.161:0/3640996787 shutdown_connections 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.853+0000 7ff2fb06e700 1 -- 172.21.15.161:0/3640996787 wait complete. 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 Processor -- start 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 -- start start 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2f4100690 0x7ff2f406a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4199b50 0x7ff2f419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 0x7ff2f419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f419f100 con 0x7ff2f4199b50 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f419f270 con 0x7ff2f4100690 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2fb06e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f419f430 con 0x7ff2f419c4d0 2024-03-12T05:04:08.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2f960b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 0x7ff2f419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2f960b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 0x7ff2f419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39346/0 (socket says 172.21.15.161:39346) 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2f960b700 1 -- 172.21.15.161:0/2282728986 learned_addr learned my addr 172.21.15.161:0/2282728986 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.854+0000 7ff2f3fff700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4199b50 0x7ff2f419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f8e0a700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2f4100690 0x7ff2f406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f3fff700 1 -- 172.21.15.161:0/2282728986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 msgr2=0x7ff2f419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f3fff700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 0x7ff2f419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f3fff700 1 -- 172.21.15.161:0/2282728986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2f4100690 msgr2=0x7ff2f406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f3fff700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2f4100690 0x7ff2f406a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f3fff700 1 -- 172.21.15.161:0/2282728986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e80095f0 con 0x7ff2f4199b50 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f960b700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 0x7ff2f419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f3fff700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4199b50 0x7ff2f419bf40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b550 tx=0x7ff2e000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0004230 con 0x7ff2f4199b50 2024-03-12T05:04:08.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e0004390 con 0x7ff2f4199b50 2024-03-12T05:04:08.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f419f710 con 0x7ff2f4199b50 2024-03-12T05:04:08.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0011a90 con 0x7ff2f4199b50 2024-03-12T05:04:08.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.855+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f419fc90 con 0x7ff2f4199b50 2024-03-12T05:04:08.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.856+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7ff2e0011c30 con 0x7ff2f4199b50 2024-03-12T05:04:08.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.856+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f418f4a0 con 0x7ff2f4199b50 2024-03-12T05:04:08.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.856+0000 7ff2f1ffb700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff2e40391e0 0x7ff2e403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:08.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.856+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff2e004b240 con 0x7ff2f4199b50 2024-03-12T05:04:08.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.857+0000 7ff2f8e0a700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff2e40391e0 0x7ff2e403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:08.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.857+0000 7ff2f8e0a700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff2e40391e0 0x7ff2e403b6a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009fd0 tx=0x7ff2e8005300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:08.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:08.862+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2e000f8d0 con 0x7ff2f4199b50 2024-03-12T05:04:09.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.068+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi156=y;smithi161=z", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2f402d870 con 0x7ff2e40391e0 2024-03-12T05:04:09.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.075+0000 7ff2f1ffb700 1 -- 172.21.15.161:0/2282728986 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff2f402d870 con 0x7ff2e40391e0 2024-03-12T05:04:09.076 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-03-12T05:04:09.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff2e40391e0 msgr2=0x7ff2e403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:09.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff2e40391e0 0x7ff2e403b6a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009fd0 tx=0x7ff2e8005300 comp rx=0 tx=0).stop 2024-03-12T05:04:09.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4199b50 msgr2=0x7ff2f419bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:09.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4199b50 0x7ff2f419bf40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b550 tx=0x7ff2e000b860 comp rx=0 tx=0).stop 2024-03-12T05:04:09.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 shutdown_connections 2024-03-12T05:04:09.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff2e40391e0 0x7ff2e403b6a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2f4100690 0x7ff2f406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2f419c4d0 0x7ff2f419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 --2- 172.21.15.161:0/2282728986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f4199b50 0x7ff2f419bf40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.078+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 >> 172.21.15.161:0/2282728986 conn(0x7ff2f40fbc50 msgr2=0x7ff2f40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.079+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 shutdown_connections 2024-03-12T05:04:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:04:09.079+0000 7ff2fb06e700 1 -- 172.21.15.161:0/2282728986 wait complete. 2024-03-12T05:04:09.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:09.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:09.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: Reconfiguring mon.c (monmap changed)... 2024-03-12T05:04:09.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:09.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:09.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:09.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: Reconfiguring daemon mon.c on smithi161 2024-03-12T05:04:09.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: Reconfiguring mon.c (monmap changed)... 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: Reconfiguring daemon mon.c on smithi161 2024-03-12T05:04:09.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: Reconfiguring mon.c (monmap changed)... 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: Reconfiguring daemon mon.c on smithi161 2024-03-12T05:04:10.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.460 DEBUG:teuthology.orchestra.run.smithi156:mgr.y> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.y.service 2024-03-12T05:04:10.465 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.z.service 2024-03-12T05:04:10.468 INFO:tasks.cephadm:Deploying OSDs... 2024-03-12T05:04:10.468 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:04:10.468 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T05:04:10.484 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T05:04:10.484 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-03-12T05:04:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-12 05:03:13.499393651 +0000 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-12 04:57:01.777978207 +0000 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-12 04:57:01.777978207 +0000 2024-03-12T05:04:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-12 04:57:01.777978207 +0000 2024-03-12T05:04:10.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T05:04:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-12T05:04:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-12T05:04:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000186075 s, 2.8 MB/s 2024-03-12T05:04:10.607 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T05:04:10.662 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-03-12T05:04:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T05:04:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:10.717 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-12T05:04:10.718 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:10.718 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:10.718 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-12 05:03:13.531392836 +0000 2024-03-12T05:04:10.718 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-12 04:57:02.118969209 +0000 2024-03-12T05:04:10.718 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-12 04:57:02.118969209 +0000 2024-03-12T05:04:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-12 04:57:02.118969209 +0000 2024-03-12T05:04:10.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T05:04:10.785 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-12T05:04:10.785 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-12T05:04:10.785 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.00021213 s, 2.4 MB/s 2024-03-12T05:04:10.786 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T05:04:10.843 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-03-12T05:04:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T05:04:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:10.900 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-12T05:04:10.900 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:10.900 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:10.900 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-12 05:03:13.561392072 +0000 2024-03-12T05:04:10.901 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-12 04:57:02.460960183 +0000 2024-03-12T05:04:10.901 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-12 04:57:02.460960183 +0000 2024-03-12T05:04:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-12 04:57:02.460960183 +0000 2024-03-12T05:04:10.901 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T05:04:10.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:10 smithi156 ceph-mon[36270]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi156=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:10.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:10 smithi156 ceph-mon[36270]: Saving service mgr spec with placement smithi100=x;smithi156=y;smithi161=z;count:3 2024-03-12T05:04:10.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:10 smithi156 ceph-mon[36270]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:10.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:10.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:10 smithi100 ceph-mon[32210]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi156=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:10.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:10 smithi100 ceph-mon[32210]: Saving service mgr spec with placement smithi100=x;smithi156=y;smithi161=z;count:3 2024-03-12T05:04:10.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:10 smithi100 ceph-mon[32210]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:10.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:10.966 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-12T05:04:10.966 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-12T05:04:10.966 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000185827 s, 2.8 MB/s 2024-03-12T05:04:10.967 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T05:04:10.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:10 smithi161 ceph-mon[36043]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi156=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:10.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:10 smithi161 ceph-mon[36043]: Saving service mgr spec with placement smithi100=x;smithi156=y;smithi161=z;count:3 2024-03-12T05:04:10.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:10 smithi161 ceph-mon[36043]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:10.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:10.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:11.023 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-12 05:03:13.590391334 +0000 2024-03-12T05:04:11.079 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-12 04:57:02.805951079 +0000 2024-03-12T05:04:11.080 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-12 04:57:02.805951079 +0000 2024-03-12T05:04:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-12 04:57:02.805951079 +0000 2024-03-12T05:04:11.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T05:04:11.145 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-12T05:04:11.145 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-12T05:04:11.145 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000195749 s, 2.6 MB/s 2024-03-12T05:04:11.146 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T05:04:11.201 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-12T05:04:11.201 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T05:04:11.220 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T05:04:11.220 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-03-12T05:04:11.278 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T05:04:11.278 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:11.278 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-12T05:04:11.278 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:11.278 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:11.279 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-12 05:03:56.460307911 +0000 2024-03-12T05:04:11.279 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-12 04:57:48.293639874 +0000 2024-03-12T05:04:11.279 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-12 04:57:48.293639874 +0000 2024-03-12T05:04:11.279 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-12 04:57:48.293639874 +0000 2024-03-12T05:04:11.279 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T05:04:11.348 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-12T05:04:11.348 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-12T05:04:11.348 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000107507 s, 4.8 MB/s 2024-03-12T05:04:11.349 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T05:04:11.408 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-03-12T05:04:11.464 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T05:04:11.464 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:11.464 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-03-12T05:04:11.465 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:11.465 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:11.465 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-12 05:03:56.460307911 +0000 2024-03-12T05:04:11.465 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-12 04:57:48.619634665 +0000 2024-03-12T05:04:11.465 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-12 04:57:48.619634665 +0000 2024-03-12T05:04:11.465 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-12 04:57:48.619634665 +0000 2024-03-12T05:04:11.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T05:04:11.532 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-12T05:04:11.532 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-12T05:04:11.532 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00010511 s, 4.9 MB/s 2024-03-12T05:04:11.533 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T05:04:11.590 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-03-12T05:04:11.646 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T05:04:11.646 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-12 05:03:56.460307911 +0000 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-12 04:57:48.964629152 +0000 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-12 04:57:48.964629152 +0000 2024-03-12T05:04:11.647 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-12 04:57:48.964629152 +0000 2024-03-12T05:04:11.647 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T05:04:11.714 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-12T05:04:11.714 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-12T05:04:11.714 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 9.7662e-05 s, 5.2 MB/s 2024-03-12T05:04:11.715 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:11.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:11.772 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-12 05:03:56.461307896 +0000 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-12 04:57:49.307623671 +0000 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-12 04:57:49.307623671 +0000 2024-03-12T05:04:11.828 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-12 04:57:49.307623671 +0000 2024-03-12T05:04:11.829 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:11.893 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:11.895 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-12T05:04:11.895 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-12T05:04:11.895 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00012073 s, 4.2 MB/s 2024-03-12T05:04:11.896 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T05:04:11.952 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-12T05:04:11.952 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T05:04:11.968 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T05:04:11.968 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-12 05:03:46.892482825 +0000 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-12 04:57:16.602071177 +0000 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-12 04:57:16.602071177 +0000 2024-03-12T05:04:12.024 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-12 04:57:16.602071177 +0000 2024-03-12T05:04:12.024 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T05:04:12.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:12.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:12.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:12.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:12.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T05:04:12.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:12.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:12.090 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-12T05:04:12.090 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-12T05:04:12.090 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000267347 s, 1.9 MB/s 2024-03-12T05:04:12.092 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T05:04:12.148 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-03-12T05:04:12.203 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-12 05:03:46.893482803 +0000 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-12 04:57:16.933063405 +0000 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-12 04:57:16.933063405 +0000 2024-03-12T05:04:12.204 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-12 04:57:16.933063405 +0000 2024-03-12T05:04:12.204 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T05:04:12.270 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-12T05:04:12.270 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-12T05:04:12.271 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00021368 s, 2.4 MB/s 2024-03-12T05:04:12.272 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T05:04:12.329 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-12 05:03:46.893482803 +0000 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-12 04:57:17.284055163 +0000 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-12 04:57:17.284055163 +0000 2024-03-12T05:04:12.385 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-12 04:57:17.284055163 +0000 2024-03-12T05:04:12.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T05:04:12.497 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-12T05:04:12.497 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-12T05:04:12.497 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000185637 s, 2.8 MB/s 2024-03-12T05:04:12.498 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T05:04:12.513 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-03-12T05:04:12.571 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T05:04:12.571 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-12 05:04:12.425944058 +0000 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-12 04:57:17.618047321 +0000 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-12 04:57:17.618047321 +0000 2024-03-12T05:04:12.572 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-12 04:57:17.618047321 +0000 2024-03-12T05:04:12.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T05:04:12.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:12 smithi161 ceph-mon[36043]: Deploying daemon mgr.z on smithi161 2024-03-12T05:04:12.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:12 smithi161 ceph-mon[36043]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:12.707 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-12T05:04:12.707 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-12T05:04:12.707 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000185677 s, 2.8 MB/s 2024-03-12T05:04:12.708 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T05:04:12.724 INFO:tasks.cephadm:Deploying osd.0 on smithi100 with /dev/vg_nvme/lv_4... 2024-03-12T05:04:12.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-12T05:04:12.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:12 smithi156 ceph-mon[36270]: Deploying daemon mgr.z on smithi161 2024-03-12T05:04:12.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:12 smithi156 ceph-mon[36270]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:12.970 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:04:12.977 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:12 smithi161 podman[38051]: 2024-03-12 05:04:12.832435491 +0000 UTC m=+0.017352077 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:04:13.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:12 smithi100 ceph-mon[32210]: Deploying daemon mgr.z on smithi161 2024-03-12T05:04:13.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:12 smithi100 ceph-mon[32210]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:13.477 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 podman[38051]: 2024-03-12 05:04:13.151668237 +0000 UTC m=+0.336584817 container init a29f183d81ad0e4e2cda2d0360e755f9cba7c094ead92839f9f048b596564a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-12T05:04:13.477 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 podman[38051]: 2024-03-12 05:04:13.154245081 +0000 UTC m=+0.339161660 container start a29f183d81ad0e4e2cda2d0360e755f9cba7c094ead92839f9f048b596564a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-12T05:04:13.477 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 ceph-mgr[38071]: -- 172.21.15.161:0/2771992722 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56103784a1a0 con 0x561037820800 2024-03-12T05:04:13.477 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 bash[38051]: a29f183d81ad0e4e2cda2d0360e755f9cba7c094ead92839f9f048b596564a7a 2024-03-12T05:04:13.477 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 systemd[1]: Started Ceph mgr.z for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:04:13.809 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:13.712+0000 7f7e88938200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:04:13.809 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:13.806+0000 7f7e88938200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:14.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: Deploying daemon mgr.y on smithi156 2024-03-12T05:04:14.564 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-mon[36043]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:14.564 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:14.465+0000 7f7e88938200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:04:14.564 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:14.562+0000 7f7e88938200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: Deploying daemon mgr.y on smithi156 2024-03-12T05:04:14.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:14 smithi100 ceph-mon[32210]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: Deploying daemon mgr.y on smithi156 2024-03-12T05:04:14.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:14 smithi156 ceph-mon[36270]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:14.862 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-12T05:04:14.862 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-12T05:04:14.863 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-12T05:04:14.863 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-12T05:04:14.863 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111991 s, 936 MB/s 2024-03-12T05:04:14.863 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 2024-03-12T05:04:14.863 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-12T05:04:14.865 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:14.710+0000 7f7e88938200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:04:14.865 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:14.810+0000 7f7e88938200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:04:15.171 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:14.896+0000 7f7e88938200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:04:15.171 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:15.170+0000 7f7e88938200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:04:15.435 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-03-12T05:04:15.459 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 systemd[1]: Starting Ceph mgr.y for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:04:15.467 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:15.282+0000 7f7e88938200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:04:15.467 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:15.369+0000 7f7e88938200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:04:15.467 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:15.466+0000 7f7e88938200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:04:15.671 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:04:15.726 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:15.647+0000 7f7e88938200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:04:15.747 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 podman[37633]: 2024-03-12 05:04:15.477347637 +0000 UTC m=+0.108158732 container create 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-12T05:04:15.747 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 podman[37633]: 2024-03-12 05:04:15.386343299 +0000 UTC m=+0.017154415 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:04:15.747 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 podman[37633]: 2024-03-12 05:04:15.662793941 +0000 UTC m=+0.293605042 container init 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-12T05:04:15.747 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 podman[37633]: 2024-03-12 05:04:15.665354732 +0000 UTC m=+0.296165840 container start 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-12T05:04:15.747 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 ceph-mgr[37652]: -- 172.21.15.156:0/366672511 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623cf5301a0 con 0x5623cf506c00 2024-03-12T05:04:15.748 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 bash[37633]: 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 2024-03-12T05:04:16.208 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:15 smithi156 systemd[1]: Started Ceph mgr.y for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:04:16.708 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:16.227+0000 7fb0cb127200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:04:16.709 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:16.323+0000 7fb0cb127200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:04:16.977 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:16.674+0000 7f7e88938200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:04:16.977 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:16.806+0000 7f7e88938200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:04:16.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-mon[36043]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:16.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:16.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:16.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:16.978 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:16.978 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:17.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-mon[36270]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:17.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:17.016 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:16 smithi100 ceph-mon[32210]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:17.016 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:17.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:17.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 -- 172.21.15.100:0/56582733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc103840 msgr2=0x7f86bc10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:17.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/56582733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc103840 0x7f86bc10a0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86b400b0d0 tx=0x7f86b400b3e0 comp rx=0 tx=0).stop 2024-03-12T05:04:17.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 -- 172.21.15.100:0/56582733 shutdown_connections 2024-03-12T05:04:17.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/56582733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc103840 0x7f86bc10a0f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/56582733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86bc101ca0 0x7f86bc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/56582733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc102ea0 0x7f86bc103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 -- 172.21.15.100:0/56582733 >> 172.21.15.100:0/56582733 conn(0x7f86bc0fd280 msgr2=0x7f86bc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 -- 172.21.15.100:0/56582733 shutdown_connections 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 -- 172.21.15.100:0/56582733 wait complete. 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.043+0000 7f86c1ae8700 1 Processor -- start 2024-03-12T05:04:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 -- start start 2024-03-12T05:04:17.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101ca0 0x7f86bc075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 0x7f86bc074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86bc103840 0x7f86bc0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc074d20 con 0x7f86bc102ea0 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc074e90 con 0x7f86bc101ca0 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86c1ae8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1a1d80 con 0x7f86bc103840 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 0x7f86bc074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 0x7f86bc074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52430/0 (socket says 172.21.15.100:52430) 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 -- 172.21.15.100:0/3189444377 learned_addr learned my addr 172.21.15.100:0/3189444377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86bbfff700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86bc103840 0x7f86bc0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86bb7fe700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101ca0 0x7f86bc075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 -- 172.21.15.100:0/3189444377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86bc103840 msgr2=0x7f86bc0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86bc103840 0x7f86bc0745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 -- 172.21.15.100:0/3189444377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101ca0 msgr2=0x7f86bc075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101ca0 0x7f86bc075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 -- 172.21.15.100:0/3189444377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009d00 con 0x7f86bc102ea0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86baffd700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 0x7f86bc074060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86b000d950 tx=0x7f86b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.044+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000f890 con 0x7f86bc102ea0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.045+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86b0010460 con 0x7f86bc102ea0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.045+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc1a2060 con 0x7f86bc102ea0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.045+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000e610 con 0x7f86bc102ea0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.045+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc1a2560 con 0x7f86bc102ea0 2024-03-12T05:04:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.045+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f86b000f9f0 con 0x7f86bc102ea0 2024-03-12T05:04:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.046+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc109000 con 0x7f86bc102ea0 2024-03-12T05:04:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.046+0000 7f86b8ff9700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f86a4038f90 0x7f86a403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.046+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f86b004b6c0 con 0x7f86bc102ea0 2024-03-12T05:04:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.046+0000 7f86bb7fe700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f86a4038f90 0x7f86a403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.046+0000 7f86bb7fe700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f86a4038f90 0x7f86a403b450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86bc102d00 tx=0x7f86ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:17.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.051+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86b0009bd0 con 0x7f86bc102ea0 2024-03-12T05:04:17.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:17.253+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f86bc0620b0 con 0x7f86a4038f90 2024-03-12T05:04:17.259 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.008+0000 7fb0cb127200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:04:17.260 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.102+0000 7fb0cb127200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:04:17.446 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:17.149+0000 7f7e88938200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:04:17.446 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:17.244+0000 7f7e88938200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:04:17.446 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:17.445+0000 7f7e88938200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:04:17.702 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.259+0000 7fb0cb127200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:04:17.702 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.355+0000 7fb0cb127200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:04:17.703 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.439+0000 7fb0cb127200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:04:17.703 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.701+0000 7fb0cb127200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:04:17.726 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:17.627+0000 7f7e88938200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:04:17.958 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.809+0000 7fb0cb127200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:04:17.958 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.894+0000 7fb0cb127200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:04:18.164 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:17.904+0000 7f7e88938200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:04:18.164 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:18.163+0000 7f7e88938200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:04:18.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:17.988+0000 7fb0cb127200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:04:18.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:18.165+0000 7fb0cb127200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:04:18.476 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:18.394+0000 7f7e88938200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: Reconfiguring mgr.x (unknown last config time)... 2024-03-12T05:04:18.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:18.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:18.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: Reconfiguring daemon mgr.x on smithi100 2024-03-12T05:04:18.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:18 smithi161 ceph-mon[36043]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:18.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: Reconfiguring mgr.x (unknown last config time)... 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: Reconfiguring daemon mgr.x on smithi100 2024-03-12T05:04:18.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:18 smithi100 ceph-mon[32210]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:18.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: Reconfiguring mgr.x (unknown last config time)... 2024-03-12T05:04:18.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T05:04:18.710 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T05:04:18.710 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:18.710 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: Reconfiguring daemon mgr.x on smithi100 2024-03-12T05:04:18.710 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:18 smithi156 ceph-mon[36270]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:19.458 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:19.138+0000 7fb0cb127200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:04:19.459 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:19.267+0000 7fb0cb127200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:04:19.476 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:19.179+0000 7f7e88938200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:04:19.896 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:19 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:19.896 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:19.609+0000 7fb0cb127200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:04:19.896 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:19.700+0000 7fb0cb127200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:04:19.896 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:19.894+0000 7fb0cb127200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:04:19.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:19 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:19.977 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:19.746+0000 7f7e88938200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:04:19.977 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:19.841+0000 7f7e88938200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:04:20.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:19 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:20.208 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:20.079+0000 7fb0cb127200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:04:20.476 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:04:20.053+0000 7f7e88938200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:04:20.627 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:20.362+0000 7fb0cb127200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:04:20.627 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:20.625+0000 7fb0cb127200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:04:20.958 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:20.861+0000 7fb0cb127200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3033048986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3ee95d-798a-4fd9-82ef-81f981a5461d"}]: dispatch 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3033048986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3ee95d-798a-4fd9-82ef-81f981a5461d"}]': finished 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: osdmap e5: 1 total, 0 up, 1 in 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: Standby manager daemon z started 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T05:04:20.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:04:20.960 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2043062806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:04:20.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:20.968+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 14) v1 ==== 91025+0+0 (secure 0 0 0) 0x7f86b0010960 con 0x7f86bc102ea0 2024-03-12T05:04:20.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:20.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3033048986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3ee95d-798a-4fd9-82ef-81f981a5461d"}]: dispatch 2024-03-12T05:04:20.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3033048986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3ee95d-798a-4fd9-82ef-81f981a5461d"}]': finished 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: osdmap e5: 1 total, 0 up, 1 in 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: Standby manager daemon z started 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:04:20.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2043062806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:04:20.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3033048986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3ee95d-798a-4fd9-82ef-81f981a5461d"}]: dispatch 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3033048986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3ee95d-798a-4fd9-82ef-81f981a5461d"}]': finished 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: osdmap e5: 1 total, 0 up, 1 in 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: Standby manager daemon z started 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3210525295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:04:20.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2043062806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:04:21.958 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:21.665+0000 7fb0cb127200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:04:22.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:21 smithi161 ceph-mon[36043]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-12T05:04:22.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:21 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T05:04:22.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:21 smithi161 ceph-mon[36043]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:22.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:21 smithi156 ceph-mon[36270]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-12T05:04:22.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:21 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T05:04:22.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:21 smithi156 ceph-mon[36270]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:22.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:21 smithi100 ceph-mon[32210]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-12T05:04:22.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:21 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T05:04:22.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:21 smithi100 ceph-mon[32210]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:22.556 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:22.242+0000 7fb0cb127200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:04:22.556 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:22.340+0000 7fb0cb127200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:04:22.556 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:04:22.554+0000 7fb0cb127200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:04:22.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:22.983+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7f86b0019d20 con 0x7f86bc102ea0 2024-03-12T05:04:23.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:22 smithi161 ceph-mon[36043]: Standby manager daemon y started 2024-03-12T05:04:23.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:22 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T05:04:23.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:22 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:04:23.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:22 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T05:04:23.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:22 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:04:23.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:22 smithi100 ceph-mon[32210]: Standby manager daemon y started 2024-03-12T05:04:23.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:22 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T05:04:23.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:22 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:04:23.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:22 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T05:04:23.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:22 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:04:23.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-mon[36270]: Standby manager daemon y started 2024-03-12T05:04:23.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T05:04:23.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:04:23.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T05:04:23.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:22 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/602551567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:04:24.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:23 smithi100 ceph-mon[32210]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-12T05:04:24.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T05:04:24.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:23 smithi100 ceph-mon[32210]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:24.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:23 smithi156 ceph-mon[36270]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-12T05:04:24.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T05:04:24.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:23 smithi156 ceph-mon[36270]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:24.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:23 smithi161 ceph-mon[36043]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-12T05:04:24.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T05:04:24.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:23 smithi161 ceph-mon[36043]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:26 smithi100 ceph-mon[32210]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:26.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:26.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:26.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:26.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:26 smithi156 ceph-mon[36270]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:26.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:26.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:26.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:26.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:26.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:26 smithi161 ceph-mon[36043]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:28 smithi156 ceph-mon[36270]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:28.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T05:04:28.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:28.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:28 smithi156 ceph-mon[36270]: Deploying daemon osd.0 on smithi100 2024-03-12T05:04:28.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:28 smithi161 ceph-mon[36043]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:28.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T05:04:28.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:28.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:28 smithi161 ceph-mon[36043]: Deploying daemon osd.0 on smithi100 2024-03-12T05:04:29.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:28 smithi100 ceph-mon[32210]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:29.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T05:04:29.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:29.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:28 smithi100 ceph-mon[32210]: Deploying daemon osd.0 on smithi100 2024-03-12T05:04:30.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:30 smithi156 ceph-mon[36270]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:30.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:30 smithi161 ceph-mon[36043]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:31.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:30 smithi100 ceph-mon[32210]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:32.962 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:32 smithi156 ceph-mon[36270]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:32.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:32 smithi161 ceph-mon[36043]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:33.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:32 smithi100 ceph-mon[32210]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:34.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:33 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:34.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:33 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:34.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:33 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:34.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:33 smithi100 ceph-mon[32210]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:34.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:33 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:34.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:33 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:34.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:33 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:34.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:33 smithi156 ceph-mon[36270]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:34.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:33 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:34.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:33 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:34.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:33 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:34.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:33 smithi161 ceph-mon[36043]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:36.418 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 0 on host 'smithi100' 2024-03-12T05:04:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.416+0000 7f86b8ff9700 1 -- 172.21.15.100:0/3189444377 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86bc0620b0 con 0x7f86a4038f90 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f86a4038f90 msgr2=0x7f86a403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f86a4038f90 0x7f86a403b450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86bc102d00 tx=0x7f86ac00b410 comp rx=0 tx=0).stop 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 msgr2=0x7f86bc074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 0x7f86bc074060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86b000d950 tx=0x7f86b000dc60 comp rx=0 tx=0).stop 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 shutdown_connections 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f86a4038f90 0x7f86a403b450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101ca0 0x7f86bc075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86bc103840 0x7f86bc0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 --2- 172.21.15.100:0/3189444377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc102ea0 0x7f86bc074060 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 >> 172.21.15.100:0/3189444377 conn(0x7f86bc0fd280 msgr2=0x7f86bc106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 shutdown_connections 2024-03-12T05:04:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:04:36.419+0000 7f86c1ae8700 1 -- 172.21.15.100:0/3189444377 wait complete. 2024-03-12T05:04:36.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:36 smithi156 ceph-mon[36270]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:36.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:36 smithi161 ceph-mon[36043]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:36.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:36.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:37.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:36 smithi100 ceph-mon[32210]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:37.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:37.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:37.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:37.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:37.076 DEBUG:teuthology.orchestra.run.smithi100:osd.0> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.0.service 2024-03-12T05:04:37.079 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_4... 2024-03-12T05:04:37.080 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-12T05:04:37.324 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:04:37.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:37 smithi156 ceph-mon[36270]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T05:04:37.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:37 smithi161 ceph-mon[36043]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T05:04:38.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:37 smithi100 ceph-mon[32210]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T05:04:38.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-mon[32210]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:38.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-mon[32210]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T05:04:38.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-mon[32210]: osdmap e6: 1 total, 0 up, 1 in 2024-03-12T05:04:38.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-mon[32210]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:04:38.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:38.822 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0[44602]: 2024-03-12T05:04:38.722+0000 7f0877e06700 -1 osd.0 0 waiting for initial osdmap 2024-03-12T05:04:38.822 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:04:38 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0[44602]: 2024-03-12T05:04:38.728+0000 7f08743fa700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:04:38.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:38 smithi161 ceph-mon[36043]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:38.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:38 smithi161 ceph-mon[36043]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T05:04:38.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:38 smithi161 ceph-mon[36043]: osdmap e6: 1 total, 0 up, 1 in 2024-03-12T05:04:38.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:38 smithi161 ceph-mon[36043]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:04:38.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00830907 s, 1.3 GB/s 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 2024-03-12T05:04:39.167 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-12T05:04:39.168 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:38 smithi156 ceph-mon[36270]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:39.168 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:38 smithi156 ceph-mon[36270]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T05:04:39.168 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:38 smithi156 ceph-mon[36270]: osdmap e6: 1 total, 0 up, 1 in 2024-03-12T05:04:39.168 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:38 smithi156 ceph-mon[36270]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:04:39.168 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:39.762 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2024-03-12T05:04:39.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:39 smithi100 ceph-mon[32210]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:04:39.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:39 smithi100 ceph-mon[32210]: osdmap e7: 1 total, 0 up, 1 in 2024-03-12T05:04:39.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:39.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:04:40.136 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:39 smithi156 ceph-mon[36270]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:04:40.136 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:39 smithi156 ceph-mon[36270]: osdmap e7: 1 total, 0 up, 1 in 2024-03-12T05:04:40.136 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:40.136 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:40.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:39 smithi161 ceph-mon[36043]: from='osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:04:40.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:39 smithi161 ceph-mon[36043]: osdmap e7: 1 total, 0 up, 1 in 2024-03-12T05:04:40.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:40.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] boot 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: osdmap e8: 1 total, 1 up, 1 in 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi100 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:04:41.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:41.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] boot 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: osdmap e8: 1 total, 1 up, 1 in 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi100 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:04:41.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:41.073 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: osd.0 [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] boot 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: osdmap e8: 1 total, 1 up, 1 in 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi100 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:41.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:41.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:41.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:41.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:04:41.228 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 -- 172.21.15.156:0/3126954267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc1079f0 msgr2=0x7fdacc109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 --2- 172.21.15.156:0/3126954267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc1079f0 0x7fdacc109e00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdac400b0a0 tx=0x7fdac400b3b0 comp rx=0 tx=0).stop 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 -- 172.21.15.156:0/3126954267 shutdown_connections 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 --2- 172.21.15.156:0/3126954267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc1079f0 0x7fdacc109e00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 --2- 172.21.15.156:0/3126954267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc102790 0x7fdacc104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 --2- 172.21.15.156:0/3126954267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdacc1050c0 0x7fdacc1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 -- 172.21.15.156:0/3126954267 >> 172.21.15.156:0/3126954267 conn(0x7fdacc0fc160 msgr2=0x7fdacc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 -- 172.21.15.156:0/3126954267 shutdown_connections 2024-03-12T05:04:41.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 -- 172.21.15.156:0/3126954267 wait complete. 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 Processor -- start 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.438+0000 7fdad24ae700 1 -- start start 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 0x7fdacc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdacc1050c0 0x7fdacc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc1079f0 0x7fdacc19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc19f970 con 0x7fdacc1050c0 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc19fab0 con 0x7fdacc102790 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc19fc40 con 0x7fdacc1079f0 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 0x7fdacc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 0x7fdacc197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50026/0 (socket says 172.21.15.156:50026) 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 -- 172.21.15.156:0/756506996 learned_addr learned my addr 172.21.15.156:0/756506996 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad0a4b700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc1079f0 0x7fdacc19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:41.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 -- 172.21.15.156:0/756506996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc1079f0 msgr2=0x7fdacc19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc1079f0 0x7fdacc19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 -- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdacc1050c0 msgr2=0x7fdacc198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdacc1050c0 0x7fdacc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 -- 172.21.15.156:0/756506996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac4009d00 con 0x7fdacc102790 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad0a4b700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc1079f0 0x7fdacc19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdacbfff700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 0x7fdacc197de0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d950 tx=0x7fdabc00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc009b40 con 0x7fdacc102790 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdacc19ff20 con 0x7fdacc102790 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdabc009ca0 con 0x7fdacc102790 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.439+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00f690 con 0x7fdacc102790 2024-03-12T05:04:41.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.440+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdacc1a0410 con 0x7fdacc102790 2024-03-12T05:04:41.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.441+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdacc0fdce0 con 0x7fdacc102790 2024-03-12T05:04:41.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.441+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7fdabc010460 con 0x7fdacc102790 2024-03-12T05:04:41.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.442+0000 7fdac97fa700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdab40a1a30 0x7fdab40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:04:41.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.442+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fdabc0cb8f0 con 0x7fdacc102790 2024-03-12T05:04:41.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.442+0000 7fdacb7fe700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdab40a1a30 0x7fdab40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:04:41.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.443+0000 7fdacb7fe700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdab40a1a30 0x7fdab40a3ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdac00097b0 tx=0x7fdac0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:04:41.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.447+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdabc096570 con 0x7fdacc102790 2024-03-12T05:04:41.650 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:41.648+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdacc02d8d0 con 0x7fdab40a1a30 2024-03-12T05:04:41.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:41 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 17088M 2024-03-12T05:04:41.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:41 smithi156 ceph-mon[36270]: osdmap e9: 1 total, 1 up, 1 in 2024-03-12T05:04:41.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:41 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:04:41.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:41 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:04:41.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:41 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:42.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:41 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 17088M 2024-03-12T05:04:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:41 smithi100 ceph-mon[32210]: osdmap e9: 1 total, 1 up, 1 in 2024-03-12T05:04:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:04:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:04:42.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:42.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:41 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 17088M 2024-03-12T05:04:42.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:41 smithi161 ceph-mon[36043]: osdmap e9: 1 total, 1 up, 1 in 2024-03-12T05:04:42.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:41 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:04:42.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:41 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:04:42.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:41 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:42.882 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:42 smithi156 ceph-mon[36270]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:42.883 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:42 smithi156 ceph-mon[36270]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:43.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:42 smithi100 ceph-mon[32210]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:43.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:42 smithi100 ceph-mon[32210]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:43.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:42 smithi161 ceph-mon[36043]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:43.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:42 smithi161 ceph-mon[36043]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:04:44.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:44 smithi156 ceph-mon[36270]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:44.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/3051230202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]: dispatch 2024-03-12T05:04:44.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:44 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]: dispatch 2024-03-12T05:04:44.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:44 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]': finished 2024-03-12T05:04:44.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:44 smithi156 ceph-mon[36270]: osdmap e10: 2 total, 1 up, 2 in 2024-03-12T05:04:44.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:04:45.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:44 smithi100 ceph-mon[32210]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:45.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/3051230202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]: dispatch 2024-03-12T05:04:45.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:44 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]: dispatch 2024-03-12T05:04:45.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:44 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]': finished 2024-03-12T05:04:45.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:44 smithi100 ceph-mon[32210]: osdmap e10: 2 total, 1 up, 2 in 2024-03-12T05:04:45.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:04:45.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:44 smithi161 ceph-mon[36043]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:45.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/3051230202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]: dispatch 2024-03-12T05:04:45.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:44 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]: dispatch 2024-03-12T05:04:45.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:44 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95723563-e515-4042-ac3c-bb298e215da0"}]': finished 2024-03-12T05:04:45.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:44 smithi161 ceph-mon[36043]: osdmap e10: 2 total, 1 up, 2 in 2024-03-12T05:04:45.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:04:46.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/2459461353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:04:46.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/2459461353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:04:46.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/2459461353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:04:47.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:46 smithi100 ceph-mon[32210]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:47.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:46 smithi156 ceph-mon[36270]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:47.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:46 smithi161 ceph-mon[36043]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:49.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:48 smithi156 ceph-mon[36270]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:49.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:48 smithi100 ceph-mon[32210]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:49.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:48 smithi161 ceph-mon[36043]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:51.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:50 smithi100 ceph-mon[32210]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:51.078 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:50 smithi156 ceph-mon[36270]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:51.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:50 smithi161 ceph-mon[36043]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:52.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T05:04:52.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:52.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T05:04:52.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:52.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T05:04:52.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:04:53.053 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:52 smithi156 ceph-mon[36270]: Deploying daemon osd.1 on smithi156 2024-03-12T05:04:53.053 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:52 smithi156 ceph-mon[36270]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:52 smithi100 ceph-mon[32210]: Deploying daemon osd.1 on smithi156 2024-03-12T05:04:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:52 smithi100 ceph-mon[32210]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:53.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:52 smithi161 ceph-mon[36043]: Deploying daemon osd.1 on smithi156 2024-03-12T05:04:53.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:52 smithi161 ceph-mon[36043]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:54.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:54 smithi156 ceph-mon[36270]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:55.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:54 smithi100 ceph-mon[32210]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:55.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:54 smithi161 ceph-mon[36043]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:57.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:56 smithi100 ceph-mon[32210]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:57.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:57.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:57.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:57.105 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:56 smithi156 ceph-mon[36270]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:57.105 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:57.105 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:57.105 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:57.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:56 smithi161 ceph-mon[36043]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:57.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:57.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:04:57.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:04:59.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:04:58 smithi100 ceph-mon[32210]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:59.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:04:58 smithi156 ceph-mon[36270]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:59.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:04:58 smithi161 ceph-mon[36043]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:04:59.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.961+0000 7fdac97fa700 1 -- 172.21.15.156:0/756506996 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdacc02d8d0 con 0x7fdab40a1a30 2024-03-12T05:04:59.962 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2024-03-12T05:04:59.964 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.963+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdab40a1a30 msgr2=0x7fdab40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.963+0000 7fdad24ae700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdab40a1a30 0x7fdab40a3ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdac00097b0 tx=0x7fdac0006d20 comp rx=0 tx=0).stop 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.963+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 msgr2=0x7fdacc197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.963+0000 7fdad24ae700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 0x7fdacc197de0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d950 tx=0x7fdabc00dd10 comp rx=0 tx=0).stop 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 shutdown_connections 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdab40a1a30 0x7fdab40a3ef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdacc102790 0x7fdacc197de0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdacc1079f0 0x7fdacc19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 --2- 172.21.15.156:0/756506996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdacc1050c0 0x7fdacc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 >> 172.21.15.156:0/756506996 conn(0x7fdacc0fc160 msgr2=0x7fdacc103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 shutdown_connections 2024-03-12T05:04:59.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:04:59.964+0000 7fdad24ae700 1 -- 172.21.15.156:0/756506996 wait complete. 2024-03-12T05:05:00.215 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:00.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:00.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:00.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:00.589 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.1.service 2024-03-12T05:05:00.591 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-03-12T05:05:00.591 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-12T05:05:00.836 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:05:01.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:01 smithi161 ceph-mon[36043]: from='osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T05:05:01.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:01 smithi161 ceph-mon[36043]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T05:05:01.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:01 smithi156 ceph-mon[36270]: from='osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T05:05:01.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:01 smithi156 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T05:05:01.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:01 smithi100 ceph-mon[32210]: from='osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T05:05:01.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:01 smithi100 ceph-mon[32210]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T05:05:02.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:02 smithi161 ceph-mon[36043]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T05:05:02.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:02 smithi161 ceph-mon[36043]: from='osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:05:02.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:02 smithi161 ceph-mon[36043]: osdmap e11: 2 total, 1 up, 2 in 2024-03-12T05:05:02.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:02.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:02 smithi161 ceph-mon[36043]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:05:02.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:02 smithi161 ceph-mon[36043]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:02.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T05:05:02.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-mon[36270]: from='osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:05:02.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-mon[36270]: osdmap e11: 2 total, 1 up, 2 in 2024-03-12T05:05:02.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:02.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:05:02.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-mon[36270]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:02.520 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-12T05:05:02.520 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-12T05:05:02.520 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-12T05:05:02.520 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-12T05:05:02.520 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113232 s, 926 MB/s 2024-03-12T05:05:02.521 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-12T05:05:02.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:02 smithi100 ceph-mon[32210]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T05:05:02.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:02 smithi100 ceph-mon[32210]: from='osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:05:02.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:02 smithi100 ceph-mon[32210]: osdmap e11: 2 total, 1 up, 2 in 2024-03-12T05:05:02.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:02.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:02 smithi100 ceph-mon[32210]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:05:02.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:02 smithi100 ceph-mon[32210]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:03.039 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1[40892]: 2024-03-12T05:05:02.856+0000 7f0ddbc4a700 -1 osd.1 0 waiting for initial osdmap 2024-03-12T05:05:03.039 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1[40892]: 2024-03-12T05:05:02.860+0000 7f0dd4236700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:05:03.127 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-03-12T05:05:03.306 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:03 smithi156 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:05:03.306 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:03 smithi156 ceph-mon[36270]: osdmap e12: 2 total, 1 up, 2 in 2024-03-12T05:05:03.307 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.307 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.307 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:03 smithi156 ceph-mon[36270]: from='osd.1 ' entity='osd.1' 2024-03-12T05:05:03.307 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:05:03.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:03 smithi161 ceph-mon[36043]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:05:03.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:03 smithi161 ceph-mon[36043]: osdmap e12: 2 total, 1 up, 2 in 2024-03-12T05:05:03.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:03 smithi161 ceph-mon[36043]: from='osd.1 ' entity='osd.1' 2024-03-12T05:05:03.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:03 smithi100 ceph-mon[32210]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:05:03.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:03 smithi100 ceph-mon[32210]: osdmap e12: 2 total, 1 up, 2 in 2024-03-12T05:05:03.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:03.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:03 smithi100 ceph-mon[32210]: from='osd.1 ' entity='osd.1' 2024-03-12T05:05:03.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:04.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:05:04.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:05:04.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi156 2024-03-12T05:05:04.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 17080M 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] boot 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: osdmap e13: 2 total, 2 up, 2 in 2024-03-12T05:05:04.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:05:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:05:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi156 2024-03-12T05:05:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 17080M 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] boot 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: osdmap e13: 2 total, 2 up, 2 in 2024-03-12T05:05:04.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:05:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:05:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi156 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 17080M 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: osd.1 [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] boot 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: osdmap e13: 2 total, 2 up, 2 in 2024-03-12T05:05:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:05:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 -- 172.21.15.161:0/1086861965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74102ea0 msgr2=0x7f5e74103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1086861965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74102ea0 0x7f5e74103300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009990 tx=0x7f5e64009ca0 comp rx=0 tx=0).stop 2024-03-12T05:05:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 -- 172.21.15.161:0/1086861965 shutdown_connections 2024-03-12T05:05:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1086861965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e74101ca0 0x7f5e741020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:04.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1086861965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74103840 0x7f5e7410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1086861965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74102ea0 0x7f5e74103300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 -- 172.21.15.161:0/1086861965 >> 172.21.15.161:0/1086861965 conn(0x7f5e740fd280 msgr2=0x7f5e740ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 -- 172.21.15.161:0/1086861965 shutdown_connections 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 -- 172.21.15.161:0/1086861965 wait complete. 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.691+0000 7f5e7c583700 1 Processor -- start 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 -- start start 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74101ca0 0x7f5e74197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e74102ea0 0x7f5e74198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 0x7f5e7419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7419f7a0 con 0x7f5e74101ca0 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7419f8e0 con 0x7f5e74102ea0 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7419faa0 con 0x7f5e74103840 2024-03-12T05:05:04.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 0x7f5e7419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 0x7f5e7419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48450/0 (socket says 172.21.15.161:48450) 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 -- 172.21.15.161:0/1003573475 learned_addr learned my addr 172.21.15.161:0/1003573475 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e79b1e700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e74102ea0 0x7f5e74198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7a31f700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74101ca0 0x7f5e74197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 -- 172.21.15.161:0/1003573475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e74102ea0 msgr2=0x7f5e74198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e74102ea0 0x7f5e74198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 -- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74101ca0 msgr2=0x7f5e74197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74101ca0 0x7f5e74197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 -- 172.21.15.161:0/1003573475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640095f0 con 0x7f5e74103840 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7ab20700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 0x7f5e7419f1d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c007ee0 tx=0x7f5e6c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c017070 con 0x7f5e74103840 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e6c00f040 con 0x7f5e74103840 2024-03-12T05:05:04.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.692+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7419fd80 con 0x7f5e74103840 2024-03-12T05:05:04.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.693+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c005650 con 0x7f5e74103840 2024-03-12T05:05:04.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.693+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e741a01c0 con 0x7f5e74103840 2024-03-12T05:05:04.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.694+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e74109130 con 0x7f5e74103840 2024-03-12T05:05:04.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.695+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7f5e6c004cd0 con 0x7f5e74103840 2024-03-12T05:05:04.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.695+0000 7f5e6b7fe700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5e600a1ab0 0x7f5e600a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:04.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.695+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f5e6c0cb830 con 0x7f5e74103840 2024-03-12T05:05:04.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.695+0000 7f5e7a31f700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5e600a1ab0 0x7f5e600a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:04.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.696+0000 7f5e7a31f700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5e600a1ab0 0x7f5e600a3f70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e74199130 tx=0x7f5e70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:05:04.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.699+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e6c096290 con 0x7f5e74103840 2024-03-12T05:05:04.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:04.897+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e7402da20 con 0x7f5e600a1ab0 2024-03-12T05:05:05.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:05.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:05.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:05.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:05 smithi161 ceph-mon[36043]: osdmap e14: 2 total, 2 up, 2 in 2024-03-12T05:05:05.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:05.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:05.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:05.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:05 smithi156 ceph-mon[36270]: osdmap e14: 2 total, 2 up, 2 in 2024-03-12T05:05:05.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:05.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:05.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:05.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:05 smithi100 ceph-mon[32210]: osdmap e14: 2 total, 2 up, 2 in 2024-03-12T05:05:06.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:06 smithi161 ceph-mon[36043]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:06.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:06 smithi161 ceph-mon[36043]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:06.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:06 smithi156 ceph-mon[36270]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:06.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:06 smithi156 ceph-mon[36270]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:06.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:06 smithi100 ceph-mon[32210]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:06.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:06 smithi100 ceph-mon[32210]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:07.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:07 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/2250086713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]: dispatch 2024-03-12T05:05:07.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:07 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]: dispatch 2024-03-12T05:05:07.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:07 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]': finished 2024-03-12T05:05:07.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:07 smithi156 ceph-mon[36270]: osdmap e15: 3 total, 2 up, 3 in 2024-03-12T05:05:07.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:07 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:07.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:07 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/2250086713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]: dispatch 2024-03-12T05:05:07.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:07 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]: dispatch 2024-03-12T05:05:07.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:07 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]': finished 2024-03-12T05:05:07.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:07 smithi161 ceph-mon[36043]: osdmap e15: 3 total, 2 up, 3 in 2024-03-12T05:05:07.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:07 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:07.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:07 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/2250086713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]: dispatch 2024-03-12T05:05:07.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:07 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]: dispatch 2024-03-12T05:05:07.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:07 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79302fd1-469d-440f-b520-378f0f3a459b"}]': finished 2024-03-12T05:05:07.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:07 smithi100 ceph-mon[32210]: osdmap e15: 3 total, 2 up, 3 in 2024-03-12T05:05:07.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:07 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:08.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:08 smithi156 ceph-mon[36270]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:08.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:08 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/1458962278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:08.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:08 smithi161 ceph-mon[36043]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:08.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:08 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/1458962278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:08.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:08 smithi100 ceph-mon[32210]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:08.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:08 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/1458962278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:10.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:10 smithi161 ceph-mon[36043]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:11.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:10 smithi100 ceph-mon[32210]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:10 smithi156 ceph-mon[36270]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:12.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:12 smithi161 ceph-mon[36043]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:13.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:12 smithi100 ceph-mon[32210]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:13.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:12 smithi156 ceph-mon[36270]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:14.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:14 smithi161 ceph-mon[36043]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:15.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:14 smithi100 ceph-mon[32210]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:15.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:14 smithi156 ceph-mon[36270]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:15.783 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T05:05:15.783 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:16.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T05:05:16.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:16.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T05:05:16.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:16.855 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:16 smithi161 ceph-mon[36043]: Deploying daemon osd.2 on smithi161 2024-03-12T05:05:16.855 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:16 smithi161 ceph-mon[36043]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:17.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:16 smithi100 ceph-mon[32210]: Deploying daemon osd.2 on smithi161 2024-03-12T05:05:17.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:16 smithi100 ceph-mon[32210]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:17.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:16 smithi156 ceph-mon[36270]: Deploying daemon osd.2 on smithi161 2024-03-12T05:05:17.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:16 smithi156 ceph-mon[36270]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:19.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:18 smithi100 ceph-mon[32210]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:19.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:18 smithi161 ceph-mon[36043]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:19.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:18 smithi156 ceph-mon[36270]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:20.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:20 smithi161 ceph-mon[36043]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:20.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:20.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:20.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:20 smithi100 ceph-mon[32210]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:21.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:20 smithi156 ceph-mon[36270]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:21.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:21.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:21.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:22.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:22 smithi161 ceph-mon[36043]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:23.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:22 smithi156 ceph-mon[36270]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:23.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:22 smithi100 ceph-mon[32210]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:23.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.260+0000 7f5e6b7fe700 1 -- 172.21.15.161:0/1003573475 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5e7402da20 con 0x7f5e600a1ab0 2024-03-12T05:05:23.261 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5e600a1ab0 msgr2=0x7f5e600a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5e600a1ab0 0x7f5e600a3f70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e74199130 tx=0x7f5e70009260 comp rx=0 tx=0).stop 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 msgr2=0x7f5e7419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 0x7f5e7419f1d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c007ee0 tx=0x7f5e6c00d3b0 comp rx=0 tx=0).stop 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 shutdown_connections 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5e600a1ab0 0x7f5e600a3f70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103840 0x7f5e7419f1d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74101ca0 0x7f5e74197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 --2- 172.21.15.161:0/1003573475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e74102ea0 0x7f5e74198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:23.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 >> 172.21.15.161:0/1003573475 conn(0x7f5e740fd280 msgr2=0x7f5e74106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:05:23.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 shutdown_connections 2024-03-12T05:05:23.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:05:23.262+0000 7f5e7c583700 1 -- 172.21.15.161:0/1003573475 wait complete. 2024-03-12T05:05:23.825 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.2.service 2024-03-12T05:05:23.828 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_3... 2024-03-12T05:05:23.828 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-12T05:05:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:05:24.226 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2[41374]: 2024-03-12T05:05:23.885+0000 7f31376bb7c0 -1 osd.2 0 log_to_monitors true 2024-03-12T05:05:24.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:24.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: from='osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T05:05:24.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T05:05:24.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: from='osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: from='osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:24.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:25.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:25 smithi100 ceph-mon[32210]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T05:05:25.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:25 smithi100 ceph-mon[32210]: osdmap e16: 3 total, 2 up, 3 in 2024-03-12T05:05:25.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:25 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:25.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:25 smithi100 ceph-mon[32210]: from='osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:05:25.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:25 smithi100 ceph-mon[32210]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:05:25.476 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2[41374]: 2024-03-12T05:05:25.255+0000 7f312c51f700 -1 osd.2 0 waiting for initial osdmap 2024-03-12T05:05:25.476 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2[41374]: 2024-03-12T05:05:25.259+0000 7f3128b13700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:05:25.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-mon[36043]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T05:05:25.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-mon[36043]: osdmap e16: 3 total, 2 up, 3 in 2024-03-12T05:05:25.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:25.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-mon[36043]: from='osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:05:25.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:25 smithi161 ceph-mon[36043]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:05:25.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:25 smithi156 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T05:05:25.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:25 smithi156 ceph-mon[36270]: osdmap e16: 3 total, 2 up, 3 in 2024-03-12T05:05:25.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:25 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:25.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:25 smithi156 ceph-mon[36270]: from='osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:05:25.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:25 smithi156 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:05:25.882 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-12T05:05:25.882 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-12T05:05:25.882 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-12T05:05:25.882 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-12T05:05:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113382 s, 925 MB/s 2024-03-12T05:05:25.883 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 2024-03-12T05:05:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-12T05:05:26.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:26 smithi100 ceph-mon[32210]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:05:26.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:26 smithi100 ceph-mon[32210]: osdmap e17: 3 total, 2 up, 3 in 2024-03-12T05:05:26.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:26 smithi100 ceph-mon[32210]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:26.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.496 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-03-12T05:05:26.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:26 smithi161 ceph-mon[36043]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:05:26.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:26 smithi161 ceph-mon[36043]: osdmap e17: 3 total, 2 up, 3 in 2024-03-12T05:05:26.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:26 smithi161 ceph-mon[36043]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:26.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:26 smithi156 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:05:26.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:26 smithi156 ceph-mon[36270]: osdmap e17: 3 total, 2 up, 3 in 2024-03-12T05:05:26.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:26 smithi156 ceph-mon[36270]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-12T05:05:26.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:26.739 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:05:27.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:05:27.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:05:27.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] boot 2024-03-12T05:05:27.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: osdmap e18: 3 total, 3 up, 3 in 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi161 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 17080M 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:27.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:27.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:27.323 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] boot 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: osdmap e18: 3 total, 3 up, 3 in 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi161 2024-03-12T05:05:27.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 17080M 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:27.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: osd.2 [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] boot 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: osdmap e18: 3 total, 3 up, 3 in 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi161 2024-03-12T05:05:27.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 17080M 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:27.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 -- 172.21.15.100:0/2741431167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8101e40 msgr2=0x7f4cc81022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 --2- 172.21.15.100:0/2741431167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8101e40 0x7f4cc81022a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80099c0 tx=0x7f4cb8009cd0 comp rx=0 tx=0).stop 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 -- 172.21.15.100:0/2741431167 shutdown_connections 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 --2- 172.21.15.100:0/2741431167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cc81027e0 0x7f4cc8109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 --2- 172.21.15.100:0/2741431167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc8100c40 0x7f4cc8101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 --2- 172.21.15.100:0/2741431167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8101e40 0x7f4cc81022a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 -- 172.21.15.100:0/2741431167 >> 172.21.15.100:0/2741431167 conn(0x7f4cc80fc200 msgr2=0x7f4cc80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 -- 172.21.15.100:0/2741431167 shutdown_connections 2024-03-12T05:05:28.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.100+0000 7f4ccd223700 1 -- 172.21.15.100:0/2741431167 wait complete. 2024-03-12T05:05:28.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 Processor -- start 2024-03-12T05:05:28.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 -- start start 2024-03-12T05:05:28.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 0x7f4cc8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cc8101e40 0x7f4cc81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc81027e0 0x7f4cc819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc819f7f0 con 0x7f4cc8100c40 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc819f930 con 0x7f4cc8101e40 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4ccd223700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc819faf0 con 0x7f4cc81027e0 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4cc6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 0x7f4cc8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4cc6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 0x7f4cc8197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50374/0 (socket says 172.21.15.100:50374) 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.101+0000 7f4cc6d9d700 1 -- 172.21.15.100:0/1315237206 learned_addr learned my addr 172.21.15.100:0/1315237206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc6d9d700 1 -- 172.21.15.100:0/1315237206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc81027e0 msgr2=0x7f4cc819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc759e700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc81027e0 0x7f4cc819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc6d9d700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc81027e0 0x7f4cc819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc6d9d700 1 -- 172.21.15.100:0/1315237206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cc8101e40 msgr2=0x7f4cc81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc659c700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cc8101e40 0x7f4cc81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc6d9d700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cc8101e40 0x7f4cc81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc6d9d700 1 -- 172.21.15.100:0/1315237206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb80095f0 con 0x7f4cc8100c40 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc759e700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc81027e0 0x7f4cc819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4cc6d9d700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 0x7f4cc8197c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c970 tx=0x7f4cbc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc004220 con 0x7f4cc8100c40 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc819fdd0 con 0x7f4cc8100c40 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cbc007d80 con 0x7f4cc8100c40 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc003da0 con 0x7f4cc8100c40 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.102+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc81a0240 con 0x7f4cc8100c40 2024-03-12T05:05:28.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.103+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc81080d0 con 0x7f4cc8100c40 2024-03-12T05:05:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.105+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7f4cbc0078a0 con 0x7f4cc8100c40 2024-03-12T05:05:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.105+0000 7f4caffff700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4cb00a1a30 0x7f4cb00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.106+0000 7f4cc659c700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4cb00a1a30 0x7f4cb00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.106+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f4cbc0cbef0 con 0x7f4cc8100c40 2024-03-12T05:05:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.106+0000 7f4cc659c700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4cb00a1a30 0x7f4cb00a3ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80038c0 tx=0x7f4cb8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:05:28.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.109+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cbc0965c0 con 0x7f4cc8100c40 2024-03-12T05:05:28.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:28.312+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc802da00 con 0x7f4cb00a1a30 2024-03-12T05:05:28.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:28 smithi100 ceph-mon[32210]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:28.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:05:28.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T05:05:28.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:28 smithi100 ceph-mon[32210]: osdmap e19: 3 total, 3 up, 3 in 2024-03-12T05:05:28.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:05:28.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:28 smithi156 ceph-mon[36270]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:28.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:05:28.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T05:05:28.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:28 smithi156 ceph-mon[36270]: osdmap e19: 3 total, 3 up, 3 in 2024-03-12T05:05:28.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:05:28.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:28 smithi161 ceph-mon[36043]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:28.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:05:28.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T05:05:28.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:28 smithi161 ceph-mon[36043]: osdmap e19: 3 total, 3 up, 3 in 2024-03-12T05:05:28.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:05:29.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:29 smithi100 ceph-mon[32210]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:29.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:29 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:29.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:29 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:29.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:29 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:29.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:29 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T05:05:29.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:29 smithi100 ceph-mon[32210]: osdmap e20: 3 total, 3 up, 3 in 2024-03-12T05:05:29.572 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49450]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T05:05:29.572 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49450]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:29.572 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49450]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:29.572 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49450]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:29.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:29 smithi156 ceph-mon[36270]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:29.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:29 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:29.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:29 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:29.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:29 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:29.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:29 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T05:05:29.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:29 smithi156 ceph-mon[36270]: osdmap e20: 3 total, 3 up, 3 in 2024-03-12T05:05:29.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:29 smithi161 ceph-mon[36043]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:29.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:29 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:29.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:29 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:29.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:29 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:29.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:29 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T05:05:29.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:29 smithi161 ceph-mon[36043]: osdmap e20: 3 total, 3 up, 3 in 2024-03-12T05:05:29.862 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49456]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T05:05:29.862 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49456]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:29.862 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49456]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:29.862 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:05:29 smithi100 sudo[49456]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:30.114 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:29 smithi156 sudo[44800]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T05:05:30.114 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:29 smithi156 sudo[44800]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:30.114 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:29 smithi156 sudo[44800]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:30.114 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:29 smithi156 sudo[44800]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:30.457 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:30 smithi156 sudo[44803]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T05:05:30.458 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:30 smithi156 sudo[44803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:30.458 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:30 smithi156 sudo[44803]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:30.458 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:05:30 smithi156 sudo[44803]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:30 smithi161 ceph-mon[36043]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:30 smithi161 ceph-mon[36043]: osdmap e21: 3 total, 3 up, 3 in 2024-03-12T05:05:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:30.633 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T05:05:30.633 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:30.633 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45026]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:30.633 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45026]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:30.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:30.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 ceph-mon[32210]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:30.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 ceph-mon[32210]: osdmap e21: 3 total, 3 up, 3 in 2024-03-12T05:05:30.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:30.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:30.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:30 smithi156 ceph-mon[36270]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:30.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:30 smithi156 ceph-mon[36270]: osdmap e21: 3 total, 3 up, 3 in 2024-03-12T05:05:30.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:30.976 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T05:05:30.976 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:30.976 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:30.976 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:05:30 smithi161 sudo[45029]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:31.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 sudo[49530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T05:05:31.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 sudo[49530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:31.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 sudo[49530]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:31.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:30 smithi100 sudo[49530]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:31.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:31.172+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4cbc096990 con 0x7f4cc8100c40 2024-03-12T05:05:31.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:31 smithi100 sudo[49533]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T05:05:31.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:31 smithi100 sudo[49533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:31.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:31 smithi100 sudo[49533]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:31.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:31 smithi100 sudo[49533]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45032]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T05:05:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45032]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45032]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T05:05:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45035]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:31.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:31 smithi161 sudo[45035]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:32.142 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:31 smithi156 sudo[44806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-12T05:05:32.143 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:31 smithi156 sudo[44806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:32.143 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:31 smithi156 sudo[44806]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:32.143 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:31 smithi156 sudo[44806]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-12T05:05:32.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/718560119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037d713e-354a-437d-9f75-3b506f6e4ed0"}]: dispatch 2024-03-12T05:05:32.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 sudo[44809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 sudo[44809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 sudo[44809]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 sudo[44809]: pam_unix(sudo:session): session closed for user root 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/718560119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037d713e-354a-437d-9f75-3b506f6e4ed0"}]: dispatch 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/718560119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037d713e-354a-437d-9f75-3b506f6e4ed0"}]: dispatch 2024-03-12T05:05:32.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:05:32.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:05:32.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/718560119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "037d713e-354a-437d-9f75-3b506f6e4ed0"}]': finished 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: osdmap e22: 4 total, 3 up, 4 in 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:33.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:33.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:33 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3885967830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:33.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/718560119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "037d713e-354a-437d-9f75-3b506f6e4ed0"}]': finished 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: osdmap e22: 4 total, 3 up, 4 in 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:33.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:33 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3885967830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/718560119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "037d713e-354a-437d-9f75-3b506f6e4ed0"}]': finished 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: osdmap e22: 4 total, 3 up, 4 in 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T05:05:33.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:33 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3885967830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:34.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:34 smithi156 ceph-mon[36270]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:34.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:34 smithi161 ceph-mon[36043]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:34.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:34 smithi100 ceph-mon[32210]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:36.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:36 smithi161 ceph-mon[36043]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-12T05:05:37.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:36 smithi100 ceph-mon[32210]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-12T05:05:37.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:36 smithi156 ceph-mon[36270]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-12T05:05:38.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:38 smithi100 ceph-mon[32210]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-12T05:05:38.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:38 smithi161 ceph-mon[36043]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-12T05:05:39.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:38 smithi156 ceph-mon[36270]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-12T05:05:39.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T05:05:39.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:40.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T05:05:40.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:40.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T05:05:40.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:40.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:40 smithi161 ceph-mon[36043]: Deploying daemon osd.3 on smithi100 2024-03-12T05:05:40.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:40 smithi161 ceph-mon[36043]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.1 KiB/s rd, 189 KiB/s wr, 13 op/s 2024-03-12T05:05:41.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:40 smithi100 ceph-mon[32210]: Deploying daemon osd.3 on smithi100 2024-03-12T05:05:41.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:40 smithi100 ceph-mon[32210]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.1 KiB/s rd, 189 KiB/s wr, 13 op/s 2024-03-12T05:05:41.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:40 smithi156 ceph-mon[36270]: Deploying daemon osd.3 on smithi100 2024-03-12T05:05:41.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:40 smithi156 ceph-mon[36270]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.1 KiB/s rd, 189 KiB/s wr, 13 op/s 2024-03-12T05:05:42.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:42 smithi161 ceph-mon[36043]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-12T05:05:43.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:42 smithi100 ceph-mon[32210]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-12T05:05:43.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:42 smithi156 ceph-mon[36270]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-12T05:05:45.068 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:44 smithi161 ceph-mon[36043]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-12T05:05:45.069 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:45.069 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:45.069 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:45.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:44 smithi100 ceph-mon[32210]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-12T05:05:45.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:45.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:45.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:45.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:44 smithi156 ceph-mon[36270]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-12T05:05:45.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:45.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:45.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:46.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:46 smithi100 ceph-mon[32210]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-12T05:05:47.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:46 smithi156 ceph-mon[36270]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-12T05:05:47.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:46 smithi161 ceph-mon[36043]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-12T05:05:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.133+0000 7f4caffff700 1 -- 172.21.15.100:0/1315237206 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4cc802da00 con 0x7f4cb00a1a30 2024-03-12T05:05:48.134 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-03-12T05:05:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4cb00a1a30 msgr2=0x7f4cb00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4cb00a1a30 0x7f4cb00a3ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80038c0 tx=0x7f4cb8000bc0 comp rx=0 tx=0).stop 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 msgr2=0x7f4cc8197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 0x7f4cc8197c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c970 tx=0x7f4cbc00cd30 comp rx=0 tx=0).stop 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 shutdown_connections 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4cb00a1a30 0x7f4cb00a3ef0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cc81027e0 0x7f4cc819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc8100c40 0x7f4cc8197c70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 --2- 172.21.15.100:0/1315237206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cc8101e40 0x7f4cc81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.135+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 >> 172.21.15.100:0/1315237206 conn(0x7f4cc80fc200 msgr2=0x7f4cc8105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.136+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 shutdown_connections 2024-03-12T05:05:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:05:48.136+0000 7f4ccd223700 1 -- 172.21.15.100:0/1315237206 wait complete. 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:48.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.709 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:48.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:48.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:48.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:48.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:48.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:05:48.780 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.3.service 2024-03-12T05:05:48.783 INFO:tasks.cephadm:Deploying osd.4 on smithi156 with /dev/vg_nvme/lv_3... 2024-03-12T05:05:48.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-12T05:05:49.028 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:05:49.438 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:49 smithi156 ceph-mon[36270]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T05:05:49.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:49 smithi100 ceph-mon[32210]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T05:05:49.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:49 smithi161 ceph-mon[36043]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T05:05:50.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:50 smithi100 ceph-mon[32210]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T05:05:50.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:50 smithi100 ceph-mon[32210]: osdmap e23: 4 total, 3 up, 4 in 2024-03-12T05:05:50.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:50 smithi100 ceph-mon[32210]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:05:50.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:50.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:50 smithi100 ceph-mon[32210]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:50.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:50 smithi156 ceph-mon[36270]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T05:05:50.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:50 smithi156 ceph-mon[36270]: osdmap e23: 4 total, 3 up, 4 in 2024-03-12T05:05:50.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:50 smithi156 ceph-mon[36270]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:05:50.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:50.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:50 smithi156 ceph-mon[36270]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:50.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:50 smithi161 ceph-mon[36043]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T05:05:50.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:50 smithi161 ceph-mon[36043]: osdmap e23: 4 total, 3 up, 4 in 2024-03-12T05:05:50.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:50 smithi161 ceph-mon[36043]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:05:50.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:50.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:50 smithi161 ceph-mon[36043]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:50.847 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-12T05:05:50.847 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-12T05:05:50.847 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-12T05:05:50.847 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-12T05:05:50.847 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00829526 s, 1.3 GB/s 2024-03-12T05:05:50.847 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-12T05:05:51.321 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:05:51.196+0000 7f28172e8700 -1 osd.3 0 waiting for initial osdmap 2024-03-12T05:05:51.322 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:05:51.202+0000 7f280f8d4700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:05:51.459 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2024-03-12T05:05:51.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:05:51.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: osdmap e24: 4 total, 3 up, 4 in 2024-03-12T05:05:51.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: osdmap e25: 4 total, 3 up, 4 in 2024-03-12T05:05:51.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:51 smithi161 ceph-mon[36043]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' 2024-03-12T05:05:51.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:05:51.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: osdmap e24: 4 total, 3 up, 4 in 2024-03-12T05:05:51.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: osdmap e25: 4 total, 3 up, 4 in 2024-03-12T05:05:51.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:51 smithi100 ceph-mon[32210]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' 2024-03-12T05:05:51.700 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: osdmap e24: 4 total, 3 up, 4 in 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: osdmap e25: 4 total, 3 up, 4 in 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:51.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:51 smithi156 ceph-mon[36270]: from='osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202]' entity='osd.3' 2024-03-12T05:05:52.670 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:05:52.670 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:05:52.670 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:52.670 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] boot 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: osdmap e26: 4 total, 4 up, 4 in 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi100 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 8544M 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:52.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] boot 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: osdmap e26: 4 total, 4 up, 4 in 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi100 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:52.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:52.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 8544M 2024-03-12T05:05:52.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:52.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:52.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: osd.3 [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] boot 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: osdmap e26: 4 total, 4 up, 4 in 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi100 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:52.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:05:52.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 8544M 2024-03-12T05:05:52.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:52.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:52.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:05:52.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:05:53.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1031001746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68100ba0 msgr2=0x7f1b68100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:53.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1031001746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68100ba0 0x7f1b68100fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1b580099c0 tx=0x7f1b58009cd0 comp rx=0 tx=0).stop 2024-03-12T05:05:53.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1031001746 shutdown_connections 2024-03-12T05:05:53.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1031001746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68100ba0 0x7f1b68100fc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:53.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1031001746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68102740 0x7f1b68108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:53.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1031001746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b68101da0 0x7f1b68102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:53.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1031001746 >> 172.21.15.156:0/1031001746 conn(0x7f1b680fc160 msgr2=0x7f1b680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:05:53.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1031001746 shutdown_connections 2024-03-12T05:05:53.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.177+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1031001746 wait complete. 2024-03-12T05:05:53.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 Processor -- start 2024-03-12T05:05:53.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 -- start start 2024-03-12T05:05:53.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68100ba0 0x7f1b6806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:53.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 0x7f1b6819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:53.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b68102740 0x7f1b6819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:53.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6819f860 con 0x7f1b68100ba0 2024-03-12T05:05:53.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6819f9d0 con 0x7f1b68101da0 2024-03-12T05:05:53.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.178+0000 7f1b6d1ab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6819fb90 con 0x7f1b68102740 2024-03-12T05:05:53.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 0x7f1b6819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:53.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b66d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68100ba0 0x7f1b6806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 0x7f1b6819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56854/0 (socket says 172.21.15.156:56854) 2024-03-12T05:05:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 -- 172.21.15.156:0/1437791662 learned_addr learned my addr 172.21.15.156:0/1437791662 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-12T05:05:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6759e700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b68102740 0x7f1b6819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 -- 172.21.15.156:0/1437791662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b68102740 msgr2=0x7f1b6819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b68102740 0x7f1b6819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 -- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68100ba0 msgr2=0x7f1b6806aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:05:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68100ba0 0x7f1b6806aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:05:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 -- 172.21.15.156:0/1437791662 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580095f0 con 0x7f1b68101da0 2024-03-12T05:05:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b66d9d700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68100ba0 0x7f1b6806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:05:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6659c700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 0x7f1b6819ca80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00b530 tx=0x7f1b5c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:05:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011830 con 0x7f1b68101da0 2024-03-12T05:05:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b5c011e70 con 0x7f1b68101da0 2024-03-12T05:05:53.189 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6819fe70 con 0x7f1b68101da0 2024-03-12T05:05:53.189 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00f2e0 con 0x7f1b68101da0 2024-03-12T05:05:53.189 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.179+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b681a0360 con 0x7f1b68101da0 2024-03-12T05:05:53.189 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.180+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b68068930 con 0x7f1b68101da0 2024-03-12T05:05:53.189 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.181+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1b5c010610 con 0x7f1b68101da0 2024-03-12T05:05:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.182+0000 7f1b4ffff700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1b500a1ac0 0x7f1b500a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:05:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.182+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f1b5c0cbbd0 con 0x7f1b68101da0 2024-03-12T05:05:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.182+0000 7f1b66d9d700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1b500a1ac0 0x7f1b500a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:05:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.183+0000 7f1b66d9d700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1b500a1ac0 0x7f1b500a3f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b580038c0 tx=0x7f1b5800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:05:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.186+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b5c095fe0 con 0x7f1b68101da0 2024-03-12T05:05:53.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:05:53.384+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b6802d910 con 0x7f1b500a1ac0 2024-03-12T05:05:53.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:53 smithi156 ceph-mon[36270]: osdmap e27: 4 total, 4 up, 4 in 2024-03-12T05:05:53.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:53.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:53.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:53.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:53 smithi161 ceph-mon[36043]: osdmap e27: 4 total, 4 up, 4 in 2024-03-12T05:05:53.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:53.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:53.977 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:54.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:53 smithi100 ceph-mon[32210]: osdmap e27: 4 total, 4 up, 4 in 2024-03-12T05:05:54.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:05:54.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:05:54.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:05:54.945 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:54 smithi156 ceph-mon[36270]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:54.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:54 smithi156 ceph-mon[36270]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:05:54.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:54 smithi156 ceph-mon[36270]: osdmap e28: 4 total, 4 up, 4 in 2024-03-12T05:05:54.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:54 smithi100 ceph-mon[32210]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:54.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:54 smithi100 ceph-mon[32210]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:05:54.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:54 smithi100 ceph-mon[32210]: osdmap e28: 4 total, 4 up, 4 in 2024-03-12T05:05:54.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:54 smithi161 ceph-mon[36043]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:05:54.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:54 smithi161 ceph-mon[36043]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:05:54.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:54 smithi161 ceph-mon[36043]: osdmap e28: 4 total, 4 up, 4 in 2024-03-12T05:05:55.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:55 smithi156 ceph-mon[36270]: osdmap e29: 4 total, 4 up, 4 in 2024-03-12T05:05:55.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:55 smithi156 ceph-mon[36270]: osdmap e30: 4 total, 4 up, 4 in 2024-03-12T05:05:55.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:55 smithi161 ceph-mon[36043]: osdmap e29: 4 total, 4 up, 4 in 2024-03-12T05:05:55.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:55 smithi161 ceph-mon[36043]: osdmap e30: 4 total, 4 up, 4 in 2024-03-12T05:05:55.988 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:55 smithi100 ceph-mon[32210]: osdmap e29: 4 total, 4 up, 4 in 2024-03-12T05:05:55.988 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:55 smithi100 ceph-mon[32210]: osdmap e30: 4 total, 4 up, 4 in 2024-03-12T05:05:56.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:05:56.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/71303535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]: dispatch 2024-03-12T05:05:56.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]: dispatch 2024-03-12T05:05:56.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]': finished 2024-03-12T05:05:56.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: osdmap e31: 5 total, 4 up, 5 in 2024-03-12T05:05:56.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:05:56.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/2439866374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:56.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:05:56.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/71303535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]: dispatch 2024-03-12T05:05:56.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]: dispatch 2024-03-12T05:05:56.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]': finished 2024-03-12T05:05:56.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: osdmap e31: 5 total, 4 up, 5 in 2024-03-12T05:05:56.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:05:56.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/2439866374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:57.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:05:57.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/71303535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]: dispatch 2024-03-12T05:05:57.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]: dispatch 2024-03-12T05:05:57.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "206a171a-ec8a-474b-9f06-40508adf5df8"}]': finished 2024-03-12T05:05:57.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: osdmap e31: 5 total, 4 up, 5 in 2024-03-12T05:05:57.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:05:57.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/2439866374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:05:58.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:05:58 smithi156 ceph-mon[36270]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:05:58.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:05:58 smithi161 ceph-mon[36043]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:05:59.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:05:58 smithi100 ceph-mon[32210]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:06:00.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:00 smithi156 ceph-mon[36270]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:06:00.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:00 smithi161 ceph-mon[36043]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:06:01.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:00 smithi100 ceph-mon[32210]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:06:02.798 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:02 smithi156 ceph-mon[36270]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:02.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:02 smithi161 ceph-mon[36043]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:03.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:02 smithi100 ceph-mon[32210]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:03.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T05:06:03.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:03.978 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T05:06:03.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:04.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T05:06:04.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:04.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:04 smithi156 ceph-mon[36270]: Deploying daemon osd.4 on smithi156 2024-03-12T05:06:04.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:04 smithi156 ceph-mon[36270]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:04.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:04 smithi161 ceph-mon[36043]: Deploying daemon osd.4 on smithi156 2024-03-12T05:06:04.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:04 smithi161 ceph-mon[36043]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:05.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:04 smithi100 ceph-mon[32210]: Deploying daemon osd.4 on smithi156 2024-03-12T05:06:05.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:04 smithi100 ceph-mon[32210]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:06.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:06 smithi156 ceph-mon[36270]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:06.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:06 smithi161 ceph-mon[36043]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:07.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:06 smithi100 ceph-mon[32210]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:08.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:08 smithi156 ceph-mon[36270]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:08.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:08.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:08.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:08.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:08 smithi161 ceph-mon[36043]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:08.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:08.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:08.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:09.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:08 smithi100 ceph-mon[32210]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:09.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:09.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:09.090 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:10.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:10 smithi161 ceph-mon[36043]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:10.986 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:10 smithi156 ceph-mon[36270]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:10 smithi100 ceph-mon[32210]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:11.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:11.998+0000 7f1b4ffff700 1 -- 172.21.15.156:0/1437791662 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b6802d910 con 0x7f1b500a1ac0 2024-03-12T05:06:11.999 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 4 on host 'smithi156' 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.000+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1b500a1ac0 msgr2=0x7f1b500a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.000+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1b500a1ac0 0x7f1b500a3f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b580038c0 tx=0x7f1b5800b540 comp rx=0 tx=0).stop 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.000+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 msgr2=0x7f1b6819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.000+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 0x7f1b6819ca80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00b530 tx=0x7f1b5c00b840 comp rx=0 tx=0).stop 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 shutdown_connections 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1b500a1ac0 0x7f1b500a3f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b68102740 0x7f1b6819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b68100ba0 0x7f1b6806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 --2- 172.21.15.156:0/1437791662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68101da0 0x7f1b6819ca80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 >> 172.21.15.156:0/1437791662 conn(0x7f1b680fc160 msgr2=0x7f1b68105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 shutdown_connections 2024-03-12T05:06:12.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:06:12.001+0000 7f1b6d1ab700 1 -- 172.21.15.156:0/1437791662 wait complete. 2024-03-12T05:06:12.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:12.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:12.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:11 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:11 smithi156 ceph-mon[36270]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:12.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:12.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:12.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:11 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:11 smithi161 ceph-mon[36043]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:12.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:12.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:12.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:11 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:12.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:11 smithi100 ceph-mon[32210]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:12.661 DEBUG:teuthology.orchestra.run.smithi156:osd.4> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.4.service 2024-03-12T05:06:12.663 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-03-12T05:06:12.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-12T05:06:12.909 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:06:13.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:13.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:13.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:13.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:12 smithi156 ceph-mon[36270]: from='osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T05:06:13.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:12 smithi156 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T05:06:13.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:13.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:13.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:13.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:12 smithi161 ceph-mon[36043]: from='osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T05:06:13.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:12 smithi161 ceph-mon[36043]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T05:06:13.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:13.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:13.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:13.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:12 smithi100 ceph-mon[32210]: from='osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T05:06:13.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:12 smithi100 ceph-mon[32210]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T05:06:14.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:13 smithi156 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T05:06:14.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:13 smithi156 ceph-mon[36270]: from='osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:06:14.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:13 smithi156 ceph-mon[36270]: osdmap e32: 5 total, 4 up, 5 in 2024-03-12T05:06:14.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:13 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:14.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:13 smithi156 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:06:14.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:13 smithi156 ceph-mon[36270]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:14.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:13 smithi161 ceph-mon[36043]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T05:06:14.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:13 smithi161 ceph-mon[36043]: from='osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:06:14.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:13 smithi161 ceph-mon[36043]: osdmap e32: 5 total, 4 up, 5 in 2024-03-12T05:06:14.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:13 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:14.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:13 smithi161 ceph-mon[36043]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:06:14.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:13 smithi161 ceph-mon[36043]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:14.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:13 smithi100 ceph-mon[32210]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T05:06:14.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:13 smithi100 ceph-mon[32210]: from='osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:06:14.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:13 smithi100 ceph-mon[32210]: osdmap e32: 5 total, 4 up, 5 in 2024-03-12T05:06:14.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:13 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:14.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:13 smithi100 ceph-mon[32210]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:06:14.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:13 smithi100 ceph-mon[32210]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:14.751 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-12T05:06:14.751 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-12T05:06:14.751 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-12T05:06:14.751 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-12T05:06:14.752 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117384 s, 893 MB/s 2024-03-12T05:06:14.752 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-12T05:06:14.923 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4[47631]: 2024-03-12T05:06:14.658+0000 7f8720a5c700 -1 osd.4 0 waiting for initial osdmap 2024-03-12T05:06:14.924 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4[47631]: 2024-03-12T05:06:14.665+0000 7f8719048700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:06:15.058 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:14 smithi161 ceph-mon[36043]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:06:15.058 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:14 smithi161 ceph-mon[36043]: osdmap e33: 5 total, 4 up, 5 in 2024-03-12T05:06:15.058 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:15.058 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:15.058 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:14 smithi161 ceph-mon[36043]: from='osd.4 ' entity='osd.4' 2024-03-12T05:06:15.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:06:15.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-mon[36270]: osdmap e33: 5 total, 4 up, 5 in 2024-03-12T05:06:15.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:15.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:15.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:14 smithi156 ceph-mon[36270]: from='osd.4 ' entity='osd.4' 2024-03-12T05:06:15.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:14 smithi100 ceph-mon[32210]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:06:15.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:14 smithi100 ceph-mon[32210]: osdmap e33: 5 total, 4 up, 5 in 2024-03-12T05:06:15.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:15.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:15.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:14 smithi100 ceph-mon[32210]: from='osd.4 ' entity='osd.4' 2024-03-12T05:06:15.342 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-03-12T05:06:15.588 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:06:16.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi156 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 8540M 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:16.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:16.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] boot 2024-03-12T05:06:16.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: osdmap e34: 5 total, 5 up, 5 in 2024-03-12T05:06:16.227 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:16.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi156 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 8540M 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:16.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] boot 2024-03-12T05:06:16.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: osdmap e34: 5 total, 5 up, 5 in 2024-03-12T05:06:16.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi156 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 8540M 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:16.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:16.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:16.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-12T05:06:16.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: osd.4 [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] boot 2024-03-12T05:06:16.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: osdmap e34: 5 total, 5 up, 5 in 2024-03-12T05:06:16.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 -- 172.21.15.161:0/998298866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90101ca0 msgr2=0x7f3f901020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 --2- 172.21.15.161:0/998298866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90101ca0 0x7f3f901020c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009990 tx=0x7f3f80009ca0 comp rx=0 tx=0).stop 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 -- 172.21.15.161:0/998298866 shutdown_connections 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 --2- 172.21.15.161:0/998298866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f90103840 0x7f3f9010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 --2- 172.21.15.161:0/998298866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90101ca0 0x7f3f901020c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 --2- 172.21.15.161:0/998298866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 0x7f3f90103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:16.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.987+0000 7f3f95a80700 1 -- 172.21.15.161:0/998298866 >> 172.21.15.161:0/998298866 conn(0x7f3f900fd280 msgr2=0x7f3f900ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.988+0000 7f3f95a80700 1 -- 172.21.15.161:0/998298866 shutdown_connections 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.988+0000 7f3f95a80700 1 -- 172.21.15.161:0/998298866 wait complete. 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.988+0000 7f3f95a80700 1 Processor -- start 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.988+0000 7f3f95a80700 1 -- start start 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.988+0000 7f3f95a80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f90101ca0 0x7f3f9019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.988+0000 7f3f95a80700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 0x7f3f9019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f95a80700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 0x7f3f901a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f95a80700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901a5e30 con 0x7f3f90101ca0 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f95a80700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901a5f70 con 0x7f3f90102ea0 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f95a80700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901a6100 con 0x7f3f90103840 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 0x7f3f901a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:16.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 0x7f3f901a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45364/0 (socket says 172.21.15.161:45364) 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 -- 172.21.15.161:0/3773632072 learned_addr learned my addr 172.21.15.161:0/3773632072 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8f7fe700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f90101ca0 0x7f3f9019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8effd700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 0x7f3f9019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 -- 172.21.15.161:0/3773632072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 msgr2=0x7f3f9019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 0x7f3f9019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 -- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f90101ca0 msgr2=0x7f3f9019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f90101ca0 0x7f3f9019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 -- 172.21.15.161:0/3773632072 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800095f0 con 0x7f3f90103840 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8effd700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 0x7f3f9019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8ffff700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 0x7f3f901a5860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f88007d40 tx=0x7f3f8800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800ad60 con 0x7f3f90103840 2024-03-12T05:06:16.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f88007e90 con 0x7f3f90103840 2024-03-12T05:06:16.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f901a63e0 con 0x7f3f90103840 2024-03-12T05:06:16.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88011740 con 0x7f3f90103840 2024-03-12T05:06:16.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.989+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f901a6910 con 0x7f3f90103840 2024-03-12T05:06:16.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.990+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f90107050 con 0x7f3f90103840 2024-03-12T05:06:16.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.991+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f3f88012420 con 0x7f3f90103840 2024-03-12T05:06:16.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.992+0000 7f3f8cff9700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3f780a1ac0 0x7f3f780a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:16.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.992+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f3f880cc610 con 0x7f3f90103840 2024-03-12T05:06:16.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.992+0000 7f3f8f7fe700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3f780a1ac0 0x7f3f780a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:16.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.993+0000 7f3f8f7fe700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3f780a1ac0 0x7f3f780a3f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f80003800 tx=0x7f3f80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:06:16.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:16.997+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f88096890 con 0x7f3f90103840 2024-03-12T05:06:17.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:17.199+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f9002d870 con 0x7f3f780a1ac0 2024-03-12T05:06:18.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:18 smithi161 ceph-mon[36043]: osdmap e35: 5 total, 5 up, 5 in 2024-03-12T05:06:18.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:18 smithi161 ceph-mon[36043]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:06:18.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:06:18.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:06:18.121 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:18.121 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:18 smithi161 ceph-mon[36043]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:18.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:18 smithi100 ceph-mon[32210]: osdmap e35: 5 total, 5 up, 5 in 2024-03-12T05:06:18.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:18 smithi100 ceph-mon[32210]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:06:18.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:06:18.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:06:18.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:18.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:18 smithi100 ceph-mon[32210]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:18.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:18 smithi156 ceph-mon[36270]: osdmap e35: 5 total, 5 up, 5 in 2024-03-12T05:06:18.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:18 smithi156 ceph-mon[36270]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:06:18.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:06:18.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:06:18.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:18.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:18 smithi156 ceph-mon[36270]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/2420190755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]: dispatch 2024-03-12T05:06:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]: dispatch 2024-03-12T05:06:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]': finished 2024-03-12T05:06:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: osdmap e36: 6 total, 5 up, 6 in 2024-03-12T05:06:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:21.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/2178390635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/2420190755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]: dispatch 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]: dispatch 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]': finished 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: osdmap e36: 6 total, 5 up, 6 in 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:21.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/2178390635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:06:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/2420190755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]: dispatch 2024-03-12T05:06:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]: dispatch 2024-03-12T05:06:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41244b3-befa-47fa-809e-fdc7b1a23f2c"}]': finished 2024-03-12T05:06:21.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: osdmap e36: 6 total, 5 up, 6 in 2024-03-12T05:06:21.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:21.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/2178390635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:06:23.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:22 smithi156 ceph-mon[36270]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:23.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:22 smithi100 ceph-mon[32210]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:23.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:22 smithi161 ceph-mon[36043]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:25.003 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:24 smithi161 ceph-mon[36043]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:25.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:24 smithi100 ceph-mon[32210]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:25.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:24 smithi156 ceph-mon[36270]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:26.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:26 smithi161 ceph-mon[36043]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:27.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:26 smithi100 ceph-mon[32210]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:27.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:26 smithi156 ceph-mon[36270]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:28.047 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T05:06:28.047 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:28.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T05:06:28.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:28.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T05:06:28.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:29.062 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:28 smithi161 ceph-mon[36043]: Deploying daemon osd.5 on smithi161 2024-03-12T05:06:29.062 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:28 smithi161 ceph-mon[36043]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:29.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:28 smithi100 ceph-mon[32210]: Deploying daemon osd.5 on smithi161 2024-03-12T05:06:29.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:28 smithi100 ceph-mon[32210]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:29.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:28 smithi156 ceph-mon[36270]: Deploying daemon osd.5 on smithi161 2024-03-12T05:06:29.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:28 smithi156 ceph-mon[36270]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:30 smithi161 ceph-mon[36043]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:31.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:30 smithi100 ceph-mon[32210]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:31.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:30 smithi156 ceph-mon[36270]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:33.051 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:32 smithi161 ceph-mon[36043]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:33.051 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:33.051 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:33.051 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:33.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:32 smithi100 ceph-mon[32210]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:33.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:33.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:33.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:33.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:32 smithi156 ceph-mon[36270]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:33.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:33.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:33.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:34.838 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:34 smithi161 ceph-mon[36043]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:35.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:34 smithi100 ceph-mon[32210]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:35.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:34 smithi156 ceph-mon[36270]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:35.911 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-03-12T05:06:35.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.912+0000 7f3f8cff9700 1 -- 172.21.15.161:0/3773632072 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3f9002d870 con 0x7f3f780a1ac0 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3f780a1ac0 msgr2=0x7f3f780a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3f780a1ac0 0x7f3f780a3f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f80003800 tx=0x7f3f80005fd0 comp rx=0 tx=0).stop 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 msgr2=0x7f3f901a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 0x7f3f901a5860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f88007d40 tx=0x7f3f8800a340 comp rx=0 tx=0).stop 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 shutdown_connections 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3f780a1ac0 0x7f3f780a3f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.914+0000 7f3f95a80700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f90103840 0x7f3f901a5860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.915+0000 7f3f95a80700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f90101ca0 0x7f3f9019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.915+0000 7f3f95a80700 1 --2- 172.21.15.161:0/3773632072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f90102ea0 0x7f3f9019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.915+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 >> 172.21.15.161:0/3773632072 conn(0x7f3f900fd280 msgr2=0x7f3f90104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:06:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.915+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 shutdown_connections 2024-03-12T05:06:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:06:35.915+0000 7f3f95a80700 1 -- 172.21.15.161:0/3773632072 wait complete. 2024-03-12T05:06:36.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:36.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:36.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:36.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:35 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:36.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:36.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:36.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:36.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:35 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:36.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:36.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:36.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:36.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:36.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:35 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:36.484 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.5.service 2024-03-12T05:06:36.486 INFO:tasks.cephadm:Deploying osd.6 on smithi100 with /dev/vg_nvme/lv_2... 2024-03-12T05:06:36.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-12T05:06:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:06:37.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:36 smithi100 ceph-mon[32210]: from='osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T05:06:37.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:36 smithi100 ceph-mon[32210]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T05:06:37.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:36 smithi156 ceph-mon[36270]: from='osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T05:06:37.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:36 smithi156 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T05:06:37.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:36 smithi161 ceph-mon[36043]: from='osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T05:06:37.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:36 smithi161 ceph-mon[36043]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T05:06:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T05:06:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: osdmap e37: 6 total, 5 up, 6 in 2024-03-12T05:06:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: from='osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:06:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:06:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:37.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:37 smithi100 ceph-mon[32210]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:06:37.959 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T05:06:37.959 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: osdmap e37: 6 total, 5 up, 6 in 2024-03-12T05:06:37.959 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: from='osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:06:37.959 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:37.959 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:06:37.959 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:37.960 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-mon[36043]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:06:37.960 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:06:37.952+0000 7f442d571700 -1 osd.5 0 waiting for initial osdmap 2024-03-12T05:06:37.960 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:06:37 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:06:37.959+0000 7f4429b65700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: osdmap e37: 6 total, 5 up, 6 in 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: from='osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:38.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:37 smithi156 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:06:38.490 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-12T05:06:38.490 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-12T05:06:38.490 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-12T05:06:38.490 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-12T05:06:38.491 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112678 s, 931 MB/s 2024-03-12T05:06:38.491 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-12T05:06:39.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-03-12T05:06:39.097 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:38 smithi100 ceph-mon[32210]: osdmap e38: 6 total, 5 up, 6 in 2024-03-12T05:06:39.097 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:39.097 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:39.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:38 smithi161 ceph-mon[36043]: osdmap e38: 6 total, 5 up, 6 in 2024-03-12T05:06:39.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:39.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:39.325 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:06:39.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:38 smithi156 ceph-mon[36270]: osdmap e38: 6 total, 5 up, 6 in 2024-03-12T05:06:39.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:39.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:40.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] boot 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: osdmap e39: 6 total, 6 up, 6 in 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi161 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:40.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 8540M 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:40.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:39 smithi100 ceph-mon[32210]: osdmap e40: 6 total, 6 up, 6 in 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] boot 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: osdmap e39: 6 total, 6 up, 6 in 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi161 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 8540M 2024-03-12T05:06:40.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:40.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:40.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:40.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:39 smithi161 ceph-mon[36043]: osdmap e40: 6 total, 6 up, 6 in 2024-03-12T05:06:40.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:06:40.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:06:40.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: osd.5 [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] boot 2024-03-12T05:06:40.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: osdmap e39: 6 total, 6 up, 6 in 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi161 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 8540M 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:40.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:06:40.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:40.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-03-12T05:06:40.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:39 smithi156 ceph-mon[36270]: osdmap e40: 6 total, 6 up, 6 in 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.698+0000 7f29e262d700 1 -- 172.21.15.100:0/429439027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 msgr2=0x7f29dc103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.698+0000 7f29e262d700 1 --2- 172.21.15.100:0/429439027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc103300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29d00099c0 tx=0x7f29d0009cd0 comp rx=0 tx=0).stop 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 -- 172.21.15.100:0/429439027 shutdown_connections 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 --2- 172.21.15.100:0/429439027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29dc101ca0 0x7f29dc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 --2- 172.21.15.100:0/429439027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29dc103840 0x7f29dc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 --2- 172.21.15.100:0/429439027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc103300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 -- 172.21.15.100:0/429439027 >> 172.21.15.100:0/429439027 conn(0x7f29dc0fd280 msgr2=0x7f29dc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 -- 172.21.15.100:0/429439027 shutdown_connections 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 -- 172.21.15.100:0/429439027 wait complete. 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 Processor -- start 2024-03-12T05:06:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.699+0000 7f29e262d700 1 -- start start 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29dc101ca0 0x7f29dc197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29dc103840 0x7f29dc19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc19f7f0 con 0x7f29dc102ea0 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc19f930 con 0x7f29dc101ca0 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc19faf0 con 0x7f29dc103840 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47966/0 (socket says 172.21.15.100:47966) 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 -- 172.21.15.100:0/692325730 learned_addr learned my addr 172.21.15.100:0/692325730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e0bca700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29dc103840 0x7f29dc19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29dbfff700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29dc101ca0 0x7f29dc197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 -- 172.21.15.100:0/692325730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29dc103840 msgr2=0x7f29dc19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29dc103840 0x7f29dc19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 -- 172.21.15.100:0/692325730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29dc101ca0 msgr2=0x7f29dc197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29dc101ca0 0x7f29dc197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 -- 172.21.15.100:0/692325730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00095f0 con 0x7f29dc102ea0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29db7fe700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc1981b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29d0003790 tx=0x7f29d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d001d070 con 0x7f29dc102ea0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29d0022470 con 0x7f29dc102ea0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc19fd70 con 0x7f29dc102ea0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000f6e0 con 0x7f29dc102ea0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.700+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc1a01c0 con 0x7f29dc102ea0 2024-03-12T05:06:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.701+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc1090b0 con 0x7f29dc102ea0 2024-03-12T05:06:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.702+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f29d00225e0 con 0x7f29dc102ea0 2024-03-12T05:06:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.703+0000 7f29d97fa700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f29c40a1ac0 0x7f29c40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:06:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.703+0000 7f29dbfff700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f29c40a1ac0 0x7f29c40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:06:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.703+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f29d00cdd80 con 0x7f29dc102ea0 2024-03-12T05:06:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.703+0000 7f29dbfff700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f29c40a1ac0 0x7f29c40a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005fd0 tx=0x7f29cc005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:06:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.706+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f29d0097fe0 con 0x7f29dc102ea0 2024-03-12T05:06:40.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:40.908+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f29dc02d8e0 con 0x7f29c40a1ac0 2024-03-12T05:06:41.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:06:41.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:06:41.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:41 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:41.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:41 smithi100 ceph-mon[32210]: osdmap e41: 6 total, 6 up, 6 in 2024-03-12T05:06:41.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:41 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:06:41.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:41 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:06:41.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:41 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:41.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:41 smithi156 ceph-mon[36270]: osdmap e41: 6 total, 6 up, 6 in 2024-03-12T05:06:41.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:41 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:06:41.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:41 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:06:41.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:41 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:41.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:41 smithi161 ceph-mon[36043]: osdmap e41: 6 total, 6 up, 6 in 2024-03-12T05:06:42.460 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:42 smithi100 ceph-mon[32210]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:06:42.460 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:42 smithi100 ceph-mon[32210]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:42.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:42 smithi156 ceph-mon[36270]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:06:42.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:42 smithi156 ceph-mon[36270]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:42.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:42 smithi161 ceph-mon[36043]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:06:42.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:42 smithi161 ceph-mon[36043]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:43.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:43 smithi156 ceph-mon[36270]: osdmap e42: 6 total, 6 up, 6 in 2024-03-12T05:06:43.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:43 smithi161 ceph-mon[36043]: osdmap e42: 6 total, 6 up, 6 in 2024-03-12T05:06:43.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:43 smithi100 ceph-mon[32210]: osdmap e42: 6 total, 6 up, 6 in 2024-03-12T05:06:44.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: osdmap e43: 6 total, 6 up, 6 in 2024-03-12T05:06:44.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2475898209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4349c6aa-c535-40fb-a462-cf066f57ea26"}]: dispatch 2024-03-12T05:06:44.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2475898209' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4349c6aa-c535-40fb-a462-cf066f57ea26"}]': finished 2024-03-12T05:06:44.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: osdmap e44: 7 total, 6 up, 7 in 2024-03-12T05:06:44.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:06:44.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:44.708 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2446840366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: osdmap e43: 6 total, 6 up, 6 in 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2475898209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4349c6aa-c535-40fb-a462-cf066f57ea26"}]: dispatch 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2475898209' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4349c6aa-c535-40fb-a462-cf066f57ea26"}]': finished 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: osdmap e44: 7 total, 6 up, 7 in 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:44.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2446840366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:06:44.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: osdmap e43: 6 total, 6 up, 6 in 2024-03-12T05:06:44.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2475898209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4349c6aa-c535-40fb-a462-cf066f57ea26"}]: dispatch 2024-03-12T05:06:44.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2475898209' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4349c6aa-c535-40fb-a462-cf066f57ea26"}]': finished 2024-03-12T05:06:44.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: osdmap e44: 7 total, 6 up, 7 in 2024-03-12T05:06:44.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:06:44.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:44.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2446840366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:06:47.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:46 smithi100 ceph-mon[32210]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-12T05:06:47.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:46 smithi156 ceph-mon[36270]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-12T05:06:47.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:46 smithi161 ceph-mon[36043]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-12T05:06:48.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:48 smithi100 ceph-mon[32210]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:06:49.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:48 smithi156 ceph-mon[36270]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:06:49.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:48 smithi161 ceph-mon[36043]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:06:51.025 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:50 smithi100 ceph-mon[32210]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-12T05:06:51.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:50 smithi156 ceph-mon[36270]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-12T05:06:51.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:50 smithi161 ceph-mon[36043]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-12T05:06:51.912 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T05:06:51.912 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:52.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T05:06:52.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:52.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T05:06:52.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:06:52.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:52 smithi100 ceph-mon[32210]: Deploying daemon osd.6 on smithi100 2024-03-12T05:06:52.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:52 smithi100 ceph-mon[32210]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-12T05:06:53.108 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:52 smithi156 ceph-mon[36270]: Deploying daemon osd.6 on smithi100 2024-03-12T05:06:53.108 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:52 smithi156 ceph-mon[36270]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-12T05:06:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:52 smithi161 ceph-mon[36043]: Deploying daemon osd.6 on smithi100 2024-03-12T05:06:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:52 smithi161 ceph-mon[36043]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-12T05:06:55.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:54 smithi100 ceph-mon[32210]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-12T05:06:55.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:54 smithi156 ceph-mon[36270]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-12T05:06:55.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:54 smithi161 ceph-mon[36043]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-12T05:06:56.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:56 smithi100 ceph-mon[32210]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-12T05:06:56.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:56.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:56.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:56 smithi156 ceph-mon[36270]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-12T05:06:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:56 smithi161 ceph-mon[36043]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-12T05:06:57.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:57.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:06:57.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:06:58.882 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:06:58 smithi100 ceph-mon[32210]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:59.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:06:58 smithi156 ceph-mon[36270]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:59.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:06:58 smithi161 ceph-mon[36043]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:06:59.966 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 6 on host 'smithi100' 2024-03-12T05:06:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.966+0000 7f29d97fa700 1 -- 172.21.15.100:0/692325730 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f29dc02d8e0 con 0x7f29c40a1ac0 2024-03-12T05:06:59.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.968+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f29c40a1ac0 msgr2=0x7f29c40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:59.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.968+0000 7f29e262d700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f29c40a1ac0 0x7f29c40a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005fd0 tx=0x7f29cc005dd0 comp rx=0 tx=0).stop 2024-03-12T05:06:59.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.968+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 msgr2=0x7f29dc1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:06:59.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.968+0000 7f29e262d700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc1981b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29d0003790 tx=0x7f29d00047b0 comp rx=0 tx=0).stop 2024-03-12T05:06:59.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 shutdown_connections 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f29c40a1ac0 0x7f29c40a3f80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29dc101ca0 0x7f29dc197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29dc103840 0x7f29dc19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 --2- 172.21.15.100:0/692325730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29dc102ea0 0x7f29dc1981b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 >> 172.21.15.100:0/692325730 conn(0x7f29dc0fd280 msgr2=0x7f29dc1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 shutdown_connections 2024-03-12T05:06:59.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:06:59.969+0000 7f29e262d700 1 -- 172.21.15.100:0/692325730 wait complete. 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:00 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:00 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:00.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:00.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:00.514 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.514 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:00.514 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.514 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:00.514 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:00 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:00.539 DEBUG:teuthology.orchestra.run.smithi100:osd.6> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.6.service 2024-03-12T05:07:00.582 INFO:tasks.cephadm:Deploying osd.7 on smithi156 with /dev/vg_nvme/lv_2... 2024-03-12T05:07:00.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-12T05:07:00.830 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:07:01.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:01 smithi156 ceph-mon[36270]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T05:07:01.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:01 smithi161 ceph-mon[36043]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T05:07:01.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:01 smithi100 ceph-mon[32210]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T05:07:02.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:02 smithi100 ceph-mon[32210]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T05:07:02.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:02 smithi100 ceph-mon[32210]: osdmap e45: 7 total, 6 up, 7 in 2024-03-12T05:07:02.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:02 smithi100 ceph-mon[32210]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:07:02.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:02.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:02 smithi100 ceph-mon[32210]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:02.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:02 smithi156 ceph-mon[36270]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T05:07:02.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:02 smithi156 ceph-mon[36270]: osdmap e45: 7 total, 6 up, 7 in 2024-03-12T05:07:02.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:02 smithi156 ceph-mon[36270]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:07:02.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:02.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:02 smithi156 ceph-mon[36270]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:02.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:02 smithi161 ceph-mon[36043]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T05:07:02.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:02 smithi161 ceph-mon[36043]: osdmap e45: 7 total, 6 up, 7 in 2024-03-12T05:07:02.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:02 smithi161 ceph-mon[36043]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:07:02.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:02.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:02 smithi161 ceph-mon[36043]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:02.724 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-12T05:07:02.724 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-12T05:07:02.724 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-12T05:07:02.724 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-12T05:07:02.725 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00842812 s, 1.2 GB/s 2024-03-12T05:07:02.725 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-12T05:07:03.169 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:07:03.028+0000 7f2a53613700 -1 osd.6 0 waiting for initial osdmap 2024-03-12T05:07:03.169 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:07:03.044+0000 7f2a4bbff700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:07:03.283 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:03 smithi156 ceph-mon[36270]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:07:03.283 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:03 smithi156 ceph-mon[36270]: osdmap e46: 7 total, 6 up, 7 in 2024-03-12T05:07:03.283 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:03.283 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:03.284 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:03 smithi156 ceph-mon[36270]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' 2024-03-12T05:07:03.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2024-03-12T05:07:03.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:03 smithi161 ceph-mon[36043]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:07:03.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:03 smithi161 ceph-mon[36043]: osdmap e46: 7 total, 6 up, 7 in 2024-03-12T05:07:03.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:03.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:03.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:03 smithi161 ceph-mon[36043]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' 2024-03-12T05:07:03.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-mon[32210]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:07:03.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-mon[32210]: osdmap e46: 7 total, 6 up, 7 in 2024-03-12T05:07:03.520 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:03.520 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:03.520 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:03 smithi100 ceph-mon[32210]: from='osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956]' entity='osd.6' 2024-03-12T05:07:03.561 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi100 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 5696M 2024-03-12T05:07:04.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] boot 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: osdmap e47: 7 total, 7 up, 7 in 2024-03-12T05:07:04.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi100 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 5696M 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:04.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:04.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:04.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] boot 2024-03-12T05:07:04.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: osdmap e47: 7 total, 7 up, 7 in 2024-03-12T05:07:04.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:04.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:07:04.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:07:04.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:04.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi100 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 5696M 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-12T05:07:04.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: osd.6 [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] boot 2024-03-12T05:07:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: osdmap e47: 7 total, 7 up, 7 in 2024-03-12T05:07:04.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:07:05.164 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 -- 172.21.15.156:0/3369100711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8103840 msgr2=0x7f24a810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:05.164 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 --2- 172.21.15.156:0/3369100711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8103840 0x7f24a810a0f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24a000b0a0 tx=0x7f24a000b3b0 comp rx=0 tx=0).stop 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 -- 172.21.15.156:0/3369100711 shutdown_connections 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 --2- 172.21.15.156:0/3369100711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8101ca0 0x7f24a81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 --2- 172.21.15.156:0/3369100711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8103840 0x7f24a810a0f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 --2- 172.21.15.156:0/3369100711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 0x7f24a8103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 -- 172.21.15.156:0/3369100711 >> 172.21.15.156:0/3369100711 conn(0x7f24a80fd280 msgr2=0x7f24a80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 -- 172.21.15.156:0/3369100711 shutdown_connections 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.162+0000 7f24aefac700 1 -- 172.21.15.156:0/3369100711 wait complete. 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 Processor -- start 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 -- start start 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8101ca0 0x7f24a8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:05.165 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 0x7f24a8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 0x7f24a819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a819f870 con 0x7f24a8102ea0 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a819f9b0 con 0x7f24a8103840 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24aefac700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a819fb70 con 0x7f24a8101ca0 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 0x7f24a819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24acd48700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8101ca0 0x7f24a8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 0x7f24a819f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46894/0 (socket says 172.21.15.156:46894) 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 -- 172.21.15.156:0/1570798144 learned_addr learned my addr 172.21.15.156:0/1570798144 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24a7fff700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 0x7f24a8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:05.166 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 -- 172.21.15.156:0/1570798144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8101ca0 msgr2=0x7f24a8197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8101ca0 0x7f24a8197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 -- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 msgr2=0x7f24a8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 0x7f24a8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 -- 172.21.15.156:0/1570798144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0009d00 con 0x7f24a8103840 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24a7fff700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 0x7f24a8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.163+0000 7f24ad549700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 0x7f24a819f2a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24a000bd60 tx=0x7f24a000bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.164+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a000e040 con 0x7f24a8103840 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.164+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a819fdf0 con 0x7f24a8103840 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.164+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a0004c60 con 0x7f24a8103840 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.164+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00052a0 con 0x7f24a8103840 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.164+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a81a01b0 con 0x7f24a8103840 2024-03-12T05:07:05.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.165+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a8109060 con 0x7f24a8103840 2024-03-12T05:07:05.168 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.166+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f24a001d9e0 con 0x7f24a8103840 2024-03-12T05:07:05.168 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.166+0000 7f24a5ffb700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f24980a1ac0 0x7f24980a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:05.169 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.167+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f24a00ce230 con 0x7f24a8103840 2024-03-12T05:07:05.169 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.167+0000 7f24acd48700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f24980a1ac0 0x7f24980a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:05.169 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.167+0000 7f24acd48700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f24980a1ac0 0x7f24980a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a8199200 tx=0x7f249c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:07:05.169 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.170+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24a00981e0 con 0x7f24a8103840 2024-03-12T05:07:05.370 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:05.370+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f24a802d950 con 0x7f24980a1ac0 2024-03-12T05:07:06.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:06 smithi156 ceph-mon[36270]: osdmap e48: 7 total, 7 up, 7 in 2024-03-12T05:07:06.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:06 smithi156 ceph-mon[36270]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:06.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:06.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:06.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:06.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:06 smithi156 ceph-mon[36270]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:06.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:06 smithi161 ceph-mon[36043]: osdmap e48: 7 total, 7 up, 7 in 2024-03-12T05:07:06.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:06 smithi161 ceph-mon[36043]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:06.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:06.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:06.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:06.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:06 smithi161 ceph-mon[36043]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:06.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:06 smithi100 ceph-mon[32210]: osdmap e48: 7 total, 7 up, 7 in 2024-03-12T05:07:06.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:06 smithi100 ceph-mon[32210]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:06.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:06.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:06.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:06.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:06 smithi100 ceph-mon[32210]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:09.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:09.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/1208529847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]: dispatch 2024-03-12T05:07:09.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]: dispatch 2024-03-12T05:07:09.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]': finished 2024-03-12T05:07:09.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: osdmap e49: 8 total, 7 up, 8 in 2024-03-12T05:07:09.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:09.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:08 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/3513051536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/1208529847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]: dispatch 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]: dispatch 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]': finished 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: osdmap e49: 8 total, 7 up, 8 in 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:09.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:08 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/3513051536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/1208529847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]: dispatch 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]: dispatch 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd43ecf-765c-412a-b351-a8b7fd3b7381"}]': finished 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: osdmap e49: 8 total, 7 up, 8 in 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:09.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:08 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/3513051536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:11.074 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:10 smithi100 ceph-mon[32210]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:10 smithi156 ceph-mon[36270]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:10 smithi161 ceph-mon[36043]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:13.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:12 smithi156 ceph-mon[36270]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:13.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:12 smithi100 ceph-mon[32210]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:13.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:12 smithi161 ceph-mon[36043]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:14.944 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:14 smithi156 ceph-mon[36270]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:15.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:14 smithi100 ceph-mon[32210]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:15.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:14 smithi161 ceph-mon[36043]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:15.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T05:07:15.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:16.056 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T05:07:16.056 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:16.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T05:07:16.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:16.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:16 smithi156 ceph-mon[36270]: Deploying daemon osd.7 on smithi156 2024-03-12T05:07:16.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:16 smithi156 ceph-mon[36270]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:17.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:16 smithi100 ceph-mon[32210]: Deploying daemon osd.7 on smithi156 2024-03-12T05:07:17.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:16 smithi100 ceph-mon[32210]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:16 smithi161 ceph-mon[36043]: Deploying daemon osd.7 on smithi156 2024-03-12T05:07:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:16 smithi161 ceph-mon[36043]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:18.932 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:18 smithi156 ceph-mon[36270]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:19.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:18 smithi100 ceph-mon[32210]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:19.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:18 smithi161 ceph-mon[36043]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:20.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:20 smithi156 ceph-mon[36270]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:20.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:20.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:20.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:20 smithi100 ceph-mon[32210]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:21.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:20 smithi161 ceph-mon[36043]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:21.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:22.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:22 smithi156 ceph-mon[36270]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:23.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:22 smithi100 ceph-mon[32210]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:23.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:22 smithi161 ceph-mon[36043]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:24.235 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 7 on host 'smithi156' 2024-03-12T05:07:24.235 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.235+0000 7f24a5ffb700 1 -- 172.21.15.156:0/1570798144 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f24a802d950 con 0x7f24980a1ac0 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f24980a1ac0 msgr2=0x7f24980a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f24980a1ac0 0x7f24980a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a8199200 tx=0x7f249c009260 comp rx=0 tx=0).stop 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 msgr2=0x7f24a819f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 0x7f24a819f2a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24a000bd60 tx=0x7f24a000bd90 comp rx=0 tx=0).stop 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 shutdown_connections 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f24980a1ac0 0x7f24980a3f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24a8103840 0x7f24a819f2a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8101ca0 0x7f24a8197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 --2- 172.21.15.156:0/1570798144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102ea0 0x7f24a8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 >> 172.21.15.156:0/1570798144 conn(0x7f24a80fd280 msgr2=0x7f24a81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 shutdown_connections 2024-03-12T05:07:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:07:24.238+0000 7f24aefac700 1 -- 172.21.15.156:0/1570798144 wait complete. 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:24 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:24.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:24.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:24.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:24.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:24 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:24.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:24 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:24.864 DEBUG:teuthology.orchestra.run.smithi156:osd.7> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.7.service 2024-03-12T05:07:24.866 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-03-12T05:07:24.867 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-12T05:07:25.115 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:07:25.523 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:25 smithi161 ceph-mon[36043]: from='osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T05:07:25.523 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:25 smithi161 ceph-mon[36043]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T05:07:25.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:25 smithi100 ceph-mon[32210]: from='osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T05:07:25.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:25 smithi100 ceph-mon[32210]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T05:07:25.911 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:25 smithi156 ceph-mon[36270]: from='osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T05:07:25.911 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:25 smithi156 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T05:07:26.526 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:26 smithi161 ceph-mon[36043]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T05:07:26.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:26 smithi161 ceph-mon[36043]: from='osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:07:26.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:26 smithi161 ceph-mon[36043]: osdmap e50: 8 total, 7 up, 8 in 2024-03-12T05:07:26.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:26.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:26 smithi161 ceph-mon[36043]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:07:26.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:26 smithi161 ceph-mon[36043]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T05:07:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-mon[36270]: from='osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:07:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-mon[36270]: osdmap e50: 8 total, 7 up, 8 in 2024-03-12T05:07:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:07:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-mon[36270]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:26.708 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:07:26.474+0000 7fd70ebd9700 -1 osd.7 0 waiting for initial osdmap 2024-03-12T05:07:26.708 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:07:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:07:26.487+0000 7fd70b1cd700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:07:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:26 smithi100 ceph-mon[32210]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T05:07:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:26 smithi100 ceph-mon[32210]: from='osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:07:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:26 smithi100 ceph-mon[32210]: osdmap e50: 8 total, 7 up, 8 in 2024-03-12T05:07:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:26 smithi100 ceph-mon[32210]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:07:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:26 smithi100 ceph-mon[32210]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-12T05:07:27.090 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-12T05:07:27.090 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-12T05:07:27.090 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-12T05:07:27.090 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129179 s, 812 MB/s 2024-03-12T05:07:27.090 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-12T05:07:27.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:27 smithi161 ceph-mon[36043]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:07:27.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:27 smithi161 ceph-mon[36043]: osdmap e51: 8 total, 7 up, 8 in 2024-03-12T05:07:27.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:27.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:27 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:27.652 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:27 smithi156 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:07:27.652 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:27 smithi156 ceph-mon[36270]: osdmap e51: 8 total, 7 up, 8 in 2024-03-12T05:07:27.652 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:27.652 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:27 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:27.678 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-03-12T05:07:27.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:27 smithi100 ceph-mon[32210]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:07:27.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:27 smithi100 ceph-mon[32210]: osdmap e51: 8 total, 7 up, 8 in 2024-03-12T05:07:27.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:27.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:27 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:27.923 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:07:28.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:07:28.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:07:28.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] boot 2024-03-12T05:07:28.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: osdmap e52: 8 total, 8 up, 8 in 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi156 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 5693M 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:28.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:28.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:07:28.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:07:28.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] boot 2024-03-12T05:07:28.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: osdmap e52: 8 total, 8 up, 8 in 2024-03-12T05:07:28.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi156 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 5693M 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:28.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:28.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: osd.7 [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] boot 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: osdmap e52: 8 total, 8 up, 8 in 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi156 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 5693M 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:28.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 -- 172.21.15.161:0/2305215070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c100ba0 msgr2=0x7fde4c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 --2- 172.21.15.161:0/2305215070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c100ba0 0x7fde4c100fc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde4400b0a0 tx=0x7fde4400b3b0 comp rx=0 tx=0).stop 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 -- 172.21.15.161:0/2305215070 shutdown_connections 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 --2- 172.21.15.161:0/2305215070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde4c102740 0x7fde4c108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 --2- 172.21.15.161:0/2305215070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c100ba0 0x7fde4c100fc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 --2- 172.21.15.161:0/2305215070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c101da0 0x7fde4c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:29.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 -- 172.21.15.161:0/2305215070 >> 172.21.15.161:0/2305215070 conn(0x7fde4c0fc160 msgr2=0x7fde4c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 -- 172.21.15.161:0/2305215070 shutdown_connections 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.405+0000 7fde530c7700 1 -- 172.21.15.161:0/2305215070 wait complete. 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 Processor -- start 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 -- start start 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde4c100ba0 0x7fde4c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 0x7fde4c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c102740 0x7fde4c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19e840 con 0x7fde4c100ba0 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19e980 con 0x7fde4c102740 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde530c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19eb40 con 0x7fde4c101da0 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 0x7fde4c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 0x7fde4c197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43036/0 (socket says 172.21.15.161:43036) 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021421059 learned_addr learned my addr 172.21.15.161:0/3021421059 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde50e63700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde4c100ba0 0x7fde4c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.406+0000 7fde51664700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c102740 0x7fde4c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021421059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c102740 msgr2=0x7fde4c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde4bfff700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c102740 0x7fde4c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde4c100ba0 msgr2=0x7fde4c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde4bfff700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde4c100ba0 0x7fde4c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde4bfff700 1 -- 172.21.15.161:0/3021421059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44009d00 con 0x7fde4c101da0 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde51664700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c102740 0x7fde4c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:07:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde4bfff700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 0x7fde4c197230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c920 tx=0x7fde3c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:07:29.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c007b00 con 0x7fde4c101da0 2024-03-12T05:07:29.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde3c00ce70 con 0x7fde4c101da0 2024-03-12T05:07:29.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c19ee20 con 0x7fde4c101da0 2024-03-12T05:07:29.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c005610 con 0x7fde4c101da0 2024-03-12T05:07:29.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.407+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c19f230 con 0x7fde4c101da0 2024-03-12T05:07:29.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.408+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c108080 con 0x7fde4c101da0 2024-03-12T05:07:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.409+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fde3c004750 con 0x7fde4c101da0 2024-03-12T05:07:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.410+0000 7fde49ffb700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fde380a1ac0 0x7fde380a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.410+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fde3c0cc800 con 0x7fde4c101da0 2024-03-12T05:07:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.410+0000 7fde50e63700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fde380a1ac0 0x7fde380a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.410+0000 7fde50e63700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fde380a1ac0 0x7fde380a3f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde4c1981d0 tx=0x7fde44006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:07:29.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.413+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde3c096550 con 0x7fde4c101da0 2024-03-12T05:07:29.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:29.621+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fde4c02d950 con 0x7fde380a1ac0 2024-03-12T05:07:29.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:29 smithi100 ceph-mon[32210]: osdmap e53: 8 total, 8 up, 8 in 2024-03-12T05:07:29.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:29 smithi156 ceph-mon[36270]: osdmap e53: 8 total, 8 up, 8 in 2024-03-12T05:07:29.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:29 smithi161 ceph-mon[36043]: osdmap e53: 8 total, 8 up, 8 in 2024-03-12T05:07:30.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:30 smithi100 ceph-mon[32210]: osdmap e54: 8 total, 8 up, 8 in 2024-03-12T05:07:30.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:30 smithi100 ceph-mon[32210]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:30.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:30.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:30.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:30.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:30 smithi100 ceph-mon[32210]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:30.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:30 smithi161 ceph-mon[36043]: osdmap e54: 8 total, 8 up, 8 in 2024-03-12T05:07:30.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:30 smithi161 ceph-mon[36043]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:30.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:30.849 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:30.849 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:30.849 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:30 smithi161 ceph-mon[36043]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:30 smithi156 ceph-mon[36270]: osdmap e54: 8 total, 8 up, 8 in 2024-03-12T05:07:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:30 smithi156 ceph-mon[36270]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:30.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:30.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:30 smithi156 ceph-mon[36270]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:31.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:31 smithi161 ceph-mon[36043]: osdmap e55: 8 total, 8 up, 8 in 2024-03-12T05:07:31.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:31 smithi100 ceph-mon[32210]: osdmap e55: 8 total, 8 up, 8 in 2024-03-12T05:07:31.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:31 smithi156 ceph-mon[36270]: osdmap e55: 8 total, 8 up, 8 in 2024-03-12T05:07:32.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: osdmap e56: 8 total, 8 up, 8 in 2024-03-12T05:07:32.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:32.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/4267213708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]: dispatch 2024-03-12T05:07:32.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]: dispatch 2024-03-12T05:07:32.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]': finished 2024-03-12T05:07:32.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: osdmap e57: 9 total, 8 up, 9 in 2024-03-12T05:07:32.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:32 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:32.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: osdmap e56: 8 total, 8 up, 8 in 2024-03-12T05:07:32.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:32.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/4267213708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]: dispatch 2024-03-12T05:07:32.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]: dispatch 2024-03-12T05:07:32.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]': finished 2024-03-12T05:07:32.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: osdmap e57: 9 total, 8 up, 9 in 2024-03-12T05:07:32.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:32 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:32.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: osdmap e56: 8 total, 8 up, 8 in 2024-03-12T05:07:32.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:32.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/4267213708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]: dispatch 2024-03-12T05:07:32.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]: dispatch 2024-03-12T05:07:32.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fc82b38-b2e7-47c3-9130-8c921510b06c"}]': finished 2024-03-12T05:07:32.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: osdmap e57: 9 total, 8 up, 9 in 2024-03-12T05:07:32.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:32 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:33.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:33 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/1965376820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:33.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:33 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/1965376820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:33.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:33 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/1965376820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:34.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:34 smithi100 ceph-mon[32210]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:34.876 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:34 smithi161 ceph-mon[36043]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:34.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:34 smithi156 ceph-mon[36270]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:36.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:36 smithi161 ceph-mon[36043]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:37.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:36 smithi100 ceph-mon[32210]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:36 smithi156 ceph-mon[36270]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:38 smithi100 ceph-mon[32210]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:38 smithi156 ceph-mon[36270]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:38 smithi161 ceph-mon[36043]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:40.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-12T05:07:40.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:40.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-12T05:07:40.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:40.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-12T05:07:40.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:40.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:40 smithi161 ceph-mon[36043]: Deploying daemon osd.8 on smithi161 2024-03-12T05:07:40.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:40 smithi161 ceph-mon[36043]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:40 smithi100 ceph-mon[32210]: Deploying daemon osd.8 on smithi161 2024-03-12T05:07:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:40 smithi100 ceph-mon[32210]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:41.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:40 smithi156 ceph-mon[36270]: Deploying daemon osd.8 on smithi161 2024-03-12T05:07:41.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:40 smithi156 ceph-mon[36270]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:43.049 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:42 smithi161 ceph-mon[36043]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:43.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:42 smithi100 ceph-mon[32210]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:43.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:42 smithi156 ceph-mon[36270]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:44 smithi100 ceph-mon[32210]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:45.124 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:44 smithi161 ceph-mon[36043]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:45.125 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:45.125 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:45.125 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:44 smithi156 ceph-mon[36270]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:47.033 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:46 smithi161 ceph-mon[36043]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:47.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:46 smithi100 ceph-mon[32210]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:47.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:46 smithi156 ceph-mon[36270]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:47.688 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-03-12T05:07:47.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.689+0000 7fde49ffb700 1 -- 172.21.15.161:0/3021421059 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fde4c02d950 con 0x7fde380a1ac0 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fde380a1ac0 msgr2=0x7fde380a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fde380a1ac0 0x7fde380a3f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde4c1981d0 tx=0x7fde44006000 comp rx=0 tx=0).stop 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 msgr2=0x7fde4c197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 0x7fde4c197230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c920 tx=0x7fde3c00cc30 comp rx=0 tx=0).stop 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 shutdown_connections 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fde380a1ac0 0x7fde380a3f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4c102740 0x7fde4c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde4c100ba0 0x7fde4c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 --2- 172.21.15.161:0/3021421059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4c101da0 0x7fde4c197230 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:47.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 >> 172.21.15.161:0/3021421059 conn(0x7fde4c0fc160 msgr2=0x7fde4c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:07:47.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 shutdown_connections 2024-03-12T05:07:47.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:07:47.692+0000 7fde530c7700 1 -- 172.21.15.161:0/3021421059 wait complete. 2024-03-12T05:07:48.313 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.8.service 2024-03-12T05:07:48.315 INFO:tasks.cephadm:Deploying osd.9 on smithi100 with /dev/vg_nvme/lv_1... 2024-03-12T05:07:48.316 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-12T05:07:48.474 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:07:48.388+0000 7f8f230117c0 -1 osd.8 0 log_to_monitors true 2024-03-12T05:07:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:07:48.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:48.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: from='osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T05:07:48.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T05:07:48.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:48 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:48.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: from='osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T05:07:48.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T05:07:48.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:48 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: from='osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:48 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:49.932 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:49 smithi100 ceph-mon[32210]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-12T05:07:49.932 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:49 smithi100 ceph-mon[32210]: osdmap e58: 9 total, 8 up, 9 in 2024-03-12T05:07:49.932 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:49 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:49.932 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:49 smithi100 ceph-mon[32210]: from='osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:07:49.932 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:49 smithi100 ceph-mon[32210]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:07:49.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:49 smithi156 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-12T05:07:49.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:49 smithi156 ceph-mon[36270]: osdmap e58: 9 total, 8 up, 9 in 2024-03-12T05:07:49.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:49 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:49.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:49 smithi156 ceph-mon[36270]: from='osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:07:49.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:49 smithi156 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:07:49.974 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:07:49.695+0000 7f8f17e75700 -1 osd.8 0 waiting for initial osdmap 2024-03-12T05:07:49.975 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:07:49.709+0000 7f8f14469700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:07:49.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-mon[36043]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-12T05:07:49.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-mon[36043]: osdmap e58: 9 total, 8 up, 9 in 2024-03-12T05:07:49.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:49.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-mon[36043]: from='osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:07:49.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:49 smithi161 ceph-mon[36043]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:07:50.488 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-12T05:07:50.488 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-12T05:07:50.489 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-12T05:07:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-12T05:07:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114861 s, 913 MB/s 2024-03-12T05:07:50.489 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 2024-03-12T05:07:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-12T05:07:50.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:50.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:07:50.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: osdmap e59: 9 total, 8 up, 9 in 2024-03-12T05:07:50.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] boot 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: osdmap e60: 9 total, 9 up, 9 in 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: osdmap e59: 9 total, 8 up, 9 in 2024-03-12T05:07:50.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] boot 2024-03-12T05:07:50.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: osdmap e60: 9 total, 9 up, 9 in 2024-03-12T05:07:50.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-12T05:07:50.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:07:50.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: osdmap e59: 9 total, 8 up, 9 in 2024-03-12T05:07:50.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:50.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: osd.8 [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] boot 2024-03-12T05:07:50.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: osdmap e60: 9 total, 9 up, 9 in 2024-03-12T05:07:50.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:07:51.160 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-03-12T05:07:51.405 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:07:51.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:51 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:07:51.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:51 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:07:51.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:51 smithi161 ceph-mon[36043]: osdmap e61: 9 total, 9 up, 9 in 2024-03-12T05:07:51.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:51 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:07:51.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:51 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:07:51.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:51 smithi100 ceph-mon[32210]: osdmap e61: 9 total, 9 up, 9 in 2024-03-12T05:07:51.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:51 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:07:51.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:51 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:07:51.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:51 smithi156 ceph-mon[36270]: osdmap e61: 9 total, 9 up, 9 in 2024-03-12T05:07:53.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 -- 172.21.15.100:0/2186818453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80102ea0 msgr2=0x7efe80103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:53.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 --2- 172.21.15.100:0/2186818453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80102ea0 0x7efe80103300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0099c0 tx=0x7efe7c009cd0 comp rx=0 tx=0).stop 2024-03-12T05:07:53.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 -- 172.21.15.100:0/2186818453 shutdown_connections 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 --2- 172.21.15.100:0/2186818453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe80101ca0 0x7efe801020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 --2- 172.21.15.100:0/2186818453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe80103840 0x7efe8010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 --2- 172.21.15.100:0/2186818453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80102ea0 0x7efe80103300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 -- 172.21.15.100:0/2186818453 >> 172.21.15.100:0/2186818453 conn(0x7efe800fd280 msgr2=0x7efe800ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 -- 172.21.15.100:0/2186818453 shutdown_connections 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 -- 172.21.15.100:0/2186818453 wait complete. 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 Processor -- start 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 -- start start 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe80101ca0 0x7efe80197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.037+0000 7efe87579700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe80102ea0 0x7efe80198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe87579700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 0x7efe8019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe87579700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8019f880 con 0x7efe80103840 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe87579700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8019f9c0 con 0x7efe80101ca0 2024-03-12T05:07:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe87579700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8019fb80 con 0x7efe80102ea0 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 0x7efe8019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 0x7efe8019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34642/0 (socket says 172.21.15.100:34642) 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 -- 172.21.15.100:0/3064522082 learned_addr learned my addr 172.21.15.100:0/3064522082 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85315700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe80101ca0 0x7efe80197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe84b14700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe80102ea0 0x7efe80198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 -- 172.21.15.100:0/3064522082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe80102ea0 msgr2=0x7efe80198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe80102ea0 0x7efe80198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 -- 172.21.15.100:0/3064522082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe80101ca0 msgr2=0x7efe80197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe80101ca0 0x7efe80197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 -- 172.21.15.100:0/3064522082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0095f0 con 0x7efe80103840 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe85b16700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 0x7efe8019f2b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe7800b970 tx=0x7efe7800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800d860 con 0x7efe80103840 2024-03-12T05:07:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8019fe60 con 0x7efe80103840 2024-03-12T05:07:53.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe7800e430 con 0x7efe80103840 2024-03-12T05:07:53.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800dea0 con 0x7efe80103840 2024-03-12T05:07:53.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.038+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe801a02a0 con 0x7efe80103840 2024-03-12T05:07:53.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.039+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe801090b0 con 0x7efe80103840 2024-03-12T05:07:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.040+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7efe7800e910 con 0x7efe80103840 2024-03-12T05:07:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.041+0000 7efe767fc700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efe6c0a1ac0 0x7efe6c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:07:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.041+0000 7efe85315700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efe6c0a1ac0 0x7efe6c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:07:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.041+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7efe78006e40 con 0x7efe80103840 2024-03-12T05:07:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.041+0000 7efe85315700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efe6c0a1ac0 0x7efe6c0a3f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe80199210 tx=0x7efe70009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:07:53.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.045+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe780963b0 con 0x7efe80103840 2024-03-12T05:07:53.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:07:53.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi161 2024-03-12T05:07:53.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 5693M 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.071 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:52 smithi100 ceph-mon[32210]: osdmap e62: 9 total, 9 up, 9 in 2024-03-12T05:07:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:07:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi161 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 5693M 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:53.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:53.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:53.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:52 smithi156 ceph-mon[36270]: osdmap e62: 9 total, 9 up, 9 in 2024-03-12T05:07:53.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:07:53.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi161 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 5693M 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:07:53.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:07:53.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:52 smithi161 ceph-mon[36043]: osdmap e62: 9 total, 9 up, 9 in 2024-03-12T05:07:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:07:53.250+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efe8002da20 con 0x7efe6c0a1ac0 2024-03-12T05:07:54.108 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:54.108 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:54.108 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:54.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:54.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:54.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:07:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:07:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:07:55.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:54 smithi100 ceph-mon[32210]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:55.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:54 smithi100 ceph-mon[32210]: pgmap v197: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:07:55.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:54 smithi100 ceph-mon[32210]: osdmap e63: 9 total, 9 up, 9 in 2024-03-12T05:07:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:54 smithi156 ceph-mon[36270]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:54 smithi156 ceph-mon[36270]: pgmap v197: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:07:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:54 smithi156 ceph-mon[36270]: osdmap e63: 9 total, 9 up, 9 in 2024-03-12T05:07:55.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:54 smithi161 ceph-mon[36043]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:07:55.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:54 smithi161 ceph-mon[36043]: pgmap v197: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:07:55.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:54 smithi161 ceph-mon[36043]: osdmap e63: 9 total, 9 up, 9 in 2024-03-12T05:07:56.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:55 smithi100 ceph-mon[32210]: osdmap e64: 9 total, 9 up, 9 in 2024-03-12T05:07:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:55 smithi156 ceph-mon[36270]: osdmap e64: 9 total, 9 up, 9 in 2024-03-12T05:07:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:55 smithi161 ceph-mon[36043]: osdmap e64: 9 total, 9 up, 9 in 2024-03-12T05:07:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:56 smithi156 ceph-mon[36270]: pgmap v200: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:07:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3664787026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f17e94a-a270-4e9e-837e-e0c5dfb2e68c"}]: dispatch 2024-03-12T05:07:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3664787026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f17e94a-a270-4e9e-837e-e0c5dfb2e68c"}]': finished 2024-03-12T05:07:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:56 smithi156 ceph-mon[36270]: osdmap e65: 10 total, 9 up, 10 in 2024-03-12T05:07:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:07:57.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1876686929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:56 smithi161 ceph-mon[36043]: pgmap v200: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:07:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3664787026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f17e94a-a270-4e9e-837e-e0c5dfb2e68c"}]: dispatch 2024-03-12T05:07:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3664787026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f17e94a-a270-4e9e-837e-e0c5dfb2e68c"}]': finished 2024-03-12T05:07:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:56 smithi161 ceph-mon[36043]: osdmap e65: 10 total, 9 up, 10 in 2024-03-12T05:07:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:07:57.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1876686929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:57.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:56 smithi100 ceph-mon[32210]: pgmap v200: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:07:57.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3664787026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f17e94a-a270-4e9e-837e-e0c5dfb2e68c"}]: dispatch 2024-03-12T05:07:57.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3664787026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f17e94a-a270-4e9e-837e-e0c5dfb2e68c"}]': finished 2024-03-12T05:07:57.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:56 smithi100 ceph-mon[32210]: osdmap e65: 10 total, 9 up, 10 in 2024-03-12T05:07:57.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:07:57.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1876686929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:07:58.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:07:57 smithi156 ceph-mon[36270]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-12T05:07:58.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:07:57 smithi161 ceph-mon[36043]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-12T05:07:58.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:07:57 smithi100 ceph-mon[32210]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-12T05:08:01.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:00 smithi100 ceph-mon[32210]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:01.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:00 smithi156 ceph-mon[36270]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:00 smithi161 ceph-mon[36043]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:02 smithi100 ceph-mon[32210]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:02 smithi156 ceph-mon[36270]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:03.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:02 smithi161 ceph-mon[36043]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:04.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-12T05:08:04.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:04.125 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-12T05:08:04.126 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:04.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-12T05:08:04.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:04 smithi100 ceph-mon[32210]: Deploying daemon osd.9 on smithi100 2024-03-12T05:08:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:04 smithi100 ceph-mon[32210]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:04 smithi156 ceph-mon[36270]: Deploying daemon osd.9 on smithi100 2024-03-12T05:08:05.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:04 smithi156 ceph-mon[36270]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:04 smithi161 ceph-mon[36043]: Deploying daemon osd.9 on smithi100 2024-03-12T05:08:05.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:04 smithi161 ceph-mon[36043]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:07.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:06 smithi100 ceph-mon[32210]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:06 smithi156 ceph-mon[36270]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:06 smithi161 ceph-mon[36043]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:09.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:08 smithi100 ceph-mon[32210]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:09.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:08 smithi156 ceph-mon[36270]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:09.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:08 smithi161 ceph-mon[36043]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:10.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:10.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:10.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:10.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:10 smithi100 ceph-mon[32210]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:10.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:10.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:10.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:10.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:10 smithi156 ceph-mon[36270]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:10.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:10.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:10.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:10.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:10 smithi161 ceph-mon[36043]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:12.916 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 9 on host 'smithi100' 2024-03-12T05:08:12.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.917+0000 7efe767fc700 1 -- 172.21.15.100:0/3064522082 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe8002da20 con 0x7efe6c0a1ac0 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efe6c0a1ac0 msgr2=0x7efe6c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efe6c0a1ac0 0x7efe6c0a3f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe80199210 tx=0x7efe70009190 comp rx=0 tx=0).stop 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 msgr2=0x7efe8019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 0x7efe8019f2b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe7800b970 tx=0x7efe7800bc80 comp rx=0 tx=0).stop 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 shutdown_connections 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efe6c0a1ac0 0x7efe6c0a3f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe80101ca0 0x7efe80197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe80103840 0x7efe8019f2b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 --2- 172.21.15.100:0/3064522082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe80102ea0 0x7efe80198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 >> 172.21.15.100:0/3064522082 conn(0x7efe800fd280 msgr2=0x7efe801061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:08:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 shutdown_connections 2024-03-12T05:08:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:08:12.919+0000 7efe87579700 1 -- 172.21.15.100:0/3064522082 wait complete. 2024-03-12T05:08:12.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:12 smithi100 ceph-mon[32210]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:12.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:12.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:12.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:12.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:12.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:12 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:12 smithi156 ceph-mon[36270]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:12 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:12 smithi161 ceph-mon[36043]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:12 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:13.594 DEBUG:teuthology.orchestra.run.smithi100:osd.9> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.9.service 2024-03-12T05:08:13.598 INFO:tasks.cephadm:Deploying osd.10 on smithi156 with /dev/vg_nvme/lv_1... 2024-03-12T05:08:13.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-12T05:08:13.847 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:08:14.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:13 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:14.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:13 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:14.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:13 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:14.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:13 smithi100 ceph-mon[32210]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-12T05:08:14.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:13 smithi100 ceph-mon[32210]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:14.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:13 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:14.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:13 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:14.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:13 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:14.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:13 smithi156 ceph-mon[36270]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-12T05:08:14.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:13 smithi156 ceph-mon[36270]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:14.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:13 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:14.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:13 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:14.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:13 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:14.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:13 smithi161 ceph-mon[36043]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-12T05:08:14.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:13 smithi161 ceph-mon[36043]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:14.939 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:14 smithi156 ceph-mon[36270]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-12T05:08:14.939 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:14 smithi156 ceph-mon[36270]: osdmap e66: 10 total, 9 up, 10 in 2024-03-12T05:08:14.939 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:14 smithi156 ceph-mon[36270]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:08:14.940 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:14 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:14 smithi100 ceph-mon[32210]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-12T05:08:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:14 smithi100 ceph-mon[32210]: osdmap e66: 10 total, 9 up, 10 in 2024-03-12T05:08:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:14 smithi100 ceph-mon[32210]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:08:15.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:14 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:15.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:14 smithi161 ceph-mon[36043]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-12T05:08:15.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:14 smithi161 ceph-mon[36043]: osdmap e66: 10 total, 9 up, 10 in 2024-03-12T05:08:15.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:14 smithi161 ceph-mon[36043]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-12T05:08:15.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:14 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:15.757 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-12T05:08:15.757 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-12T05:08:15.758 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-12T05:08:15.758 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-12T05:08:15.758 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00840301 s, 1.2 GB/s 2024-03-12T05:08:15.758 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-12T05:08:16.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-mon[32210]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:08:16.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-mon[32210]: osdmap e67: 10 total, 9 up, 10 in 2024-03-12T05:08:16.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-mon[32210]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:16.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-mon[32210]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' 2024-03-12T05:08:16.070 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:08:15.724+0000 7fd85b685700 -1 osd.9 0 waiting for initial osdmap 2024-03-12T05:08:16.070 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:08:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:08:15.743+0000 7fd85246e700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:08:16.095 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:15 smithi156 ceph-mon[36270]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:08:16.096 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:15 smithi156 ceph-mon[36270]: osdmap e67: 10 total, 9 up, 10 in 2024-03-12T05:08:16.096 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.096 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:15 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.096 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:15 smithi156 ceph-mon[36270]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:16.096 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:15 smithi156 ceph-mon[36270]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' 2024-03-12T05:08:16.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:15 smithi161 ceph-mon[36043]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-12T05:08:16.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:15 smithi161 ceph-mon[36043]: osdmap e67: 10 total, 9 up, 10 in 2024-03-12T05:08:16.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:15 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:15 smithi161 ceph-mon[36043]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-03-12T05:08:16.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:15 smithi161 ceph-mon[36043]: from='osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860]' entity='osd.9' 2024-03-12T05:08:16.407 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2024-03-12T05:08:16.652 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:08:16.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi100 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:16.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 4272M 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] boot 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: osdmap e68: 10 total, 10 up, 10 in 2024-03-12T05:08:16.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:17.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi100 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 4272M 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:17.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] boot 2024-03-12T05:08:17.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: osdmap e68: 10 total, 10 up, 10 in 2024-03-12T05:08:17.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:17.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:08:17.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi100 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 4272M 2024-03-12T05:08:17.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:17.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:17.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:17.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: osd.9 [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] boot 2024-03-12T05:08:17.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: osdmap e68: 10 total, 10 up, 10 in 2024-03-12T05:08:17.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:08:18.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 -- 172.21.15.156:0/2443158239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78240ffd70 msgr2=0x7f78241001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:18.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 --2- 172.21.15.156:0/2443158239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78240ffd70 0x7f78241001f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f780c009990 tx=0x7f780c009ca0 comp rx=0 tx=0).stop 2024-03-12T05:08:18.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 -- 172.21.15.156:0/2443158239 shutdown_connections 2024-03-12T05:08:18.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 --2- 172.21.15.156:0/2443158239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7824100730 0x7f7824100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:18.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 --2- 172.21.15.156:0/2443158239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7824102270 0x7f7824102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:18.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 --2- 172.21.15.156:0/2443158239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78240ffd70 0x7f78241001f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 -- 172.21.15.156:0/2443158239 >> 172.21.15.156:0/2443158239 conn(0x7f78240fb910 msgr2=0x7f78240fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 -- 172.21.15.156:0/2443158239 shutdown_connections 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 -- 172.21.15.156:0/2443158239 wait complete. 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 Processor -- start 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.069+0000 7f782a5be700 1 -- start start 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f782a5be700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78240ffd70 0x7f7824199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f782a5be700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 0x7f782419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f782a5be700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7824102270 0x7f78241a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f782a5be700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78241a18e0 con 0x7f7824102270 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f782a5be700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78241a1a20 con 0x7f7824100730 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f782a5be700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78241a1bb0 con 0x7f78240ffd70 2024-03-12T05:08:18.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 0x7f782419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 0x7f782419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58312/0 (socket says 172.21.15.156:58312) 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 -- 172.21.15.156:0/2284509734 learned_addr learned my addr 172.21.15.156:0/2284509734 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f7823fff700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78240ffd70 0x7f7824199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f7828b5b700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7824102270 0x7f78241a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 -- 172.21.15.156:0/2284509734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78240ffd70 msgr2=0x7f7824199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78240ffd70 0x7f7824199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 -- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7824102270 msgr2=0x7f78241a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7824102270 0x7f78241a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 -- 172.21.15.156:0/2284509734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0095f0 con 0x7f7824100730 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.070+0000 7f78237fe700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 0x7f782419a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f780c00b5c0 tx=0x7f780c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.071+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c01d070 con 0x7f7824100730 2024-03-12T05:08:18.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.071+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f780c00fb80 con 0x7f7824100730 2024-03-12T05:08:18.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.071+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78241a1e30 con 0x7f7824100730 2024-03-12T05:08:18.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.071+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c017650 con 0x7f7824100730 2024-03-12T05:08:18.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.071+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78241a21e0 con 0x7f7824100730 2024-03-12T05:08:18.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.072+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782410bc20 con 0x7f7824100730 2024-03-12T05:08:18.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.073+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f780c00fcf0 con 0x7f7824100730 2024-03-12T05:08:18.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.073+0000 7f78217fa700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78100a1ac0 0x7f78100a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:18.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.073+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f780c0ce7e0 con 0x7f7824100730 2024-03-12T05:08:18.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.074+0000 7f7823fff700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78100a1ac0 0x7f78100a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:18.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.074+0000 7f7823fff700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78100a1ac0 0x7f78100a3f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f781400ac50 tx=0x7f781400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:08:18.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.077+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f780c098280 con 0x7f7824100730 2024-03-12T05:08:18.279 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:18.280+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f782402d950 con 0x7f78100a1ac0 2024-03-12T05:08:19.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:18 smithi156 ceph-mon[36270]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:19.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:18 smithi156 ceph-mon[36270]: osdmap e69: 10 total, 10 up, 10 in 2024-03-12T05:08:19.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:08:19.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:08:19.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:18 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:19.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:18 smithi100 ceph-mon[32210]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:19.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:18 smithi100 ceph-mon[32210]: osdmap e69: 10 total, 10 up, 10 in 2024-03-12T05:08:19.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:08:19.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:08:19.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:18 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:19.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:18 smithi161 ceph-mon[36043]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:19.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:18 smithi161 ceph-mon[36043]: osdmap e69: 10 total, 10 up, 10 in 2024-03-12T05:08:19.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:08:19.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:08:19.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:18 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:20.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:19 smithi156 ceph-mon[36270]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:08:20.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:19 smithi100 ceph-mon[32210]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:08:20.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:19 smithi161 ceph-mon[36043]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:08:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:20 smithi100 ceph-mon[32210]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:21.132 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:20 smithi156 ceph-mon[36270]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:21.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:20 smithi161 ceph-mon[36043]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/987071817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]: dispatch 2024-03-12T05:08:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:21 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]: dispatch 2024-03-12T05:08:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:21 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]': finished 2024-03-12T05:08:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:21 smithi100 ceph-mon[32210]: osdmap e70: 11 total, 10 up, 11 in 2024-03-12T05:08:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:21 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.156:0/4265763521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:08:22.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:21 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/987071817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]: dispatch 2024-03-12T05:08:22.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:21 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]: dispatch 2024-03-12T05:08:22.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:21 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]': finished 2024-03-12T05:08:22.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:21 smithi156 ceph-mon[36270]: osdmap e70: 11 total, 10 up, 11 in 2024-03-12T05:08:22.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:21 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:22.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:21 smithi156 ceph-mon[36270]: from='client.? 172.21.15.156:0/4265763521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:08:22.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:21 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/987071817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]: dispatch 2024-03-12T05:08:22.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:21 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]: dispatch 2024-03-12T05:08:22.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:21 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359d96d0-445b-4e93-8811-5e0407d42957"}]': finished 2024-03-12T05:08:22.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:21 smithi161 ceph-mon[36043]: osdmap e70: 11 total, 10 up, 11 in 2024-03-12T05:08:22.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:21 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:22.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:21 smithi161 ceph-mon[36043]: from='client.? 172.21.15.156:0/4265763521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:08:23.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:22 smithi100 ceph-mon[32210]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:23.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:22 smithi156 ceph-mon[36270]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:23.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:22 smithi161 ceph-mon[36043]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:24.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:24 smithi156 ceph-mon[36270]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:25.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:24 smithi100 ceph-mon[32210]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:25.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:24 smithi161 ceph-mon[36043]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:26.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:26 smithi156 ceph-mon[36270]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:27.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:26 smithi100 ceph-mon[32210]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:27.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:26 smithi161 ceph-mon[36043]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:28.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:28 smithi156 ceph-mon[36270]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:28.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-12T05:08:28.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:28 smithi100 ceph-mon[32210]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-12T05:08:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:28 smithi161 ceph-mon[36043]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-12T05:08:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:29.859 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:29 smithi156 ceph-mon[36270]: Deploying daemon osd.10 on smithi156 2024-03-12T05:08:30.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:29 smithi100 ceph-mon[32210]: Deploying daemon osd.10 on smithi156 2024-03-12T05:08:30.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:29 smithi161 ceph-mon[36043]: Deploying daemon osd.10 on smithi156 2024-03-12T05:08:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:30 smithi100 ceph-mon[32210]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:31.107 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:30 smithi156 ceph-mon[36270]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:30 smithi161 ceph-mon[36043]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:33.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:32 smithi100 ceph-mon[32210]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:33.105 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:32 smithi156 ceph-mon[36270]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:33.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:32 smithi161 ceph-mon[36043]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:34.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:34.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:34.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:34.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:34 smithi156 ceph-mon[36270]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:34.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:34.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:34.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:34.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:34 smithi161 ceph-mon[36043]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:34.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:34.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:34.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:34.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:34 smithi100 ceph-mon[32210]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:36.692 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.693+0000 7f78217fa700 1 -- 172.21.15.156:0/2284509734 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f782402d950 con 0x7f78100a1ac0 2024-03-12T05:08:36.692 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 10 on host 'smithi156' 2024-03-12T05:08:36.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78100a1ac0 msgr2=0x7f78100a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:36.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78100a1ac0 0x7f78100a3f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f781400ac50 tx=0x7f781400a380 comp rx=0 tx=0).stop 2024-03-12T05:08:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 msgr2=0x7f782419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 0x7f782419a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f780c00b5c0 tx=0x7f780c00f690 comp rx=0 tx=0).stop 2024-03-12T05:08:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 shutdown_connections 2024-03-12T05:08:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78100a1ac0 0x7f78100a3f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7824102270 0x7f78241a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78240ffd70 0x7f7824199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:36.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 --2- 172.21.15.156:0/2284509734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7824100730 0x7f782419a290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:36.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 >> 172.21.15.156:0/2284509734 conn(0x7f78240fb910 msgr2=0x7f78240fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:08:36.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 shutdown_connections 2024-03-12T05:08:36.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-12T05:08:36.696+0000 7f782a5be700 1 -- 172.21.15.156:0/2284509734 wait complete. 2024-03-12T05:08:36.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:36 smithi156 ceph-mon[36270]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:36.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:36.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:37.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:36 smithi100 ceph-mon[32210]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:37.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:37.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:37.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:36 smithi161 ceph-mon[36043]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:37.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:37.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:37.372 DEBUG:teuthology.orchestra.run.smithi156:osd.10> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.10.service 2024-03-12T05:08:37.374 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-03-12T05:08:37.374 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-12T05:08:37.621 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:08:38.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:37 smithi161 ceph-mon[36043]: from='osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T05:08:38.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:37 smithi161 ceph-mon[36043]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T05:08:38.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:37 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:38.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:37 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:38.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:37 smithi100 ceph-mon[32210]: from='osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T05:08:38.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:37 smithi100 ceph-mon[32210]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T05:08:38.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:37 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:38.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:37 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:38.120 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:37 smithi156 ceph-mon[36270]: from='osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T05:08:38.121 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:37 smithi156 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-12T05:08:38.121 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:37 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:38.121 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:37 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:39.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:39.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-12T05:08:39.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: from='osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:08:39.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: osdmap e71: 11 total, 10 up, 11 in 2024-03-12T05:08:39.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:39.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:08:39.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:08:39.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: osdmap e72: 11 total, 10 up, 11 in 2024-03-12T05:08:39.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:39.047 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:08:38.759+0000 7f553a0dc700 -1 osd.10 0 waiting for initial osdmap 2024-03-12T05:08:39.047 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:08:38 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:08:38.773+0000 7f55366d0700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:08:39.049 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:39.049 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-12T05:08:39.049 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: from='osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:08:39.049 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: osdmap e71: 11 total, 10 up, 11 in 2024-03-12T05:08:39.049 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:39.050 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:08:39.050 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:08:39.050 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: osdmap e72: 11 total, 10 up, 11 in 2024-03-12T05:08:39.050 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-12T05:08:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: from='osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:08:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: osdmap e71: 11 total, 10 up, 11 in 2024-03-12T05:08:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:39.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-12T05:08:39.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-12T05:08:39.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: osdmap e72: 11 total, 10 up, 11 in 2024-03-12T05:08:39.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113709 s, 922 MB/s 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 2024-03-12T05:08:39.590 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-12T05:08:40.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.173 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.173 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.210 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.210 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-03-12T05:08:40.483 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:08:40.920 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:08:40.920 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:08:40.920 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:40.920 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] boot 2024-03-12T05:08:40.921 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: osdmap e73: 11 total, 11 up, 11 in 2024-03-12T05:08:40.921 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:40.921 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:40 smithi156 ceph-mon[36270]: osdmap e74: 11 total, 11 up, 11 in 2024-03-12T05:08:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:08:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:08:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] boot 2024-03-12T05:08:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: osdmap e73: 11 total, 11 up, 11 in 2024-03-12T05:08:41.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:41.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:40 smithi100 ceph-mon[32210]: osdmap e74: 11 total, 11 up, 11 in 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: osd.10 [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] boot 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: osdmap e73: 11 total, 11 up, 11 in 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:08:41.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:40 smithi161 ceph-mon[36043]: osdmap e74: 11 total, 11 up, 11 in 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3213688175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4102730 msgr2=0x7f73a4108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3213688175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4102730 0x7f73a4108fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f739c00b0a0 tx=0x7f739c00b3b0 comp rx=0 tx=0).stop 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3213688175 shutdown_connections 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3213688175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4100b90 0x7f73a4100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3213688175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4102730 0x7f73a4108fe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3213688175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a4101d90 0x7f73a41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3213688175 >> 172.21.15.161:0/3213688175 conn(0x7f73a40fc130 msgr2=0x7f73a40fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3213688175 shutdown_connections 2024-03-12T05:08:42.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.193+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3213688175 wait complete. 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 Processor -- start 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 -- start start 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 0x7f73a4104b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a4101d90 0x7f73a41031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4102730 0x7f73a4103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a4103ea0 con 0x7f73a4101d90 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a4104010 con 0x7f73a4102730 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a8fa0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a41a1e30 con 0x7f73a4100b90 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 0x7f73a4104b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 0x7f73a4104b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58010/0 (socket says 172.21.15.161:58010) 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.194+0000 7f73a259c700 1 -- 172.21.15.161:0/3470346613 learned_addr learned my addr 172.21.15.161:0/3470346613 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:08:42.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a1d9b700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a4101d90 0x7f73a41031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a2d9d700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4102730 0x7f73a4103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a259c700 1 -- 172.21.15.161:0/3470346613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4102730 msgr2=0x7f73a4103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a259c700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4102730 0x7f73a4103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a259c700 1 -- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a4101d90 msgr2=0x7f73a41031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a259c700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a4101d90 0x7f73a41031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a259c700 1 -- 172.21.15.161:0/3470346613 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009d00 con 0x7f73a4100b90 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a2d9d700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4102730 0x7f73a4103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a259c700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 0x7f73a4104b90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f739800b500 tx=0x7f739800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398011830 con 0x7f73a4100b90 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7398011e70 con 0x7f73a4100b90 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a41a2110 con 0x7f73a4100b90 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800f2e0 con 0x7f73a4100b90 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.195+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a41a2690 con 0x7f73a4100b90 2024-03-12T05:08:42.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.196+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a4068930 con 0x7f73a4100b90 2024-03-12T05:08:42.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.197+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f73980105f0 con 0x7f73a4100b90 2024-03-12T05:08:42.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.198+0000 7f73937fe700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f738c0a1ac0 0x7f738c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:08:42.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.198+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f73980cc810 con 0x7f73a4100b90 2024-03-12T05:08:42.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.198+0000 7f73a1d9b700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f738c0a1ac0 0x7f738c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:08:42.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.199+0000 7f73a1d9b700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f738c0a1ac0 0x7f738c0a3f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7394005950 tx=0x7f739400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:08:42.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.202+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7398096050 con 0x7f73a4100b90 2024-03-12T05:08:42.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi156 2024-03-12T05:08:42.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 4270M 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:42.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:42.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: osdmap e75: 11 total, 11 up, 11 in 2024-03-12T05:08:42.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:42 smithi100 ceph-mon[32210]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:42.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:08:42.405+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f73a406a8c0 con 0x7f738c0a1ac0 2024-03-12T05:08:42.403 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi156 2024-03-12T05:08:42.403 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 4270M 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:42.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:42.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: osdmap e75: 11 total, 11 up, 11 in 2024-03-12T05:08:42.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:42 smithi161 ceph-mon[36043]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:42.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi156 2024-03-12T05:08:42.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 4270M 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:42.457 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:08:42.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:42.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: osdmap e75: 11 total, 11 up, 11 in 2024-03-12T05:08:42.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:42 smithi156 ceph-mon[36270]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:43.307 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:43 smithi161 ceph-mon[36043]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:08:43.307 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:43 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:08:43.307 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:43 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:08:43.307 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:43 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:43.307 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:43 smithi161 ceph-mon[36043]: osdmap e76: 11 total, 11 up, 11 in 2024-03-12T05:08:43.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:43 smithi100 ceph-mon[32210]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:08:43.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:43 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:08:43.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:43 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:08:43.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:43 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:43.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:43 smithi100 ceph-mon[32210]: osdmap e76: 11 total, 11 up, 11 in 2024-03-12T05:08:43.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:43 smithi156 ceph-mon[36270]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:08:43.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:43 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T05:08:43.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:43 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T05:08:43.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:43 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:43.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:43 smithi156 ceph-mon[36270]: osdmap e76: 11 total, 11 up, 11 in 2024-03-12T05:08:44.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:44 smithi156 ceph-mon[36270]: osdmap e77: 11 total, 11 up, 11 in 2024-03-12T05:08:44.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:44 smithi156 ceph-mon[36270]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-03-12T05:08:44.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:44 smithi161 ceph-mon[36043]: osdmap e77: 11 total, 11 up, 11 in 2024-03-12T05:08:44.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:44 smithi161 ceph-mon[36043]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-03-12T05:08:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:44 smithi100 ceph-mon[32210]: osdmap e77: 11 total, 11 up, 11 in 2024-03-12T05:08:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:44 smithi100 ceph-mon[32210]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-03-12T05:08:45.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/546286636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]: dispatch 2024-03-12T05:08:45.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:45 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]: dispatch 2024-03-12T05:08:45.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:45 smithi156 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]': finished 2024-03-12T05:08:45.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:45 smithi156 ceph-mon[36270]: osdmap e78: 12 total, 11 up, 12 in 2024-03-12T05:08:45.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:45 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:08:45.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.161:0/2216195618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:08:45.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/546286636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]: dispatch 2024-03-12T05:08:45.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:45 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]: dispatch 2024-03-12T05:08:45.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:45 smithi161 ceph-mon[36043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]': finished 2024-03-12T05:08:45.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:45 smithi161 ceph-mon[36043]: osdmap e78: 12 total, 11 up, 12 in 2024-03-12T05:08:45.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:45 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:08:45.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.161:0/2216195618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:08:46.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/546286636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]: dispatch 2024-03-12T05:08:46.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:45 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]: dispatch 2024-03-12T05:08:46.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:45 smithi100 ceph-mon[32210]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230"}]': finished 2024-03-12T05:08:46.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:45 smithi100 ceph-mon[32210]: osdmap e78: 12 total, 11 up, 12 in 2024-03-12T05:08:46.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:45 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:08:46.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.161:0/2216195618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T05:08:46.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:46 smithi156 ceph-mon[36270]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:08:46.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:46 smithi161 ceph-mon[36043]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:08:47.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:46 smithi100 ceph-mon[32210]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-12T05:08:48.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:48 smithi156 ceph-mon[36270]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:08:48.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:48 smithi161 ceph-mon[36043]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:08:49.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:48 smithi100 ceph-mon[32210]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-12T05:08:50.944 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:50 smithi161 ceph-mon[36043]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-12T05:08:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:50 smithi100 ceph-mon[32210]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-12T05:08:51.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:50 smithi156 ceph-mon[36270]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-12T05:08:52.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:52 smithi161 ceph-mon[36043]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:52.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-12T05:08:52.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:53.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:52 smithi100 ceph-mon[32210]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:53.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-12T05:08:53.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:52 smithi156 ceph-mon[36270]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-12T05:08:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:08:53.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:53 smithi161 ceph-mon[36043]: Deploying daemon osd.11 on smithi161 2024-03-12T05:08:54.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:53 smithi100 ceph-mon[32210]: Deploying daemon osd.11 on smithi161 2024-03-12T05:08:54.107 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:53 smithi156 ceph-mon[36270]: Deploying daemon osd.11 on smithi161 2024-03-12T05:08:54.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:54 smithi100 ceph-mon[32210]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:54.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:54 smithi161 ceph-mon[36043]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:55.120 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:54 smithi156 ceph-mon[36270]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:56.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:56 smithi161 ceph-mon[36043]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:57.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:56 smithi100 ceph-mon[32210]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:56 smithi156 ceph-mon[36270]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:58.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:58 smithi161 ceph-mon[36043]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:58.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:58 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:58.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:58 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:58.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:08:58 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:59.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:58 smithi100 ceph-mon[32210]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:59.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:58 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:59.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:58 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:59.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:08:58 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:08:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:58 smithi156 ceph-mon[36270]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:08:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:58 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:58 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:08:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:08:58 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:00.921 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:00 smithi161 ceph-mon[36043]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:01.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:00 smithi100 ceph-mon[32210]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:01.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:00 smithi156 ceph-mon[36270]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:01.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.820+0000 7f73937fe700 1 -- 172.21.15.161:0/3470346613 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f73a406a8c0 con 0x7f738c0a1ac0 2024-03-12T05:09:01.819 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f738c0a1ac0 msgr2=0x7f738c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f738c0a1ac0 0x7f738c0a3f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7394005950 tx=0x7f739400a300 comp rx=0 tx=0).stop 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 msgr2=0x7f73a4104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 0x7f73a4104b90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f739800b500 tx=0x7f739800b810 comp rx=0 tx=0).stop 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 shutdown_connections 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f738c0a1ac0 0x7f738c0a3f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4102730 0x7f73a4103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a4100b90 0x7f73a4104b90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 --2- 172.21.15.161:0/3470346613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a4101d90 0x7f73a41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 >> 172.21.15.161:0/3470346613 conn(0x7f73a40fc130 msgr2=0x7f73a406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:01.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 shutdown_connections 2024-03-12T05:09:01.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:01.823+0000 7f73a8fa0700 1 -- 172.21.15.161:0/3470346613 wait complete. 2024-03-12T05:09:02.474 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.11.service 2024-03-12T05:09:02.476 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-12T05:09:02.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T05:09:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: from='osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:02 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: from='osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:02 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: from='osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:03.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:02 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:04.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:03 smithi100 ceph-mon[32210]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-12T05:09:04.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:03 smithi100 ceph-mon[32210]: osdmap e79: 12 total, 11 up, 12 in 2024-03-12T05:09:04.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:04.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:03 smithi100 ceph-mon[32210]: from='osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:09:04.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:03 smithi100 ceph-mon[32210]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:09:04.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:03 smithi161 ceph-mon[36043]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-12T05:09:04.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:03 smithi161 ceph-mon[36043]: osdmap e79: 12 total, 11 up, 12 in 2024-03-12T05:09:04.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:04.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:03 smithi161 ceph-mon[36043]: from='osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:09:04.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:03 smithi161 ceph-mon[36043]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:09:04.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:03 smithi156 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-12T05:09:04.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:03 smithi156 ceph-mon[36270]: osdmap e79: 12 total, 11 up, 12 in 2024-03-12T05:09:04.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:04.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:03 smithi156 ceph-mon[36270]: from='osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:09:04.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:03 smithi156 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 -- 172.21.15.100:0/1267152487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4103840 msgr2=0x7fccc410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 --2- 172.21.15.100:0/1267152487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4103840 0x7fccc410a0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b0d0 tx=0x7fccbc00b3e0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 -- 172.21.15.100:0/1267152487 shutdown_connections 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 --2- 172.21.15.100:0/1267152487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4103840 0x7fccc410a0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 --2- 172.21.15.100:0/1267152487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4101ca0 0x7fccc41020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 --2- 172.21.15.100:0/1267152487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccc4102ea0 0x7fccc4103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 -- 172.21.15.100:0/1267152487 >> 172.21.15.100:0/1267152487 conn(0x7fccc40fd280 msgr2=0x7fccc40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 -- 172.21.15.100:0/1267152487 shutdown_connections 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.476+0000 7fccc90dd700 1 -- 172.21.15.100:0/1267152487 wait complete. 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 Processor -- start 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 -- start start 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 0x7fccc4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccc4102ea0 0x7fccc41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4103840 0x7fccc419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419f900 con 0x7fccc4101ca0 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419fa40 con 0x7fccc4102ea0 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc90dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419fc00 con 0x7fccc4103840 2024-03-12T05:09:04.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 0x7fccc4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 0x7fccc4197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39368/0 (socket says 172.21.15.100:39368) 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 -- 172.21.15.100:0/3341215596 learned_addr learned my addr 172.21.15.100:0/3341215596 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc359e700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4103840 0x7fccc419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccba59c700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccc4102ea0 0x7fccc41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 -- 172.21.15.100:0/3341215596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4103840 msgr2=0x7fccc419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4103840 0x7fccc419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 -- 172.21.15.100:0/3341215596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccc4102ea0 msgr2=0x7fccc41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccc4102ea0 0x7fccc41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.477+0000 7fccc2d9d700 1 -- 172.21.15.100:0/3341215596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc009d00 con 0x7fccc4101ca0 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.478+0000 7fccc2d9d700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 0x7fccc4197db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fccb400d980 tx=0x7fccb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.478+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4004020 con 0x7fccc4101ca0 2024-03-12T05:09:04.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.478+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccb400f690 con 0x7fccc4101ca0 2024-03-12T05:09:04.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.478+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4003b40 con 0x7fccc4101ca0 2024-03-12T05:09:04.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.478+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc419fee0 con 0x7fccc4101ca0 2024-03-12T05:09:04.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.478+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc41a0370 con 0x7fccc4101ca0 2024-03-12T05:09:04.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.479+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4109000 con 0x7fccc4101ca0 2024-03-12T05:09:04.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.480+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fccb400fb20 con 0x7fccc4101ca0 2024-03-12T05:09:04.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.480+0000 7fccc0d99700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fccb00a1ac0 0x7fccb00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:04.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.481+0000 7fccba59c700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fccb00a1ac0 0x7fccb00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:04.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.481+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fccb40cccd0 con 0x7fccc4101ca0 2024-03-12T05:09:04.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.481+0000 7fccba59c700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fccb00a1ac0 0x7fccb00a3f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fccac009a10 tx=0x7fccac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:04.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.484+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fccb40964a0 con 0x7fccc4101ca0 2024-03-12T05:09:04.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.675+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fccc410d660 con 0x7fccc4101ca0 2024-03-12T05:09:04.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.676+0000 7fccc0d99700 1 -- 172.21.15.100:0/3341215596 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7fccb4099b60 con 0x7fccc4101ca0 2024-03-12T05:09:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:04.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fccb00a1ac0 msgr2=0x7fccb00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:04.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fccb00a1ac0 0x7fccb00a3f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fccac009a10 tx=0x7fccac009190 comp rx=0 tx=0).stop 2024-03-12T05:09:04.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 msgr2=0x7fccc4197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:04.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 0x7fccc4197db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fccb400d980 tx=0x7fccb400dc90 comp rx=0 tx=0).stop 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 shutdown_connections 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fccb00a1ac0 0x7fccb00a3f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4103840 0x7fccc419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc4101ca0 0x7fccc4197db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 --2- 172.21.15.100:0/3341215596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccc4102ea0 0x7fccc41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 >> 172.21.15.100:0/3341215596 conn(0x7fccc40fd280 msgr2=0x7fccc4106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 shutdown_connections 2024-03-12T05:09:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:04.678+0000 7fccc90dd700 1 -- 172.21.15.100:0/3341215596 wait complete. 2024-03-12T05:09:04.724 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:09:04.662+0000 7f27d130c700 -1 osd.11 0 waiting for initial osdmap 2024-03-12T05:09:04.724 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:09:04.681+0000 7f27c98f8700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: osdmap e80: 12 total, 11 up, 12 in 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: from='osd.11 ' entity='osd.11' 2024-03-12T05:09:04.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:04 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3341215596' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: osdmap e80: 12 total, 11 up, 12 in 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: from='osd.11 ' entity='osd.11' 2024-03-12T05:09:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:04 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3341215596' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T05:09:05.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-12T05:09:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: osdmap e80: 12 total, 11 up, 12 in 2024-03-12T05:09:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: from='osd.11 ' entity='osd.11' 2024-03-12T05:09:05.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:04 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3341215596' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T05:09:05.321 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1710220119,"num_in_osds":12,"osd_in_since":1710220124,"num_remapped_pgs":0} 2024-03-12T05:09:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:05 smithi100 ceph-mon[32210]: purged_snaps scrub starts 2024-03-12T05:09:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:05 smithi100 ceph-mon[32210]: purged_snaps scrub ok 2024-03-12T05:09:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:05 smithi100 ceph-mon[32210]: osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] boot 2024-03-12T05:09:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:05 smithi100 ceph-mon[32210]: osdmap e81: 12 total, 12 up, 12 in 2024-03-12T05:09:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:05 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:06.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:05 smithi156 ceph-mon[36270]: purged_snaps scrub starts 2024-03-12T05:09:06.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:05 smithi156 ceph-mon[36270]: purged_snaps scrub ok 2024-03-12T05:09:06.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:06.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:05 smithi156 ceph-mon[36270]: osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] boot 2024-03-12T05:09:06.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:05 smithi156 ceph-mon[36270]: osdmap e81: 12 total, 12 up, 12 in 2024-03-12T05:09:06.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:05 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:06.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:05 smithi161 ceph-mon[36043]: purged_snaps scrub starts 2024-03-12T05:09:06.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:05 smithi161 ceph-mon[36043]: purged_snaps scrub ok 2024-03-12T05:09:06.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:06.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:05 smithi161 ceph-mon[36043]: osd.11 [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] boot 2024-03-12T05:09:06.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:05 smithi161 ceph-mon[36043]: osdmap e81: 12 total, 12 up, 12 in 2024-03-12T05:09:06.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:05 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:09:06.322 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T05:09:06.564 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:09:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:06 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:07.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:06 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:09:07.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:07.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:06 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:08.016 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:07 smithi100 ceph-mon[32210]: Detected new or changed devices on smithi161 2024-03-12T05:09:08.016 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:07 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 4270M 2024-03-12T05:09:08.016 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:07 smithi100 ceph-mon[32210]: osdmap e82: 12 total, 12 up, 12 in 2024-03-12T05:09:08.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:07 smithi156 ceph-mon[36270]: Detected new or changed devices on smithi161 2024-03-12T05:09:08.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:07 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 4270M 2024-03-12T05:09:08.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:07 smithi156 ceph-mon[36270]: osdmap e82: 12 total, 12 up, 12 in 2024-03-12T05:09:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:07 smithi161 ceph-mon[36043]: Detected new or changed devices on smithi161 2024-03-12T05:09:08.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:07 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 4270M 2024-03-12T05:09:08.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:07 smithi161 ceph-mon[36043]: osdmap e82: 12 total, 12 up, 12 in 2024-03-12T05:09:08.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 -- 172.21.15.100:0/3721205311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc81027e0 msgr2=0x7f5cc8109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:08.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- 172.21.15.100:0/3721205311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc81027e0 0x7f5cc8109090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000b0d0 tx=0x7f5cc000b3e0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 -- 172.21.15.100:0/3721205311 shutdown_connections 2024-03-12T05:09:08.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- 172.21.15.100:0/3721205311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 0x7f5cc8101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- 172.21.15.100:0/3721205311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc81027e0 0x7f5cc8109090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- 172.21.15.100:0/3721205311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc8101e40 0x7f5cc81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 -- 172.21.15.100:0/3721205311 >> 172.21.15.100:0/3721205311 conn(0x7f5cc80fc200 msgr2=0x7f5cc80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:08.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 -- 172.21.15.100:0/3721205311 shutdown_connections 2024-03-12T05:09:08.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 -- 172.21.15.100:0/3721205311 wait complete. 2024-03-12T05:09:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 Processor -- start 2024-03-12T05:09:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 -- start start 2024-03-12T05:09:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 0x7f5cc8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 0x7f5cc81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.286+0000 7f5ccf771700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc81027e0 0x7f5cc819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccf771700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819f820 con 0x7f5cc8101e40 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccf771700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819f960 con 0x7f5cc8100c40 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccf771700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819fb20 con 0x7f5cc81027e0 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 0x7f5cc81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 0x7f5cc81981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39376/0 (socket says 172.21.15.100:39376) 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 -- 172.21.15.100:0/893290066 learned_addr learned my addr 172.21.15.100:0/893290066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccdd0e700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc81027e0 0x7f5cc819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccd50d700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 0x7f5cc8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 -- 172.21.15.100:0/893290066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc81027e0 msgr2=0x7f5cc819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc81027e0 0x7f5cc819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 -- 172.21.15.100:0/893290066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 msgr2=0x7f5cc8197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 0x7f5cc8197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 -- 172.21.15.100:0/893290066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0009d00 con 0x7f5cc8101e40 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccd50d700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 0x7f5cc8197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cccd0c700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 0x7f5cc81981e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800c920 tx=0x7f5cb800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8007b00 con 0x7f5cc8101e40 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cb800ce70 con 0x7f5cc8101e40 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8109b00 con 0x7f5cc8101e40 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8005610 con 0x7f5cc8101e40 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.287+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc810a000 con 0x7f5cc8101e40 2024-03-12T05:09:08.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.288+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8068930 con 0x7f5cc8101e40 2024-03-12T05:09:08.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.290+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5cb8004750 con 0x7f5cc8101e40 2024-03-12T05:09:08.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.290+0000 7f5cbe7fc700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1ac0 0x7f5cb40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:08.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.290+0000 7f5ccd50d700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1ac0 0x7f5cb40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:08.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.291+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5cb80cd020 con 0x7f5cc8101e40 2024-03-12T05:09:08.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.291+0000 7f5ccd50d700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1ac0 0x7f5cb40a3f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8198b80 tx=0x7f5cc400a2b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:08.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.294+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cb80966b0 con 0x7f5cc8101e40 2024-03-12T05:09:08.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.488+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5cc810d680 con 0x7f5cc8101e40 2024-03-12T05:09:08.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.489+0000 7f5cbe7fc700 1 -- 172.21.15.100:0/893290066 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5cb8099d70 con 0x7f5cc8101e40 2024-03-12T05:09:08.487 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1ac0 msgr2=0x7f5cb40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1ac0 0x7f5cb40a3f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8198b80 tx=0x7f5cc400a2b0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 msgr2=0x7f5cc81981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 0x7f5cc81981e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800c920 tx=0x7f5cb800cc30 comp rx=0 tx=0).stop 2024-03-12T05:09:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 shutdown_connections 2024-03-12T05:09:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1ac0 0x7f5cb40a3f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc8100c40 0x7f5cc8197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc81027e0 0x7f5cc819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 --2- 172.21.15.100:0/893290066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc8101e40 0x7f5cc81981e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:08.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 >> 172.21.15.100:0/893290066 conn(0x7f5cc80fc200 msgr2=0x7f5cc8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:08.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 shutdown_connections 2024-03-12T05:09:08.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:08.491+0000 7f5ccf771700 1 -- 172.21.15.100:0/893290066 wait complete. 2024-03-12T05:09:09.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:08 smithi100 ceph-mon[32210]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:09.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:08 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/893290066' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T05:09:09.094 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1710220145,"num_in_osds":12,"osd_in_since":1710220124,"num_remapped_pgs":0} 2024-03-12T05:09:09.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T05:09:09.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:08 smithi156 ceph-mon[36270]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:09.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:08 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/893290066' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T05:09:09.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:08 smithi161 ceph-mon[36043]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:09.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:08 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/893290066' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T05:09:09.335 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:09:10.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 -- 172.21.15.100:0/2885893923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554102680 msgr2=0x7fe55410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:10.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 --2- 172.21.15.100:0/2885893923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554102680 0x7fe55410eac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00b0d0 tx=0x7fe54c00b3e0 comp rx=0 tx=0).stop 2024-03-12T05:09:10.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 -- 172.21.15.100:0/2885893923 shutdown_connections 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 --2- 172.21.15.100:0/2885893923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554102680 0x7fe55410eac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 --2- 172.21.15.100:0/2885893923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 0x7fe554108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 --2- 172.21.15.100:0/2885893923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554101cc0 0x7fe554102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 -- 172.21.15.100:0/2885893923 >> 172.21.15.100:0/2885893923 conn(0x7fe554078da0 msgr2=0x7fe5540791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 -- 172.21.15.100:0/2885893923 shutdown_connections 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.826+0000 7fe55befa700 1 -- 172.21.15.100:0/2885893923 wait complete. 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 Processor -- start 2024-03-12T05:09:10.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 -- start start 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 0x7fe554198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554102680 0x7fe554198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 0x7fe55419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55419d1a0 con 0x7fe554101cc0 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55419d310 con 0x7fe554102680 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe55befa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55419d4d0 con 0x7fe554107ca0 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe559c96700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 0x7fe554198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:10.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe559c96700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 0x7fe554198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45482/0 (socket says 172.21.15.100:45482) 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.827+0000 7fe559c96700 1 -- 172.21.15.100:0/2142958400 learned_addr learned my addr 172.21.15.100:0/2142958400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559495700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554102680 0x7fe554198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe55a497700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 0x7fe55419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559c96700 1 -- 172.21.15.100:0/2142958400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 msgr2=0x7fe55419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559c96700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 0x7fe55419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559c96700 1 -- 172.21.15.100:0/2142958400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554102680 msgr2=0x7fe554198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559c96700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554102680 0x7fe554198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:10.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559c96700 1 -- 172.21.15.100:0/2142958400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009d00 con 0x7fe554101cc0 2024-03-12T05:09:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559495700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554102680 0x7fe554198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe55a497700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 0x7fe55419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe559c96700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 0x7fe554198310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe55000b530 tx=0x7fe55000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550011830 con 0x7fe554101cc0 2024-03-12T05:09:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55419d7b0 con 0x7fe554101cc0 2024-03-12T05:09:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe550011e70 con 0x7fe554101cc0 2024-03-12T05:09:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55000f2e0 con 0x7fe554101cc0 2024-03-12T05:09:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.828+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5541a5010 con 0x7fe554101cc0 2024-03-12T05:09:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.829+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55410a180 con 0x7fe554101cc0 2024-03-12T05:09:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.830+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe5500105f0 con 0x7fe554101cc0 2024-03-12T05:09:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.831+0000 7fe54affd700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5400a1ac0 0x7fe5400a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.831+0000 7fe559495700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5400a1ac0 0x7fe5400a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.831+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe5500ccf40 con 0x7fe554101cc0 2024-03-12T05:09:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.831+0000 7fe559495700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5400a1ac0 0x7fe5400a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe544005fd0 tx=0x7fe544005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:10.834+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5500965f0 con 0x7fe554101cc0 2024-03-12T05:09:11.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.025+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe554111d80 con 0x7fe554101cc0 2024-03-12T05:09:11.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.026+0000 7fe54affd700 1 -- 172.21.15.100:0/2142958400 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18486 (secure 0 0 0) 0x7fe550010380 con 0x7fe554101cc0 2024-03-12T05:09:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:11.025 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":82,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:09:07.135605+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.028+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5400a1ac0 msgr2=0x7fe5400a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.028+0000 7fe55befa700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5400a1ac0 0x7fe5400a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe544005fd0 tx=0x7fe544005ee0 comp rx=0 tx=0).stop 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.028+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 msgr2=0x7fe554198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.028+0000 7fe55befa700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 0x7fe554198310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe55000b530 tx=0x7fe55000b8f0 comp rx=0 tx=0).stop 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 shutdown_connections 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5400a1ac0 0x7fe5400a3f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe554107ca0 0x7fe55419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe554101cc0 0x7fe554198310 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 --2- 172.21.15.100:0/2142958400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe554102680 0x7fe554198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:11.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 >> 172.21.15.100:0/2142958400 conn(0x7fe554078da0 msgr2=0x7fe5541008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:11.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 shutdown_connections 2024-03-12T05:09:11.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:11.029+0000 7fe55befa700 1 -- 172.21.15.100:0/2142958400 wait complete. 2024-03-12T05:09:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:10 smithi156 ceph-mon[36270]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:10 smithi161 ceph-mon[36043]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:11.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:10 smithi100 ceph-mon[32210]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:11.637 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:09:11.638 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-12T05:09:11.877 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:09:11.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:11 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2142958400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:12.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:11 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2142958400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:12.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:11 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2142958400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:13.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:12 smithi100 ceph-mon[32210]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:13.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:12 smithi156 ceph-mon[36270]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:13.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:12 smithi161 ceph-mon[36043]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:13.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 -- 172.21.15.100:0/3029485378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 msgr2=0x7f691810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:13.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 --2- 172.21.15.100:0/3029485378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f691810da40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f691000b0a0 tx=0x7f691000b3b0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 -- 172.21.15.100:0/3029485378 shutdown_connections 2024-03-12T05:09:13.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 --2- 172.21.15.100:0/3029485378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f691810da40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 --2- 172.21.15.100:0/3029485378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6918106d60 0x7f6918107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 --2- 172.21.15.100:0/3029485378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918100e50 0x7f69181012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 -- 172.21.15.100:0/3029485378 >> 172.21.15.100:0/3029485378 conn(0x7f69180779d0 msgr2=0x7f6918077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 -- 172.21.15.100:0/3029485378 shutdown_connections 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.432+0000 7f691dea7700 1 -- 172.21.15.100:0/3029485378 wait complete. 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 Processor -- start 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 -- start start 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6918100e50 0x7f6918199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f6918199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918106d60 0x7f69181a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6918199ff0 con 0x7f6918101810 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69181a47d0 con 0x7f6918106d60 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f691dea7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69181a4910 con 0x7f6918100e50 2024-03-12T05:09:13.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f6918199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f6918199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45496/0 (socket says 172.21.15.100:45496) 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 -- 172.21.15.100:0/3492050334 learned_addr learned my addr 172.21.15.100:0/3492050334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f6917fff700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918106d60 0x7f69181a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f69177fe700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6918100e50 0x7f6918199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 -- 172.21.15.100:0/3492050334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6918100e50 msgr2=0x7f6918199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6918100e50 0x7f6918199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 -- 172.21.15.100:0/3492050334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918106d60 msgr2=0x7f69181a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918106d60 0x7f69181a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f690ffff700 1 -- 172.21.15.100:0/3492050334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910009d00 con 0x7f6918101810 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.433+0000 7f6917fff700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918106d60 0x7f69181a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.434+0000 7f690ffff700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f6918199750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f690000c950 tx=0x7f690000cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.434+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900007ac0 con 0x7f6918101810 2024-03-12T05:09:13.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.434+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69181a4bc0 con 0x7f6918101810 2024-03-12T05:09:13.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.434+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6900004d10 con 0x7f6918101810 2024-03-12T05:09:13.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.434+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900005610 con 0x7f6918101810 2024-03-12T05:09:13.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.434+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69181a50c0 con 0x7f6918101810 2024-03-12T05:09:13.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.435+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6918109190 con 0x7f6918101810 2024-03-12T05:09:13.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.436+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6900004750 con 0x7f6918101810 2024-03-12T05:09:13.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.436+0000 7f69157fa700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f69040a1ac0 0x7f69040a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:13.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.437+0000 7f69177fe700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f69040a1ac0 0x7f69040a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:13.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.437+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69000ccfe0 con 0x7f6918101810 2024-03-12T05:09:13.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.437+0000 7f69177fe700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f69040a1ac0 0x7f69040a3f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:13.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.440+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6900096690 con 0x7f6918101810 2024-03-12T05:09:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.629+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f691806a9a0 con 0x7f6918101810 2024-03-12T05:09:13.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.629+0000 7f69157fa700 1 -- 172.21.15.100:0/3492050334 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6900099d50 con 0x7f6918101810 2024-03-12T05:09:13.628 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f69040a1ac0 msgr2=0x7f69040a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f69040a1ac0 0x7f69040a3f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908008040 comp rx=0 tx=0).stop 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 msgr2=0x7f6918199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f6918199750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f690000c950 tx=0x7f690000cd10 comp rx=0 tx=0).stop 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 shutdown_connections 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f69040a1ac0 0x7f69040a3f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6918106d60 0x7f69181a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6918100e50 0x7f6918199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 --2- 172.21.15.100:0/3492050334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6918101810 0x7f6918199750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:13.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 >> 172.21.15.100:0/3492050334 conn(0x7f69180779d0 msgr2=0x7f69180ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:13.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 shutdown_connections 2024-03-12T05:09:13.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:13.632+0000 7f691dea7700 1 -- 172.21.15.100:0/3492050334 wait complete. 2024-03-12T05:09:14.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:13 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3492050334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:14.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:13 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3492050334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:13 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3492050334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:14.315 INFO:tasks.cephadm:Adding mds.a on smithi100 2024-03-12T05:09:14.315 INFO:tasks.cephadm:Adding mds.d on smithi100 2024-03-12T05:09:14.315 INFO:tasks.cephadm:Adding mds.g on smithi100 2024-03-12T05:09:14.315 INFO:tasks.cephadm:Adding mds.j on smithi100 2024-03-12T05:09:14.315 INFO:tasks.cephadm:Adding mds.b on smithi156 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.e on smithi156 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.h on smithi156 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.k on smithi156 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-03-12T05:09:14.316 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-03-12T05:09:14.316 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph orch apply mds all '12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-03-12T05:09:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:09:14.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:14 smithi161 ceph-mon[36043]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:15.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:14 smithi156 ceph-mon[36270]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:15.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:14 smithi100 ceph-mon[32210]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:16.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 -- 172.21.15.161:0/1207959166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c105090 msgr2=0x7fbb4c107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:16.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/1207959166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c105090 0x7fbb4c107480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009990 tx=0x7fbb40009ca0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 -- 172.21.15.161:0/1207959166 shutdown_connections 2024-03-12T05:09:16.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/1207959166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c102760 0x7fbb4c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/1207959166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c1079c0 0x7fbb4c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/1207959166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c105090 0x7fbb4c107480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 -- 172.21.15.161:0/1207959166 >> 172.21.15.161:0/1207959166 conn(0x7fbb4c0fc190 msgr2=0x7fbb4c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 -- 172.21.15.161:0/1207959166 shutdown_connections 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 -- 172.21.15.161:0/1207959166 wait complete. 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.132+0000 7fbb52a3b700 1 Processor -- start 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 -- start start 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 0x7fbb4c1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c105090 0x7fbb4c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c1079c0 0x7fbb4c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1013d0 con 0x7fbb4c105090 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c101540 con 0x7fbb4c1079c0 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb52a3b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1a1c40 con 0x7fbb4c102760 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 0x7fbb4c1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:16.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 0x7fbb4c1020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34310/0 (socket says 172.21.15.161:34310) 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 -- 172.21.15.161:0/2197790023 learned_addr learned my addr 172.21.15.161:0/2197790023 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4b7fe700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c105090 0x7fbb4c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb50fd8700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c1079c0 0x7fbb4c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 -- 172.21.15.161:0/2197790023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c1079c0 msgr2=0x7fbb4c100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c1079c0 0x7fbb4c100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 -- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c105090 msgr2=0x7fbb4c100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c105090 0x7fbb4c100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4bfff700 1 -- 172.21.15.161:0/2197790023 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400095f0 con 0x7fbb4c102760 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.133+0000 7fbb4b7fe700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c105090 0x7fbb4c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.134+0000 7fbb4bfff700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 0x7fbb4c1020c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d950 tx=0x7fbb3c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.134+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c009b40 con 0x7fbb4c102760 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.134+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c1a1f20 con 0x7fbb4c102760 2024-03-12T05:09:16.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.134+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c009ca0 con 0x7fbb4c102760 2024-03-12T05:09:16.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.134+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00f6b0 con 0x7fbb4c102760 2024-03-12T05:09:16.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.134+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c1a2340 con 0x7fbb4c102760 2024-03-12T05:09:16.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.135+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c0fdd10 con 0x7fbb4c102760 2024-03-12T05:09:16.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.136+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbb3c010460 con 0x7fbb4c102760 2024-03-12T05:09:16.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.136+0000 7fbb497fa700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb340a1ac0 0x7fbb340a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:16.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.136+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb3c0cd2a0 con 0x7fbb4c102760 2024-03-12T05:09:16.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.137+0000 7fbb4b7fe700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb340a1ac0 0x7fbb340a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:16.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.137+0000 7fbb4b7fe700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb340a1ac0 0x7fbb340a3f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000b5c0 tx=0x7fbb40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:16.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.141+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb3c096950 con 0x7fbb4c102760 2024-03-12T05:09:16.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.350+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4c02da10 con 0x7fbb340a1ac0 2024-03-12T05:09:16.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.358+0000 7fbb497fa700 1 -- 172.21.15.161:0/2197790023 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fbb4c02da10 con 0x7fbb340a1ac0 2024-03-12T05:09:16.356 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.360+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb340a1ac0 msgr2=0x7fbb340a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.360+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb340a1ac0 0x7fbb340a3f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000b5c0 tx=0x7fbb40005fb0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.360+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 msgr2=0x7fbb4c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.360+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 0x7fbb4c1020c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d950 tx=0x7fbb3c00dd10 comp rx=0 tx=0).stop 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 shutdown_connections 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb340a1ac0 0x7fbb340a3f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c1079c0 0x7fbb4c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c102760 0x7fbb4c1020c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 --2- 172.21.15.161:0/2197790023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb4c105090 0x7fbb4c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 >> 172.21.15.161:0/2197790023 conn(0x7fbb4c0fc190 msgr2=0x7fbb4c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 shutdown_connections 2024-03-12T05:09:16.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-12T05:09:16.361+0000 7fbb52a3b700 1 -- 172.21.15.161:0/2197790023 wait complete. 2024-03-12T05:09:17.052 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:16 smithi161 ceph-mon[36043]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:17.052 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:17.052 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:16 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:17.053 DEBUG:teuthology.orchestra.run.smithi100:mds.a> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.a.service 2024-03-12T05:09:17.056 DEBUG:teuthology.orchestra.run.smithi100:mds.d> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.d.service 2024-03-12T05:09:17.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:16 smithi100 ceph-mon[32210]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:17.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:17.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:16 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:17.069 DEBUG:teuthology.orchestra.run.smithi100:mds.g> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.g.service 2024-03-12T05:09:17.112 DEBUG:teuthology.orchestra.run.smithi100:mds.j> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.j.service 2024-03-12T05:09:17.155 DEBUG:teuthology.orchestra.run.smithi156:mds.b> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.b.service 2024-03-12T05:09:17.157 DEBUG:teuthology.orchestra.run.smithi156:mds.e> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.e.service 2024-03-12T05:09:17.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:16 smithi156 ceph-mon[36270]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:17.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:17.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:16 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:17.181 DEBUG:teuthology.orchestra.run.smithi156:mds.h> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.h.service 2024-03-12T05:09:17.224 DEBUG:teuthology.orchestra.run.smithi156:mds.k> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.k.service 2024-03-12T05:09:17.267 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.c.service 2024-03-12T05:09:17.269 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.f.service 2024-03-12T05:09:17.312 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.i.service 2024-03-12T05:09:17.355 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.l.service 2024-03-12T05:09:17.397 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-12T05:09:17.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:09:17.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1435594313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 msgr2=0x7f04dc102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1435594313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc102bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009900 tx=0x7f04d802bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:17.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1435594313 shutdown_connections 2024-03-12T05:09:17.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1435594313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04dc0ffbe0 0x7f04dc109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1435594313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04dc0ff220 0x7f04dc0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1435594313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc102bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.523+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1435594313 >> 172.21.15.100:0/1435594313 conn(0x7f04dc0fadc0 msgr2=0x7f04dc0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1435594313 shutdown_connections 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1435594313 wait complete. 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 Processor -- start 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 -- start start 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04dc0ff220 0x7f04dc1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04dc0ffbe0 0x7f04dc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc19eda0 con 0x7f04dc1027b0 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc19ef10 con 0x7f04dc0ffbe0 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e3a9a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc19f0d0 con 0x7f04dc0ff220 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e2037640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e2037640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45506/0 (socket says 172.21.15.100:45506) 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e2037640 1 -- 172.21.15.100:0/1797976109 learned_addr learned my addr 172.21.15.100:0/1797976109 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e1035640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04dc0ffbe0 0x7f04dc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.524+0000 7f04e1836640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04dc0ff220 0x7f04dc1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e2037640 1 -- 172.21.15.100:0/1797976109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04dc0ff220 msgr2=0x7f04dc1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e2037640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04dc0ff220 0x7f04dc1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e2037640 1 -- 172.21.15.100:0/1797976109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04dc0ffbe0 msgr2=0x7f04dc1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e2037640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04dc0ffbe0 0x7f04dc1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e2037640 1 -- 172.21.15.100:0/1797976109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d8009560 con 0x7f04dc1027b0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e2037640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc19e860 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009fd0 tx=0x7f04d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400bea0 con 0x7f04dc1027b0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04d4009940 con 0x7f04dc1027b0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc19f3b0 con 0x7f04dc1027b0 2024-03-12T05:09:17.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d40106c0 con 0x7f04dc1027b0 2024-03-12T05:09:17.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.525+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc19f8c0 con 0x7f04dc1027b0 2024-03-12T05:09:17.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.526+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc1086f0 con 0x7f04dc1027b0 2024-03-12T05:09:17.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.527+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f04d4004240 con 0x7f04dc1027b0 2024-03-12T05:09:17.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.528+0000 7f04d2ffd640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04c80a1ab0 0x7f04c80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.528+0000 7f04e1836640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04c80a1ab0 0x7f04c80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.528+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f04d40ccb60 con 0x7f04dc1027b0 2024-03-12T05:09:17.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.528+0000 7f04e1836640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04c80a1ab0 0x7f04c80a3f70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005ab0 tx=0x7f04d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:17.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.531+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04d4096340 con 0x7f04dc1027b0 2024-03-12T05:09:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.704+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04dc02cdd0 con 0x7f04dc1027b0 2024-03-12T05:09:17.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.705+0000 7f04d2ffd640 1 -- 172.21.15.100:0/1797976109 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18486 (secure 0 0 0) 0x7f04d4020090 con 0x7f04dc1027b0 2024-03-12T05:09:17.703 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:17.703 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":82,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:09:07.135605+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04c80a1ab0 msgr2=0x7f04c80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04c80a1ab0 0x7f04c80a3f70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005ab0 tx=0x7f04d8005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 msgr2=0x7f04dc19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc19e860 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009fd0 tx=0x7f04d400b260 comp rx=0 tx=0).stop 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 shutdown_connections 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04c80a1ab0 0x7f04c80a3f70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04dc1027b0 0x7f04dc19e860 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04dc0ffbe0 0x7f04dc1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 --2- 172.21.15.100:0/1797976109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04dc0ff220 0x7f04dc1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 >> 172.21.15.100:0/1797976109 conn(0x7f04dc0fadc0 msgr2=0x7f04dc1058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:17.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 shutdown_connections 2024-03-12T05:09:17.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.707+0000 7f04e3a9a640 1 -- 172.21.15.100:0/1797976109 wait complete. 2024-03-12T05:09:17.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:09:17.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:09:17.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 -- 172.21.15.100:0/4259645704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 msgr2=0x7f1490101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 --2- 172.21.15.100:0/4259645704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f1490101700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1484009900 tx=0x7f148402bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:17.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 -- 172.21.15.100:0/4259645704 shutdown_connections 2024-03-12T05:09:17.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 --2- 172.21.15.100:0/4259645704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1490101c40 0x7f14901084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 --2- 172.21.15.100:0/4259645704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f1490101700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 --2- 172.21.15.100:0/4259645704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1490100090 0x7f14901004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 -- 172.21.15.100:0/4259645704 >> 172.21.15.100:0/4259645704 conn(0x7f14900fb620 msgr2=0x7f14900fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 -- 172.21.15.100:0/4259645704 shutdown_connections 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.822+0000 7f1497cb5640 1 -- 172.21.15.100:0/4259645704 wait complete. 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 Processor -- start 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 -- start start 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1490100090 0x7f149006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f149019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1490101c40 0x7f149019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149019eda0 con 0x7f1490101280 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149019ef10 con 0x7f1490100090 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149019f0d0 con 0x7f1490101c40 2024-03-12T05:09:17.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f149019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f149019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45536/0 (socket says 172.21.15.100:45536) 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 -- 172.21.15.100:0/587399713 learned_addr learned my addr 172.21.15.100:0/587399713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495a51640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1490100090 0x7f149006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1496252640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1490101c40 0x7f149019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 -- 172.21.15.100:0/587399713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1490101c40 msgr2=0x7f149019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1490101c40 0x7f149019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 -- 172.21.15.100:0/587399713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1490100090 msgr2=0x7f149006a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1490100090 0x7f149006a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 -- 172.21.15.100:0/587399713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484009560 con 0x7f1490101280 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1495250640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f149019bfa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1484006010 tx=0x7f14840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14840044e0 con 0x7f1490101280 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149019f2f0 con 0x7f1490101280 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.823+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14840054e0 con 0x7f1490101280 2024-03-12T05:09:17.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.824+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148403e450 con 0x7f1490101280 2024-03-12T05:09:17.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.824+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149019f850 con 0x7f1490101280 2024-03-12T05:09:17.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.824+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1490107480 con 0x7f1490101280 2024-03-12T05:09:17.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.825+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f14840035f0 con 0x7f1490101280 2024-03-12T05:09:17.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.826+0000 7f1482ffd640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f147c0a1a30 0x7f147c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:17.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.826+0000 7f1495a51640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f147c0a1a30 0x7f147c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:17.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.826+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14840ef340 con 0x7f1490101280 2024-03-12T05:09:17.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.827+0000 7f1495a51640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f147c0a1a30 0x7f147c0a3ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14901010e0 tx=0x7f148c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:17.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:17.831+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14840b89f0 con 0x7f1490101280 2024-03-12T05:09:18.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.006+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f149010bca0 con 0x7f1490101280 2024-03-12T05:09:18.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.006+0000 7f1482ffd640 1 -- 172.21.15.100:0/587399713 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14840bc0b0 con 0x7f1490101280 2024-03-12T05:09:18.005 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f147c0a1a30 msgr2=0x7f147c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f147c0a1a30 0x7f147c0a3ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14901010e0 tx=0x7f148c00a2d0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 msgr2=0x7f149019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f149019bfa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1484006010 tx=0x7f14840042f0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 shutdown_connections 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f147c0a1a30 0x7f147c0a3ef0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1490101c40 0x7f149019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1490101280 0x7f149019bfa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 --2- 172.21.15.100:0/587399713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1490100090 0x7f149006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.008+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 >> 172.21.15.100:0/587399713 conn(0x7f14900fb620 msgr2=0x7f1490104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.009+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 shutdown_connections 2024-03-12T05:09:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.009+0000 7f1497cb5640 1 -- 172.21.15.100:0/587399713 wait complete. 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:18.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:17 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1797976109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:18.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 -- 172.21.15.100:0/3772578451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10bbb0 msgr2=0x7fd9dc10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/3772578451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10bbb0 0x7fd9dc10bfd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009900 tx=0x7fd9cc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 -- 172.21.15.100:0/3772578451 shutdown_connections 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/3772578451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9dc10d760 0x7fd9dc113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/3772578451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/3772578451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10bbb0 0x7fd9dc10bfd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 -- 172.21.15.100:0/3772578451 >> 172.21.15.100:0/3772578451 conn(0x7fd9dc06bbe0 msgr2=0x7fd9dc06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 -- 172.21.15.100:0/3772578451 shutdown_connections 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 -- 172.21.15.100:0/3772578451 wait complete. 2024-03-12T05:09:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 Processor -- start 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 -- start start 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9dc10bbb0 0x7fd9dc1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.121+0000 7fd9e1a62640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10d760 0x7fd9dc1a9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9e1a62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a97f0 con 0x7fd9dc10cda0 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9e1a62640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a9960 con 0x7fd9dc10d760 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9e1a62640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a9b20 con 0x7fd9dc10bbb0 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc1a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45548/0 (socket says 172.21.15.100:45548) 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 -- 172.21.15.100:0/1861318060 learned_addr learned my addr 172.21.15.100:0/1861318060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9db7fe640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9dc10bbb0 0x7fd9dc1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9dbfff640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10d760 0x7fd9dc1a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 -- 172.21.15.100:0/1861318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9dc10bbb0 msgr2=0x7fd9dc1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9dc10bbb0 0x7fd9dc1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 -- 172.21.15.100:0/1861318060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10d760 msgr2=0x7fd9dc1a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10d760 0x7fd9dc1a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 -- 172.21.15.100:0/1861318060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc009560 con 0x7fd9dc10cda0 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9dbfff640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10d760 0x7fd9dc1a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9daffd640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc1a2250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c890 tx=0x7fd9d000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004220 con 0x7fd9dc10cda0 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9d0007e90 con 0x7fd9dc10cda0 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc1a9e00 con 0x7fd9dc10cda0 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004a00 con 0x7fd9dc10cda0 2024-03-12T05:09:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.122+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1aa300 con 0x7fd9dc10cda0 2024-03-12T05:09:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.123+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc112f50 con 0x7fd9dc10cda0 2024-03-12T05:09:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.124+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fd9d00079b0 con 0x7fd9dc10cda0 2024-03-12T05:09:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.125+0000 7fd9d8ff9640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd9c40a1a30 0x7fd9c40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.125+0000 7fd9db7fe640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd9c40a1a30 0x7fd9c40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.125+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9d00cccf0 con 0x7fd9dc10cda0 2024-03-12T05:09:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.125+0000 7fd9db7fe640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd9c40a1a30 0x7fd9c40a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc1a31e0 tx=0x7fd9cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:18.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.129+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9d00d1070 con 0x7fd9dc10cda0 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:18.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:17 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1797976109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:18.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:09:18.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:18.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:17 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1797976109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:18.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.338+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9dc117800 con 0x7fd9dc10cda0 2024-03-12T05:09:18.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.338+0000 7fd9d8ff9640 1 -- 172.21.15.100:0/1861318060 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fd9d0096310 con 0x7fd9dc10cda0 2024-03-12T05:09:18.336 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:18.337 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-12T05:09:18.337 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 1 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd9c40a1a30 msgr2=0x7fd9c40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd9c40a1a30 0x7fd9c40a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc1a31e0 tx=0x7fd9cc005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 msgr2=0x7fd9dc1a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc1a2250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c890 tx=0x7fd9d000cd50 comp rx=0 tx=0).stop 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 shutdown_connections 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd9c40a1a30 0x7fd9c40a3ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9dc10d760 0x7fd9dc1a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9dc10cda0 0x7fd9dc1a2250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 --2- 172.21.15.100:0/1861318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9dc10bbb0 0x7fd9dc1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 >> 172.21.15.100:0/1861318060 conn(0x7fd9dc06bbe0 msgr2=0x7fd9dc110120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:18.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 shutdown_connections 2024-03-12T05:09:18.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:18.340+0000 7fd9e1a62640 1 -- 172.21.15.100:0/1861318060 wait complete. 2024-03-12T05:09:19.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:18 smithi161 ceph-mon[36043]: Deploying daemon mds.l on smithi161 2024-03-12T05:09:19.017 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:18 smithi161 ceph-mon[36043]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:19.017 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:18 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/587399713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:19.017 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:18 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1861318060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:19.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:18 smithi156 ceph-mon[36270]: Deploying daemon mds.l on smithi161 2024-03-12T05:09:19.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:18 smithi156 ceph-mon[36270]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:19.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:18 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/587399713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:19.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:18 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1861318060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:19.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:18 smithi100 ceph-mon[32210]: Deploying daemon mds.l on smithi161 2024-03-12T05:09:19.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:18 smithi100 ceph-mon[32210]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:19.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:18 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/587399713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:19.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:18 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1861318060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:19.411 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 systemd[1]: Starting Ceph mds.l for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 podman[66375]: 2024-03-12 05:09:19.316444613 +0000 UTC m=+0.017727343 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 podman[66375]: 2024-03-12 05:09:19.418296016 +0000 UTC m=+0.119578788 container create 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 podman[66375]: 2024-03-12 05:09:19.62909828 +0000 UTC m=+0.330381010 container init 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 podman[66375]: 2024-03-12 05:09:19.63177761 +0000 UTC m=+0.333060341 container start 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True) 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 ceph-mds[66394]: -- 172.21.15.161:0/2941182278 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55af734b61a0 con 0x55af7348c800 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l[66390]: starting mds.l at 2024-03-12T05:09:19.711 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 bash[66375]: 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c 2024-03-12T05:09:19.974 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:09:19 smithi161 systemd[1]: Started Ceph mds.l for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:20.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:20.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 -- 172.21.15.100:0/2966751993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70101cd0 msgr2=0x7f9b70108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 --2- 172.21.15.100:0/2966751993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70101cd0 0x7f9b70108560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009ca0 tx=0x7f9b6802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 -- 172.21.15.100:0/2966751993 shutdown_connections 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 --2- 172.21.15.100:0/2966751993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70101cd0 0x7f9b70108560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 --2- 172.21.15.100:0/2966751993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70101310 0x7f9b70101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 --2- 172.21.15.100:0/2966751993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b70100120 0x7f9b70100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 -- 172.21.15.100:0/2966751993 >> 172.21.15.100:0/2966751993 conn(0x7f9b700fb6b0 msgr2=0x7f9b700fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 -- 172.21.15.100:0/2966751993 shutdown_connections 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.466+0000 7f9b75777640 1 -- 172.21.15.100:0/2966751993 wait complete. 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 Processor -- start 2024-03-12T05:09:20.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 -- start start 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 0x7f9b70197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70101310 0x7f9b701977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b70101cd0 0x7f9b7019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019ed80 con 0x7f9b70100120 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019eef0 con 0x7f9b70101310 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b75777640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019f0b0 con 0x7f9b70101cd0 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 0x7f9b70197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 0x7f9b70197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53122/0 (socket says 172.21.15.100:53122) 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 -- 172.21.15.100:0/1751523712 learned_addr learned my addr 172.21.15.100:0/1751523712 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b67fff640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70101310 0x7f9b701977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:20.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 -- 172.21.15.100:0/1751523712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b70101cd0 msgr2=0x7f9b7019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b70101cd0 0x7f9b7019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 -- 172.21.15.100:0/1751523712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70101310 msgr2=0x7f9b701977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70101310 0x7f9b701977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 -- 172.21.15.100:0/1751523712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68009900 con 0x7f9b70100120 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6effd640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 0x7f9b70197290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000d950 tx=0x7f9b6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.467+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60009c30 con 0x7f9b70100120 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.468+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7019f390 con 0x7f9b70100120 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.468+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b60009d90 con 0x7f9b70100120 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.468+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6000f720 con 0x7f9b70100120 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.468+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7019f850 con 0x7f9b70100120 2024-03-12T05:09:20.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.468+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7010bac0 con 0x7f9b70100120 2024-03-12T05:09:20.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.470+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9b60010a60 con 0x7f9b70100120 2024-03-12T05:09:20.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.470+0000 7f9b6cff9640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b5c0a1a30 0x7f9b5c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:20.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.470+0000 7f9b67fff640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b5c0a1a30 0x7f9b5c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:20.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.470+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9b600ccdc0 con 0x7f9b70100120 2024-03-12T05:09:20.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.470+0000 7f9b67fff640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b5c0a1a30 0x7f9b5c0a3ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b701987a0 tx=0x7f9b58005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:20.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.474+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b60096470 con 0x7f9b70100120 2024-03-12T05:09:20.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.683+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b7010bc50 con 0x7f9b70100120 2024-03-12T05:09:20.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.683+0000 7f9b6cff9640 1 -- 172.21.15.100:0/1751523712 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f9b60021540 con 0x7f9b70100120 2024-03-12T05:09:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:20.682 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T05:09:20.682 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-03-12T05:09:20.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.685+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b5c0a1a30 msgr2=0x7f9b5c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:20.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.685+0000 7f9b75777640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b5c0a1a30 0x7f9b5c0a3ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b701987a0 tx=0x7f9b58005c60 comp rx=0 tx=0).stop 2024-03-12T05:09:20.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.685+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 msgr2=0x7f9b70197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:20.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.685+0000 7f9b75777640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 0x7f9b70197290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000d950 tx=0x7f9b6000de10 comp rx=0 tx=0).stop 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 shutdown_connections 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b5c0a1a30 0x7f9b5c0a3ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b70101cd0 0x7f9b7019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70101310 0x7f9b701977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 --2- 172.21.15.100:0/1751523712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b70100120 0x7f9b70197290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 >> 172.21.15.100:0/1751523712 conn(0x7f9b700fb6b0 msgr2=0x7f9b700fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 shutdown_connections 2024-03-12T05:09:20.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:20.686+0000 7f9b75777640 1 -- 172.21.15.100:0/1751523712 wait complete. 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6832/2628600051,v1:172.21.15.161:6833/2628600051] up:boot 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: fsmap 1 up:standby 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:20.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:20.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:20.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:20.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:20 smithi156 ceph-mon[36270]: Deploying daemon mds.k on smithi156 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6832/2628600051,v1:172.21.15.161:6833/2628600051] up:boot 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: fsmap 1 up:standby 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:20.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:20.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:20.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:20.975 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:20 smithi161 ceph-mon[36043]: Deploying daemon mds.k on smithi156 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6832/2628600051,v1:172.21.15.161:6833/2628600051] up:boot 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: fsmap 1 up:standby 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:21.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:21.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:20 smithi100 ceph-mon[32210]: Deploying daemon mds.k on smithi156 2024-03-12T05:09:21.954 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:21 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1751523712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:21.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:21 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1751523712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:22.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1751523712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:22.206 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 podman[65928]: 2024-03-12 05:09:22.17036285 +0000 UTC m=+0.116102791 container create 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=) 2024-03-12T05:09:22.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:22.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:22 smithi156 ceph-mon[36270]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:22.706 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 podman[65928]: 2024-03-12 05:09:22.5566195 +0000 UTC m=+0.502359446 container init 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:09:22.706 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 podman[65928]: 2024-03-12 05:09:22.559138056 +0000 UTC m=+0.504878004 container start 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-12T05:09:22.706 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 ceph-mds[65947]: -- 172.21.15.156:0/745889869 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b610761a0 con 0x559b6104c800 2024-03-12T05:09:22.706 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k[65943]: starting mds.k at 2024-03-12T05:09:22.706 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 bash[65928]: 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f 2024-03-12T05:09:22.706 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:09:22 smithi156 systemd[1]: Started Ceph mds.k for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 -- 172.21.15.100:0/559507068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c106ee0 msgr2=0x7fbd3c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 --2- 172.21.15.100:0/559507068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c1092d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ca0 tx=0x7fbd3402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 -- 172.21.15.100:0/559507068 shutdown_connections 2024-03-12T05:09:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 --2- 172.21.15.100:0/559507068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c1092d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:22.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 --2- 172.21.15.100:0/559507068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 --2- 172.21.15.100:0/559507068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c101c80 0x7fbd3c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 -- 172.21.15.100:0/559507068 >> 172.21.15.100:0/559507068 conn(0x7fbd3c0fb610 msgr2=0x7fbd3c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 -- 172.21.15.100:0/559507068 shutdown_connections 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 -- 172.21.15.100:0/559507068 wait complete. 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 Processor -- start 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.813+0000 7fbd43379640 1 -- start start 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd43379640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3c101c80 0x7fbd3c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd43379640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd43379640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd43379640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19ed00 con 0x7fbd3c1045b0 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd43379640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19ee70 con 0x7fbd3c101c80 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd43379640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19f030 con 0x7fbd3c106ee0 2024-03-12T05:09:22.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53132/0 (socket says 172.21.15.100:53132) 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 -- 172.21.15.100:0/1258619692 learned_addr learned my addr 172.21.15.100:0/1258619692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd41115640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3c101c80 0x7fbd3c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd41916640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 -- 172.21.15.100:0/1258619692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c106ee0 msgr2=0x7fbd3c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 -- 172.21.15.100:0/1258619692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3c101c80 msgr2=0x7fbd3c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3c101c80 0x7fbd3c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 -- 172.21.15.100:0/1258619692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009900 con 0x7fbd3c1045b0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd41916640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd40914640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c197750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00d770 tx=0x7fbd2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.814+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c009c30 con 0x7fbd3c1045b0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.815+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c00de80 con 0x7fbd3c1045b0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.815+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c19f310 con 0x7fbd3c1045b0 2024-03-12T05:09:22.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.815+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00f6b0 con 0x7fbd3c1045b0 2024-03-12T05:09:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.815+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c19f870 con 0x7fbd3c1045b0 2024-03-12T05:09:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.816+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c0fd100 con 0x7fbd3c1045b0 2024-03-12T05:09:22.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.817+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbd2c010460 con 0x7fbd3c1045b0 2024-03-12T05:09:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.817+0000 7fbd327fc640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd280a1ab0 0x7fbd280a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.817+0000 7fbd41115640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd280a1ab0 0x7fbd280a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.817+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd2c0ccc60 con 0x7fbd3c1045b0 2024-03-12T05:09:22.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.818+0000 7fbd41115640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd280a1ab0 0x7fbd280a3f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800bbe0 tx=0x7fbd3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:22.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:22.822+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd2c096310 con 0x7fbd3c1045b0 2024-03-12T05:09:22.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:22 smithi161 ceph-mon[36043]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:23.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.052+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd3c02ce40 con 0x7fbd3c1045b0 2024-03-12T05:09:23.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.053+0000 7fbd327fc640 1 -- 172.21.15.100:0/1258619692 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fbd2c017860 con 0x7fbd3c1045b0 2024-03-12T05:09:23.051 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:23.051 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T05:09:23.052 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-03-12T05:09:23.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd280a1ab0 msgr2=0x7fbd280a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:23.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd280a1ab0 0x7fbd280a3f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800bbe0 tx=0x7fbd3800b390 comp rx=0 tx=0).stop 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 msgr2=0x7fbd3c197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c197750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00d770 tx=0x7fbd2c00dc30 comp rx=0 tx=0).stop 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 shutdown_connections 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd280a1ab0 0x7fbd280a3f70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3c106ee0 0x7fbd3c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c1045b0 0x7fbd3c197750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 --2- 172.21.15.100:0/1258619692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3c101c80 0x7fbd3c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 >> 172.21.15.100:0/1258619692 conn(0x7fbd3c0fb610 msgr2=0x7fbd3c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 shutdown_connections 2024-03-12T05:09:23.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:23.055+0000 7fbd43379640 1 -- 172.21.15.100:0/1258619692 wait complete. 2024-03-12T05:09:23.063 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:22 smithi100 ceph-mon[32210]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6832/692677489,v1:172.21.15.156:6833/692677489] up:boot 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: fsmap 2 up:standby 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: Deploying daemon mds.j on smithi100 2024-03-12T05:09:23.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:23 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1258619692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6832/692677489,v1:172.21.15.156:6833/692677489] up:boot 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: fsmap 2 up:standby 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: Deploying daemon mds.j on smithi100 2024-03-12T05:09:23.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:23 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1258619692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:24.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6832/692677489,v1:172.21.15.156:6833/692677489] up:boot 2024-03-12T05:09:24.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: fsmap 2 up:standby 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: Deploying daemon mds.j on smithi100 2024-03-12T05:09:24.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:23 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1258619692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:24.919 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:24 smithi156 ceph-mon[36270]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:24.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:24 smithi161 ceph-mon[36043]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:25.020 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:24 smithi100 ceph-mon[32210]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:25.021 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:24 smithi100 systemd[1]: Starting Ceph mds.j for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:25.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 -- 172.21.15.100:0/463939534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 msgr2=0x7f1d38102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 --2- 172.21.15.100:0/463939534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38102810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009900 tx=0x7f1d2802bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 -- 172.21.15.100:0/463939534 shutdown_connections 2024-03-12T05:09:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 --2- 172.21.15.100:0/463939534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d38102d50 0x7f1d381095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 --2- 172.21.15.100:0/463939534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38102810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 --2- 172.21.15.100:0/463939534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d381011a0 0x7f1d381015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 -- 172.21.15.100:0/463939534 >> 172.21.15.100:0/463939534 conn(0x7f1d380fc730 msgr2=0x7f1d380feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 -- 172.21.15.100:0/463939534 shutdown_connections 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.176+0000 7f1d40423640 1 -- 172.21.15.100:0/463939534 wait complete. 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 Processor -- start 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 -- start start 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d381011a0 0x7f1d38199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d38102d50 0x7f1d381a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381a0f20 con 0x7f1d38102390 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381a1090 con 0x7f1d38102d50 2024-03-12T05:09:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d40423640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381a1250 con 0x7f1d381011a0 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53152/0 (socket says 172.21.15.100:53152) 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 -- 172.21.15.100:0/3293062335 learned_addr learned my addr 172.21.15.100:0/3293062335 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3e1bf640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d381011a0 0x7f1d38199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3e9c0640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d38102d50 0x7f1d381a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 -- 172.21.15.100:0/3293062335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d381011a0 msgr2=0x7f1d38199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d381011a0 0x7f1d38199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 -- 172.21.15.100:0/3293062335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d38102d50 msgr2=0x7f1d381a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d38102d50 0x7f1d381a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 -- 172.21.15.100:0/3293062335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28009560 con 0x7f1d38102390 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d3d9be640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38199970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d28006010 tx=0x7f1d280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.177+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280044e0 con 0x7f1d38102390 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.178+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d280054e0 con 0x7f1d38102390 2024-03-12T05:09:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.178+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d381a14d0 con 0x7f1d38102390 2024-03-12T05:09:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.178+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2803e450 con 0x7f1d38102390 2024-03-12T05:09:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.178+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d381a19e0 con 0x7f1d38102390 2024-03-12T05:09:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.179+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3810cbb0 con 0x7f1d38102390 2024-03-12T05:09:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.180+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1d280035f0 con 0x7f1d38102390 2024-03-12T05:09:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.180+0000 7f1d2f7fe640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1d240a1a30 0x7f1d240a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.180+0000 7f1d3e1bf640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1d240a1a30 0x7f1d240a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.180+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d280ef3c0 con 0x7f1d38102390 2024-03-12T05:09:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.181+0000 7f1d3e1bf640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1d240a1a30 0x7f1d240a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d34005d00 tx=0x7f1d34005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.184+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d280b8a70 con 0x7f1d38102390 2024-03-12T05:09:25.280 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 podman[70942]: 2024-03-12 05:09:24.926406721 +0000 UTC m=+0.018482112 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:25.280 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 podman[70942]: 2024-03-12 05:09:25.043232481 +0000 UTC m=+0.135307870 container create d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-12T05:09:25.280 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 podman[70942]: 2024-03-12 05:09:25.254755049 +0000 UTC m=+0.346830456 container init d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-12T05:09:25.280 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 podman[70942]: 2024-03-12 05:09:25.258087053 +0000 UTC m=+0.350162436 container start d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-12T05:09:25.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.413+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d3810cd40 con 0x7f1d38102390 2024-03-12T05:09:25.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.413+0000 7f1d2f7fe640 1 -- 172.21.15.100:0/3293062335 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f1d280bc130 con 0x7f1d38102390 2024-03-12T05:09:25.413 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:25.413 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T05:09:25.413 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1d240a1a30 msgr2=0x7f1d240a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1d240a1a30 0x7f1d240a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d34005d00 tx=0x7f1d34005c50 comp rx=0 tx=0).stop 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 msgr2=0x7f1d38199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38199970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d28006010 tx=0x7f1d280042f0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 shutdown_connections 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1d240a1a30 0x7f1d240a3ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d38102d50 0x7f1d381a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d38102390 0x7f1d38199970 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 --2- 172.21.15.100:0/3293062335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d381011a0 0x7f1d38199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 >> 172.21.15.100:0/3293062335 conn(0x7f1d380fc730 msgr2=0x7f1d380733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 shutdown_connections 2024-03-12T05:09:25.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:25.415+0000 7f1d40423640 1 -- 172.21.15.100:0/3293062335 wait complete. 2024-03-12T05:09:25.568 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 ceph-mds[71000]: -- 172.21.15.100:0/514122511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628f1e7a1a0 con 0x5628f1e50c00 2024-03-12T05:09:25.569 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j[70977]: starting mds.j at 2024-03-12T05:09:25.569 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 bash[70942]: d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef 2024-03-12T05:09:25.569 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:09:25 smithi100 systemd[1]: Started Ceph mds.j for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:26.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3293062335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:26.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: Deploying daemon mds.d on smithi100 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.100:6834/3900408401,v1:172.21.15.100:6835/3900408401] up:boot 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: fsmap 3 up:standby 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T05:09:26.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:26 smithi156 ceph-mon[36270]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3293062335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:26.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:26.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: Deploying daemon mds.d on smithi100 2024-03-12T05:09:26.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.100:6834/3900408401,v1:172.21.15.100:6835/3900408401] up:boot 2024-03-12T05:09:26.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: fsmap 3 up:standby 2024-03-12T05:09:26.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T05:09:26.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:26 smithi161 ceph-mon[36043]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:26.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3293062335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:26.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:26.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:26.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: Deploying daemon mds.d on smithi100 2024-03-12T05:09:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.100:6834/3900408401,v1:172.21.15.100:6835/3900408401] up:boot 2024-03-12T05:09:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: fsmap 3 up:standby 2024-03-12T05:09:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T05:09:26.820 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:26 smithi100 ceph-mon[32210]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:27.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 -- 172.21.15.100:0/423985776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 msgr2=0x7f6ed01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 --2- 172.21.15.100:0/423985776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01004b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009900 tx=0x7f6ec002bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 -- 172.21.15.100:0/423985776 shutdown_connections 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 --2- 172.21.15.100:0/423985776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ed0101c40 0x7f6ed01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 --2- 172.21.15.100:0/423985776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed0101280 0x7f6ed0101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 --2- 172.21.15.100:0/423985776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01004b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 -- 172.21.15.100:0/423985776 >> 172.21.15.100:0/423985776 conn(0x7f6ed00fb600 msgr2=0x7f6ed00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 -- 172.21.15.100:0/423985776 shutdown_connections 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.539+0000 7f6ed5179640 1 -- 172.21.15.100:0/423985776 wait complete. 2024-03-12T05:09:27.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 Processor -- start 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 -- start start 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed0101280 0x7f6ed0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ed0101c40 0x7f6ed01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01a0ee0 con 0x7f6ed0100090 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01a1050 con 0x7f6ed0101280 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.540+0000 7f6ed5179640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01a1210 con 0x7f6ed0101c40 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53180/0 (socket says 172.21.15.100:53180) 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 -- 172.21.15.100:0/648790634 learned_addr learned my addr 172.21.15.100:0/648790634 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6ecf59e640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ed0101c40 0x7f6ed01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:27.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6ece59c640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed0101280 0x7f6ed0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 -- 172.21.15.100:0/648790634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ed0101c40 msgr2=0x7f6ed01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ed0101c40 0x7f6ed01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 -- 172.21.15.100:0/648790634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed0101280 msgr2=0x7f6ed0199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed0101280 0x7f6ed0199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 -- 172.21.15.100:0/648790634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0009560 con 0x7f6ed0100090 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eced9d640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01993f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000c00 tx=0x7f6ec00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00044c0 con 0x7f6ed0100090 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ec00052d0 con 0x7f6ed0100090 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed01a1490 con 0x7f6ed0100090 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0003e80 con 0x7f6ed0100090 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.541+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed01a19a0 con 0x7f6ed0100090 2024-03-12T05:09:27.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.543+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed010bab0 con 0x7f6ed0100090 2024-03-12T05:09:27.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.544+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6ec0005440 con 0x7f6ed0100090 2024-03-12T05:09:27.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.545+0000 7f6eb7fff640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6eb80a1a30 0x7f6eb80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:27.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.545+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ec00ef7e0 con 0x7f6ed0100090 2024-03-12T05:09:27.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.545+0000 7f6ece59c640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6eb80a1a30 0x7f6eb80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:27.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.545+0000 7f6ece59c640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6eb80a1a30 0x7f6eb80a3ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ed019a900 tx=0x7f6ec4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:27.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.551+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ec00b8fb0 con 0x7f6ed0100090 2024-03-12T05:09:27.568 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 systemd[1]: Starting Ceph mds.d for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.779+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ed010bc90 con 0x7f6ed0100090 2024-03-12T05:09:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.780+0000 7f6eb7fff640 1 -- 172.21.15.100:0/648790634 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f6ec0043b80 con 0x7f6ed0100090 2024-03-12T05:09:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:27.778 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T05:09:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 4 2024-03-12T05:09:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6eb80a1a30 msgr2=0x7f6eb80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6eb80a1a30 0x7f6eb80a3ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ed019a900 tx=0x7f6ec4005f90 comp rx=0 tx=0).stop 2024-03-12T05:09:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 msgr2=0x7f6ed01993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01993f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000c00 tx=0x7f6ec00040c0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 shutdown_connections 2024-03-12T05:09:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6eb80a1a30 0x7f6eb80a3ef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ed0101c40 0x7f6ed01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed0101280 0x7f6ed0199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 --2- 172.21.15.100:0/648790634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed0100090 0x7f6ed01993f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 >> 172.21.15.100:0/648790634 conn(0x7f6ed00fb600 msgr2=0x7f6ed00fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 shutdown_connections 2024-03-12T05:09:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:27.782+0000 7f6ed5179640 1 -- 172.21.15.100:0/648790634 wait complete. 2024-03-12T05:09:27.991 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 podman[71345]: 2024-03-12 05:09:27.603565932 +0000 UTC m=+0.018568680 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:27.991 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 podman[71345]: 2024-03-12 05:09:27.715582144 +0000 UTC m=+0.130584874 container create 929a7f2a1b639b7123cd788aca3ea14624f352c1da0fe09496d48cb57fc4545d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-12T05:09:27.991 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 podman[71345]: 2024-03-12 05:09:27.893011552 +0000 UTC m=+0.308014286 container init 929a7f2a1b639b7123cd788aca3ea14624f352c1da0fe09496d48cb57fc4545d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:09:27.991 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 podman[71345]: 2024-03-12 05:09:27.895562364 +0000 UTC m=+0.310565095 container start 929a7f2a1b639b7123cd788aca3ea14624f352c1da0fe09496d48cb57fc4545d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-12T05:09:27.991 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 ceph-mds[71366]: -- 172.21.15.100:0/3171345924 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560cc51ee1a0 con 0x560cc51c4800 2024-03-12T05:09:27.992 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d[71362]: starting mds.d at 2024-03-12T05:09:28.318 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 bash[71345]: 929a7f2a1b639b7123cd788aca3ea14624f352c1da0fe09496d48cb57fc4545d 2024-03-12T05:09:28.319 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:09:27 smithi100 systemd[1]: Started Ceph mds.d for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:28.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/648790634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:28.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:28 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/648790634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:29.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:28 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/648790634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:29.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:28 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:29.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:29.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 -- 172.21.15.100:0/839755398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4101ce0 msgr2=0x7fbde41040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 --2- 172.21.15.100:0/839755398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4101ce0 0x7fbde41040d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009900 tx=0x7fbdd402bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 -- 172.21.15.100:0/839755398 shutdown_connections 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 --2- 172.21.15.100:0/839755398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbde4106f40 0x7fbde4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 --2- 172.21.15.100:0/839755398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde4104610 0x7fbde4106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 --2- 172.21.15.100:0/839755398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4101ce0 0x7fbde41040d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 -- 172.21.15.100:0/839755398 >> 172.21.15.100:0/839755398 conn(0x7fbde40fb630 msgr2=0x7fbde40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 -- 172.21.15.100:0/839755398 shutdown_connections 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.909+0000 7fbde9eae640 1 -- 172.21.15.100:0/839755398 wait complete. 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 Processor -- start 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 -- start start 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde4101ce0 0x7fbde41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 0x7fbde4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbde4106f40 0x7fbde419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde419edd0 con 0x7fbde4104610 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde419ef40 con 0x7fbde4106f40 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde9eae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde419f100 con 0x7fbde4101ce0 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 0x7fbde4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 0x7fbde4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53208/0 (socket says 172.21.15.100:53208) 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde2ffd640 1 -- 172.21.15.100:0/4171501787 learned_addr learned my addr 172.21.15.100:0/4171501787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde3fff640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbde4106f40 0x7fbde419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde2ffd640 1 -- 172.21.15.100:0/4171501787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde4101ce0 msgr2=0x7fbde41972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.910+0000 7fbde37fe640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde4101ce0 0x7fbde41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde2ffd640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde4101ce0 0x7fbde41972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde2ffd640 1 -- 172.21.15.100:0/4171501787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbde4106f40 msgr2=0x7fbde419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:29.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde2ffd640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbde4106f40 0x7fbde419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde2ffd640 1 -- 172.21.15.100:0/4171501787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009560 con 0x7fbde4104610 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde2ffd640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 0x7fbde4197820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b450 tx=0x7fbdd800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8011890 con 0x7fbde4104610 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdd80103e0 con 0x7fbde4104610 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800f580 con 0x7fbde4104610 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde419f3e0 con 0x7fbde4104610 2024-03-12T05:09:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.911+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde419f8a0 con 0x7fbde4104610 2024-03-12T05:09:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.912+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde40fd120 con 0x7fbde4104610 2024-03-12T05:09:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.913+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbdd80108e0 con 0x7fbde4104610 2024-03-12T05:09:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.913+0000 7fbde0ff9640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbdcc0a1a30 0x7fbdcc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.914+0000 7fbde37fe640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbdcc0a1a30 0x7fbdcc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.914+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbdd80ccd60 con 0x7fbde4104610 2024-03-12T05:09:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.914+0000 7fbde37fe640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbdcc0a1a30 0x7fbdcc0a3ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdd402c6a0 tx=0x7fbdd4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:29.918+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbdd8096410 con 0x7fbde4104610 2024-03-12T05:09:29.918 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:29 smithi156 ceph-mon[36270]: Deploying daemon mds.e on smithi156 2024-03-12T05:09:29.918 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:29 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.100:6836/1446338988,v1:172.21.15.100:6837/1446338988] up:boot 2024-03-12T05:09:29.918 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:29 smithi156 ceph-mon[36270]: fsmap 4 up:standby 2024-03-12T05:09:29.918 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:29 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T05:09:30.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:29 smithi100 ceph-mon[32210]: Deploying daemon mds.e on smithi156 2024-03-12T05:09:30.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:29 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.100:6836/1446338988,v1:172.21.15.100:6837/1446338988] up:boot 2024-03-12T05:09:30.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:29 smithi100 ceph-mon[32210]: fsmap 4 up:standby 2024-03-12T05:09:30.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:29 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T05:09:30.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.133+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbde410cc30 con 0x7fbde4104610 2024-03-12T05:09:30.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.134+0000 7fbde0ff9640 1 -- 172.21.15.100:0/4171501787 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3707 (secure 0 0 0) 0x7fbdd8015e90 con 0x7fbde4104610 2024-03-12T05:09:30.132 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:30.133 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-12T05:09:30.133 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 5 2024-03-12T05:09:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbdcc0a1a30 msgr2=0x7fbdcc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbdcc0a1a30 0x7fbdcc0a3ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdd402c6a0 tx=0x7fbdd4005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 msgr2=0x7fbde4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 0x7fbde4197820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b450 tx=0x7fbdd800b910 comp rx=0 tx=0).stop 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 shutdown_connections 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbdcc0a1a30 0x7fbdcc0a3ef0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbde4106f40 0x7fbde419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde4104610 0x7fbde4197820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 --2- 172.21.15.100:0/4171501787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde4101ce0 0x7fbde41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 >> 172.21.15.100:0/4171501787 conn(0x7fbde40fb630 msgr2=0x7fbde4102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 shutdown_connections 2024-03-12T05:09:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:30.136+0000 7fbde9eae640 1 -- 172.21.15.100:0/4171501787 wait complete. 2024-03-12T05:09:30.205 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 systemd[1]: Starting Ceph mds.e for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:30.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:29 smithi161 ceph-mon[36043]: Deploying daemon mds.e on smithi156 2024-03-12T05:09:30.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:29 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.100:6836/1446338988,v1:172.21.15.100:6837/1446338988] up:boot 2024-03-12T05:09:30.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:29 smithi161 ceph-mon[36043]: fsmap 4 up:standby 2024-03-12T05:09:30.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:29 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T05:09:30.614 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 podman[66244]: 2024-03-12 05:09:30.315167028 +0000 UTC m=+0.104179374 container create 06b18cfc8214a8c46ffb328b6388a929516e409bf3e40eb702475d1e0b800a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True) 2024-03-12T05:09:30.615 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 podman[66244]: 2024-03-12 05:09:30.228148505 +0000 UTC m=+0.017160849 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:30.615 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 podman[66244]: 2024-03-12 05:09:30.534762888 +0000 UTC m=+0.323775218 container init 06b18cfc8214a8c46ffb328b6388a929516e409bf3e40eb702475d1e0b800a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-12T05:09:30.615 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 podman[66244]: 2024-03-12 05:09:30.537731583 +0000 UTC m=+0.326743910 container start 06b18cfc8214a8c46ffb328b6388a929516e409bf3e40eb702475d1e0b800a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_CLEAN=True) 2024-03-12T05:09:30.615 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mds[66263]: -- 172.21.15.156:0/1043288513 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56424015e1a0 con 0x564240135000 2024-03-12T05:09:30.615 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e[66259]: starting mds.e at 2024-03-12T05:09:30.615 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 bash[66244]: 06b18cfc8214a8c46ffb328b6388a929516e409bf3e40eb702475d1e0b800a04 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:09:30 smithi156 systemd[1]: Started Ceph mds.e for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4171501787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:30.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:30.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:30.957 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:30 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4171501787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:31.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:30 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4171501787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:31.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:30 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:31.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:31 smithi161 ceph-mon[36043]: Deploying daemon mds.f on smithi161 2024-03-12T05:09:31.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:31 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6834/2592644588,v1:172.21.15.156:6835/2592644588] up:boot 2024-03-12T05:09:31.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:31 smithi161 ceph-mon[36043]: fsmap 5 up:standby 2024-03-12T05:09:31.974 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:31 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:09:32.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:31 smithi100 ceph-mon[32210]: Deploying daemon mds.f on smithi161 2024-03-12T05:09:32.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:31 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6834/2592644588,v1:172.21.15.156:6835/2592644588] up:boot 2024-03-12T05:09:32.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:31 smithi100 ceph-mon[32210]: fsmap 5 up:standby 2024-03-12T05:09:32.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:31 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:09:32.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:32.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:31 smithi156 ceph-mon[36270]: Deploying daemon mds.f on smithi161 2024-03-12T05:09:32.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:31 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6834/2592644588,v1:172.21.15.156:6835/2592644588] up:boot 2024-03-12T05:09:32.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:31 smithi156 ceph-mon[36270]: fsmap 5 up:standby 2024-03-12T05:09:32.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:31 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 -- 172.21.15.100:0/3302922678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac40ff830 msgr2=0x7fdac40ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 --2- 172.21.15.100:0/3302922678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac40ff830 0x7fdac40ffc50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab402bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 -- 172.21.15.100:0/3302922678 shutdown_connections 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 --2- 172.21.15.100:0/3302922678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdac41013e0 0x7fdac4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 --2- 172.21.15.100:0/3302922678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdac4100a20 0x7fdac4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 --2- 172.21.15.100:0/3302922678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac40ff830 0x7fdac40ffc50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 -- 172.21.15.100:0/3302922678 >> 172.21.15.100:0/3302922678 conn(0x7fdac40fada0 msgr2=0x7fdac40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 -- 172.21.15.100:0/3302922678 shutdown_connections 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.261+0000 7fdaca24d640 1 -- 172.21.15.100:0/3302922678 wait complete. 2024-03-12T05:09:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 Processor -- start 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 -- start start 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdac40ff830 0x7fdac4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 0x7fdac41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdac41013e0 0x7fdac419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac419ed50 con 0x7fdac4100a20 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac419eec0 con 0x7fdac40ff830 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdaca24d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac419f080 con 0x7fdac41013e0 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 0x7fdac41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 0x7fdac41977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55332/0 (socket says 172.21.15.100:55332) 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdac2ffd640 1 -- 172.21.15.100:0/2160872630 learned_addr learned my addr 172.21.15.100:0/2160872630 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.262+0000 7fdac3fff640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdac41013e0 0x7fdac419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:32.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac37fe640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdac40ff830 0x7fdac4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac2ffd640 1 -- 172.21.15.100:0/2160872630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdac41013e0 msgr2=0x7fdac419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac2ffd640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdac41013e0 0x7fdac419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac2ffd640 1 -- 172.21.15.100:0/2160872630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdac40ff830 msgr2=0x7fdac4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac2ffd640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdac40ff830 0x7fdac4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac2ffd640 1 -- 172.21.15.100:0/2160872630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4009560 con 0x7fdac4100a20 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac2ffd640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 0x7fdac41977a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d950 tx=0x7fdab800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8009c30 con 0x7fdac4100a20 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac419f360 con 0x7fdac4100a20 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdab8009d90 con 0x7fdac4100a20 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800f7e0 con 0x7fdac4100a20 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.263+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac419f870 con 0x7fdac4100a20 2024-03-12T05:09:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.264+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac4106b50 con 0x7fdac4100a20 2024-03-12T05:09:32.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.265+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fdab8010ad0 con 0x7fdac4100a20 2024-03-12T05:09:32.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.266+0000 7fdac0ff9640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdaac0a1a30 0x7fdaac0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:32.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.266+0000 7fdac37fe640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdaac0a1a30 0x7fdaac0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:32.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.266+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdab80ccc20 con 0x7fdac4100a20 2024-03-12T05:09:32.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.266+0000 7fdac37fe640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdaac0a1a30 0x7fdaac0a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdab4006010 tx=0x7fdab4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:32.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.270+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdab80962b0 con 0x7fdac4100a20 2024-03-12T05:09:32.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.481+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdac410ce20 con 0x7fdac4100a20 2024-03-12T05:09:32.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.481+0000 7fdac0ff9640 1 -- 172.21.15.100:0/2160872630 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4501 (secure 0 0 0) 0x7fdab8099970 con 0x7fdac4100a20 2024-03-12T05:09:32.479 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:32.479 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-12T05:09:32.480 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-03-12T05:09:32.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdaac0a1a30 msgr2=0x7fdaac0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:32.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdaac0a1a30 0x7fdaac0a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdab4006010 tx=0x7fdab4000bc0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 msgr2=0x7fdac41977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:32.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 0x7fdac41977a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d950 tx=0x7fdab800de10 comp rx=0 tx=0).stop 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 shutdown_connections 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdaac0a1a30 0x7fdaac0a3ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdac41013e0 0x7fdac419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac4100a20 0x7fdac41977a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 --2- 172.21.15.100:0/2160872630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdac40ff830 0x7fdac4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 >> 172.21.15.100:0/2160872630 conn(0x7fdac40fada0 msgr2=0x7fdac4103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 shutdown_connections 2024-03-12T05:09:32.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:32.483+0000 7fdaca24d640 1 -- 172.21.15.100:0/2160872630 wait complete. 2024-03-12T05:09:32.724 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:32 smithi161 systemd[1]: Starting Ceph mds.f for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:33.045 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:32 smithi161 ceph-mon[36043]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:33.046 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:32 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2160872630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:33.046 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:32 smithi161 podman[66690]: 2024-03-12 05:09:32.820371614 +0000 UTC m=+0.096551044 container create 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-12T05:09:33.046 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:32 smithi161 podman[66690]: 2024-03-12 05:09:32.740814921 +0000 UTC m=+0.016994366 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:33.046 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:33 smithi161 podman[66690]: 2024-03-12 05:09:33.023062512 +0000 UTC m=+0.299241951 container init 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:09:33.046 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:33 smithi161 podman[66690]: 2024-03-12 05:09:33.025975704 +0000 UTC m=+0.302155138 container start 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-12T05:09:33.046 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:33 smithi161 ceph-mds[66709]: -- 172.21.15.161:0/3017658708 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556f0e9021a0 con 0x556f0e8d8c00 2024-03-12T05:09:33.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:32 smithi100 ceph-mon[32210]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:33.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:32 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2160872630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:33.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:32 smithi156 ceph-mon[36270]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:33.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:32 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2160872630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:33.473 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f[66705]: starting mds.f at 2024-03-12T05:09:33.474 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:33 smithi161 bash[66690]: 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 2024-03-12T05:09:33.474 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:09:33 smithi161 systemd[1]: Started Ceph mds.f for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: Deploying daemon mds.c on smithi161 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6834/692407308,v1:172.21.15.161:6835/692407308] up:boot 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: fsmap 6 up:standby 2024-03-12T05:09:34.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:34 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:09:34.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: Deploying daemon mds.c on smithi161 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6834/692407308,v1:172.21.15.161:6835/692407308] up:boot 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: fsmap 6 up:standby 2024-03-12T05:09:34.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:34 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:09:34.493 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:34.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:34.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: Deploying daemon mds.c on smithi161 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6834/692407308,v1:172.21.15.161:6835/692407308] up:boot 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: fsmap 6 up:standby 2024-03-12T05:09:34.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:34 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 -- 172.21.15.100:0/82584024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c1000a0 msgr2=0x7f460c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 --2- 172.21.15.100:0/82584024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c1000a0 0x7f460c1004c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4604009ca0 tx=0x7f460402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 -- 172.21.15.100:0/82584024 shutdown_connections 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 --2- 172.21.15.100:0/82584024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c101c50 0x7f460c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 --2- 172.21.15.100:0/82584024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c101290 0x7f460c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 --2- 172.21.15.100:0/82584024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c1000a0 0x7f460c1004c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 -- 172.21.15.100:0/82584024 >> 172.21.15.100:0/82584024 conn(0x7f460c0fb610 msgr2=0x7f460c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:34.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 -- 172.21.15.100:0/82584024 shutdown_connections 2024-03-12T05:09:34.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 -- 172.21.15.100:0/82584024 wait complete. 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 Processor -- start 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.609+0000 7f4614240640 1 -- start start 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c1000a0 0x7f460c068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c101290 0x7f460c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 0x7f460c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c19ed40 con 0x7f460c101c50 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c19eeb0 con 0x7f460c1000a0 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c19f070 con 0x7f460c101290 2024-03-12T05:09:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 0x7f460c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 0x7f460c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55348/0 (socket says 172.21.15.100:55348) 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 -- 172.21.15.100:0/3283610787 learned_addr learned my addr 172.21.15.100:0/3283610787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46117db640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c101290 0x7f460c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4611fdc640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c1000a0 0x7f460c068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 -- 172.21.15.100:0/3283610787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c101290 msgr2=0x7f460c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c101290 0x7f460c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 -- 172.21.15.100:0/3283610787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c1000a0 msgr2=0x7f460c068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c1000a0 0x7f460c068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 -- 172.21.15.100:0/3283610787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604009900 con 0x7f460c101c50 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4611fdc640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c1000a0 0x7f460c068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f46127dd640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 0x7f460c19e5b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00d750 tx=0x7f45fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc009bd0 con 0x7f460c101c50 2024-03-12T05:09:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45fc00de60 con 0x7f460c101c50 2024-03-12T05:09:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c19f350 con 0x7f460c101c50 2024-03-12T05:09:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc004e70 con 0x7f460c101c50 2024-03-12T05:09:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.610+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c19f860 con 0x7f460c101c50 2024-03-12T05:09:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.611+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c1073c0 con 0x7f460c101c50 2024-03-12T05:09:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.612+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f45fc005020 con 0x7f460c101c50 2024-03-12T05:09:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.613+0000 7f460affd640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f45f40a1a30 0x7f45f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.613+0000 7f4611fdc640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f45f40a1a30 0x7f45f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.613+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45fc0cc8e0 con 0x7f460c101c50 2024-03-12T05:09:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.613+0000 7f4611fdc640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f45f40a1a30 0x7f45f40a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f460402c700 tx=0x7f46040326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:34.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.617+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45fc099ac0 con 0x7f460c101c50 2024-03-12T05:09:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.827+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460c10bd00 con 0x7f460c101c50 2024-03-12T05:09:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.828+0000 7f460affd640 1 -- 172.21.15.100:0/3283610787 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f45fc099650 con 0x7f460c101c50 2024-03-12T05:09:34.826 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-12T05:09:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 7 2024-03-12T05:09:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f45f40a1a30 msgr2=0x7f45f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f45f40a1a30 0x7f45f40a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f460402c700 tx=0x7f46040326d0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 msgr2=0x7f460c19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 0x7f460c19e5b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00d750 tx=0x7f45fc00dc10 comp rx=0 tx=0).stop 2024-03-12T05:09:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 shutdown_connections 2024-03-12T05:09:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f45f40a1a30 0x7f45f40a3ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c50 0x7f460c19e5b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c101290 0x7f460c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 --2- 172.21.15.100:0/3283610787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f460c1000a0 0x7f460c068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 >> 172.21.15.100:0/3283610787 conn(0x7f460c0fb610 msgr2=0x7f460c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 shutdown_connections 2024-03-12T05:09:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:34.830+0000 7f4614240640 1 -- 172.21.15.100:0/3283610787 wait complete. 2024-03-12T05:09:35.312 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:35 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3283610787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:35.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:35 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3283610787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:35.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:35 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3283610787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:35.973 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:09:35 smithi161 systemd[1]: Started Ceph mds.c for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:36.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 -- 172.21.15.100:0/2580539848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102440 msgr2=0x7fda781028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 --2- 172.21.15.100:0/2580539848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102440 0x7fda781028c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fda68009900 tx=0x7fda6802bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 -- 172.21.15.100:0/2580539848 shutdown_connections 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 --2- 172.21.15.100:0/2580539848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda78102e00 0x7fda78109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 --2- 172.21.15.100:0/2580539848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102440 0x7fda781028c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 --2- 172.21.15.100:0/2580539848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda780ff470 0x7fda780ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:36.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 -- 172.21.15.100:0/2580539848 >> 172.21.15.100:0/2580539848 conn(0x7fda780fada0 msgr2=0x7fda780fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 -- 172.21.15.100:0/2580539848 shutdown_connections 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.955+0000 7fda801ac640 1 -- 172.21.15.100:0/2580539848 wait complete. 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 Processor -- start 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 -- start start 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda780ff470 0x7fda78197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda78102440 0x7fda78197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 0x7fda7819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7819ed40 con 0x7fda78102e00 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7819eeb0 con 0x7fda780ff470 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda801ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7819f070 con 0x7fda78102440 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 0x7fda7819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:36.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 0x7fda7819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55360/0 (socket says 172.21.15.100:55360) 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 -- 172.21.15.100:0/1517599980 learned_addr learned my addr 172.21.15.100:0/1517599980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7d747640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda78102440 0x7fda78197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7df48640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda780ff470 0x7fda78197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 -- 172.21.15.100:0/1517599980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda78102440 msgr2=0x7fda78197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda78102440 0x7fda78197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 -- 172.21.15.100:0/1517599980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda780ff470 msgr2=0x7fda78197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda780ff470 0x7fda78197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.956+0000 7fda7e749640 1 -- 172.21.15.100:0/1517599980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68009560 con 0x7fda78102e00 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.957+0000 7fda7e749640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 0x7fda7819e800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda70009fd0 tx=0x7fda7000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.957+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000bea0 con 0x7fda78102e00 2024-03-12T05:09:36.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.957+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda70009940 con 0x7fda78102e00 2024-03-12T05:09:36.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.957+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7819f350 con 0x7fda78102e00 2024-03-12T05:09:36.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.957+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda700106c0 con 0x7fda78102e00 2024-03-12T05:09:36.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.957+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7819f830 con 0x7fda78102e00 2024-03-12T05:09:36.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.958+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda78108640 con 0x7fda78102e00 2024-03-12T05:09:36.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.959+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fda70004240 con 0x7fda78102e00 2024-03-12T05:09:36.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.959+0000 7fda6effd640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fda640a1ab0 0x7fda640a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:36.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.960+0000 7fda7df48640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fda640a1ab0 0x7fda640a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:36.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.960+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fda700ccc10 con 0x7fda78102e00 2024-03-12T05:09:36.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.960+0000 7fda7df48640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fda640a1ab0 0x7fda640a3f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda780695b0 tx=0x7fda74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:36.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:36.963+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda70096340 con 0x7fda78102e00 2024-03-12T05:09:37.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:37.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: Deploying daemon mds.h on smithi156 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6836/854337146,v1:172.21.15.161:6837/854337146] up:boot 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: fsmap 7 up:standby 2024-03-12T05:09:37.102 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:36 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:09:37.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.170+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda7810cbd0 con 0x7fda78102e00 2024-03-12T05:09:37.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.170+0000 7fda6effd640 1 -- 172.21.15.100:0/1517599980 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7fda70020090 con 0x7fda78102e00 2024-03-12T05:09:37.171 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:37.171 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-12T05:09:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-03-12T05:09:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fda640a1ab0 msgr2=0x7fda640a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fda640a1ab0 0x7fda640a3f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda780695b0 tx=0x7fda74005f90 comp rx=0 tx=0).stop 2024-03-12T05:09:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 msgr2=0x7fda7819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 0x7fda7819e800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda70009fd0 tx=0x7fda7000b260 comp rx=0 tx=0).stop 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 shutdown_connections 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fda640a1ab0 0x7fda640a3f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda78102e00 0x7fda7819e800 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda78102440 0x7fda78197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 --2- 172.21.15.100:0/1517599980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda780ff470 0x7fda78197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 >> 172.21.15.100:0/1517599980 conn(0x7fda780fada0 msgr2=0x7fda78105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:37.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 shutdown_connections 2024-03-12T05:09:37.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:37.172+0000 7fda801ac640 1 -- 172.21.15.100:0/1517599980 wait complete. 2024-03-12T05:09:37.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:37.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:37.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:37.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:37.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: Deploying daemon mds.h on smithi156 2024-03-12T05:09:37.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6836/854337146,v1:172.21.15.161:6837/854337146] up:boot 2024-03-12T05:09:37.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: fsmap 7 up:standby 2024-03-12T05:09:37.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:36 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: Deploying daemon mds.h on smithi156 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6836/854337146,v1:172.21.15.161:6837/854337146] up:boot 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: fsmap 7 up:standby 2024-03-12T05:09:37.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:36 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:09:38.173 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:37 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1517599980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:38.173 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:37 smithi156 systemd[1]: Starting Ceph mds.h for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:38.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:37 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1517599980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:38.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:37 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1517599980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:38.435 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 podman[66556]: 2024-03-12 05:09:38.078284701 +0000 UTC m=+0.017772020 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:38.435 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 podman[66556]: 2024-03-12 05:09:38.176262728 +0000 UTC m=+0.115750040 container create 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-12T05:09:38.435 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 podman[66556]: 2024-03-12 05:09:38.411957211 +0000 UTC m=+0.351444535 container init 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-12T05:09:38.435 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 podman[66556]: 2024-03-12 05:09:38.414605451 +0000 UTC m=+0.354092762 container start 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:09:38.705 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mds[66576]: -- 172.21.15.156:0/4026780339 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562bbaa6e1a0 con 0x562bbaa44800 2024-03-12T05:09:38.706 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h[66572]: starting mds.h at 2024-03-12T05:09:38.706 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 bash[66556]: 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e 2024-03-12T05:09:38.706 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:09:38 smithi156 systemd[1]: Started Ceph mds.h for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:39.184 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:39.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:39.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:38 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:39.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:39.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:39.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:39.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:38 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:39.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:39.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:38 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 -- 172.21.15.100:0/3492728759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd01027b0 msgr2=0x7f2dd0102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/3492728759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd01027b0 0x7f2dd0102bd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009900 tx=0x7f2dc002bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 -- 172.21.15.100:0/3492728759 shutdown_connections 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/3492728759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd00ffbe0 0x7f2dd0109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/3492728759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2dd00ff220 0x7f2dd00ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/3492728759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd01027b0 0x7f2dd0102bd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 -- 172.21.15.100:0/3492728759 >> 172.21.15.100:0/3492728759 conn(0x7f2dd00fadc0 msgr2=0x7f2dd00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 -- 172.21.15.100:0/3492728759 shutdown_connections 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 -- 172.21.15.100:0/3492728759 wait complete. 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.299+0000 7f2dd82f4640 1 Processor -- start 2024-03-12T05:09:39.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 -- start start 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 0x7f2dd0101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd00ffbe0 0x7f2dd01004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2dd01027b0 0x7f2dd0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0101190 con 0x7f2dd00ff220 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0101300 con 0x7f2dd00ffbe0 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd01a12e0 con 0x7f2dd01027b0 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 0x7f2dd0101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 0x7f2dd0101eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55372/0 (socket says 172.21.15.100:55372) 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 -- 172.21.15.100:0/1644673444 learned_addr learned my addr 172.21.15.100:0/1644673444 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6891640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2dd01027b0 0x7f2dd0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd588f640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd00ffbe0 0x7f2dd01004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 -- 172.21.15.100:0/1644673444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2dd01027b0 msgr2=0x7f2dd0100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2dd01027b0 0x7f2dd0100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 -- 172.21.15.100:0/1644673444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd00ffbe0 msgr2=0x7f2dd01004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd00ffbe0 0x7f2dd01004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 -- 172.21.15.100:0/1644673444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc0009560 con 0x7f2dd00ff220 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd6090640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 0x7f2dd0101eb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0036040 tx=0x7f2dc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004df0 con 0x7f2dd00ff220 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd01a1530 con 0x7f2dd00ff220 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0005430 con 0x7f2dd00ff220 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc003e450 con 0x7f2dd00ff220 2024-03-12T05:09:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.300+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd01a1a40 con 0x7f2dd00ff220 2024-03-12T05:09:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.302+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd0108670 con 0x7f2dd00ff220 2024-03-12T05:09:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.303+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f2dc00055a0 con 0x7f2dd00ff220 2024-03-12T05:09:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.304+0000 7f2dc77fe640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2dbc0a1a30 0x7f2dbc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.304+0000 7f2dd588f640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2dbc0a1a30 0x7f2dbc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.304+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2dc00ef9f0 con 0x7f2dd00ff220 2024-03-12T05:09:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.304+0000 7f2dd588f640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2dbc0a1a30 0x7f2dbc0a3ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005fd0 tx=0x7f2dcc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.307+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2dc00b90a0 con 0x7f2dd00ff220 2024-03-12T05:09:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.516+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dd010cc10 con 0x7f2dd00ff220 2024-03-12T05:09:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.517+0000 7f2dc77fe640 1 -- 172.21.15.100:0/1644673444 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f2dc00bc760 con 0x7f2dd00ff220 2024-03-12T05:09:39.515 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:39.515 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24439,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-12T05:09:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2dbc0a1a30 msgr2=0x7f2dbc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2dbc0a1a30 0x7f2dbc0a3ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005fd0 tx=0x7f2dcc005850 comp rx=0 tx=0).stop 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 msgr2=0x7f2dd0101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 0x7f2dd0101eb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0036040 tx=0x7f2dc0004270 comp rx=0 tx=0).stop 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 shutdown_connections 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2dbc0a1a30 0x7f2dbc0a3ef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2dd01027b0 0x7f2dd0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd00ffbe0 0x7f2dd01004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 --2- 172.21.15.100:0/1644673444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00ff220 0x7f2dd0101eb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 >> 172.21.15.100:0/1644673444 conn(0x7f2dd00fadc0 msgr2=0x7f2dd0105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 shutdown_connections 2024-03-12T05:09:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:39.519+0000 7f2dd82f4640 1 -- 172.21.15.100:0/1644673444 wait complete. 2024-03-12T05:09:40.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:39 smithi156 ceph-mon[36270]: Deploying daemon mds.i on smithi161 2024-03-12T05:09:40.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:39 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] up:boot 2024-03-12T05:09:40.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:39 smithi156 ceph-mon[36270]: fsmap 8 up:standby 2024-03-12T05:09:40.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:39 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:09:40.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:39 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1644673444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:40.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:39 smithi161 ceph-mon[36043]: Deploying daemon mds.i on smithi161 2024-03-12T05:09:40.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:39 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] up:boot 2024-03-12T05:09:40.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:39 smithi161 ceph-mon[36043]: fsmap 8 up:standby 2024-03-12T05:09:40.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:39 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:09:40.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:39 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1644673444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:40.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:39 smithi100 ceph-mon[32210]: Deploying daemon mds.i on smithi161 2024-03-12T05:09:40.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:39 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] up:boot 2024-03-12T05:09:40.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:39 smithi100 ceph-mon[32210]: fsmap 8 up:standby 2024-03-12T05:09:40.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:39 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:09:40.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:39 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1644673444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:40.853 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:40 smithi161 systemd[1]: Starting Ceph mds.i for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:41.114 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:40 smithi161 ceph-mon[36043]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:41.114 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:40 smithi161 podman[67321]: 2024-03-12 05:09:40.851221436 +0000 UTC m=+0.017289357 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:41.114 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:40 smithi161 podman[67321]: 2024-03-12 05:09:40.967892309 +0000 UTC m=+0.133960276 container create f73d71f5907b029ae1349020bc8a31cf5c4a8255b36c6910e84316e0b7d602b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-12T05:09:41.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:40 smithi156 ceph-mon[36270]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:41.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:40 smithi100 ceph-mon[32210]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:41.474 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:41 smithi161 podman[67321]: 2024-03-12 05:09:41.19546865 +0000 UTC m=+0.361536571 container init f73d71f5907b029ae1349020bc8a31cf5c4a8255b36c6910e84316e0b7d602b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-12T05:09:41.474 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:41 smithi161 podman[67321]: 2024-03-12 05:09:41.198302745 +0000 UTC m=+0.364370666 container start f73d71f5907b029ae1349020bc8a31cf5c4a8255b36c6910e84316e0b7d602b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.license=GPLv2) 2024-03-12T05:09:41.474 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:41 smithi161 ceph-mds[67340]: -- 172.21.15.161:0/2099549219 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e4d04001a0 con 0x55e4d03d6800 2024-03-12T05:09:41.474 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:41 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i[67336]: starting mds.i at 2024-03-12T05:09:41.474 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:41 smithi161 bash[67321]: f73d71f5907b029ae1349020bc8a31cf5c4a8255b36c6910e84316e0b7d602b7 2024-03-12T05:09:41.474 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:09:41 smithi161 systemd[1]: Started Ceph mds.i for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:41.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:41.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.633+0000 7f4a807d0640 1 -- 172.21.15.100:0/490655916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78104550 msgr2=0x7f4a78106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:41.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.633+0000 7f4a807d0640 1 --2- 172.21.15.100:0/490655916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78104550 0x7f4a78106940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009900 tx=0x7f4a6802bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:41.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- 172.21.15.100:0/490655916 shutdown_connections 2024-03-12T05:09:41.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 --2- 172.21.15.100:0/490655916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78106e80 0x7f4a781092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 --2- 172.21.15.100:0/490655916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78104550 0x7f4a78106940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 --2- 172.21.15.100:0/490655916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a78101c20 0x7f4a78104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- 172.21.15.100:0/490655916 >> 172.21.15.100:0/490655916 conn(0x7f4a780fb610 msgr2=0x7f4a780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- 172.21.15.100:0/490655916 shutdown_connections 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- 172.21.15.100:0/490655916 wait complete. 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 Processor -- start 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- start start 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a78101c20 0x7f4a78197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78104550 0x7f4a781977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 0x7f4a7819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7819ed60 con 0x7f4a78106e80 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7819eed0 con 0x7f4a78104550 2024-03-12T05:09:41.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.634+0000 7f4a807d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7819f090 con 0x7f4a78101c20 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 0x7f4a7819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7dd6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78104550 0x7f4a781977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 0x7f4a7819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47128/0 (socket says 172.21.15.100:47128) 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 -- 172.21.15.100:0/3067714832 learned_addr learned my addr 172.21.15.100:0/3067714832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7e56c640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a78101c20 0x7f4a78197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 -- 172.21.15.100:0/3067714832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a78101c20 msgr2=0x7f4a78197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a78101c20 0x7f4a78197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 -- 172.21.15.100:0/3067714832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78104550 msgr2=0x7f4a781977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78104550 0x7f4a781977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 -- 172.21.15.100:0/3067714832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009560 con 0x7f4a78106e80 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a7ed6d640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 0x7f4a7819e820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000eaa0 tx=0x7f4a7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000cdb0 con 0x7f4a78106e80 2024-03-12T05:09:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a70005020 con 0x7f4a78106e80 2024-03-12T05:09:41.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7819f370 con 0x7f4a78106e80 2024-03-12T05:09:41.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a700116b0 con 0x7f4a78106e80 2024-03-12T05:09:41.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.635+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7819f850 con 0x7f4a78106e80 2024-03-12T05:09:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.636+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a780fd100 con 0x7f4a78106e80 2024-03-12T05:09:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.638+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4a70005520 con 0x7f4a78106e80 2024-03-12T05:09:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.639+0000 7f4a6f7fe640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4a640a1ab0 0x7f4a640a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.639+0000 7f4a7e56c640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4a640a1ab0 0x7f4a640a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.639+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a700cd060 con 0x7f4a78106e80 2024-03-12T05:09:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.639+0000 7f4a7e56c640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4a640a1ab0 0x7f4a640a3f70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009bd0 tx=0x7f4a74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.643+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a700967c0 con 0x7f4a78106e80 2024-03-12T05:09:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.865+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a7810cbb0 con 0x7f4a78106e80 2024-03-12T05:09:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.866+0000 7f4a6f7fe640 1 -- 172.21.15.100:0/3067714832 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f4a70099e80 con 0x7f4a78106e80 2024-03-12T05:09:41.864 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:41.864 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24439,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T05:09:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 10 2024-03-12T05:09:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4a640a1ab0 msgr2=0x7f4a640a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4a640a1ab0 0x7f4a640a3f70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009bd0 tx=0x7f4a74005f90 comp rx=0 tx=0).stop 2024-03-12T05:09:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 msgr2=0x7f4a7819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 0x7f4a7819e820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000eaa0 tx=0x7f4a7000ef60 comp rx=0 tx=0).stop 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 shutdown_connections 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4a640a1ab0 0x7f4a640a3f70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78106e80 0x7f4a7819e820 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a78104550 0x7f4a781977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 --2- 172.21.15.100:0/3067714832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a78101c20 0x7f4a78197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 >> 172.21.15.100:0/3067714832 conn(0x7f4a780fb610 msgr2=0x7f4a78102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 shutdown_connections 2024-03-12T05:09:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:41.868+0000 7f4a807d0640 1 -- 172.21.15.100:0/3067714832 wait complete. 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:42.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: Deploying daemon mds.g on smithi100 2024-03-12T05:09:42.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:42.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] up:boot 2024-03-12T05:09:42.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: fsmap 9 up:standby 2024-03-12T05:09:42.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:09:42.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:42 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3067714832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: Deploying daemon mds.g on smithi100 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] up:boot 2024-03-12T05:09:42.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: fsmap 9 up:standby 2024-03-12T05:09:42.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:09:42.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:42 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3067714832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:42.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: Deploying daemon mds.g on smithi100 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] up:boot 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: fsmap 9 up:standby 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:09:42.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:42 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3067714832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:43.569 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 systemd[1]: Starting Ceph mds.g for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:43.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:43.946 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 podman[71905]: 2024-03-12 05:09:43.598852334 +0000 UTC m=+0.017827533 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:43.946 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 podman[71905]: 2024-03-12 05:09:43.702059417 +0000 UTC m=+0.121034670 container create 5dc8a96ab3105d9bac8f7e1ebd94bdc3ea5bafd3d50649c32775381132af7fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-12T05:09:43.947 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 podman[71905]: 2024-03-12 05:09:43.946527936 +0000 UTC m=+0.365503138 container init 5dc8a96ab3105d9bac8f7e1ebd94bdc3ea5bafd3d50649c32775381132af7fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:09:43.947 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 podman[71905]: 2024-03-12 05:09:43.949376595 +0000 UTC m=+0.368351787 container start 5dc8a96ab3105d9bac8f7e1ebd94bdc3ea5bafd3d50649c32775381132af7fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-12T05:09:43.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 -- 172.21.15.100:0/1528403641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0101c70 msgr2=0x7f3ff0108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:43.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 --2- 172.21.15.100:0/1528403641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0101c70 0x7f3ff0108500 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009ca0 tx=0x7f3fe802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:43.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 -- 172.21.15.100:0/1528403641 shutdown_connections 2024-03-12T05:09:43.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 --2- 172.21.15.100:0/1528403641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0101c70 0x7f3ff0108500 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 --2- 172.21.15.100:0/1528403641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff01012b0 0x7f3ff0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 --2- 172.21.15.100:0/1528403641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff01000c0 0x7f3ff01004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 -- 172.21.15.100:0/1528403641 >> 172.21.15.100:0/1528403641 conn(0x7f3ff00fb630 msgr2=0x7f3ff00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 -- 172.21.15.100:0/1528403641 shutdown_connections 2024-03-12T05:09:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.989+0000 7f3ff480e640 1 -- 172.21.15.100:0/1528403641 wait complete. 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 Processor -- start 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 -- start start 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 0x7f3ff0069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff01012b0 0x7f3ff019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff0101c70 0x7f3ff019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019ec50 con 0x7f3ff01000c0 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019edc0 con 0x7f3ff0101c70 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3ff480e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019ef80 con 0x7f3ff01012b0 2024-03-12T05:09:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 0x7f3ff0069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 0x7f3ff0069ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47160/0 (socket says 172.21.15.100:47160) 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 -- 172.21.15.100:0/2722552002 learned_addr learned my addr 172.21.15.100:0/2722552002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3feed9d640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff0101c70 0x7f3ff019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fedd9b640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff01012b0 0x7f3ff019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 -- 172.21.15.100:0/2722552002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff01012b0 msgr2=0x7f3ff019be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff01012b0 0x7f3ff019be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 -- 172.21.15.100:0/2722552002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff0101c70 msgr2=0x7f3ff019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff0101c70 0x7f3ff019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3fee59c640 1 -- 172.21.15.100:0/2722552002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009900 con 0x7f3ff01000c0 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.990+0000 7f3feed9d640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff0101c70 0x7f3ff019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.991+0000 7f3fee59c640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 0x7f3ff0069ff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c970 tx=0x7f3fe000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.991+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0007c10 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.991+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fe0007d70 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.991+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff019f200 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.991+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000f8a0 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.991+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff019f760 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.992+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff0107310 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.993+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f3fe0018af0 con 0x7f3ff01000c0 2024-03-12T05:09:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.993+0000 7f3fdf7fe640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3fd80a1ab0 0x7f3fd80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:43.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.993+0000 7f3fedd9b640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3fd80a1ab0 0x7f3fd80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:43.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.993+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3fe0013070 con 0x7f3ff01000c0 2024-03-12T05:09:43.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.994+0000 7f3fedd9b640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3fd80a1ab0 0x7f3fd80a3f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005e10 tx=0x7f3fe4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:43.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:43.998+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fe0096a00 con 0x7f3ff01000c0 2024-03-12T05:09:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.222+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ff010bad0 con 0x7f3ff01000c0 2024-03-12T05:09:44.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.222+0000 7f3fdf7fe640 1 -- 172.21.15.100:0/2722552002 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f3fe009a0c0 con 0x7f3ff01000c0 2024-03-12T05:09:44.220 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:44.220 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24439,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T05:09:44.221 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 10 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3fd80a1ab0 msgr2=0x7f3fd80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3fd80a1ab0 0x7f3fd80a3f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005e10 tx=0x7f3fe4005da0 comp rx=0 tx=0).stop 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 msgr2=0x7f3ff0069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 0x7f3ff0069ff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c970 tx=0x7f3fe000ce30 comp rx=0 tx=0).stop 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 shutdown_connections 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3fd80a1ab0 0x7f3fd80a3f70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff0101c70 0x7f3ff019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff01012b0 0x7f3ff019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 --2- 172.21.15.100:0/2722552002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff01000c0 0x7f3ff0069ff0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 >> 172.21.15.100:0/2722552002 conn(0x7f3ff00fb630 msgr2=0x7f3ff01044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 shutdown_connections 2024-03-12T05:09:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:44.224+0000 7f3ff480e640 1 -- 172.21.15.100:0/2722552002 wait complete. 2024-03-12T05:09:44.232 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 ceph-mds[71944]: -- 172.21.15.100:0/3683914881 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dc5778e1a0 con 0x55dc57765000 2024-03-12T05:09:44.232 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g[71920]: starting mds.g at 2024-03-12T05:09:44.232 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:44 smithi100 bash[71905]: 5dc8a96ab3105d9bac8f7e1ebd94bdc3ea5bafd3d50649c32775381132af7fd9 2024-03-12T05:09:44.233 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:09:44 smithi100 systemd[1]: Started Ceph mds.g for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:45.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2722552002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:45.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:45.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2722552002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:45.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:45.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:45.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:45.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2722552002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:45.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:45 smithi100 ceph-mon[32210]: Deploying daemon mds.a on smithi100 2024-03-12T05:09:45.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:45 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.100:6838/2856725609,v1:172.21.15.100:6839/2856725609] up:boot 2024-03-12T05:09:45.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:45 smithi100 ceph-mon[32210]: fsmap 10 up:standby 2024-03-12T05:09:45.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:45 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T05:09:46.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:45 smithi156 ceph-mon[36270]: Deploying daemon mds.a on smithi100 2024-03-12T05:09:46.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:45 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.100:6838/2856725609,v1:172.21.15.100:6839/2856725609] up:boot 2024-03-12T05:09:46.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:45 smithi156 ceph-mon[36270]: fsmap 10 up:standby 2024-03-12T05:09:46.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:45 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T05:09:46.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:45 smithi161 ceph-mon[36043]: Deploying daemon mds.a on smithi100 2024-03-12T05:09:46.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:45 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.100:6838/2856725609,v1:172.21.15.100:6839/2856725609] up:boot 2024-03-12T05:09:46.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:45 smithi161 ceph-mon[36043]: fsmap 10 up:standby 2024-03-12T05:09:46.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:45 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T05:09:46.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:46.262 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 systemd[1]: Starting Ceph mds.a for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 -- 172.21.15.100:0/1332354581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f18101290 msgr2=0x7f0f18101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 --2- 172.21.15.100:0/1332354581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f18101290 0x7f0f18101710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009900 tx=0x7f0f0c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 -- 172.21.15.100:0/1332354581 shutdown_connections 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 --2- 172.21.15.100:0/1332354581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f18101c50 0x7f0f181084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 --2- 172.21.15.100:0/1332354581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f18101290 0x7f0f18101710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 --2- 172.21.15.100:0/1332354581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f181000a0 0x7f0f181004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.351+0000 7f0f1d464640 1 -- 172.21.15.100:0/1332354581 >> 172.21.15.100:0/1332354581 conn(0x7f0f180fb610 msgr2=0x7f0f180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.352+0000 7f0f1d464640 1 -- 172.21.15.100:0/1332354581 shutdown_connections 2024-03-12T05:09:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.352+0000 7f0f1d464640 1 -- 172.21.15.100:0/1332354581 wait complete. 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.352+0000 7f0f1d464640 1 Processor -- start 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 -- start start 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 0x7f0f18197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f18101290 0x7f0f18197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18101c50 0x7f0f1819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1819ed00 con 0x7f0f181000a0 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1819ee70 con 0x7f0f18101290 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f1d464640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1819f030 con 0x7f0f18101c50 2024-03-12T05:09:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 0x7f0f18197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 0x7f0f18197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47184/0 (socket says 172.21.15.100:47184) 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 -- 172.21.15.100:0/2550603979 learned_addr learned my addr 172.21.15.100:0/2550603979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f167fc640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f18101290 0x7f0f18197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f177fe640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18101c50 0x7f0f1819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 -- 172.21.15.100:0/2550603979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18101c50 msgr2=0x7f0f1819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18101c50 0x7f0f1819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 -- 172.21.15.100:0/2550603979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f18101290 msgr2=0x7f0f18197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f18101290 0x7f0f18197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 -- 172.21.15.100:0/2550603979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c009560 con 0x7f0f181000a0 2024-03-12T05:09:46.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0f16ffd640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 0x7f0f18197210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e9c0 tx=0x7f0f0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.353+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080096e0 con 0x7f0f181000a0 2024-03-12T05:09:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.354+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1819f310 con 0x7f0f181000a0 2024-03-12T05:09:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.354+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f0800cd80 con 0x7f0f181000a0 2024-03-12T05:09:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.354+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08010430 con 0x7f0f181000a0 2024-03-12T05:09:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.354+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1819f7f0 con 0x7f0f181000a0 2024-03-12T05:09:46.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.355+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f18067f70 con 0x7f0f181000a0 2024-03-12T05:09:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.357+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f0f08003680 con 0x7f0f181000a0 2024-03-12T05:09:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.357+0000 7f0efffff640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f000a1a30 0x7f0f000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.357+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f08014070 con 0x7f0f181000a0 2024-03-12T05:09:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.357+0000 7f0f167fc640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f000a1a30 0x7f0f000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.358+0000 7f0f167fc640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f000a1a30 0x7f0f000a3ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c000c00 tx=0x7f0f0c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:46.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.360+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f08096b40 con 0x7f0f181000a0 2024-03-12T05:09:46.562 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 podman[72273]: 2024-03-12 05:09:46.334652378 +0000 UTC m=+0.022984891 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:46.562 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 podman[72273]: 2024-03-12 05:09:46.439779801 +0000 UTC m=+0.128112310 container create 45a474f3c504279a1e4f9286edd0019739cd0861201fe14158074c878be1f4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-12T05:09:46.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.583+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f1810bb00 con 0x7f0f181000a0 2024-03-12T05:09:46.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.584+0000 7f0efffff640 1 -- 172.21.15.100:0/2550603979 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7f0f0809a200 con 0x7f0f181000a0 2024-03-12T05:09:46.582 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:46.582 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24439,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T05:09:46.582 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 11 2024-03-12T05:09:46.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f000a1a30 msgr2=0x7f0f000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:46.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f000a1a30 0x7f0f000a3ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c000c00 tx=0x7f0f0c02c7e0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 msgr2=0x7f0f18197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 0x7f0f18197210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e9c0 tx=0x7f0f0800ee80 comp rx=0 tx=0).stop 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 shutdown_connections 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f000a1a30 0x7f0f000a3ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18101c50 0x7f0f1819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f18101290 0x7f0f18197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 --2- 172.21.15.100:0/2550603979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f181000a0 0x7f0f18197210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 >> 172.21.15.100:0/2550603979 conn(0x7f0f180fb610 msgr2=0x7f0f180fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 shutdown_connections 2024-03-12T05:09:46.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:46.586+0000 7f0f1d464640 1 -- 172.21.15.100:0/2550603979 wait complete. 2024-03-12T05:09:46.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:46 smithi100 ceph-mon[32210]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:46.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:46 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2550603979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:46.819 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 podman[72273]: 2024-03-12 05:09:46.759307567 +0000 UTC m=+0.447640089 container init 45a474f3c504279a1e4f9286edd0019739cd0861201fe14158074c878be1f4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397) 2024-03-12T05:09:46.819 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 podman[72273]: 2024-03-12 05:09:46.762162788 +0000 UTC m=+0.450495303 container start 45a474f3c504279a1e4f9286edd0019739cd0861201fe14158074c878be1f4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:09:46.819 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 ceph-mds[72322]: -- 172.21.15.100:0/3792977132 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56120f31e1a0 con 0x56120f2f4800 2024-03-12T05:09:46.819 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a[72314]: starting mds.a at 2024-03-12T05:09:47.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:46 smithi156 ceph-mon[36270]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:47.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:46 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2550603979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:47.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:46 smithi161 ceph-mon[36043]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:47.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:46 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2550603979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:47.240 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 bash[72273]: 45a474f3c504279a1e4f9286edd0019739cd0861201fe14158074c878be1f4bf 2024-03-12T05:09:47.240 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:09:46 smithi100 systemd[1]: Started Ceph mds.a for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: Deploying daemon mds.b on smithi156 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.100:6840/3942756808,v1:172.21.15.100:6841/3942756808] up:boot 2024-03-12T05:09:48.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: fsmap 11 up:standby 2024-03-12T05:09:48.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:47 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T05:09:48.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: Deploying daemon mds.b on smithi156 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.100:6840/3942756808,v1:172.21.15.100:6841/3942756808] up:boot 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: fsmap 11 up:standby 2024-03-12T05:09:48.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:47 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T05:09:48.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: Deploying daemon mds.b on smithi156 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.100:6840/3942756808,v1:172.21.15.100:6841/3942756808] up:boot 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: fsmap 11 up:standby 2024-03-12T05:09:48.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:47 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T05:09:48.596 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:48.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.714+0000 7f91bc647640 1 -- 172.21.15.100:0/3987297678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b41000a0 msgr2=0x7f91b41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.714+0000 7f91bc647640 1 --2- 172.21.15.100:0/3987297678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b41000a0 0x7f91b41004c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009900 tx=0x7f91b002bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- 172.21.15.100:0/3987297678 shutdown_connections 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 --2- 172.21.15.100:0/3987297678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b4101c50 0x7f91b41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 --2- 172.21.15.100:0/3987297678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91b4101290 0x7f91b4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 --2- 172.21.15.100:0/3987297678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b41000a0 0x7f91b41004c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- 172.21.15.100:0/3987297678 >> 172.21.15.100:0/3987297678 conn(0x7f91b40fb610 msgr2=0x7f91b40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- 172.21.15.100:0/3987297678 shutdown_connections 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- 172.21.15.100:0/3987297678 wait complete. 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 Processor -- start 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- start start 2024-03-12T05:09:48.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91b41000a0 0x7f91b4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 0x7f91b41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b4101c50 0x7f91b419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b419ed50 con 0x7f91b4101290 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b419eec0 con 0x7f91b41000a0 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91bc647640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b419f080 con 0x7f91b4101c50 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.715+0000 7f91b9be2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 0x7f91b41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 0x7f91b41977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47218/0 (socket says 172.21.15.100:47218) 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 -- 172.21.15.100:0/3399493885 learned_addr learned my addr 172.21.15.100:0/3399493885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91ba3e3640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91b41000a0 0x7f91b4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91babe4640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b4101c50 0x7f91b419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 -- 172.21.15.100:0/3399493885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b4101c50 msgr2=0x7f91b419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b4101c50 0x7f91b419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 -- 172.21.15.100:0/3399493885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91b41000a0 msgr2=0x7f91b4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91b41000a0 0x7f91b4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 -- 172.21.15.100:0/3399493885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0009560 con 0x7f91b4101290 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91b9be2640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 0x7f91b41977a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e9f0 tx=0x7f91a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4009700 con 0x7f91b4101290 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91a400cd80 con 0x7f91b4101290 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b419f360 con 0x7f91b4101290 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4018920 con 0x7f91b4101290 2024-03-12T05:09:48.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.716+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b419f8c0 con 0x7f91b4101290 2024-03-12T05:09:48.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.717+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b4107440 con 0x7f91b4101290 2024-03-12T05:09:48.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.718+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f91a4018a80 con 0x7f91b4101290 2024-03-12T05:09:48.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.719+0000 7f91ab7fe640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f91a00a1ab0 0x7f91a00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:48.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.719+0000 7f91ba3e3640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f91a00a1ab0 0x7f91a00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:48.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.719+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f91a4014070 con 0x7f91b4101290 2024-03-12T05:09:48.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.719+0000 7f91ba3e3640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f91a00a1ab0 0x7f91a00a3f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91b0005ab0 tx=0x7f91b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.723+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91a4052d20 con 0x7f91b4101290 2024-03-12T05:09:48.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.932+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91b410cb70 con 0x7f91b4101290 2024-03-12T05:09:48.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.933+0000 7f91ab7fe640 1 -- 172.21.15.100:0/3399493885 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9263 (secure 0 0 0) 0x7f91a40969b0 con 0x7f91b4101290 2024-03-12T05:09:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:48.931 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24439,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-12T05:09:48.931 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 12 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f91a00a1ab0 msgr2=0x7f91a00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f91a00a1ab0 0x7f91a00a3f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91b0005ab0 tx=0x7f91b0005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 msgr2=0x7f91b41977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 0x7f91b41977a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91a400e9f0 tx=0x7f91a400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 shutdown_connections 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f91a00a1ab0 0x7f91a00a3f70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b4101c50 0x7f91b419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b4101290 0x7f91b41977a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 --2- 172.21.15.100:0/3399493885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91b41000a0 0x7f91b4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:48.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 >> 172.21.15.100:0/3399493885 conn(0x7f91b40fb610 msgr2=0x7f91b4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:48.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 shutdown_connections 2024-03-12T05:09:48.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:48.935+0000 7f91bc647640 1 -- 172.21.15.100:0/3399493885 wait complete. 2024-03-12T05:09:49.251 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:48 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3399493885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:49.251 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 systemd[1]: Starting Ceph mds.b for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:09:49.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:48 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3399493885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:49.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:48 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3399493885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:49.577 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 podman[66861]: 2024-03-12 05:09:49.372169763 +0000 UTC m=+0.018211917 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:09:49.577 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 podman[66861]: 2024-03-12 05:09:49.471896732 +0000 UTC m=+0.117938876 container create 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-12T05:09:49.955 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 podman[66861]: 2024-03-12 05:09:49.665870701 +0000 UTC m=+0.311912846 container init 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-12T05:09:49.955 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 podman[66861]: 2024-03-12 05:09:49.668648652 +0000 UTC m=+0.314690806 container start 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T05:09:49.956 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 ceph-mds[66881]: -- 172.21.15.156:0/3571970696 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2a6ad81a0 con 0x55b2a6aae800 2024-03-12T05:09:49.956 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: starting mds.b at 2024-03-12T05:09:49.956 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 bash[66861]: 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 2024-03-12T05:09:49.956 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:49 smithi156 systemd[1]: Started Ceph mds.b for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:09:50.945 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 -- 172.21.15.100:0/3283590818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1501000a0 msgr2=0x7ff1501004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 --2- 172.21.15.100:0/3283590818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1501000a0 0x7ff1501004c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff140009900 tx=0x7ff14002bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 -- 172.21.15.100:0/3283590818 shutdown_connections 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 --2- 172.21.15.100:0/3283590818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff150101c50 0x7ff1501084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 --2- 172.21.15.100:0/3283590818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff150101290 0x7ff150101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 --2- 172.21.15.100:0/3283590818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1501000a0 0x7ff1501004c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 -- 172.21.15.100:0/3283590818 >> 172.21.15.100:0/3283590818 conn(0x7ff1500fb610 msgr2=0x7ff1500fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 -- 172.21.15.100:0/3283590818 shutdown_connections 2024-03-12T05:09:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.052+0000 7ff155c0e640 1 -- 172.21.15.100:0/3283590818 wait complete. 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 Processor -- start 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 -- start start 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1501000a0 0x7ff1501972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 0x7ff150197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff150101c50 0x7ff15019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019ed70 con 0x7ff150101290 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019eee0 con 0x7ff150101c50 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff155c0e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019f0a0 con 0x7ff1501000a0 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff14effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 0x7ff150197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff14effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 0x7ff150197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54086/0 (socket says 172.21.15.100:54086) 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff14effd640 1 -- 172.21.15.100:0/1984873715 learned_addr learned my addr 172.21.15.100:0/1984873715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:51.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.053+0000 7ff14f7fe640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1501000a0 0x7ff1501972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14ffff640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff150101c50 0x7ff15019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14effd640 1 -- 172.21.15.100:0/1984873715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1501000a0 msgr2=0x7ff1501972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14effd640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1501000a0 0x7ff1501972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14effd640 1 -- 172.21.15.100:0/1984873715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff150101c50 msgr2=0x7ff15019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14effd640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff150101c50 0x7ff15019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14effd640 1 -- 172.21.15.100:0/1984873715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140009560 con 0x7ff150101290 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14effd640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 0x7ff150197830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff14400d950 tx=0x7ff14400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144009a30 con 0x7ff150101290 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15019f380 con 0x7ff150101290 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff144009b90 con 0x7ff150101290 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144004a00 con 0x7ff150101290 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.054+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15019f890 con 0x7ff150101290 2024-03-12T05:09:51.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.055+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15010bab0 con 0x7ff150101290 2024-03-12T05:09:51.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.056+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff144009d00 con 0x7ff150101290 2024-03-12T05:09:51.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.056+0000 7ff14cff9640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1380a1a30 0x7ff1380a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.057+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff1440ccaa0 con 0x7ff150101290 2024-03-12T05:09:51.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.057+0000 7ff14f7fe640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1380a1a30 0x7ff1380a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.057+0000 7ff14f7fe640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1380a1a30 0x7ff1380a3ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff140006010 tx=0x7ff140000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:51.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.061+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff144096150 con 0x7ff150101290 2024-03-12T05:09:51.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:51.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] up:boot 2024-03-12T05:09:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: fsmap 12 up:standby 2024-03-12T05:09:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:09:51.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:50 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:51.099 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:51.099 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.099 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.099 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.099 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.099 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] up:boot 2024-03-12T05:09:51.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: fsmap 12 up:standby 2024-03-12T05:09:51.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:09:51.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:50 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] up:boot 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: fsmap 12 up:standby 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:09:51.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:50 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.272+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff15019fb10 con 0x7ff150101290 2024-03-12T05:09:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.272+0000 7ff14cff9640 1 -- 172.21.15.100:0/1984873715 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10058 (secure 0 0 0) 0x7ff1440cae60 con 0x7ff150101290 2024-03-12T05:09:51.270 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24439,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-12T05:09:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 13 2024-03-12T05:09:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.274+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1380a1a30 msgr2=0x7ff1380a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.274+0000 7ff155c0e640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1380a1a30 0x7ff1380a3ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff140006010 tx=0x7ff140000bc0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.274+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 msgr2=0x7ff150197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.274+0000 7ff155c0e640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 0x7ff150197830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff14400d950 tx=0x7ff14400de10 comp rx=0 tx=0).stop 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.274+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 shutdown_connections 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.274+0000 7ff155c0e640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1380a1a30 0x7ff1380a3ef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.275+0000 7ff155c0e640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff150101c50 0x7ff15019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.275+0000 7ff155c0e640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff150101290 0x7ff150197830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.275+0000 7ff155c0e640 1 --2- 172.21.15.100:0/1984873715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1501000a0 0x7ff1501972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.275+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 >> 172.21.15.100:0/1984873715 conn(0x7ff1500fb610 msgr2=0x7ff1500fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:51.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.275+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 shutdown_connections 2024-03-12T05:09:51.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.275+0000 7ff155c0e640 1 -- 172.21.15.100:0/1984873715 wait complete. 2024-03-12T05:09:51.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:09:51.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 -- 172.21.15.100:0/114759661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410e540 msgr2=0x7f87641109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 --2- 172.21.15.100:0/114759661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410e540 0x7f87641109e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f875c009ca0 tx=0x7f875c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 -- 172.21.15.100:0/114759661 shutdown_connections 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 --2- 172.21.15.100:0/114759661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410e540 0x7f87641109e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 --2- 172.21.15.100:0/114759661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f876410bbc0 0x7f876410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 --2- 172.21.15.100:0/114759661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 0x7f876410b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 -- 172.21.15.100:0/114759661 >> 172.21.15.100:0/114759661 conn(0x7f876406dff0 msgr2=0x7f8764070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 -- 172.21.15.100:0/114759661 shutdown_connections 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.395+0000 7f876a935640 1 -- 172.21.15.100:0/114759661 wait complete. 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 Processor -- start 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 -- start start 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 0x7f876419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 0x7f876419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f876410e540 0x7f87641a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641a6450 con 0x7f876410bbc0 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641a65c0 con 0x7f8764109290 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f876a935640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641a6780 con 0x7f876410e540 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 0x7f876419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f8763fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 0x7f876419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 0x7f876419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54102/0 (socket says 172.21.15.100:54102) 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 -- 172.21.15.100:0/3869826962 learned_addr learned my addr 172.21.15.100:0/3869826962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f8768ed2640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f876410e540 0x7f87641a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 -- 172.21.15.100:0/3869826962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f876410e540 msgr2=0x7f87641a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f876410e540 0x7f87641a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 -- 172.21.15.100:0/3869826962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 msgr2=0x7f876419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 0x7f876419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f87637fe640 1 -- 172.21.15.100:0/3869826962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009900 con 0x7f876410bbc0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.396+0000 7f8763fff640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 0x7f876419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.397+0000 7f87637fe640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 0x7f876419eea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f875800d770 tx=0x7f875800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.397+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758009c30 con 0x7f876410bbc0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.397+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87641a6a60 con 0x7f876410bbc0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.397+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f875800de80 con 0x7f876410bbc0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.397+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758004e80 con 0x7f876410bbc0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.397+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87641a6f20 con 0x7f876410bbc0 2024-03-12T05:09:51.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.398+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764070af0 con 0x7f876410bbc0 2024-03-12T05:09:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.399+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f8758004fe0 con 0x7f876410bbc0 2024-03-12T05:09:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.400+0000 7f87617fa640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f874c0a1a30 0x7f874c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.400+0000 7f8763fff640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f874c0a1a30 0x7f874c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.400+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87580ccd40 con 0x7f876410bbc0 2024-03-12T05:09:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.400+0000 7f8763fff640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f874c0a1a30 0x7f874c0a3ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8764072270 tx=0x7f875400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:51.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.403+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87580964a0 con 0x7f876410bbc0 2024-03-12T05:09:51.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.607+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f876402cda0 con 0x7f876410bbc0 2024-03-12T05:09:51.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.608+0000 7f87617fa640 1 -- 172.21.15.100:0/3869826962 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18486 (secure 0 0 0) 0x7f8758017090 con 0x7f876410bbc0 2024-03-12T05:09:51.607 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:51.607 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":82,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:09:07.135605+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:09:51.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f874c0a1a30 msgr2=0x7f874c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f874c0a1a30 0x7f874c0a3ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8764072270 tx=0x7f875400b390 comp rx=0 tx=0).stop 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 msgr2=0x7f876419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 0x7f876419eea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f875800d770 tx=0x7f875800dc30 comp rx=0 tx=0).stop 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 shutdown_connections 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f874c0a1a30 0x7f874c0a3ef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f876410e540 0x7f87641a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f876410bbc0 0x7f876419eea0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 --2- 172.21.15.100:0/3869826962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8764109290 0x7f876419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.610+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 >> 172.21.15.100:0/3869826962 conn(0x7f876406dff0 msgr2=0x7f8764109670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.611+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 shutdown_connections 2024-03-12T05:09:51.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.611+0000 7f876a935640 1 -- 172.21.15.100:0/3869826962 wait complete. 2024-03-12T05:09:51.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:09:51.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:09:51.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 -- 172.21.15.100:0/2215141764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101190 msgr2=0x7f48081015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 --2- 172.21.15.100:0/2215141764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101190 0x7f48081015b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009900 tx=0x7f47f802bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 -- 172.21.15.100:0/2215141764 shutdown_connections 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 --2- 172.21.15.100:0/2215141764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4808102d40 0x7f48081095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 --2- 172.21.15.100:0/2215141764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4808102380 0x7f4808102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 --2- 172.21.15.100:0/2215141764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101190 0x7f48081015b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 -- 172.21.15.100:0/2215141764 >> 172.21.15.100:0/2215141764 conn(0x7f48080fc6e0 msgr2=0x7f48080feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.748+0000 7f480e9bc640 1 -- 172.21.15.100:0/2215141764 shutdown_connections 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 -- 172.21.15.100:0/2215141764 wait complete. 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 Processor -- start 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 -- start start 2024-03-12T05:09:51.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4808101190 0x7f4808197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4808102380 0x7f48081977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 0x7f480819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480819ed60 con 0x7f4808102d40 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480819eed0 con 0x7f4808101190 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.749+0000 7f480e9bc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480819f090 con 0x7f4808102380 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 0x7f480819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f4807fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4808101190 0x7f4808197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 0x7f480819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54120/0 (socket says 172.21.15.100:54120) 2024-03-12T05:09:51.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 -- 172.21.15.100:0/12148320 learned_addr learned my addr 172.21.15.100:0/12148320 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f48077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4808102380 0x7f48081977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 -- 172.21.15.100:0/12148320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4808102380 msgr2=0x7f48081977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4808102380 0x7f48081977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 -- 172.21.15.100:0/12148320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4808101190 msgr2=0x7f4808197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4808101190 0x7f4808197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 -- 172.21.15.100:0/12148320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8009560 con 0x7f4808102d40 2024-03-12T05:09:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.750+0000 7f480cf59640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 0x7f480819e820 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f480000eb10 tx=0x7f480000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:51.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.751+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004b20 con 0x7f4808102d40 2024-03-12T05:09:51.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.751+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480819f370 con 0x7f4808102d40 2024-03-12T05:09:51.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.751+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480819f8d0 con 0x7f4808102d40 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.751+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4800021920 con 0x7f4808102d40 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.751+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800010890 con 0x7f4808102d40 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.752+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f40052c0 con 0x7f4808102d40 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.753+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4800021440 con 0x7f4808102d40 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.754+0000 7f48057fa640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47f00a1a30 0x7f47f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.754+0000 7f4807fff640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47f00a1a30 0x7f47f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.754+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4800014070 con 0x7f4808102d40 2024-03-12T05:09:51.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.754+0000 7f4807fff640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47f00a1a30 0x7f47f00a3ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47f802c240 tx=0x7f47f802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:51.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.758+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4800096af0 con 0x7f4808102d40 2024-03-12T05:09:51.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.956+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47f4005840 con 0x7f4808102d40 2024-03-12T05:09:51.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.957+0000 7f48057fa640 1 -- 172.21.15.100:0/12148320 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f480009a1b0 con 0x7f4808102d40 2024-03-12T05:09:51.955 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47f00a1a30 msgr2=0x7f47f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47f00a1a30 0x7f47f00a3ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47f802c240 tx=0x7f47f802c7e0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 msgr2=0x7f480819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 0x7f480819e820 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f480000eb10 tx=0x7f480000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 shutdown_connections 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47f00a1a30 0x7f47f00a3ef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808102d40 0x7f480819e820 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4808102380 0x7f48081977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 --2- 172.21.15.100:0/12148320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4808101190 0x7f4808197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:51.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 >> 172.21.15.100:0/12148320 conn(0x7f48080fc6e0 msgr2=0x7f4808105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:51.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 shutdown_connections 2024-03-12T05:09:51.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:51.959+0000 7f480e9bc640 1 -- 172.21.15.100:0/12148320 wait complete. 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1984873715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3869826962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:51 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:51.969 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-12T05:09:52.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1527222406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6081023b0 msgr2=0x7fa608102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1527222406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6081023b0 0x7fa608102830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60402bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:52.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1527222406 shutdown_connections 2024-03-12T05:09:52.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1527222406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa608102d70 0x7fa608109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1527222406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6081023b0 0x7fa608102830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1527222406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 0x7fa6081015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.097+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1527222406 >> 172.21.15.100:0/1527222406 conn(0x7fa6080fc730 msgr2=0x7fa6080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1527222406 shutdown_connections 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1527222406 wait complete. 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 Processor -- start 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 -- start start 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 0x7fa608197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6081023b0 0x7fa6081977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 0x7fa60819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60819ed80 con 0x7fa608102d70 2024-03-12T05:09:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60819eef0 con 0x7fa6081023b0 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60f3e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60819f0b0 con 0x7fa6081011c0 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60d986640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 0x7fa60819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60d986640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 0x7fa60819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54140/0 (socket says 172.21.15.100:54140) 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60d986640 1 -- 172.21.15.100:0/1798400093 learned_addr learned my addr 172.21.15.100:0/1798400093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60d185640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 0x7fa608197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.098+0000 7fa60c984640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6081023b0 0x7fa6081977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d986640 1 -- 172.21.15.100:0/1798400093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 msgr2=0x7fa608197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d986640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 0x7fa608197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d986640 1 -- 172.21.15.100:0/1798400093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6081023b0 msgr2=0x7fa6081977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d986640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6081023b0 0x7fa6081977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d986640 1 -- 172.21.15.100:0/1798400093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604009560 con 0x7fa608102d70 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d185640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 0x7fa608197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60d986640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 0x7fa60819e840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa600009fd0 tx=0x7fa60000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000bea0 con 0x7fa608102d70 2024-03-12T05:09:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa600009940 con 0x7fa608102d70 2024-03-12T05:09:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60819f390 con 0x7fa608102d70 2024-03-12T05:09:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000106c0 con 0x7fa608102d70 2024-03-12T05:09:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.099+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60819f820 con 0x7fa608102d70 2024-03-12T05:09:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.100+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa608108560 con 0x7fa608102d70 2024-03-12T05:09:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.101+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa600004240 con 0x7fa608102d70 2024-03-12T05:09:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.102+0000 7fa5fe7fc640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa5f40a1a30 0x7fa5f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.102+0000 7fa60d185640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa5f40a1a30 0x7fa5f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.102+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6000ccbb0 con 0x7fa608102d70 2024-03-12T05:09:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.102+0000 7fa60d185640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa5f40a1a30 0x7fa5f40a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa608102210 tx=0x7fa5f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:52.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.110+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa600096310 con 0x7fa608102d70 2024-03-12T05:09:52.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1984873715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:52.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3869826962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:52.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:51 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1984873715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:09:52.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3869826962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:52.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:52.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:51 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:52.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.304+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa60802ce40 con 0x7fa608102d70 2024-03-12T05:09:52.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.304+0000 7fa5fe7fc640 1 -- 172.21.15.100:0/1798400093 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa6000999d0 con 0x7fa608102d70 2024-03-12T05:09:52.302 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:52.302 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-03-12T05:09:52.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:09:52.303 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-03-12T05:09:52.303 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-03-12T05:09:52.303 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:09:52.303 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-03-12T05:09:52.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa5f40a1a30 msgr2=0x7fa5f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa5f40a1a30 0x7fa5f40a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa608102210 tx=0x7fa5f8005f90 comp rx=0 tx=0).stop 2024-03-12T05:09:52.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 msgr2=0x7fa60819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 0x7fa60819e840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa600009fd0 tx=0x7fa60000b260 comp rx=0 tx=0).stop 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 shutdown_connections 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa5f40a1a30 0x7fa5f40a3ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa608102d70 0x7fa60819e840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6081023b0 0x7fa6081977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 --2- 172.21.15.100:0/1798400093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6081011c0 0x7fa608197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 >> 172.21.15.100:0/1798400093 conn(0x7fa6080fc730 msgr2=0x7fa608105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 shutdown_connections 2024-03-12T05:09:52.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.306+0000 7fa60f3e9640 1 -- 172.21.15.100:0/1798400093 wait complete. 2024-03-12T05:09:52.315 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-12T05:09:52.316 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-12T05:09:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1574666905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68100080 msgr2=0x7fbb681004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1574666905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68100080 0x7fbb681004a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009900 tx=0x7fbb5c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1574666905 shutdown_connections 2024-03-12T05:09:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1574666905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68101c30 0x7fbb681084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1574666905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb68101270 0x7fbb681016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1574666905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68100080 0x7fbb681004a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1574666905 >> 172.21.15.100:0/1574666905 conn(0x7fbb680fb610 msgr2=0x7fbb680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1574666905 shutdown_connections 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.426+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1574666905 wait complete. 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 Processor -- start 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 -- start start 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb68100080 0x7fbb68197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68101270 0x7fbb681977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 0x7fbb6819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6819ed50 con 0x7fbb68101c30 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6819eec0 con 0x7fbb68100080 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6ec0e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6819f080 con 0x7fbb68101270 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 0x7fbb6819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 0x7fbb6819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54158/0 (socket says 172.21.15.100:54158) 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 -- 172.21.15.100:0/1221378850 learned_addr learned my addr 172.21.15.100:0/1221378850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6c9aa640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb68100080 0x7fbb68197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb67fff640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68101270 0x7fbb681977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 -- 172.21.15.100:0/1221378850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68101270 msgr2=0x7fbb681977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68101270 0x7fbb681977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 -- 172.21.15.100:0/1221378850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb68100080 msgr2=0x7fbb68197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb68100080 0x7fbb68197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.427+0000 7fbb6d1ab640 1 -- 172.21.15.100:0/1221378850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c009560 con 0x7fbb68101c30 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.428+0000 7fbb6d1ab640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 0x7fbb6819e810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000eb10 tx=0x7fbb6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.428+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60004b20 con 0x7fbb68101c30 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.428+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6819f360 con 0x7fbb68101c30 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.428+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb60004c80 con 0x7fbb68101c30 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.428+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60010670 con 0x7fbb68101c30 2024-03-12T05:09:52.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.428+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6819f8c0 con 0x7fbb68101c30 2024-03-12T05:09:52.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.429+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6810ba90 con 0x7fbb68101c30 2024-03-12T05:09:52.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.430+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbb60004df0 con 0x7fbb68101c30 2024-03-12T05:09:52.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.431+0000 7fbb65ffb640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb580a1a30 0x7fbb580a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:52.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.431+0000 7fbb6c9aa640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb580a1a30 0x7fbb580a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:52.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.431+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb60014070 con 0x7fbb68101c30 2024-03-12T05:09:52.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.431+0000 7fbb6c9aa640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb580a1a30 0x7fbb580a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c005ab0 tx=0x7fbb5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:52.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.434+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb60096d90 con 0x7fbb68101c30 2024-03-12T05:09:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.625+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb6819fd60 con 0x7fbb68101c30 2024-03-12T05:09:52.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.919+0000 7fbb65ffb640 1 -- 172.21.15.100:0/1221378850 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb6009a450 con 0x7fbb68101c30 2024-03-12T05:09:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-03-12T05:09:52.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb580a1a30 msgr2=0x7fbb580a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb580a1a30 0x7fbb580a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c005ab0 tx=0x7fbb5c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 msgr2=0x7fbb6819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 0x7fbb6819e810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000eb10 tx=0x7fbb6000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 shutdown_connections 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbb580a1a30 0x7fbb580a3ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb68101c30 0x7fbb6819e810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68101270 0x7fbb681977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 --2- 172.21.15.100:0/1221378850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb68100080 0x7fbb68197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 >> 172.21.15.100:0/1221378850 conn(0x7fbb680fb610 msgr2=0x7fbb680fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 shutdown_connections 2024-03-12T05:09:52.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:52.923+0000 7fbb6ec0e640 1 -- 172.21.15.100:0/1221378850 wait complete. 2024-03-12T05:09:52.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 -- 172.21.15.100:0/3086092842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 msgr2=0x7f274c1082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 --2- 172.21.15.100:0/3086092842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c1082a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2744009ca0 tx=0x7f274402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 -- 172.21.15.100:0/3086092842 shutdown_connections 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 --2- 172.21.15.100:0/3086092842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c1082a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 --2- 172.21.15.100:0/3086092842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 0x7f274c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 --2- 172.21.15.100:0/3086092842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f274c0ffe70 0x7f274c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 -- 172.21.15.100:0/3086092842 >> 172.21.15.100:0/3086092842 conn(0x7f274c0fb420 msgr2=0x7f274c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 -- 172.21.15.100:0/3086092842 shutdown_connections 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.047+0000 7f2750f9e640 1 -- 172.21.15.100:0/3086092842 wait complete. 2024-03-12T05:09:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.048+0000 7f2750f9e640 1 Processor -- start 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.048+0000 7f2750f9e640 1 -- start start 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.048+0000 7f2750f9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f274c0ffe70 0x7f274c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.048+0000 7f2750f9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 0x7f274c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2750f9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2750f9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19d970 con 0x7f274c101980 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2750f9e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19dae0 con 0x7f274c0ffe70 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2750f9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19dca0 con 0x7f274c100fc0 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54186/0 (socket says 172.21.15.100:54186) 2024-03-12T05:09:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f274c0ffe70 0x7f274c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 -- 172.21.15.100:0/4110194623 learned_addr learned my addr 172.21.15.100:0/4110194623 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274affd640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 0x7f274c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 -- 172.21.15.100:0/4110194623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 msgr2=0x7f274c1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 0x7f274c1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 -- 172.21.15.100:0/4110194623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f274c0ffe70 msgr2=0x7f274c195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f274c0ffe70 0x7f274c195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 -- 172.21.15.100:0/4110194623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744009900 con 0x7f274c101980 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274affd640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 0x7f274c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f274bfff640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c19d430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2744037040 tx=0x7f2744034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744032eb0 con 0x7f274c101980 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c19df20 con 0x7f274c101980 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2744003e50 con 0x7f274c101980 2024-03-12T05:09:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744040450 con 0x7f274c101980 2024-03-12T05:09:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.049+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c19e340 con 0x7f274c101980 2024-03-12T05:09:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.050+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c107120 con 0x7f274c101980 2024-03-12T05:09:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.051+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f274400a040 con 0x7f274c101980 2024-03-12T05:09:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.052+0000 7f2748ff9640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f27340a1a30 0x7f27340a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.052+0000 7f274b7fe640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f27340a1a30 0x7f27340a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.052+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f27440f0080 con 0x7f274c101980 2024-03-12T05:09:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.052+0000 7f274b7fe640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f27340a1a30 0x7f27340a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f273c00bbe0 tx=0x7f273c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:53.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.056+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27440b95c0 con 0x7f274c101980 2024-03-12T05:09:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:52 smithi100 ceph-mon[32210]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/12148320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:52 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1798400093' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-12T05:09:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:52 smithi156 ceph-mon[36270]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/12148320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:52 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:53.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1798400093' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-12T05:09:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:52 smithi161 ceph-mon[36043]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/12148320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:09:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:52 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:09:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1798400093' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-12T05:09:53.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.234+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f274c02cf20 con 0x7f274c101980 2024-03-12T05:09:53.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.924+0000 7f2748ff9640 1 -- 172.21.15.100:0/4110194623 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f27440bcc80 con 0x7f274c101980 2024-03-12T05:09:53.922 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f27340a1a30 msgr2=0x7f27340a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f27340a1a30 0x7f27340a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f273c00bbe0 tx=0x7f273c00b390 comp rx=0 tx=0).stop 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 msgr2=0x7f274c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c19d430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2744037040 tx=0x7f2744034d40 comp rx=0 tx=0).stop 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 shutdown_connections 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f27340a1a30 0x7f27340a3ef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c101980 0x7f274c19d430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f274c100fc0 0x7f274c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 --2- 172.21.15.100:0/4110194623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f274c0ffe70 0x7f274c195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 >> 172.21.15.100:0/4110194623 conn(0x7f274c0fb420 msgr2=0x7f274c1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:53.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 shutdown_connections 2024-03-12T05:09:53.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:53.926+0000 7f2750f9e640 1 -- 172.21.15.100:0/4110194623 wait complete. 2024-03-12T05:09:53.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-12T05:09:54.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 -- 172.21.15.100:0/94177916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c073b60 msgr2=0x7fb00c073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:54.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 --2- 172.21.15.100:0/94177916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c073b60 0x7fb00c073fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb000009900 tx=0x7fb00002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 -- 172.21.15.100:0/94177916 shutdown_connections 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 --2- 172.21.15.100:0/94177916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c109060 0x7fb00c10b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 --2- 172.21.15.100:0/94177916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c073b60 0x7fb00c073fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 --2- 172.21.15.100:0/94177916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb00c075140 0x7fb00c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 -- 172.21.15.100:0/94177916 >> 172.21.15.100:0/94177916 conn(0x7fb00c0fc720 msgr2=0x7fb00c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 -- 172.21.15.100:0/94177916 shutdown_connections 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.053+0000 7fb011015640 1 -- 172.21.15.100:0/94177916 wait complete. 2024-03-12T05:09:54.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 Processor -- start 2024-03-12T05:09:54.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 -- start start 2024-03-12T05:09:54.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb00c073b60 0x7fb00c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:54.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 0x7fb00c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:54.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c109060 0x7fb00c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:54.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a0f80 con 0x7fb00c075140 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a10f0 con 0x7fb00c109060 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb011015640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a12b0 con 0x7fb00c073b60 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb00a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 0x7fb00c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb00a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 0x7fb00c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54202/0 (socket says 172.21.15.100:54202) 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb00a59c640 1 -- 172.21.15.100:0/4261385515 learned_addr learned my addr 172.21.15.100:0/4261385515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb00ad9d640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb00c073b60 0x7fb00c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.054+0000 7fb00b59e640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c109060 0x7fb00c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb00a59c640 1 -- 172.21.15.100:0/4261385515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb00c073b60 msgr2=0x7fb00c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb00a59c640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb00c073b60 0x7fb00c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb00a59c640 1 -- 172.21.15.100:0/4261385515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c109060 msgr2=0x7fb00c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb00a59c640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c109060 0x7fb00c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb00a59c640 1 -- 172.21.15.100:0/4261385515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000009560 con 0x7fb00c075140 2024-03-12T05:09:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb00a59c640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 0x7fb00c1999d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb000006010 tx=0x7fb0000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000044e0 con 0x7fb00c075140 2024-03-12T05:09:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0000054e0 con 0x7fb00c075140 2024-03-12T05:09:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c1a1530 con 0x7fb00c075140 2024-03-12T05:09:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00003e450 con 0x7fb00c075140 2024-03-12T05:09:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.055+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1a1a40 con 0x7fb00c075140 2024-03-12T05:09:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.056+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c107e20 con 0x7fb00c075140 2024-03-12T05:09:54.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.058+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fb0000035f0 con 0x7fb00c075140 2024-03-12T05:09:54.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.058+0000 7faff3fff640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faff40a1a30 0x7faff40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:54.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.058+0000 7fb00ad9d640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faff40a1a30 0x7faff40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:54.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.058+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fb0000f0200 con 0x7fb00c075140 2024-03-12T05:09:54.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.059+0000 7fb00ad9d640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faff40a1a30 0x7faff40a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faffc005fd0 tx=0x7faffc009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:54.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.062+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0000b95c0 con 0x7fb00c075140 2024-03-12T05:09:54.125 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1221378850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1221378850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: osdmap e83: 12 total, 12 up, 12 in 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4110194623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.126 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1221378850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-12T05:09:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1221378850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: osdmap e83: 12 total, 12 up, 12 in 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4110194623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.273+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb00c02ce40 con 0x7fb00c075140 2024-03-12T05:09:54.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1221378850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-12T05:09:54.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1221378850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-12T05:09:54.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: osdmap e83: 12 total, 12 up, 12 in 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4110194623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:09:54.923 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-03-12T05:09:54.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.925+0000 7faff3fff640 1 -- 172.21.15.100:0/4261385515 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb0000bcc80 con 0x7fb00c075140 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faff40a1a30 msgr2=0x7faff40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faff40a1a30 0x7faff40a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faffc005fd0 tx=0x7faffc009760 comp rx=0 tx=0).stop 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 msgr2=0x7fb00c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 0x7fb00c1999d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb000006010 tx=0x7fb0000042f0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 shutdown_connections 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faff40a1a30 0x7faff40a3ef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c109060 0x7fb00c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb00c075140 0x7fb00c1999d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 --2- 172.21.15.100:0/4261385515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb00c073b60 0x7fb00c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.927+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 >> 172.21.15.100:0/4261385515 conn(0x7fb00c0fc720 msgr2=0x7fb00c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.928+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 shutdown_connections 2024-03-12T05:09:54.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:54.928+0000 7fb011015640 1 -- 172.21.15.100:0/4261385515 wait complete. 2024-03-12T05:09:54.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:09:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 -- 172.21.15.100:0/3113875388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810bbb0 msgr2=0x7fdf7810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 --2- 172.21.15.100:0/3113875388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810bbb0 0x7fdf7810bfd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009900 tx=0x7fdf6c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 -- 172.21.15.100:0/3113875388 shutdown_connections 2024-03-12T05:09:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 --2- 172.21.15.100:0/3113875388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810d760 0x7fdf78113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 --2- 172.21.15.100:0/3113875388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7810cda0 0x7fdf7810d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 --2- 172.21.15.100:0/3113875388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810bbb0 0x7fdf7810bfd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 -- 172.21.15.100:0/3113875388 >> 172.21.15.100:0/3113875388 conn(0x7fdf7806bbe0 msgr2=0x7fdf7806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 -- 172.21.15.100:0/3113875388 shutdown_connections 2024-03-12T05:09:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.045+0000 7fdf7f062640 1 -- 172.21.15.100:0/3113875388 wait complete. 2024-03-12T05:09:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 Processor -- start 2024-03-12T05:09:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 -- start start 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7810bbb0 0x7fdf781a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810cda0 0x7fdf781a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 0x7fdf781a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf781a9790 con 0x7fdf7810d760 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf781a9900 con 0x7fdf7810bbb0 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf781a9ac0 con 0x7fdf7810cda0 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 0x7fdf781a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 0x7fdf781a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54214/0 (socket says 172.21.15.100:54214) 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 -- 172.21.15.100:0/2773028827 learned_addr learned my addr 172.21.15.100:0/2773028827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7cdfe640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7810bbb0 0x7fdf781a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf77fff640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810cda0 0x7fdf781a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 -- 172.21.15.100:0/2773028827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810cda0 msgr2=0x7fdf781a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810cda0 0x7fdf781a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 -- 172.21.15.100:0/2773028827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7810bbb0 msgr2=0x7fdf781a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7810bbb0 0x7fdf781a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 -- 172.21.15.100:0/2773028827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c009560 con 0x7fdf7810d760 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf77fff640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810cda0 0x7fdf781a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7d5ff640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 0x7fdf781a9220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000eaa0 tx=0x7fdf7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000cdb0 con 0x7fdf7810d760 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf70005020 con 0x7fdf7810d760 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf781a9da0 con 0x7fdf7810d760 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.046+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700116b0 con 0x7fdf7810d760 2024-03-12T05:09:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.047+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf781aa270 con 0x7fdf7810d760 2024-03-12T05:09:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.048+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf78112ed0 con 0x7fdf7810d760 2024-03-12T05:09:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.049+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fdf70005520 con 0x7fdf7810d760 2024-03-12T05:09:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.049+0000 7fdf75ffb640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf680a1ab0 0x7fdf680a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.049+0000 7fdf7cdfe640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf680a1ab0 0x7fdf680a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.050+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdf700cd440 con 0x7fdf7810d760 2024-03-12T05:09:55.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.050+0000 7fdf7cdfe640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf680a1ab0 0x7fdf680a3f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005ab0 tx=0x7fdf6c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.054+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf700967a0 con 0x7fdf7810d760 2024-03-12T05:09:55.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: pgmap v278: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:55.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4110194623' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-12T05:09:55.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: osdmap e84: 12 total, 12 up, 12 in 2024-03-12T05:09:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4261385515' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-12T05:09:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-12T05:09:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-12T05:09:55.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:54 smithi156 ceph-mon[36270]: osdmap e85: 12 total, 12 up, 12 in 2024-03-12T05:09:55.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: pgmap v278: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:55.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4110194623' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-12T05:09:55.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: osdmap e84: 12 total, 12 up, 12 in 2024-03-12T05:09:55.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4261385515' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-12T05:09:55.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-12T05:09:55.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-12T05:09:55.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:54 smithi161 ceph-mon[36043]: osdmap e85: 12 total, 12 up, 12 in 2024-03-12T05:09:55.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.229+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf78117760 con 0x7fdf7810d760 2024-03-12T05:09:55.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.230+0000 7fdf75ffb640 1 -- 172.21.15.100:0/2773028827 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22262 (secure 0 0 0) 0x7fdf70099e60 con 0x7fdf7810d760 2024-03-12T05:09:55.228 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:55.229 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":85,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:09:54.274405+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf680a1ab0 msgr2=0x7fdf680a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf680a1ab0 0x7fdf680a3f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005ab0 tx=0x7fdf6c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 msgr2=0x7fdf781a9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 0x7fdf781a9220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000eaa0 tx=0x7fdf7000ef60 comp rx=0 tx=0).stop 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 shutdown_connections 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf680a1ab0 0x7fdf680a3f70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf7810d760 0x7fdf781a9220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf7810cda0 0x7fdf781a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.232+0000 7fdf7f062640 1 --2- 172.21.15.100:0/2773028827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7810bbb0 0x7fdf781a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.233+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 >> 172.21.15.100:0/2773028827 conn(0x7fdf7806bbe0 msgr2=0x7fdf781100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.233+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 shutdown_connections 2024-03-12T05:09:55.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.233+0000 7fdf7f062640 1 -- 172.21.15.100:0/2773028827 wait complete. 2024-03-12T05:09:55.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: pgmap v278: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:09:55.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4110194623' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-12T05:09:55.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: osdmap e84: 12 total, 12 up, 12 in 2024-03-12T05:09:55.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4261385515' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-12T05:09:55.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-12T05:09:55.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-12T05:09:55.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:54 smithi100 ceph-mon[32210]: osdmap e85: 12 total, 12 up, 12 in 2024-03-12T05:09:55.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 -- 172.21.15.100:0/722349992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904101c20 msgr2=0x7f7904104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 --2- 172.21.15.100:0/722349992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904101c20 0x7f7904104010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009900 tx=0x7f78f802bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 -- 172.21.15.100:0/722349992 shutdown_connections 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 --2- 172.21.15.100:0/722349992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904106e80 0x7f79041092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 --2- 172.21.15.100:0/722349992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904104550 0x7f7904106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 --2- 172.21.15.100:0/722349992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904101c20 0x7f7904104010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.345+0000 7f790ad48640 1 -- 172.21.15.100:0/722349992 >> 172.21.15.100:0/722349992 conn(0x7f79040fb610 msgr2=0x7f79040fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 -- 172.21.15.100:0/722349992 shutdown_connections 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 -- 172.21.15.100:0/722349992 wait complete. 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 Processor -- start 2024-03-12T05:09:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 -- start start 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101c20 0x7f79041972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 0x7f79041977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904106e80 0x7f790419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419ed90 con 0x7f7904104550 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419ef00 con 0x7f7904101c20 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f790ad48640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419f0c0 con 0x7f7904106e80 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f7903fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 0x7f79041977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f7903fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 0x7f79041977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54226/0 (socket says 172.21.15.100:54226) 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f7903fff640 1 -- 172.21.15.100:0/3598604844 learned_addr learned my addr 172.21.15.100:0/3598604844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.346+0000 7f7908ae4640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101c20 0x7f79041972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7903fff640 1 -- 172.21.15.100:0/3598604844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904106e80 msgr2=0x7f790419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7903fff640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904106e80 0x7f790419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7903fff640 1 -- 172.21.15.100:0/3598604844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101c20 msgr2=0x7f79041972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7903fff640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101c20 0x7f79041972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7903fff640 1 -- 172.21.15.100:0/3598604844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8009560 con 0x7f7904104550 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7908ae4640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101c20 0x7f79041972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7903fff640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 0x7f79041977e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f78f000d740 tx=0x7f78f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0009c30 con 0x7f7904104550 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790419f3a0 con 0x7f7904104550 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78f000de50 con 0x7f7904104550 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0010460 con 0x7f7904104550 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.347+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790419f8b0 con 0x7f7904104550 2024-03-12T05:09:55.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.348+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79040fd100 con 0x7f7904104550 2024-03-12T05:09:55.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.349+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f78f0004f10 con 0x7f7904104550 2024-03-12T05:09:55.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.350+0000 7f7901ffb640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78f40a1a30 0x7f78f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.350+0000 7f7908ae4640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78f40a1a30 0x7f78f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.350+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78f00ccd10 con 0x7f7904104550 2024-03-12T05:09:55.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.350+0000 7f7908ae4640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78f40a1a30 0x7f78f40a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000c00 tx=0x7f78f8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.354+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78f0096070 con 0x7f7904104550 2024-03-12T05:09:55.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.529+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f790410cd60 con 0x7f7904104550 2024-03-12T05:09:55.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.531+0000 7f7901ffb640 1 -- 172.21.15.100:0/3598604844 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22262 (secure 0 0 0) 0x7f78f0017090 con 0x7f7904104550 2024-03-12T05:09:55.529 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:09:55.529 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":85,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:09:54.274405+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:09:55.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78f40a1a30 msgr2=0x7f78f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78f40a1a30 0x7f78f40a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000c00 tx=0x7f78f8005be0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 msgr2=0x7f79041977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 0x7f79041977e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f78f000d740 tx=0x7f78f000dc00 comp rx=0 tx=0).stop 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 shutdown_connections 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f78f40a1a30 0x7f78f40a3ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904106e80 0x7f790419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7904104550 0x7f79041977e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 --2- 172.21.15.100:0/3598604844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7904101c20 0x7f79041972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 >> 172.21.15.100:0/3598604844 conn(0x7f79040fb610 msgr2=0x7f7904102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.533+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 shutdown_connections 2024-03-12T05:09:55.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.534+0000 7f790ad48640 1 -- 172.21.15.100:0/3598604844 wait complete. 2024-03-12T05:09:55.542 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 -- 172.21.15.100:0/3037452102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f01013f0 msgr2=0x7fe1f0107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 --2- 172.21.15.100:0/3037452102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f01013f0 0x7fe1f0107c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800adb0 tx=0x7fe1e802cff0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 -- 172.21.15.100:0/3037452102 shutdown_connections 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 --2- 172.21.15.100:0/3037452102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f01013f0 0x7fe1f0107c80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 --2- 172.21.15.100:0/3037452102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f0100a30 0x7fe1f0100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 --2- 172.21.15.100:0/3037452102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1f00ff840 0x7fe1f00ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 -- 172.21.15.100:0/3037452102 >> 172.21.15.100:0/3037452102 conn(0x7fe1f00fadb0 msgr2=0x7fe1f00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 -- 172.21.15.100:0/3037452102 shutdown_connections 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 -- 172.21.15.100:0/3037452102 wait complete. 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.649+0000 7fe1f833e640 1 Processor -- start 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 -- start start 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1f00ff840 0x7fe1f01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 0x7fe1f0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f01013f0 0x7fe1f01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01a0ee0 con 0x7fe1f0100a30 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01a1050 con 0x7fe1f00ff840 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01a1210 con 0x7fe1f01013f0 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 0x7fe1f0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 0x7fe1f0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54242/0 (socket says 172.21.15.100:54242) 2024-03-12T05:09:55.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 -- 172.21.15.100:0/1920824653 learned_addr learned my addr 172.21.15.100:0/1920824653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f68db640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f01013f0 0x7fe1f01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 -- 172.21.15.100:0/1920824653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f01013f0 msgr2=0x7fe1f01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f01013f0 0x7fe1f01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 -- 172.21.15.100:0/1920824653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1f00ff840 msgr2=0x7fe1f01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f60da640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1f00ff840 0x7fe1f01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1f00ff840 0x7fe1f01993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 -- 172.21.15.100:0/1920824653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e800aa10 con 0x7fe1f0100a30 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f68db640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f01013f0 0x7fe1f01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f58d9640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 0x7fe1f0199930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00d950 tx=0x7fe1ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec009c30 con 0x7fe1f0100a30 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1ec009d90 con 0x7fe1f0100a30 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f01a14f0 con 0x7fe1f0100a30 2024-03-12T05:09:55.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00f7e0 con 0x7fe1f0100a30 2024-03-12T05:09:55.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.650+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f01a1a50 con 0x7fe1f0100a30 2024-03-12T05:09:55.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.651+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f0106a90 con 0x7fe1f0100a30 2024-03-12T05:09:55.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.652+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe1ec010ad0 con 0x7fe1f0100a30 2024-03-12T05:09:55.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.653+0000 7fe1e77fe640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1dc0a1ab0 0x7fe1dc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:55.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.653+0000 7fe1f60da640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1dc0a1ab0 0x7fe1dc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:55.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.653+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe1ec0cced0 con 0x7fe1f0100a30 2024-03-12T05:09:55.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.653+0000 7fe1f60da640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1dc0a1ab0 0x7fe1dc0a3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0005fd0 tx=0x7fe1e0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:55.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.657+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1ec096230 con 0x7fe1f0100a30 2024-03-12T05:09:55.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:55.875+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe1f002cda0 con 0x7fe1f0100a30 2024-03-12T05:09:56.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4261385515' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-12T05:09:56.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2773028827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3598604844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: osdmap e86: 12 total, 12 up, 12 in 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] up:active 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: pgmap v282: 97 pgs: 15 creating+peering, 57 unknown, 25 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-03-12T05:09:56.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1920824653' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-12T05:09:56.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4261385515' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-12T05:09:56.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2773028827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3598604844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: osdmap e86: 12 total, 12 up, 12 in 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] up:active 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: pgmap v282: 97 pgs: 15 creating+peering, 57 unknown, 25 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-03-12T05:09:56.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1920824653' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-12T05:09:56.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4261385515' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2773028827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3598604844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: osdmap e86: 12 total, 12 up, 12 in 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] up:active 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: pgmap v282: 97 pgs: 15 creating+peering, 57 unknown, 25 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-03-12T05:09:56.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1920824653' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-12T05:09:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.704+0000 7fe1e77fe640 1 -- 172.21.15.100:0/1920824653 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe1ec0170b0 con 0x7fe1f0100a30 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1dc0a1ab0 msgr2=0x7fe1dc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1dc0a1ab0 0x7fe1dc0a3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0005fd0 tx=0x7fe1e0005c70 comp rx=0 tx=0).stop 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 msgr2=0x7fe1f0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 0x7fe1f0199930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00d950 tx=0x7fe1ec00de10 comp rx=0 tx=0).stop 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 shutdown_connections 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1dc0a1ab0 0x7fe1dc0a3f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1f01013f0 0x7fe1f01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0100a30 0x7fe1f0199930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 --2- 172.21.15.100:0/1920824653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1f00ff840 0x7fe1f01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 >> 172.21.15.100:0/1920824653 conn(0x7fe1f00fadb0 msgr2=0x7fe1f0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 shutdown_connections 2024-03-12T05:09:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.707+0000 7fe1f833e640 1 -- 172.21.15.100:0/1920824653 wait complete. 2024-03-12T05:09:56.718 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-12T05:09:56.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-12T05:09:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 -- 172.21.15.100:0/3172459925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c100090 msgr2=0x7f9c7c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3172459925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c100090 0x7f9c7c1004b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009900 tx=0x7f9c6c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 -- 172.21.15.100:0/3172459925 shutdown_connections 2024-03-12T05:09:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3172459925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c7c101c40 0x7f9c7c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3172459925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c7c101280 0x7f9c7c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3172459925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c100090 0x7f9c7c1004b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 -- 172.21.15.100:0/3172459925 >> 172.21.15.100:0/3172459925 conn(0x7f9c7c0fb620 msgr2=0x7f9c7c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 -- 172.21.15.100:0/3172459925 shutdown_connections 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.831+0000 7f9c81c88640 1 -- 172.21.15.100:0/3172459925 wait complete. 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 Processor -- start 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 -- start start 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c7c100090 0x7f9c7c06a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c7c101280 0x7f9c7c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 0x7f9c7c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19ed00 con 0x7f9c7c101c40 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19ee70 con 0x7f9c7c101280 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c81c88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19f030 con 0x7f9c7c100090 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 0x7f9c7c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:56.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 0x7f9c7c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54262/0 (socket says 172.21.15.100:54262) 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 -- 172.21.15.100:0/3202367766 learned_addr learned my addr 172.21.15.100:0/3202367766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7affd640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c7c101280 0x7f9c7c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7b7fe640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c7c100090 0x7f9c7c06a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 -- 172.21.15.100:0/3202367766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c7c100090 msgr2=0x7f9c7c06a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c7c100090 0x7f9c7c06a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 -- 172.21.15.100:0/3202367766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c7c101280 msgr2=0x7f9c7c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c7c101280 0x7f9c7c19bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.832+0000 7f9c7bfff640 1 -- 172.21.15.100:0/3202367766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c009560 con 0x7f9c7c101c40 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.833+0000 7f9c7bfff640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 0x7f9c7c19e570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400cce0 tx=0x7f9c7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.833+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400f450 con 0x7f9c7c101c40 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.833+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7c19f310 con 0x7f9c7c101c40 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.833+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c7400fa90 con 0x7f9c7c101c40 2024-03-12T05:09:56.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.833+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74018b90 con 0x7f9c7c101c40 2024-03-12T05:09:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.833+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7c19f7d0 con 0x7f9c7c101c40 2024-03-12T05:09:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.834+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7c1073d0 con 0x7f9c7c101c40 2024-03-12T05:09:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.835+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9c7400fc00 con 0x7f9c7c101c40 2024-03-12T05:09:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.835+0000 7f9c78ff9640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9c640a1a30 0x7f9c640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.836+0000 7f9c7b7fe640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9c640a1a30 0x7f9c640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.836+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c740cda90 con 0x7f9c7c101c40 2024-03-12T05:09:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.836+0000 7f9c7b7fe640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9c640a1a30 0x7f9c640a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c006010 tx=0x7f9c6c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:56.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:56.840+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c74096df0 con 0x7f9c7c101c40 2024-03-12T05:09:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.053+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f9c7c02ce40 con 0x7f9c7c101c40 2024-03-12T05:09:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:56 smithi100 ceph-mon[32210]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-12T05:09:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:56 smithi100 ceph-mon[32210]: Cluster is now healthy 2024-03-12T05:09:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1920824653' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-12T05:09:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:56 smithi100 ceph-mon[32210]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:57.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:56 smithi156 ceph-mon[36270]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-12T05:09:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:56 smithi156 ceph-mon[36270]: Cluster is now healthy 2024-03-12T05:09:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1920824653' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-12T05:09:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:56 smithi156 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:56 smithi161 ceph-mon[36043]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-12T05:09:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:56 smithi161 ceph-mon[36043]: Cluster is now healthy 2024-03-12T05:09:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1920824653' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-12T05:09:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:56 smithi161 ceph-mon[36043]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.936+0000 7f9c78ff9640 1 -- 172.21.15.100:0/3202367766 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7f9c7409a4b0 con 0x7f9c7c101c40 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9c640a1a30 msgr2=0x7f9c640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9c640a1a30 0x7f9c640a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c006010 tx=0x7f9c6c000bc0 comp rx=0 tx=0).stop 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 msgr2=0x7f9c7c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 0x7f9c7c19e570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400cce0 tx=0x7f9c7400a620 comp rx=0 tx=0).stop 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 shutdown_connections 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9c640a1a30 0x7f9c640a3ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c7c101c40 0x7f9c7c19e570 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c7c101280 0x7f9c7c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 --2- 172.21.15.100:0/3202367766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c7c100090 0x7f9c7c06a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 >> 172.21.15.100:0/3202367766 conn(0x7f9c7c0fb620 msgr2=0x7f9c7c1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 shutdown_connections 2024-03-12T05:09:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:57.939+0000 7f9c81c88640 1 -- 172.21.15.100:0/3202367766 wait complete. 2024-03-12T05:09:57.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 -- 172.21.15.100:0/3196230933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c100090 msgr2=0x7f473c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- 172.21.15.100:0/3196230933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c100090 0x7f473c1004b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f472c009900 tx=0x7f472c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 -- 172.21.15.100:0/3196230933 shutdown_connections 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- 172.21.15.100:0/3196230933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c101c40 0x7f473c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- 172.21.15.100:0/3196230933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f473c101280 0x7f473c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- 172.21.15.100:0/3196230933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c100090 0x7f473c1004b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 -- 172.21.15.100:0/3196230933 >> 172.21.15.100:0/3196230933 conn(0x7f473c0fb620 msgr2=0x7f473c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 -- 172.21.15.100:0/3196230933 shutdown_connections 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 -- 172.21.15.100:0/3196230933 wait complete. 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 Processor -- start 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 -- start start 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f473c100090 0x7f473c06a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c101280 0x7f473c19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.058+0000 7f47433f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 0x7f473c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47433f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19ee70 con 0x7f473c101c40 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47433f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19efe0 con 0x7f473c100090 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47433f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19f1a0 con 0x7f473c101280 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 0x7f473c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 0x7f473c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54282/0 (socket says 172.21.15.100:54282) 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 -- 172.21.15.100:0/3591410197 learned_addr learned my addr 172.21.15.100:0/3591410197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f474098f640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c101280 0x7f473c19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741190640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f473c100090 0x7f473c06a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 -- 172.21.15.100:0/3591410197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c101280 msgr2=0x7f473c19c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c101280 0x7f473c19c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 -- 172.21.15.100:0/3591410197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f473c100090 msgr2=0x7f473c06a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f473c100090 0x7f473c06a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 -- 172.21.15.100:0/3591410197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c009560 con 0x7f473c101c40 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f4741991640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 0x7f473c19e690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4734009fd0 tx=0x7f473400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400bdd0 con 0x7f473c101c40 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4734009940 con 0x7f473c101c40 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c19f420 con 0x7f473c101c40 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340106c0 con 0x7f473c101c40 2024-03-12T05:09:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.059+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c19f8e0 con 0x7f473c101c40 2024-03-12T05:09:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.060+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c0683c0 con 0x7f473c101c40 2024-03-12T05:09:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.061+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4734004190 con 0x7f473c101c40 2024-03-12T05:09:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.062+0000 7f47327fc640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47280a1a30 0x7f47280a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.062+0000 7f4741190640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47280a1a30 0x7f47280a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.062+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f47340cd080 con 0x7f473c101c40 2024-03-12T05:09:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.062+0000 7f4741190640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47280a1a30 0x7f47280a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f472c005c40 tx=0x7f472c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.066+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47340963e0 con 0x7f473c101c40 2024-03-12T05:09:58.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:57 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3202367766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-12T05:09:58.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:57 smithi156 ceph-mon[36270]: pgmap v283: 97 pgs: 15 creating+peering, 8 unknown, 74 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-03-12T05:09:58.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:57 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3202367766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-12T05:09:58.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:57 smithi161 ceph-mon[36043]: pgmap v283: 97 pgs: 15 creating+peering, 8 unknown, 74 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-03-12T05:09:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.278+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f473c107430 con 0x7f473c101c40 2024-03-12T05:09:58.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3202367766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-12T05:09:58.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:57 smithi100 ceph-mon[32210]: pgmap v283: 97 pgs: 15 creating+peering, 8 unknown, 74 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-03-12T05:09:58.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.944+0000 7f47327fc640 1 -- 172.21.15.100:0/3591410197 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4734099aa0 con 0x7f473c101c40 2024-03-12T05:09:58.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.946+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47280a1a30 msgr2=0x7f47280a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.946+0000 7f47433f4640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47280a1a30 0x7f47280a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f472c005c40 tx=0x7f472c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.946+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 msgr2=0x7f473c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.946+0000 7f47433f4640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 0x7f473c19e690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4734009fd0 tx=0x7f473400def0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 shutdown_connections 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f47280a1a30 0x7f47280a3ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c101c40 0x7f473c19e690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c101280 0x7f473c19c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 --2- 172.21.15.100:0/3591410197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f473c100090 0x7f473c06a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 >> 172.21.15.100:0/3591410197 conn(0x7f473c0fb620 msgr2=0x7f473c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 shutdown_connections 2024-03-12T05:09:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:58.947+0000 7f47433f4640 1 -- 172.21.15.100:0/3591410197 wait complete. 2024-03-12T05:09:58.960 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-12T05:09:59.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 -- 172.21.15.100:0/135025400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101c50 msgr2=0x7fcdc81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 --2- 172.21.15.100:0/135025400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101c50 0x7fcdc81084e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009ca0 tx=0x7fcdc002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 -- 172.21.15.100:0/135025400 shutdown_connections 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 --2- 172.21.15.100:0/135025400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101c50 0x7fcdc81084e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 --2- 172.21.15.100:0/135025400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc8101290 0x7fcdc8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 --2- 172.21.15.100:0/135025400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc81000a0 0x7fcdc81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 -- 172.21.15.100:0/135025400 >> 172.21.15.100:0/135025400 conn(0x7fcdc80fb630 msgr2=0x7fcdc80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 -- 172.21.15.100:0/135025400 shutdown_connections 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.070+0000 7fcdccc55640 1 -- 172.21.15.100:0/135025400 wait complete. 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 Processor -- start 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 -- start start 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc81000a0 0x7fcdc81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:59.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 0x7fcdc8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc8101c50 0x7fcdc819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc819edb0 con 0x7fcdc8101290 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc819ef20 con 0x7fcdc81000a0 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdccc55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc819f0e0 con 0x7fcdc8101c50 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.071+0000 7fcdbdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 0x7fcdc8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 0x7fcdc8197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54308/0 (socket says 172.21.15.100:54308) 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 -- 172.21.15.100:0/1167303352 learned_addr learned my addr 172.21.15.100:0/1167303352 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdc659c640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc81000a0 0x7fcdc81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdc6d9d640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc8101c50 0x7fcdc819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 -- 172.21.15.100:0/1167303352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc8101c50 msgr2=0x7fcdc819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc8101c50 0x7fcdc819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 -- 172.21.15.100:0/1167303352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc81000a0 msgr2=0x7fcdc81972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:59.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc81000a0 0x7fcdc81972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 -- 172.21.15.100:0/1167303352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc0009900 con 0x7fcdc8101290 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdc6d9d640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc8101c50 0x7fcdc819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdc659c640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc81000a0 0x7fcdc81972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbdd9b640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 0x7fcdc8197800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b4a0 tx=0x7fcdb000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0010610 con 0x7fcdc8101290 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0010c50 con 0x7fcdc8101290 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc819f3c0 con 0x7fcdc8101290 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0017e20 con 0x7fcdc8101290 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.072+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc819f950 con 0x7fcdc8101290 2024-03-12T05:09:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.073+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc810bb90 con 0x7fcdc8101290 2024-03-12T05:09:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.074+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fcdb0017480 con 0x7fcdc8101290 2024-03-12T05:09:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.075+0000 7fcdbffff640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcdb40a1a30 0x7fcdb40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:09:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.075+0000 7fcdc659c640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcdb40a1a30 0x7fcdb40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:09:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.075+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcdb00cc9c0 con 0x7fcdc8101290 2024-03-12T05:09:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.075+0000 7fcdc659c640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcdb40a1a30 0x7fcdb40a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcdc81010f0 tx=0x7fcdb8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:09:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.079+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdb0095d20 con 0x7fcdc8101290 2024-03-12T05:09:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-12T05:09:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3202367766' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-12T05:09:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-12T05:09:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-12T05:09:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-12T05:09:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:58 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3591410197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-12T05:09:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-12T05:09:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3202367766' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-12T05:09:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-12T05:09:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:59.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-12T05:09:59.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-12T05:09:59.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:58 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3591410197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-12T05:09:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.289+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fcdc810bd20 con 0x7fcdc8101290 2024-03-12T05:09:59.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-12T05:09:59.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3202367766' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-12T05:09:59.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-12T05:09:59.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-12T05:09:59.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-12T05:09:59.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-12T05:09:59.322 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:58 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3591410197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-12T05:09:59.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.957+0000 7fcdbffff640 1 -- 172.21.15.100:0/1167303352 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcdb00993e0 con 0x7fcdc8101290 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcdb40a1a30 msgr2=0x7fcdb40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcdb40a1a30 0x7fcdb40a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcdc81010f0 tx=0x7fcdb8008040 comp rx=0 tx=0).stop 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 msgr2=0x7fcdc8197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 0x7fcdc8197800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b4a0 tx=0x7fcdb000b960 comp rx=0 tx=0).stop 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 shutdown_connections 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fcdb40a1a30 0x7fcdb40a3ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdc8101c50 0x7fcdc819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8101290 0x7fcdc8197800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 --2- 172.21.15.100:0/1167303352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc81000a0 0x7fcdc81972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 >> 172.21.15.100:0/1167303352 conn(0x7fcdc80fb630 msgr2=0x7fcdc8069790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:09:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 shutdown_connections 2024-03-12T05:09:59.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:09:59.960+0000 7fcdccc55640 1 -- 172.21.15.100:0/1167303352 wait complete. 2024-03-12T05:09:59.971 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-12T05:09:59.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-12T05:10:00.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 -- 172.21.15.100:0/1977085881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288101c00 msgr2=0x7fe288103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1977085881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288101c00 0x7fe288103ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe284009900 tx=0x7fe28402bc30 comp rx=0 tx=0).stop 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 -- 172.21.15.100:0/1977085881 shutdown_connections 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1977085881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe288106e60 0x7fe2881092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1977085881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe288104530 0x7fe288106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1977085881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288101c00 0x7fe288103ff0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 -- 172.21.15.100:0/1977085881 >> 172.21.15.100:0/1977085881 conn(0x7fe2880fb610 msgr2=0x7fe2880fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 -- 172.21.15.100:0/1977085881 shutdown_connections 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.083+0000 7fe28f86f640 1 -- 172.21.15.100:0/1977085881 wait complete. 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 Processor -- start 2024-03-12T05:10:00.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 -- start start 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe288101c00 0x7fe2881971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 0x7fe2881976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe288106e60 0x7fe28819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28819ec90 con 0x7fe288104530 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28819ee00 con 0x7fe288101c00 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28f86f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28819efc0 con 0x7fe288106e60 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 0x7fe2881976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 0x7fe2881976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54330/0 (socket says 172.21.15.100:54330) 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 -- 172.21.15.100:0/1329941721 learned_addr learned my addr 172.21.15.100:0/1329941721 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 -- 172.21.15.100:0/1329941721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe288106e60 msgr2=0x7fe28819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe288106e60 0x7fe28819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 -- 172.21.15.100:0/1329941721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe288101c00 msgr2=0x7fe2881971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28d60b640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe288101c00 0x7fe2881971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe288101c00 0x7fe2881971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.084+0000 7fe28ce0a640 1 -- 172.21.15.100:0/1329941721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe284009560 con 0x7fe288104530 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.085+0000 7fe28ce0a640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 0x7fe2881976e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe27800c970 tx=0x7fe27800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.085+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278004440 con 0x7fe288104530 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.085+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28819f2a0 con 0x7fe288104530 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.085+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe278007d40 con 0x7fe288104530 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.085+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27800f450 con 0x7fe288104530 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.085+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28819f7b0 con 0x7fe288104530 2024-03-12T05:10:00.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.086+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2880fd100 con 0x7fe288104530 2024-03-12T05:10:00.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.087+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe278007860 con 0x7fe288104530 2024-03-12T05:10:00.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.087+0000 7fe27e7fc640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe2740a1a30 0x7fe2740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.087+0000 7fe28d60b640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe2740a1a30 0x7fe2740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.087+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2780cd4b0 con 0x7fe288104530 2024-03-12T05:10:00.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.088+0000 7fe28d60b640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe2740a1a30 0x7fe2740a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe28402c6a0 tx=0x7fe284005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:00.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.092+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe278096810 con 0x7fe288104530 2024-03-12T05:10:00.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3591410197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-12T05:10:00.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] up:active 2024-03-12T05:10:00.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: Cluster is now healthy 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1167303352' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-mon[36270]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: 2024-03-12T05:09:59.993+0000 7f5160104700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-12T05:10:00.206 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: 2024-03-12T05:09:59.993+0000 7f5160104700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-12T05:10:00.207 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:09:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: 2024-03-12T05:09:59.993+0000 7f5160104700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-12T05:10:00.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3591410197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-12T05:10:00.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] up:active 2024-03-12T05:10:00.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-12T05:10:00.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-12T05:10:00.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: Cluster is now healthy 2024-03-12T05:10:00.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-12T05:10:00.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-12T05:10:00.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-12T05:10:00.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1167303352' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-12T05:10:00.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:09:59 smithi161 ceph-mon[36043]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2024-03-12T05:10:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.289+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fe28802cf50 con 0x7fe2740a1a30 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3591410197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] up:active 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: Cluster is now healthy 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-12T05:10:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-12T05:10:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-12T05:10:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1167303352' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-12T05:10:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:09:59 smithi100 ceph-mon[32210]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2024-03-12T05:10:00.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.317+0000 7fe27e7fc640 1 -- 172.21.15.100:0/1329941721 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe28802cf50 con 0x7fe2740a1a30 2024-03-12T05:10:00.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe2740a1a30 msgr2=0x7fe2740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe2740a1a30 0x7fe2740a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe28402c6a0 tx=0x7fe284005fd0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 msgr2=0x7fe2881976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 0x7fe2881976e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe27800c970 tx=0x7fe27800ce30 comp rx=0 tx=0).stop 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 shutdown_connections 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe2740a1a30 0x7fe2740a3ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe288106e60 0x7fe28819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe288104530 0x7fe2881976e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 --2- 172.21.15.100:0/1329941721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe288101c00 0x7fe2881971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 >> 172.21.15.100:0/1329941721 conn(0x7fe2880fb610 msgr2=0x7fe288102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 shutdown_connections 2024-03-12T05:10:00.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.319+0000 7fe28f86f640 1 -- 172.21.15.100:0/1329941721 wait complete. 2024-03-12T05:10:00.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-12T05:10:00.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 -- 172.21.15.100:0/3033435399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c01045b0 msgr2=0x7f01c01069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 --2- 172.21.15.100:0/3033435399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c01045b0 0x7f01c01069a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009900 tx=0x7f01b402bc30 comp rx=0 tx=0).stop 2024-03-12T05:10:00.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 -- 172.21.15.100:0/3033435399 shutdown_connections 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 --2- 172.21.15.100:0/3033435399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c0106ee0 0x7f01c01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 --2- 172.21.15.100:0/3033435399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c01045b0 0x7f01c01069a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 --2- 172.21.15.100:0/3033435399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c0101c80 0x7f01c0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 -- 172.21.15.100:0/3033435399 >> 172.21.15.100:0/3033435399 conn(0x7f01c00fb630 msgr2=0x7f01c00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 -- 172.21.15.100:0/3033435399 shutdown_connections 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.437+0000 7f01c510d640 1 -- 172.21.15.100:0/3033435399 wait complete. 2024-03-12T05:10:00.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 Processor -- start 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 -- start start 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 0x7f01c0197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c01045b0 0x7f01c0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c0106ee0 0x7f01c019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019ee00 con 0x7f01c0101c80 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019ef70 con 0x7f01c0106ee0 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01c510d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019f130 con 0x7f01c01045b0 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 0x7f01c0197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 0x7f01c0197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54374/0 (socket says 172.21.15.100:54374) 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01bed9d640 1 -- 172.21.15.100:0/1042779927 learned_addr learned my addr 172.21.15.100:0/1042779927 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01be59c640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c01045b0 0x7f01c0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.438+0000 7f01bf59e640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c0106ee0 0x7f01c019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01bed9d640 1 -- 172.21.15.100:0/1042779927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c01045b0 msgr2=0x7f01c0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01bed9d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c01045b0 0x7f01c0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01bed9d640 1 -- 172.21.15.100:0/1042779927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c0106ee0 msgr2=0x7f01c019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01bed9d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c0106ee0 0x7f01c019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01bed9d640 1 -- 172.21.15.100:0/1042779927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4009560 con 0x7f01c0101c80 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01bed9d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 0x7f01c0197310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b4a0 tx=0x7f01b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0010610 con 0x7f01c0101c80 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c019f410 con 0x7f01c0101c80 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01b0010c50 con 0x7f01c0101c80 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0017e20 con 0x7f01c0101c80 2024-03-12T05:10:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.439+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c019f970 con 0x7f01c0101c80 2024-03-12T05:10:00.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.440+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c00fd120 con 0x7f01c0101c80 2024-03-12T05:10:00.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.441+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f01b0017480 con 0x7f01c0101c80 2024-03-12T05:10:00.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.442+0000 7f01a7fff640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f01a80a1a30 0x7f01a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.442+0000 7f01be59c640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f01a80a1a30 0x7f01a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.442+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f01b00cccd0 con 0x7f01c0101c80 2024-03-12T05:10:00.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.442+0000 7f01be59c640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f01a80a1a30 0x7f01a80a3ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005ab0 tx=0x7f01b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.446+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01b0096030 con 0x7f01c0101c80 2024-03-12T05:10:00.568 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:10:00.294+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:10:00.569 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:10:00.294+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:10:00.569 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:10:00.294+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:10:00.569 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:10:00.294+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:10:00.569 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:10:00.294+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:10:00.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.645+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f01c002ce40 con 0x7f01a80a1a30 2024-03-12T05:10:00.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.668+0000 7f01a7fff640 1 -- 172.21.15.100:0/1042779927 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f01c002ce40 con 0x7f01a80a1a30 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f01a80a1a30 msgr2=0x7f01a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f01a80a1a30 0x7f01a80a3ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005ab0 tx=0x7f01b4005fd0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 msgr2=0x7f01c0197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 0x7f01c0197310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b4a0 tx=0x7f01b000b960 comp rx=0 tx=0).stop 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 shutdown_connections 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f01a80a1a30 0x7f01a80a3ef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c0106ee0 0x7f01c019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c01045b0 0x7f01c0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 --2- 172.21.15.100:0/1042779927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c0101c80 0x7f01c0197310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 >> 172.21.15.100:0/1042779927 conn(0x7f01c00fb630 msgr2=0x7f01c0102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 shutdown_connections 2024-03-12T05:10:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.670+0000 7f01c510d640 1 -- 172.21.15.100:0/1042779927 wait complete. 2024-03-12T05:10:00.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-12T05:10:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.787+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2089741493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 msgr2=0x7fbe20102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.787+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2089741493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20102800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009900 tx=0x7fbe1002bc30 comp rx=0 tx=0).stop 2024-03-12T05:10:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2089741493 shutdown_connections 2024-03-12T05:10:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2089741493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20102d40 0x7fbe201095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2089741493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20102800 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2089741493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe20101190 0x7fbe201015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2089741493 >> 172.21.15.100:0/2089741493 conn(0x7fbe200fc720 msgr2=0x7fbe200feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2089741493 shutdown_connections 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2089741493 wait complete. 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 Processor -- start 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- start start 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20101190 0x7fbe20197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe20102d40 0x7fbe2019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019ed20 con 0x7fbe20102380 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019ee90 con 0x7fbe20102d40 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe27cf1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019f050 con 0x7fbe20101190 2024-03-12T05:10:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe2528c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe2528c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35972/0 (socket says 172.21.15.100:35972) 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.788+0000 7fbe2528c640 1 -- 172.21.15.100:0/2447451473 learned_addr learned my addr 172.21.15.100:0/2447451473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe25a8d640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20101190 0x7fbe20197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe2528c640 1 -- 172.21.15.100:0/2447451473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20101190 msgr2=0x7fbe20197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe2528c640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20101190 0x7fbe20197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe2528c640 1 -- 172.21.15.100:0/2447451473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe20102d40 msgr2=0x7fbe2019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe2528c640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe20102d40 0x7fbe2019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe2528c640 1 -- 172.21.15.100:0/2447451473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe10009560 con 0x7fbe20102380 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe2528c640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20197770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe10006010 tx=0x7fbe100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe100044e0 con 0x7fbe20102380 2024-03-12T05:10:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe100054e0 con 0x7fbe20102380 2024-03-12T05:10:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2019f2d0 con 0x7fbe20102380 2024-03-12T05:10:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1003e450 con 0x7fbe20102380 2024-03-12T05:10:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.789+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2019f7e0 con 0x7fbe20102380 2024-03-12T05:10:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.790+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe201085a0 con 0x7fbe20102380 2024-03-12T05:10:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.791+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbe100035f0 con 0x7fbe20102380 2024-03-12T05:10:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.792+0000 7fbe16ffd640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe0c0a1a30 0x7fbe0c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.792+0000 7fbe25a8d640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe0c0a1a30 0x7fbe0c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.792+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbe100ef690 con 0x7fbe20102380 2024-03-12T05:10:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.792+0000 7fbe25a8d640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe0c0a1a30 0x7fbe0c0a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0058c0 tx=0x7fbe1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:00.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:00.796+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe100b89f0 con 0x7fbe20102380 2024-03-12T05:10:00.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.001+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe2002cee0 con 0x7fbe0c0a1a30 2024-03-12T05:10:01.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.005+0000 7fbe16ffd640 1 -- 172.21.15.100:0/2447451473 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fbe2002cee0 con 0x7fbe0c0a1a30 2024-03-12T05:10:01.003 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/qa/sv_0/0d7eb361-251f-4a1e-b75b-2ac216476aa0 2024-03-12T05:10:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe0c0a1a30 msgr2=0x7fbe0c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe0c0a1a30 0x7fbe0c0a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0058c0 tx=0x7fbe1c00a2d0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 msgr2=0x7fbe20197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20197770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe10006010 tx=0x7fbe100042f0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 shutdown_connections 2024-03-12T05:10:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbe0c0a1a30 0x7fbe0c0a3ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe20102d40 0x7fbe2019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20102380 0x7fbe20197770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 --2- 172.21.15.100:0/2447451473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20101190 0x7fbe20197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 >> 172.21.15.100:0/2447451473 conn(0x7fbe200fc720 msgr2=0x7fbe20105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:01.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 shutdown_connections 2024-03-12T05:10:01.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.007+0000 7fbe27cf1640 1 -- 172.21.15.100:0/2447451473 wait complete. 2024-03-12T05:10:01.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-12T05:10:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 -- 172.21.15.100:0/2067683766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101270 msgr2=0x7fbcdc1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 --2- 172.21.15.100:0/2067683766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101270 0x7fbcdc1016f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009900 tx=0x7fbcd002bc30 comp rx=0 tx=0).stop 2024-03-12T05:10:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 -- 172.21.15.100:0/2067683766 shutdown_connections 2024-03-12T05:10:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 --2- 172.21.15.100:0/2067683766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc101c30 0x7fbcdc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 --2- 172.21.15.100:0/2067683766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101270 0x7fbcdc1016f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 --2- 172.21.15.100:0/2067683766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 0x7fbcdc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 -- 172.21.15.100:0/2067683766 >> 172.21.15.100:0/2067683766 conn(0x7fbcdc0fb610 msgr2=0x7fbcdc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 -- 172.21.15.100:0/2067683766 shutdown_connections 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.124+0000 7fbce39b5640 1 -- 172.21.15.100:0/2067683766 wait complete. 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 Processor -- start 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 -- start start 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 0x7fbcdc197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc101270 0x7fbcdc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 0x7fbcdc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc19ecf0 con 0x7fbcdc101c30 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc19ee60 con 0x7fbcdc100080 2024-03-12T05:10:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce39b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc19f020 con 0x7fbcdc101270 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce1f52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 0x7fbcdc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce1f52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 0x7fbcdc19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35980/0 (socket says 172.21.15.100:35980) 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce1f52640 1 -- 172.21.15.100:0/3542953897 learned_addr learned my addr 172.21.15.100:0/3542953897 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.125+0000 7fbce1751640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 0x7fbcdc197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1f52640 1 -- 172.21.15.100:0/3542953897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc101270 msgr2=0x7fbcdc197740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1f52640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc101270 0x7fbcdc197740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1f52640 1 -- 172.21.15.100:0/3542953897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 msgr2=0x7fbcdc197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1f52640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 0x7fbcdc197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1f52640 1 -- 172.21.15.100:0/3542953897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0009560 con 0x7fbcdc101c30 2024-03-12T05:10:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1751640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 0x7fbcdc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce1f52640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 0x7fbcdc19e7b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400eaa0 tx=0x7fbcd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400cdb0 con 0x7fbcdc101c30 2024-03-12T05:10:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcd4005020 con 0x7fbcdc101c30 2024-03-12T05:10:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc19f300 con 0x7fbcdc101c30 2024-03-12T05:10:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40116b0 con 0x7fbcdc101c30 2024-03-12T05:10:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.126+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc19f810 con 0x7fbcdc101c30 2024-03-12T05:10:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.127+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc10ba90 con 0x7fbcdc101c30 2024-03-12T05:10:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.128+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbcd4005520 con 0x7fbcdc101c30 2024-03-12T05:10:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.129+0000 7fbcce7fc640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbcc80a1a30 0x7fbcc80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.129+0000 7fbce1751640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbcc80a1a30 0x7fbcc80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.129+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbcd40cd950 con 0x7fbcdc101c30 2024-03-12T05:10:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.129+0000 7fbce1751640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbcc80a1a30 0x7fbcc80a3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc1010d0 tx=0x7fbcd8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.133+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcd4096cb0 con 0x7fbcdc101c30 2024-03-12T05:10:01.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1167303352' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-12T05:10:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] up:active 2024-03-12T05:10:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-12T05:10:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:10:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: overall HEALTH_OK 2024-03-12T05:10:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:01.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:00 smithi161 ceph-mon[36043]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:01.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1167303352' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-12T05:10:01.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] up:active 2024-03-12T05:10:01.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-12T05:10:01.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:10:01.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: overall HEALTH_OK 2024-03-12T05:10:01.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:01.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:00 smithi100 ceph-mon[32210]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:01.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.334+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcdc0fd7e0 con 0x7fbcc80a1a30 2024-03-12T05:10:01.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.358+0000 7fbcce7fc640 1 -- 172.21.15.100:0/3542953897 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbcdc0fd7e0 con 0x7fbcc80a1a30 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbcc80a1a30 msgr2=0x7fbcc80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbcc80a1a30 0x7fbcc80a3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc1010d0 tx=0x7fbcd8005d30 comp rx=0 tx=0).stop 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 msgr2=0x7fbcdc19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 0x7fbcdc19e7b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400eaa0 tx=0x7fbcd400ef60 comp rx=0 tx=0).stop 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 shutdown_connections 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbcc80a1a30 0x7fbcc80a3ef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcdc101c30 0x7fbcdc19e7b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc101270 0x7fbcdc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 --2- 172.21.15.100:0/3542953897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcdc100080 0x7fbcdc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 >> 172.21.15.100:0/3542953897 conn(0x7fbcdc0fb610 msgr2=0x7fbcdc0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 shutdown_connections 2024-03-12T05:10:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.360+0000 7fbce39b5640 1 -- 172.21.15.100:0/3542953897 wait complete. 2024-03-12T05:10:01.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-12T05:10:01.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1167303352' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-12T05:10:01.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] up:active 2024-03-12T05:10:01.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-12T05:10:01.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:10:01.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: overall HEALTH_OK 2024-03-12T05:10:01.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:01.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:00 smithi156 ceph-mon[36270]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:01.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 -- 172.21.15.100:0/3187031941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 msgr2=0x7fa8f81015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 --2- 172.21.15.100:0/3187031941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f81015d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009900 tx=0x7fa8e802bc30 comp rx=0 tx=0).stop 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 -- 172.21.15.100:0/3187031941 shutdown_connections 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 --2- 172.21.15.100:0/3187031941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 0x7fa8f81095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 --2- 172.21.15.100:0/3187031941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f81023a0 0x7fa8f8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 --2- 172.21.15.100:0/3187031941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f81015d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 -- 172.21.15.100:0/3187031941 >> 172.21.15.100:0/3187031941 conn(0x7fa8f80fc720 msgr2=0x7fa8f80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 -- 172.21.15.100:0/3187031941 shutdown_connections 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 -- 172.21.15.100:0/3187031941 wait complete. 2024-03-12T05:10:01.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 Processor -- start 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.477+0000 7fa8fe977640 1 -- start start 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f81023a0 0x7fa8f81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 0x7fa8f819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f819ed80 con 0x7fa8f81011b0 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f819eef0 con 0x7fa8f81023a0 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f819f0b0 con 0x7fa8f8102d60 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f8197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36006/0 (socket says 172.21.15.100:36006) 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 -- 172.21.15.100:0/2247755283 learned_addr learned my addr 172.21.15.100:0/2247755283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 -- 172.21.15.100:0/2247755283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 msgr2=0x7fa8f819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fcf14640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 0x7fa8f819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 0x7fa8f819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 -- 172.21.15.100:0/2247755283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f81023a0 msgr2=0x7fa8f81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f81023a0 0x7fa8f81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 -- 172.21.15.100:0/2247755283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8009560 con 0x7fa8f81011b0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fcf14640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 0x7fa8f819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f7fff640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f8197290 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8036040 tx=0x7fa8e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80044e0 con 0x7fa8f81011b0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8e80054e0 con 0x7fa8f81011b0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f819f330 con 0x7fa8f81011b0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.478+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e803e450 con 0x7fa8f81011b0 2024-03-12T05:10:01.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.479+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f819f840 con 0x7fa8f81011b0 2024-03-12T05:10:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.480+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f8108550 con 0x7fa8f81011b0 2024-03-12T05:10:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.480+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa8e8003710 con 0x7fa8f81011b0 2024-03-12T05:10:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.481+0000 7fa8f57fa640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa8e00a1a30 0x7fa8e00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.481+0000 7fa8f77fe640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa8e00a1a30 0x7fa8e00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.481+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8e80ef7d0 con 0x7fa8f81011b0 2024-03-12T05:10:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.481+0000 7fa8f77fe640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa8e00a1a30 0x7fa8e00a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa8f81987a0 tx=0x7fa8ec005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:01.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.485+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8e80b8b30 con 0x7fa8f81011b0 2024-03-12T05:10:01.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.683+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8f802ce40 con 0x7fa8e00a1a30 2024-03-12T05:10:01.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.687+0000 7fa8f57fa640 1 -- 172.21.15.100:0/2247755283 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fa8f802ce40 con 0x7fa8e00a1a30 2024-03-12T05:10:01.685 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/qa/sv_1/b1f3ef68-e29d-48c5-a957-f42b5b34b829 2024-03-12T05:10:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa8e00a1a30 msgr2=0x7fa8e00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa8e00a1a30 0x7fa8e00a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa8f81987a0 tx=0x7fa8ec005bf0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 msgr2=0x7fa8f8197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f8197290 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8036040 tx=0x7fa8e80042f0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 shutdown_connections 2024-03-12T05:10:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa8e00a1a30 0x7fa8e00a3ef0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8102d60 0x7fa8f819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f81023a0 0x7fa8f81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 --2- 172.21.15.100:0/2247755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f81011b0 0x7fa8f8197290 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 >> 172.21.15.100:0/2247755283 conn(0x7fa8f80fc720 msgr2=0x7fa8f8105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 shutdown_connections 2024-03-12T05:10:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.689+0000 7fa8fe977640 1 -- 172.21.15.100:0/2247755283 wait complete. 2024-03-12T05:10:01.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 -- 172.21.15.100:0/4189885017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ffbc0 msgr2=0x7fa270109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4189885017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ffbc0 0x7fa270109770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa268009ca0 tx=0x7fa26802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 -- 172.21.15.100:0/4189885017 shutdown_connections 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4189885017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ffbc0 0x7fa270109770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4189885017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ff200 0x7fa2700ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4189885017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa270102790 0x7fa270102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 -- 172.21.15.100:0/4189885017 >> 172.21.15.100:0/4189885017 conn(0x7fa2700fada0 msgr2=0x7fa2700fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 -- 172.21.15.100:0/4189885017 shutdown_connections 2024-03-12T05:10:01.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.801+0000 7fa277fd0640 1 -- 172.21.15.100:0/4189885017 wait complete. 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 Processor -- start 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 -- start start 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ff200 0x7fa270197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2700ffbc0 0x7fa270197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 0x7fa27019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019ec30 con 0x7fa270102790 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019eda0 con 0x7fa2700ffbc0 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa277fd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019ef60 con 0x7fa2700ff200 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27656d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 0x7fa27019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27656d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 0x7fa27019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36028/0 (socket says 172.21.15.100:36028) 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27656d640 1 -- 172.21.15.100:0/4072375123 learned_addr learned my addr 172.21.15.100:0/4072375123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27556b640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2700ffbc0 0x7fa270197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa275d6c640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ff200 0x7fa270197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27656d640 1 -- 172.21.15.100:0/4072375123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ff200 msgr2=0x7fa270197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27656d640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ff200 0x7fa270197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.802+0000 7fa27656d640 1 -- 172.21.15.100:0/4072375123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2700ffbc0 msgr2=0x7fa270197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa27656d640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2700ffbc0 0x7fa270197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa27656d640 1 -- 172.21.15.100:0/4072375123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268009900 con 0x7fa270102790 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa27556b640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2700ffbc0 0x7fa270197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa27656d640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 0x7fa27019e6f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa268037040 tx=0x7fa2680346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268032e70 con 0x7fa270102790 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa268003e50 con 0x7fa270102790 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27019f1e0 con 0x7fa270102790 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268040450 con 0x7fa270102790 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.803+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27019f6e0 con 0x7fa270102790 2024-03-12T05:10:01.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.804+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa270108580 con 0x7fa270102790 2024-03-12T05:10:01.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.805+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa26800a040 con 0x7fa270102790 2024-03-12T05:10:01.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.806+0000 7fa266ffd640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa25c0a1ab0 0x7fa25c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:01.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.806+0000 7fa275d6c640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa25c0a1ab0 0x7fa25c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:01.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.806+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa2680f0240 con 0x7fa270102790 2024-03-12T05:10:01.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.806+0000 7fa275d6c640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa25c0a1ab0 0x7fa25c0a3f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005fd0 tx=0x7fa26c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:01.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:01.810+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2680b95a0 con 0x7fa270102790 2024-03-12T05:10:02.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.025+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa27010cbf0 con 0x7fa270102790 2024-03-12T05:10:02.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.026+0000 7fa266ffd640 1 -- 172.21.15.100:0/4072375123 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fa27010cbf0 con 0x7fa270102790 2024-03-12T05:10:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:02.024 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:09:59.961590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:10:02.024 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:10:02.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa25c0a1ab0 msgr2=0x7fa25c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa25c0a1ab0 0x7fa25c0a3f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005fd0 tx=0x7fa26c00b5b0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 msgr2=0x7fa27019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 0x7fa27019e6f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa268037040 tx=0x7fa2680346f0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 shutdown_connections 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa25c0a1ab0 0x7fa25c0a3f70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270102790 0x7fa27019e6f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2700ffbc0 0x7fa270197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 --2- 172.21.15.100:0/4072375123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2700ff200 0x7fa270197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 >> 172.21.15.100:0/4072375123 conn(0x7fa2700fada0 msgr2=0x7fa270105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 shutdown_connections 2024-03-12T05:10:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.028+0000 7fa277fd0640 1 -- 172.21.15.100:0/4072375123 wait complete. 2024-03-12T05:10:02.037 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:10:02.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 -- 172.21.15.100:0/1794786046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 msgr2=0x7f5f24113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 --2- 172.21.15.100:0/1794786046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f24113ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009ca0 tx=0x7f5f1c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 -- 172.21.15.100:0/1794786046 shutdown_connections 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 --2- 172.21.15.100:0/1794786046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f24113ff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 --2- 172.21.15.100:0/1794786046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2410cda0 0x7f5f2410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 --2- 172.21.15.100:0/1794786046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2410bbb0 0x7f5f2410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 -- 172.21.15.100:0/1794786046 >> 172.21.15.100:0/1794786046 conn(0x7f5f2406bbe0 msgr2=0x7f5f2406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 -- 172.21.15.100:0/1794786046 shutdown_connections 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.141+0000 7f5f2a062640 1 -- 172.21.15.100:0/1794786046 wait complete. 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 Processor -- start 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 -- start start 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2410bbb0 0x7f5f241a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2410cda0 0x7f5f241a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f241a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241a9710 con 0x7f5f2410d760 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241a9880 con 0x7f5f2410cda0 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f2a062640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241a9a40 con 0x7f5f2410bbb0 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f23fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f241a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f23fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f241a91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36048/0 (socket says 172.21.15.100:36048) 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f23fff640 1 -- 172.21.15.100:0/102583328 learned_addr learned my addr 172.21.15.100:0/102583328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f237fe640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2410bbb0 0x7f5f241a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.142+0000 7f5f22ffd640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2410cda0 0x7f5f241a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f23fff640 1 -- 172.21.15.100:0/102583328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2410bbb0 msgr2=0x7f5f241a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f23fff640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2410bbb0 0x7f5f241a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f23fff640 1 -- 172.21.15.100:0/102583328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2410cda0 msgr2=0x7f5f241a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f23fff640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2410cda0 0x7f5f241a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f23fff640 1 -- 172.21.15.100:0/102583328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c009900 con 0x7f5f2410d760 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f23fff640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f241a91a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c037040 tx=0x7f5f1c034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c032eb0 con 0x7f5f2410d760 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f241a9cc0 con 0x7f5f2410d760 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f1c003e50 con 0x7f5f2410d760 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c040450 con 0x7f5f2410d760 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.143+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f241aa170 con 0x7f5f2410d760 2024-03-12T05:10:02.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.144+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24112ed0 con 0x7f5f2410d760 2024-03-12T05:10:02.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.145+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5f1c00a040 con 0x7f5f2410d760 2024-03-12T05:10:02.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.145+0000 7f5f20ff9640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5f0c0a1a30 0x7f5f0c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.146+0000 7f5f237fe640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5f0c0a1a30 0x7f5f0c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.146+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5f1c0f02d0 con 0x7f5f2410d760 2024-03-12T05:10:02.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.146+0000 7f5f237fe640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5f0c0a1a30 0x7f5f0c0a3ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f241a3100 tx=0x7f5f1400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:02.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.150+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f1c0b95a0 con 0x7f5f2410d760 2024-03-12T05:10:02.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:01 smithi161 ceph-mon[36043]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:01 smithi161 ceph-mon[36043]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:01 smithi161 ceph-mon[36043]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:01 smithi161 ceph-mon[36043]: pgmap v285: 97 pgs: 97 active+clean; 600 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7.6 KiB/s wr, 16 op/s 2024-03-12T05:10:02.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:01 smithi100 ceph-mon[32210]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:01 smithi100 ceph-mon[32210]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:01 smithi100 ceph-mon[32210]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:01 smithi100 ceph-mon[32210]: pgmap v285: 97 pgs: 97 active+clean; 600 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7.6 KiB/s wr, 16 op/s 2024-03-12T05:10:02.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.323+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f241177e0 con 0x7f5f2410d760 2024-03-12T05:10:02.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.324+0000 7f5f20ff9640 1 -- 172.21.15.100:0/102583328 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22249 (secure 0 0 0) 0x7f5f1c0bb4d0 con 0x7f5f2410d760 2024-03-12T05:10:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:02.322 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":86,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:09:55.687555+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:10:02.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5f0c0a1a30 msgr2=0x7f5f0c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5f0c0a1a30 0x7f5f0c0a3ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f241a3100 tx=0x7f5f1400a2d0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 msgr2=0x7f5f241a91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f241a91a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c037040 tx=0x7f5f1c034d40 comp rx=0 tx=0).stop 2024-03-12T05:10:02.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 shutdown_connections 2024-03-12T05:10:02.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5f0c0a1a30 0x7f5f0c0a3ef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2410d760 0x7f5f241a91a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2410cda0 0x7f5f241a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 --2- 172.21.15.100:0/102583328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2410bbb0 0x7f5f241a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 >> 172.21.15.100:0/102583328 conn(0x7f5f2406bbe0 msgr2=0x7f5f241100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 shutdown_connections 2024-03-12T05:10:02.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.326+0000 7f5f2a062640 1 -- 172.21.15.100:0/102583328 wait complete. 2024-03-12T05:10:02.335 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-12T05:10:02.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 -- 172.21.15.100:0/2225935754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48441000b0 msgr2=0x7f48441004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 --2- 172.21.15.100:0/2225935754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48441000b0 0x7f48441004d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f483c009900 tx=0x7f483c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 -- 172.21.15.100:0/2225935754 shutdown_connections 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 --2- 172.21.15.100:0/2225935754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4844101c60 0x7f48441084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 --2- 172.21.15.100:0/2225935754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441012a0 0x7f4844101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 --2- 172.21.15.100:0/2225935754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48441000b0 0x7f48441004d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 -- 172.21.15.100:0/2225935754 >> 172.21.15.100:0/2225935754 conn(0x7f48440fb660 msgr2=0x7f48440fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 -- 172.21.15.100:0/2225935754 shutdown_connections 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.439+0000 7f4848e15640 1 -- 172.21.15.100:0/2225935754 wait complete. 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 Processor -- start 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 -- start start 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48441000b0 0x7f48441961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441012a0 0x7f4844196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 0x7f484419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484419dcc0 con 0x7f4844101c60 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484419de30 con 0x7f48441012a0 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4848e15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484419dff0 con 0x7f48441000b0 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 0x7f484419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 0x7f484419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36072/0 (socket says 172.21.15.100:36072) 2024-03-12T05:10:02.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 -- 172.21.15.100:0/113778603 learned_addr learned my addr 172.21.15.100:0/113778603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4841d9b640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441012a0 0x7f4844196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f484259c640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48441000b0 0x7f48441961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 -- 172.21.15.100:0/113778603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48441000b0 msgr2=0x7f48441961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48441000b0 0x7f48441961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 -- 172.21.15.100:0/113778603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441012a0 msgr2=0x7f4844196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441012a0 0x7f4844196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 -- 172.21.15.100:0/113778603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c009560 con 0x7f4844101c60 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f4842d9d640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 0x7f484419d780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4834009fd0 tx=0x7f483400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.440+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834011830 con 0x7f4844101c60 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.441+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484419e2d0 con 0x7f4844101c60 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.441+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4834011e70 con 0x7f4844101c60 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.441+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400f2e0 con 0x7f4844101c60 2024-03-12T05:10:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.441+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484419e860 con 0x7f4844101c60 2024-03-12T05:10:02.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.442+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48441073d0 con 0x7f4844101c60 2024-03-12T05:10:02.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.442+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f48340105f0 con 0x7f4844101c60 2024-03-12T05:10:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.443+0000 7f48337fe640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f482c0a1a30 0x7f482c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.443+0000 7f484259c640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f482c0a1a30 0x7f482c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.443+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48340ccb30 con 0x7f4844101c60 2024-03-12T05:10:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.443+0000 7f484259c640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f482c0a1a30 0x7f482c0a3ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f483c006010 tx=0x7f483c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:02.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.447+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4834095e90 con 0x7f4844101c60 2024-03-12T05:10:02.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:01 smithi156 ceph-mon[36270]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:01 smithi156 ceph-mon[36270]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:01 smithi156 ceph-mon[36270]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:10:02.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:01 smithi156 ceph-mon[36270]: pgmap v285: 97 pgs: 97 active+clean; 600 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7.6 KiB/s wr, 16 op/s 2024-03-12T05:10:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.618+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f484402ce10 con 0x7f482c0a1a30 2024-03-12T05:10:02.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.625+0000 7f48337fe640 1 -- 172.21.15.100:0/113778603 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258725 (secure 0 0 0) 0x7f484402ce10 con 0x7f482c0a1a30 2024-03-12T05:10:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:10:02.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f482c0a1a30 msgr2=0x7f482c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f482c0a1a30 0x7f482c0a3ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f483c006010 tx=0x7f483c000bc0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 msgr2=0x7f484419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 0x7f484419d780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4834009fd0 tx=0x7f483400b8e0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 shutdown_connections 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f482c0a1a30 0x7f482c0a3ef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4844101c60 0x7f484419d780 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441012a0 0x7f4844196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 --2- 172.21.15.100:0/113778603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48441000b0 0x7f48441961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 >> 172.21.15.100:0/113778603 conn(0x7f48440fb660 msgr2=0x7f48441045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 shutdown_connections 2024-03-12T05:10:02.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.628+0000 7f4848e15640 1 -- 172.21.15.100:0/113778603 wait complete. 2024-03-12T05:10:02.638 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-03-12T05:10:01.710706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614709,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":155,"num_read_kb":135,"num_write":287,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":277,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374384,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567440,"statfs":{"total":1151940427776,"available":1151557058560,"internally_reserved":0,"allocated":13037568,"data_stored":7090262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24341,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":106,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.783319"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568924+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:09:56.568924+0000","last_peered":"2024-03-12T05:09:56.568924+0000","last_clean":"2024-03-12T05:09:56.568924+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:09:56.568924+0000","last_undegraded":"2024-03-12T05:09:56.568924+0000","last_fullsized":"2024-03-12T05:09:56.568924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568996+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:09:56.568996+0000","last_peered":"2024-03-12T05:09:56.568996+0000","last_clean":"2024-03-12T05:09:56.568996+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:09:56.568996+0000","last_undegraded":"2024-03-12T05:09:56.568996+0000","last_fullsized":"2024-03-12T05:09:56.568996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568759+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:09:56.568759+0000","last_peered":"2024-03-12T05:09:56.568759+0000","last_clean":"2024-03-12T05:09:56.568759+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:09:56.568759+0000","last_undegraded":"2024-03-12T05:09:56.568759+0000","last_fullsized":"2024-03-12T05:09:56.568759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569233+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:09:56.569233+0000","last_peered":"2024-03-12T05:09:56.569233+0000","last_clean":"2024-03-12T05:09:56.569233+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:09:56.569233+0000","last_undegraded":"2024-03-12T05:09:56.569233+0000","last_fullsized":"2024-03-12T05:09:56.569233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932439+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:09:55.932439+0000","last_peered":"2024-03-12T05:09:55.932439+0000","last_clean":"2024-03-12T05:09:55.932439+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:09:55.932439+0000","last_undegraded":"2024-03-12T05:09:55.932439+0000","last_fullsized":"2024-03-12T05:09:55.932439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569309+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:09:56.569309+0000","last_peered":"2024-03-12T05:09:56.569309+0000","last_clean":"2024-03-12T05:09:56.569309+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:09:56.569309+0000","last_undegraded":"2024-03-12T05:09:56.569309+0000","last_fullsized":"2024-03-12T05:09:56.569309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182884+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:09:56.182884+0000","last_peered":"2024-03-12T05:09:56.182884+0000","last_clean":"2024-03-12T05:09:56.182884+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:09:56.182884+0000","last_undegraded":"2024-03-12T05:09:56.182884+0000","last_fullsized":"2024-03-12T05:09:56.182884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690432+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:09:55.690432+0000","last_peered":"2024-03-12T05:09:55.690432+0000","last_clean":"2024-03-12T05:09:55.690432+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:09:55.690432+0000","last_undegraded":"2024-03-12T05:09:55.690432+0000","last_fullsized":"2024-03-12T05:09:55.690432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691522+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:09:55.691522+0000","last_peered":"2024-03-12T05:09:55.691522+0000","last_clean":"2024-03-12T05:09:55.691522+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:09:55.691522+0000","last_undegraded":"2024-03-12T05:09:55.691522+0000","last_fullsized":"2024-03-12T05:09:55.691522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569564+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:09:56.569564+0000","last_peered":"2024-03-12T05:09:56.569564+0000","last_clean":"2024-03-12T05:09:56.569564+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:09:56.569564+0000","last_undegraded":"2024-03-12T05:09:56.569564+0000","last_fullsized":"2024-03-12T05:09:56.569564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183238+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:09:56.183238+0000","last_peered":"2024-03-12T05:09:56.183238+0000","last_clean":"2024-03-12T05:09:56.183238+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:09:56.183238+0000","last_undegraded":"2024-03-12T05:09:56.183238+0000","last_fullsized":"2024-03-12T05:09:56.183238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569036+0000","last_change":"2024-03-12T05:09:54.703962+0000","last_active":"2024-03-12T05:09:56.569036+0000","last_peered":"2024-03-12T05:09:56.569036+0000","last_clean":"2024-03-12T05:09:56.569036+0000","last_became_active":"2024-03-12T05:09:54.702987+0000","last_became_peered":"2024-03-12T05:09:54.702987+0000","last_unstale":"2024-03-12T05:09:56.569036+0000","last_undegraded":"2024-03-12T05:09:56.569036+0000","last_fullsized":"2024-03-12T05:09:56.569036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:55:37.300456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568240+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:09:56.568240+0000","last_peered":"2024-03-12T05:09:56.568240+0000","last_clean":"2024-03-12T05:09:56.568240+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:09:56.568240+0000","last_undegraded":"2024-03-12T05:09:56.568240+0000","last_fullsized":"2024-03-12T05:09:56.568240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568523+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:09:56.568523+0000","last_peered":"2024-03-12T05:09:56.568523+0000","last_clean":"2024-03-12T05:09:56.568523+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:09:56.568523+0000","last_undegraded":"2024-03-12T05:09:56.568523+0000","last_fullsized":"2024-03-12T05:09:56.568523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568340+0000","last_change":"2024-03-12T05:09:54.712692+0000","last_active":"2024-03-12T05:09:56.568340+0000","last_peered":"2024-03-12T05:09:56.568340+0000","last_clean":"2024-03-12T05:09:56.568340+0000","last_became_active":"2024-03-12T05:09:54.712362+0000","last_became_peered":"2024-03-12T05:09:54.712362+0000","last_unstale":"2024-03-12T05:09:56.568340+0000","last_undegraded":"2024-03-12T05:09:56.568340+0000","last_fullsized":"2024-03-12T05:09:56.568340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:50.828861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691030+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:09:55.691030+0000","last_peered":"2024-03-12T05:09:55.691030+0000","last_clean":"2024-03-12T05:09:55.691030+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:09:55.691030+0000","last_undegraded":"2024-03-12T05:09:55.691030+0000","last_fullsized":"2024-03-12T05:09:55.691030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569150+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:09:56.569150+0000","last_peered":"2024-03-12T05:09:56.569150+0000","last_clean":"2024-03-12T05:09:56.569150+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:09:56.569150+0000","last_undegraded":"2024-03-12T05:09:56.569150+0000","last_fullsized":"2024-03-12T05:09:56.569150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738891+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:09:55.738891+0000","last_peered":"2024-03-12T05:09:55.738891+0000","last_clean":"2024-03-12T05:09:55.738891+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:09:55.738891+0000","last_undegraded":"2024-03-12T05:09:55.738891+0000","last_fullsized":"2024-03-12T05:09:55.738891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569317+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:09:56.569317+0000","last_peered":"2024-03-12T05:09:56.569317+0000","last_clean":"2024-03-12T05:09:56.569317+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:09:56.569317+0000","last_undegraded":"2024-03-12T05:09:56.569317+0000","last_fullsized":"2024-03-12T05:09:56.569317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738880+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:09:55.738880+0000","last_peered":"2024-03-12T05:09:55.738880+0000","last_clean":"2024-03-12T05:09:55.738880+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:09:55.738880+0000","last_undegraded":"2024-03-12T05:09:55.738880+0000","last_fullsized":"2024-03-12T05:09:55.738880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690994+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:09:55.690994+0000","last_peered":"2024-03-12T05:09:55.690994+0000","last_clean":"2024-03-12T05:09:55.690994+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:09:55.690994+0000","last_undegraded":"2024-03-12T05:09:55.690994+0000","last_fullsized":"2024-03-12T05:09:55.690994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691091+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:09:55.691091+0000","last_peered":"2024-03-12T05:09:55.691091+0000","last_clean":"2024-03-12T05:09:55.691091+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:09:55.691091+0000","last_undegraded":"2024-03-12T05:09:55.691091+0000","last_fullsized":"2024-03-12T05:09:55.691091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568815+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:09:56.568815+0000","last_peered":"2024-03-12T05:09:56.568815+0000","last_clean":"2024-03-12T05:09:56.568815+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:09:56.568815+0000","last_undegraded":"2024-03-12T05:09:56.568815+0000","last_fullsized":"2024-03-12T05:09:56.568815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.961167+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:09:58.961167+0000","last_peered":"2024-03-12T05:09:58.961167+0000","last_clean":"2024-03-12T05:09:58.961167+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:09:58.961167+0000","last_undegraded":"2024-03-12T05:09:58.961167+0000","last_fullsized":"2024-03-12T05:09:58.961167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691317+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:09:55.691317+0000","last_peered":"2024-03-12T05:09:55.691317+0000","last_clean":"2024-03-12T05:09:55.691317+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:09:55.691317+0000","last_undegraded":"2024-03-12T05:09:55.691317+0000","last_fullsized":"2024-03-12T05:09:55.691317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932705+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:09:55.932705+0000","last_peered":"2024-03-12T05:09:55.932705+0000","last_clean":"2024-03-12T05:09:55.932705+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:09:55.932705+0000","last_undegraded":"2024-03-12T05:09:55.932705+0000","last_fullsized":"2024-03-12T05:09:55.932705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.692097+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:09:55.692097+0000","last_peered":"2024-03-12T05:09:55.692097+0000","last_clean":"2024-03-12T05:09:55.692097+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:09:55.692097+0000","last_undegraded":"2024-03-12T05:09:55.692097+0000","last_fullsized":"2024-03-12T05:09:55.692097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.953353+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:09:57.953353+0000","last_peered":"2024-03-12T05:09:57.953353+0000","last_clean":"2024-03-12T05:09:57.953353+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:09:57.953353+0000","last_undegraded":"2024-03-12T05:09:57.953353+0000","last_fullsized":"2024-03-12T05:09:57.953353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932663+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:09:55.932663+0000","last_peered":"2024-03-12T05:09:55.932663+0000","last_clean":"2024-03-12T05:09:55.932663+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:09:55.932663+0000","last_undegraded":"2024-03-12T05:09:55.932663+0000","last_fullsized":"2024-03-12T05:09:55.932663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.959417+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:09:58.959417+0000","last_peered":"2024-03-12T05:09:58.959417+0000","last_clean":"2024-03-12T05:09:58.959417+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:09:58.959417+0000","last_undegraded":"2024-03-12T05:09:58.959417+0000","last_fullsized":"2024-03-12T05:09:58.959417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691832+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:09:55.691832+0000","last_peered":"2024-03-12T05:09:55.691832+0000","last_clean":"2024-03-12T05:09:55.691832+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:09:55.691832+0000","last_undegraded":"2024-03-12T05:09:55.691832+0000","last_fullsized":"2024-03-12T05:09:55.691832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.961325+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:09:58.961325+0000","last_peered":"2024-03-12T05:09:58.961325+0000","last_clean":"2024-03-12T05:09:58.961325+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:09:58.961325+0000","last_undegraded":"2024-03-12T05:09:58.961325+0000","last_fullsized":"2024-03-12T05:09:58.961325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690492+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:09:55.690492+0000","last_peered":"2024-03-12T05:09:55.690492+0000","last_clean":"2024-03-12T05:09:55.690492+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:09:55.690492+0000","last_undegraded":"2024-03-12T05:09:55.690492+0000","last_fullsized":"2024-03-12T05:09:55.690492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691737+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:09:55.691737+0000","last_peered":"2024-03-12T05:09:55.691737+0000","last_clean":"2024-03-12T05:09:55.691737+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:09:55.691737+0000","last_undegraded":"2024-03-12T05:09:55.691737+0000","last_fullsized":"2024-03-12T05:09:55.691737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738733+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:09:55.738733+0000","last_peered":"2024-03-12T05:09:55.738733+0000","last_clean":"2024-03-12T05:09:55.738733+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:09:55.738733+0000","last_undegraded":"2024-03-12T05:09:55.738733+0000","last_fullsized":"2024-03-12T05:09:55.738733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952664+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:09:57.952664+0000","last_peered":"2024-03-12T05:09:57.952664+0000","last_clean":"2024-03-12T05:09:57.952664+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:09:57.952664+0000","last_undegraded":"2024-03-12T05:09:57.952664+0000","last_fullsized":"2024-03-12T05:09:57.952664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691424+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:09:55.691424+0000","last_peered":"2024-03-12T05:09:55.691424+0000","last_clean":"2024-03-12T05:09:55.691424+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:09:55.691424+0000","last_undegraded":"2024-03-12T05:09:55.691424+0000","last_fullsized":"2024-03-12T05:09:55.691424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.951457+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:09:57.951457+0000","last_peered":"2024-03-12T05:09:57.951457+0000","last_clean":"2024-03-12T05:09:57.951457+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:09:57.951457+0000","last_undegraded":"2024-03-12T05:09:57.951457+0000","last_fullsized":"2024-03-12T05:09:57.951457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568914+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:09:56.568914+0000","last_peered":"2024-03-12T05:09:56.568914+0000","last_clean":"2024-03-12T05:09:56.568914+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:09:56.568914+0000","last_undegraded":"2024-03-12T05:09:56.568914+0000","last_fullsized":"2024-03-12T05:09:56.568914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.692170+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:09:55.692170+0000","last_peered":"2024-03-12T05:09:55.692170+0000","last_clean":"2024-03-12T05:09:55.692170+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:09:55.692170+0000","last_undegraded":"2024-03-12T05:09:55.692170+0000","last_fullsized":"2024-03-12T05:09:55.692170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691408+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:09:55.691408+0000","last_peered":"2024-03-12T05:09:55.691408+0000","last_clean":"2024-03-12T05:09:55.691408+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:09:55.691408+0000","last_undegraded":"2024-03-12T05:09:55.691408+0000","last_fullsized":"2024-03-12T05:09:55.691408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:10:00.661217+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:10:00.661217+0000","last_peered":"2024-03-12T05:10:00.661217+0000","last_clean":"2024-03-12T05:10:00.661217+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:10:00.661217+0000","last_undegraded":"2024-03-12T05:10:00.661217+0000","last_fullsized":"2024-03-12T05:10:00.661217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932675+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:09:55.932675+0000","last_peered":"2024-03-12T05:09:55.932675+0000","last_clean":"2024-03-12T05:09:55.932675+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:09:55.932675+0000","last_undegraded":"2024-03-12T05:09:55.932675+0000","last_fullsized":"2024-03-12T05:09:55.932675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569197+0000","last_change":"2024-03-12T05:09:53.945301+0000","last_active":"2024-03-12T05:09:56.569197+0000","last_peered":"2024-03-12T05:09:56.569197+0000","last_clean":"2024-03-12T05:09:56.569197+0000","last_became_active":"2024-03-12T05:09:53.944944+0000","last_became_peered":"2024-03-12T05:09:53.944944+0000","last_unstale":"2024-03-12T05:09:56.569197+0000","last_undegraded":"2024-03-12T05:09:56.569197+0000","last_fullsized":"2024-03-12T05:09:56.569197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:08.438529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568052+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:09:56.568052+0000","last_peered":"2024-03-12T05:09:56.568052+0000","last_clean":"2024-03-12T05:09:56.568052+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:09:56.568052+0000","last_undegraded":"2024-03-12T05:09:56.568052+0000","last_fullsized":"2024-03-12T05:09:56.568052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952888+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:09:57.952888+0000","last_peered":"2024-03-12T05:09:57.952888+0000","last_clean":"2024-03-12T05:09:57.952888+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:09:57.952888+0000","last_undegraded":"2024-03-12T05:09:57.952888+0000","last_fullsized":"2024-03-12T05:09:57.952888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183214+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:09:56.183214+0000","last_peered":"2024-03-12T05:09:56.183214+0000","last_clean":"2024-03-12T05:09:56.183214+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:09:56.183214+0000","last_undegraded":"2024-03-12T05:09:56.183214+0000","last_fullsized":"2024-03-12T05:09:56.183214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:10:00.974118+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:10:00.974118+0000","last_peered":"2024-03-12T05:10:00.974118+0000","last_clean":"2024-03-12T05:10:00.974118+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:10:00.974118+0000","last_undegraded":"2024-03-12T05:10:00.974118+0000","last_fullsized":"2024-03-12T05:10:00.974118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182920+0000","last_change":"2024-03-12T05:09:54.714174+0000","last_active":"2024-03-12T05:09:56.182920+0000","last_peered":"2024-03-12T05:09:56.182920+0000","last_clean":"2024-03-12T05:09:56.182920+0000","last_became_active":"2024-03-12T05:09:54.713866+0000","last_became_peered":"2024-03-12T05:09:54.713866+0000","last_unstale":"2024-03-12T05:09:56.182920+0000","last_undegraded":"2024-03-12T05:09:56.182920+0000","last_fullsized":"2024-03-12T05:09:56.182920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:48:18.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.961841+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:09:58.961841+0000","last_peered":"2024-03-12T05:09:58.961841+0000","last_clean":"2024-03-12T05:09:58.961841+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:09:58.961841+0000","last_undegraded":"2024-03-12T05:09:58.961841+0000","last_fullsized":"2024-03-12T05:09:58.961841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691853+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:09:55.691853+0000","last_peered":"2024-03-12T05:09:55.691853+0000","last_clean":"2024-03-12T05:09:55.691853+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:09:55.691853+0000","last_undegraded":"2024-03-12T05:09:55.691853+0000","last_fullsized":"2024-03-12T05:09:55.691853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932926+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:09:55.932926+0000","last_peered":"2024-03-12T05:09:55.932926+0000","last_clean":"2024-03-12T05:09:55.932926+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:09:55.932926+0000","last_undegraded":"2024-03-12T05:09:55.932926+0000","last_fullsized":"2024-03-12T05:09:55.932926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568768+0000","last_change":"2024-03-12T05:09:54.709959+0000","last_active":"2024-03-12T05:09:56.568768+0000","last_peered":"2024-03-12T05:09:56.568768+0000","last_clean":"2024-03-12T05:09:56.568768+0000","last_became_active":"2024-03-12T05:09:54.709628+0000","last_became_peered":"2024-03-12T05:09:54.709628+0000","last_unstale":"2024-03-12T05:09:56.568768+0000","last_undegraded":"2024-03-12T05:09:56.568768+0000","last_fullsized":"2024-03-12T05:09:56.568768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:39:54.818675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738771+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:09:55.738771+0000","last_peered":"2024-03-12T05:09:55.738771+0000","last_clean":"2024-03-12T05:09:55.738771+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:09:55.738771+0000","last_undegraded":"2024-03-12T05:09:55.738771+0000","last_fullsized":"2024-03-12T05:09:55.738771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569283+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:09:56.569283+0000","last_peered":"2024-03-12T05:09:56.569283+0000","last_clean":"2024-03-12T05:09:56.569283+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:09:56.569283+0000","last_undegraded":"2024-03-12T05:09:56.569283+0000","last_fullsized":"2024-03-12T05:09:56.569283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182849+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:09:56.182849+0000","last_peered":"2024-03-12T05:09:56.182849+0000","last_clean":"2024-03-12T05:09:56.182849+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:09:56.182849+0000","last_undegraded":"2024-03-12T05:09:56.182849+0000","last_fullsized":"2024-03-12T05:09:56.182849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569316+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:09:56.569316+0000","last_peered":"2024-03-12T05:09:56.569316+0000","last_clean":"2024-03-12T05:09:56.569316+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:09:56.569316+0000","last_undegraded":"2024-03-12T05:09:56.569316+0000","last_fullsized":"2024-03-12T05:09:56.569316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691962+0000","last_change":"2024-03-12T05:09:54.706799+0000","last_active":"2024-03-12T05:09:55.691962+0000","last_peered":"2024-03-12T05:09:55.691962+0000","last_clean":"2024-03-12T05:09:55.691962+0000","last_became_active":"2024-03-12T05:09:54.706491+0000","last_became_peered":"2024-03-12T05:09:54.706491+0000","last_unstale":"2024-03-12T05:09:55.691962+0000","last_undegraded":"2024-03-12T05:09:55.691962+0000","last_fullsized":"2024-03-12T05:09:55.691962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:28:09.404200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.962577+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:09:58.962577+0000","last_peered":"2024-03-12T05:09:58.962577+0000","last_clean":"2024-03-12T05:09:58.962577+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:09:58.962577+0000","last_undegraded":"2024-03-12T05:09:58.962577+0000","last_fullsized":"2024-03-12T05:09:58.962577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690742+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:09:55.690742+0000","last_peered":"2024-03-12T05:09:55.690742+0000","last_clean":"2024-03-12T05:09:55.690742+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:09:55.690742+0000","last_undegraded":"2024-03-12T05:09:55.690742+0000","last_fullsized":"2024-03-12T05:09:55.690742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:59.967508+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:09:59.967508+0000","last_peered":"2024-03-12T05:09:59.967508+0000","last_clean":"2024-03-12T05:09:59.967508+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:09:59.967508+0000","last_undegraded":"2024-03-12T05:09:59.967508+0000","last_fullsized":"2024-03-12T05:09:59.967508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182566+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:09:56.182566+0000","last_peered":"2024-03-12T05:09:56.182566+0000","last_clean":"2024-03-12T05:09:56.182566+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:09:56.182566+0000","last_undegraded":"2024-03-12T05:09:56.182566+0000","last_fullsized":"2024-03-12T05:09:56.182566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.567835+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:09:56.567835+0000","last_peered":"2024-03-12T05:09:56.567835+0000","last_clean":"2024-03-12T05:09:56.567835+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:09:56.567835+0000","last_undegraded":"2024-03-12T05:09:56.567835+0000","last_fullsized":"2024-03-12T05:09:56.567835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183079+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:09:56.183079+0000","last_peered":"2024-03-12T05:09:56.183079+0000","last_clean":"2024-03-12T05:09:56.183079+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:09:56.183079+0000","last_undegraded":"2024-03-12T05:09:56.183079+0000","last_fullsized":"2024-03-12T05:09:56.183079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.950856+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:09:57.950856+0000","last_peered":"2024-03-12T05:09:57.950856+0000","last_clean":"2024-03-12T05:09:57.950856+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:09:57.950856+0000","last_undegraded":"2024-03-12T05:09:57.950856+0000","last_fullsized":"2024-03-12T05:09:57.950856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568333+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:09:56.568333+0000","last_peered":"2024-03-12T05:09:56.568333+0000","last_clean":"2024-03-12T05:09:56.568333+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:09:56.568333+0000","last_undegraded":"2024-03-12T05:09:56.568333+0000","last_fullsized":"2024-03-12T05:09:56.568333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.931568+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:09:55.931568+0000","last_peered":"2024-03-12T05:09:55.931568+0000","last_clean":"2024-03-12T05:09:55.931568+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:09:55.931568+0000","last_undegraded":"2024-03-12T05:09:55.931568+0000","last_fullsized":"2024-03-12T05:09:55.931568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691046+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:09:55.691046+0000","last_peered":"2024-03-12T05:09:55.691046+0000","last_clean":"2024-03-12T05:09:55.691046+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:09:55.691046+0000","last_undegraded":"2024-03-12T05:09:55.691046+0000","last_fullsized":"2024-03-12T05:09:55.691046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.951151+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:09:57.951151+0000","last_peered":"2024-03-12T05:09:57.951151+0000","last_clean":"2024-03-12T05:09:57.951151+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:09:57.951151+0000","last_undegraded":"2024-03-12T05:09:57.951151+0000","last_fullsized":"2024-03-12T05:09:57.951151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569016+0000","last_change":"2024-03-12T05:09:54.709130+0000","last_active":"2024-03-12T05:09:56.569016+0000","last_peered":"2024-03-12T05:09:56.569016+0000","last_clean":"2024-03-12T05:09:56.569016+0000","last_became_active":"2024-03-12T05:09:54.708690+0000","last_became_peered":"2024-03-12T05:09:54.708690+0000","last_unstale":"2024-03-12T05:09:56.569016+0000","last_undegraded":"2024-03-12T05:09:56.569016+0000","last_fullsized":"2024-03-12T05:09:56.569016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:04.765154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952186+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:09:57.952186+0000","last_peered":"2024-03-12T05:09:57.952186+0000","last_clean":"2024-03-12T05:09:57.952186+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:09:57.952186+0000","last_undegraded":"2024-03-12T05:09:57.952186+0000","last_fullsized":"2024-03-12T05:09:57.952186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932292+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:09:55.932292+0000","last_peered":"2024-03-12T05:09:55.932292+0000","last_clean":"2024-03-12T05:09:55.932292+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:09:55.932292+0000","last_undegraded":"2024-03-12T05:09:55.932292+0000","last_fullsized":"2024-03-12T05:09:55.932292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690526+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:09:55.690526+0000","last_peered":"2024-03-12T05:09:55.690526+0000","last_clean":"2024-03-12T05:09:55.690526+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:09:55.690526+0000","last_undegraded":"2024-03-12T05:09:55.690526+0000","last_fullsized":"2024-03-12T05:09:55.690526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568547+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:09:56.568547+0000","last_peered":"2024-03-12T05:09:56.568547+0000","last_clean":"2024-03-12T05:09:56.568547+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:09:56.568547+0000","last_undegraded":"2024-03-12T05:09:56.568547+0000","last_fullsized":"2024-03-12T05:09:56.568547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569020+0000","last_change":"2024-03-12T05:09:53.942272+0000","last_active":"2024-03-12T05:09:56.569020+0000","last_peered":"2024-03-12T05:09:56.569020+0000","last_clean":"2024-03-12T05:09:56.569020+0000","last_became_active":"2024-03-12T05:09:53.941874+0000","last_became_peered":"2024-03-12T05:09:53.941874+0000","last_unstale":"2024-03-12T05:09:56.569020+0000","last_undegraded":"2024-03-12T05:09:56.569020+0000","last_fullsized":"2024-03-12T05:09:56.569020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:40:21.770248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690805+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:09:55.690805+0000","last_peered":"2024-03-12T05:09:55.690805+0000","last_clean":"2024-03-12T05:09:55.690805+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:09:55.690805+0000","last_undegraded":"2024-03-12T05:09:55.690805+0000","last_fullsized":"2024-03-12T05:09:55.690805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952626+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:09:57.952626+0000","last_peered":"2024-03-12T05:09:57.952626+0000","last_clean":"2024-03-12T05:09:57.952626+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:09:57.952626+0000","last_undegraded":"2024-03-12T05:09:57.952626+0000","last_fullsized":"2024-03-12T05:09:57.952626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183233+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:09:56.183233+0000","last_peered":"2024-03-12T05:09:56.183233+0000","last_clean":"2024-03-12T05:09:56.183233+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:09:56.183233+0000","last_undegraded":"2024-03-12T05:09:56.183233+0000","last_fullsized":"2024-03-12T05:09:56.183233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569374+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:09:56.569374+0000","last_peered":"2024-03-12T05:09:56.569374+0000","last_clean":"2024-03-12T05:09:56.569374+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:09:56.569374+0000","last_undegraded":"2024-03-12T05:09:56.569374+0000","last_fullsized":"2024-03-12T05:09:56.569374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183256+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:09:56.183256+0000","last_peered":"2024-03-12T05:09:56.183256+0000","last_clean":"2024-03-12T05:09:56.183256+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:09:56.183256+0000","last_undegraded":"2024-03-12T05:09:56.183256+0000","last_fullsized":"2024-03-12T05:09:56.183256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691954+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:09:55.691954+0000","last_peered":"2024-03-12T05:09:55.691954+0000","last_clean":"2024-03-12T05:09:55.691954+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:09:55.691954+0000","last_undegraded":"2024-03-12T05:09:55.691954+0000","last_fullsized":"2024-03-12T05:09:55.691954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183198+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:09:56.183198+0000","last_peered":"2024-03-12T05:09:56.183198+0000","last_clean":"2024-03-12T05:09:56.183198+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:09:56.183198+0000","last_undegraded":"2024-03-12T05:09:56.183198+0000","last_fullsized":"2024-03-12T05:09:56.183198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568166+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:09:56.568166+0000","last_peered":"2024-03-12T05:09:56.568166+0000","last_clean":"2024-03-12T05:09:56.568166+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:09:56.568166+0000","last_undegraded":"2024-03-12T05:09:56.568166+0000","last_fullsized":"2024-03-12T05:09:56.568166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183229+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:09:56.183229+0000","last_peered":"2024-03-12T05:09:56.183229+0000","last_clean":"2024-03-12T05:09:56.183229+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:09:56.183229+0000","last_undegraded":"2024-03-12T05:09:56.183229+0000","last_fullsized":"2024-03-12T05:09:56.183229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568623+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:09:56.568623+0000","last_peered":"2024-03-12T05:09:56.568623+0000","last_clean":"2024-03-12T05:09:56.568623+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:09:56.568623+0000","last_undegraded":"2024-03-12T05:09:56.568623+0000","last_fullsized":"2024-03-12T05:09:56.568623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569212+0000","last_change":"2024-03-12T05:09:54.707255+0000","last_active":"2024-03-12T05:09:56.569212+0000","last_peered":"2024-03-12T05:09:56.569212+0000","last_clean":"2024-03-12T05:09:56.569212+0000","last_became_active":"2024-03-12T05:09:54.706932+0000","last_became_peered":"2024-03-12T05:09:54.706932+0000","last_unstale":"2024-03-12T05:09:56.569212+0000","last_undegraded":"2024-03-12T05:09:56.569212+0000","last_fullsized":"2024-03-12T05:09:56.569212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:27:00.222615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569026+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:09:56.569026+0000","last_peered":"2024-03-12T05:09:56.569026+0000","last_clean":"2024-03-12T05:09:56.569026+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:09:56.569026+0000","last_undegraded":"2024-03-12T05:09:56.569026+0000","last_fullsized":"2024-03-12T05:09:56.569026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568670+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:09:56.568670+0000","last_peered":"2024-03-12T05:09:56.568670+0000","last_clean":"2024-03-12T05:09:56.568670+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:09:56.568670+0000","last_undegraded":"2024-03-12T05:09:56.568670+0000","last_fullsized":"2024-03-12T05:09:56.568670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568114+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:09:56.568114+0000","last_peered":"2024-03-12T05:09:56.568114+0000","last_clean":"2024-03-12T05:09:56.568114+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:09:56.568114+0000","last_undegraded":"2024-03-12T05:09:56.568114+0000","last_fullsized":"2024-03-12T05:09:56.568114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568761+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:09:56.568761+0000","last_peered":"2024-03-12T05:09:56.568761+0000","last_clean":"2024-03-12T05:09:56.568761+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:09:56.568761+0000","last_undegraded":"2024-03-12T05:09:56.568761+0000","last_fullsized":"2024-03-12T05:09:56.568761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569470+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:09:56.569470+0000","last_peered":"2024-03-12T05:09:56.569470+0000","last_clean":"2024-03-12T05:09:56.569470+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:09:56.569470+0000","last_undegraded":"2024-03-12T05:09:56.569470+0000","last_fullsized":"2024-03-12T05:09:56.569470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691063+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:09:55.691063+0000","last_peered":"2024-03-12T05:09:55.691063+0000","last_clean":"2024-03-12T05:09:55.691063+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:09:55.691063+0000","last_undegraded":"2024-03-12T05:09:55.691063+0000","last_fullsized":"2024-03-12T05:09:55.691063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.692022+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:09:55.692022+0000","last_peered":"2024-03-12T05:09:55.692022+0000","last_clean":"2024-03-12T05:09:55.692022+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:09:55.692022+0000","last_undegraded":"2024-03-12T05:09:55.692022+0000","last_fullsized":"2024-03-12T05:09:55.692022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690518+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:09:55.690518+0000","last_peered":"2024-03-12T05:09:55.690518+0000","last_clean":"2024-03-12T05:09:55.690518+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:09:55.690518+0000","last_undegraded":"2024-03-12T05:09:55.690518+0000","last_fullsized":"2024-03-12T05:09:55.690518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691151+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:09:55.691151+0000","last_peered":"2024-03-12T05:09:55.691151+0000","last_clean":"2024-03-12T05:09:55.691151+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:09:55.691151+0000","last_undegraded":"2024-03-12T05:09:55.691151+0000","last_fullsized":"2024-03-12T05:09:55.691151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568256+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:09:56.568256+0000","last_peered":"2024-03-12T05:09:56.568256+0000","last_clean":"2024-03-12T05:09:56.568256+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:09:56.568256+0000","last_undegraded":"2024-03-12T05:09:56.568256+0000","last_fullsized":"2024-03-12T05:09:56.568256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568844+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:09:56.568844+0000","last_peered":"2024-03-12T05:09:56.568844+0000","last_clean":"2024-03-12T05:09:56.568844+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:09:56.568844+0000","last_undegraded":"2024-03-12T05:09:56.568844+0000","last_fullsized":"2024-03-12T05:09:56.568844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24224,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":105,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":33411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1035709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1033723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40408,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704744,"statfs":{"total":95995035648,"available":95953657856,"internally_reserved":0,"allocated":933888,"data_stored":443763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":442255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":442795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724587,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1033995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:10:02.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 -- 172.21.15.100:0/3705037981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd328104f70 msgr2=0x7fd32810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/3705037981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd328104f70 0x7fd32810b800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd320009ca0 tx=0x7fd32002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 -- 172.21.15.100:0/3705037981 shutdown_connections 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/3705037981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd328104f70 0x7fd32810b800 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/3705037981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3281045b0 0x7fd328104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/3705037981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3281033c0 0x7fd3281037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 -- 172.21.15.100:0/3705037981 >> 172.21.15.100:0/3705037981 conn(0x7fd3280fe970 msgr2=0x7fd328100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 -- 172.21.15.100:0/3705037981 shutdown_connections 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 -- 172.21.15.100:0/3705037981 wait complete. 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 Processor -- start 2024-03-12T05:10:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.782+0000 7fd32ff2c640 1 -- start start 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3281033c0 0x7fd3281994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 0x7fd3281999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd328104f70 0x7fd3281a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3281a0fa0 con 0x7fd3281045b0 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3281a1110 con 0x7fd3281033c0 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3281a12d0 con 0x7fd328104f70 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 0x7fd3281999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 0x7fd3281999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36098/0 (socket says 172.21.15.100:36098) 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 -- 172.21.15.100:0/95048837 learned_addr learned my addr 172.21.15.100:0/95048837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32e4c9640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd328104f70 0x7fd3281a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 -- 172.21.15.100:0/95048837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd328104f70 msgr2=0x7fd3281a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd328104f70 0x7fd3281a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 -- 172.21.15.100:0/95048837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3281033c0 msgr2=0x7fd3281994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3281033c0 0x7fd3281994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 -- 172.21.15.100:0/95048837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320009900 con 0x7fd3281045b0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32d4c7640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 0x7fd3281999f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd31800b4a0 tx=0x7fd31800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318010610 con 0x7fd3281045b0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd318010c50 con 0x7fd3281045b0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3281a15b0 con 0x7fd3281045b0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318017e20 con 0x7fd3281045b0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.783+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3281a1ac0 con 0x7fd3281045b0 2024-03-12T05:10:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.784+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32810a6e0 con 0x7fd3281045b0 2024-03-12T05:10:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.785+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fd318017480 con 0x7fd3281045b0 2024-03-12T05:10:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.786+0000 7fd31effd640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd3140a1a30 0x7fd3140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.786+0000 7fd32dcc8640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd3140a1a30 0x7fd3140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.786+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3180cc9b0 con 0x7fd3281045b0 2024-03-12T05:10:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.786+0000 7fd32dcc8640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd3140a1a30 0x7fd3140a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd328104410 tx=0x7fd324005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.790+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd318095d10 con 0x7fd3281045b0 2024-03-12T05:10:02.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.959+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd32802ceb0 con 0x7fd3140a1a30 2024-03-12T05:10:02.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.968+0000 7fd31effd640 1 -- 172.21.15.100:0/95048837 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258725 (secure 0 0 0) 0x7fd32802ceb0 con 0x7fd3140a1a30 2024-03-12T05:10:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:02.971 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:10:02.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd3140a1a30 msgr2=0x7fd3140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd3140a1a30 0x7fd3140a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd328104410 tx=0x7fd324005ed0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 msgr2=0x7fd3281999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:02.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 0x7fd3281999f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd31800b4a0 tx=0x7fd31800b960 comp rx=0 tx=0).stop 2024-03-12T05:10:02.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 shutdown_connections 2024-03-12T05:10:02.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd3140a1a30 0x7fd3140a3ef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd328104f70 0x7fd3281a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3281045b0 0x7fd3281999f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 --2- 172.21.15.100:0/95048837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3281033c0 0x7fd3281994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:02.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 >> 172.21.15.100:0/95048837 conn(0x7fd3280fe970 msgr2=0x7fd3281078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:02.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 shutdown_connections 2024-03-12T05:10:02.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:02.971+0000 7fd32ff2c640 1 -- 172.21.15.100:0/95048837 wait complete. 2024-03-12T05:10:02.986 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-03-12T05:10:01.710706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614709,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":155,"num_read_kb":135,"num_write":287,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":277,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374384,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567440,"statfs":{"total":1151940427776,"available":1151557058560,"internally_reserved":0,"allocated":13037568,"data_stored":7090262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24341,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":106,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.783319"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568924+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:09:56.568924+0000","last_peered":"2024-03-12T05:09:56.568924+0000","last_clean":"2024-03-12T05:09:56.568924+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:09:56.568924+0000","last_undegraded":"2024-03-12T05:09:56.568924+0000","last_fullsized":"2024-03-12T05:09:56.568924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568996+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:09:56.568996+0000","last_peered":"2024-03-12T05:09:56.568996+0000","last_clean":"2024-03-12T05:09:56.568996+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:09:56.568996+0000","last_undegraded":"2024-03-12T05:09:56.568996+0000","last_fullsized":"2024-03-12T05:09:56.568996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568759+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:09:56.568759+0000","last_peered":"2024-03-12T05:09:56.568759+0000","last_clean":"2024-03-12T05:09:56.568759+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:09:56.568759+0000","last_undegraded":"2024-03-12T05:09:56.568759+0000","last_fullsized":"2024-03-12T05:09:56.568759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569233+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:09:56.569233+0000","last_peered":"2024-03-12T05:09:56.569233+0000","last_clean":"2024-03-12T05:09:56.569233+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:09:56.569233+0000","last_undegraded":"2024-03-12T05:09:56.569233+0000","last_fullsized":"2024-03-12T05:09:56.569233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932439+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:09:55.932439+0000","last_peered":"2024-03-12T05:09:55.932439+0000","last_clean":"2024-03-12T05:09:55.932439+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:09:55.932439+0000","last_undegraded":"2024-03-12T05:09:55.932439+0000","last_fullsized":"2024-03-12T05:09:55.932439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569309+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:09:56.569309+0000","last_peered":"2024-03-12T05:09:56.569309+0000","last_clean":"2024-03-12T05:09:56.569309+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:09:56.569309+0000","last_undegraded":"2024-03-12T05:09:56.569309+0000","last_fullsized":"2024-03-12T05:09:56.569309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182884+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:09:56.182884+0000","last_peered":"2024-03-12T05:09:56.182884+0000","last_clean":"2024-03-12T05:09:56.182884+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:09:56.182884+0000","last_undegraded":"2024-03-12T05:09:56.182884+0000","last_fullsized":"2024-03-12T05:09:56.182884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690432+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:09:55.690432+0000","last_peered":"2024-03-12T05:09:55.690432+0000","last_clean":"2024-03-12T05:09:55.690432+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:09:55.690432+0000","last_undegraded":"2024-03-12T05:09:55.690432+0000","last_fullsized":"2024-03-12T05:09:55.690432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691522+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:09:55.691522+0000","last_peered":"2024-03-12T05:09:55.691522+0000","last_clean":"2024-03-12T05:09:55.691522+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:09:55.691522+0000","last_undegraded":"2024-03-12T05:09:55.691522+0000","last_fullsized":"2024-03-12T05:09:55.691522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569564+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:09:56.569564+0000","last_peered":"2024-03-12T05:09:56.569564+0000","last_clean":"2024-03-12T05:09:56.569564+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:09:56.569564+0000","last_undegraded":"2024-03-12T05:09:56.569564+0000","last_fullsized":"2024-03-12T05:09:56.569564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183238+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:09:56.183238+0000","last_peered":"2024-03-12T05:09:56.183238+0000","last_clean":"2024-03-12T05:09:56.183238+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:09:56.183238+0000","last_undegraded":"2024-03-12T05:09:56.183238+0000","last_fullsized":"2024-03-12T05:09:56.183238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569036+0000","last_change":"2024-03-12T05:09:54.703962+0000","last_active":"2024-03-12T05:09:56.569036+0000","last_peered":"2024-03-12T05:09:56.569036+0000","last_clean":"2024-03-12T05:09:56.569036+0000","last_became_active":"2024-03-12T05:09:54.702987+0000","last_became_peered":"2024-03-12T05:09:54.702987+0000","last_unstale":"2024-03-12T05:09:56.569036+0000","last_undegraded":"2024-03-12T05:09:56.569036+0000","last_fullsized":"2024-03-12T05:09:56.569036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:55:37.300456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568240+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:09:56.568240+0000","last_peered":"2024-03-12T05:09:56.568240+0000","last_clean":"2024-03-12T05:09:56.568240+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:09:56.568240+0000","last_undegraded":"2024-03-12T05:09:56.568240+0000","last_fullsized":"2024-03-12T05:09:56.568240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568523+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:09:56.568523+0000","last_peered":"2024-03-12T05:09:56.568523+0000","last_clean":"2024-03-12T05:09:56.568523+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:09:56.568523+0000","last_undegraded":"2024-03-12T05:09:56.568523+0000","last_fullsized":"2024-03-12T05:09:56.568523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568340+0000","last_change":"2024-03-12T05:09:54.712692+0000","last_active":"2024-03-12T05:09:56.568340+0000","last_peered":"2024-03-12T05:09:56.568340+0000","last_clean":"2024-03-12T05:09:56.568340+0000","last_became_active":"2024-03-12T05:09:54.712362+0000","last_became_peered":"2024-03-12T05:09:54.712362+0000","last_unstale":"2024-03-12T05:09:56.568340+0000","last_undegraded":"2024-03-12T05:09:56.568340+0000","last_fullsized":"2024-03-12T05:09:56.568340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:46:50.828861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691030+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:09:55.691030+0000","last_peered":"2024-03-12T05:09:55.691030+0000","last_clean":"2024-03-12T05:09:55.691030+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:09:55.691030+0000","last_undegraded":"2024-03-12T05:09:55.691030+0000","last_fullsized":"2024-03-12T05:09:55.691030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569150+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:09:56.569150+0000","last_peered":"2024-03-12T05:09:56.569150+0000","last_clean":"2024-03-12T05:09:56.569150+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:09:56.569150+0000","last_undegraded":"2024-03-12T05:09:56.569150+0000","last_fullsized":"2024-03-12T05:09:56.569150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738891+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:09:55.738891+0000","last_peered":"2024-03-12T05:09:55.738891+0000","last_clean":"2024-03-12T05:09:55.738891+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:09:55.738891+0000","last_undegraded":"2024-03-12T05:09:55.738891+0000","last_fullsized":"2024-03-12T05:09:55.738891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569317+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:09:56.569317+0000","last_peered":"2024-03-12T05:09:56.569317+0000","last_clean":"2024-03-12T05:09:56.569317+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:09:56.569317+0000","last_undegraded":"2024-03-12T05:09:56.569317+0000","last_fullsized":"2024-03-12T05:09:56.569317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738880+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:09:55.738880+0000","last_peered":"2024-03-12T05:09:55.738880+0000","last_clean":"2024-03-12T05:09:55.738880+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:09:55.738880+0000","last_undegraded":"2024-03-12T05:09:55.738880+0000","last_fullsized":"2024-03-12T05:09:55.738880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690994+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:09:55.690994+0000","last_peered":"2024-03-12T05:09:55.690994+0000","last_clean":"2024-03-12T05:09:55.690994+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:09:55.690994+0000","last_undegraded":"2024-03-12T05:09:55.690994+0000","last_fullsized":"2024-03-12T05:09:55.690994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691091+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:09:55.691091+0000","last_peered":"2024-03-12T05:09:55.691091+0000","last_clean":"2024-03-12T05:09:55.691091+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:09:55.691091+0000","last_undegraded":"2024-03-12T05:09:55.691091+0000","last_fullsized":"2024-03-12T05:09:55.691091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568815+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:09:56.568815+0000","last_peered":"2024-03-12T05:09:56.568815+0000","last_clean":"2024-03-12T05:09:56.568815+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:09:56.568815+0000","last_undegraded":"2024-03-12T05:09:56.568815+0000","last_fullsized":"2024-03-12T05:09:56.568815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.961167+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:09:58.961167+0000","last_peered":"2024-03-12T05:09:58.961167+0000","last_clean":"2024-03-12T05:09:58.961167+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:09:58.961167+0000","last_undegraded":"2024-03-12T05:09:58.961167+0000","last_fullsized":"2024-03-12T05:09:58.961167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691317+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:09:55.691317+0000","last_peered":"2024-03-12T05:09:55.691317+0000","last_clean":"2024-03-12T05:09:55.691317+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:09:55.691317+0000","last_undegraded":"2024-03-12T05:09:55.691317+0000","last_fullsized":"2024-03-12T05:09:55.691317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932705+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:09:55.932705+0000","last_peered":"2024-03-12T05:09:55.932705+0000","last_clean":"2024-03-12T05:09:55.932705+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:09:55.932705+0000","last_undegraded":"2024-03-12T05:09:55.932705+0000","last_fullsized":"2024-03-12T05:09:55.932705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.692097+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:09:55.692097+0000","last_peered":"2024-03-12T05:09:55.692097+0000","last_clean":"2024-03-12T05:09:55.692097+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:09:55.692097+0000","last_undegraded":"2024-03-12T05:09:55.692097+0000","last_fullsized":"2024-03-12T05:09:55.692097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.953353+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:09:57.953353+0000","last_peered":"2024-03-12T05:09:57.953353+0000","last_clean":"2024-03-12T05:09:57.953353+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:09:57.953353+0000","last_undegraded":"2024-03-12T05:09:57.953353+0000","last_fullsized":"2024-03-12T05:09:57.953353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932663+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:09:55.932663+0000","last_peered":"2024-03-12T05:09:55.932663+0000","last_clean":"2024-03-12T05:09:55.932663+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:09:55.932663+0000","last_undegraded":"2024-03-12T05:09:55.932663+0000","last_fullsized":"2024-03-12T05:09:55.932663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.959417+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:09:58.959417+0000","last_peered":"2024-03-12T05:09:58.959417+0000","last_clean":"2024-03-12T05:09:58.959417+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:09:58.959417+0000","last_undegraded":"2024-03-12T05:09:58.959417+0000","last_fullsized":"2024-03-12T05:09:58.959417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691832+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:09:55.691832+0000","last_peered":"2024-03-12T05:09:55.691832+0000","last_clean":"2024-03-12T05:09:55.691832+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:09:55.691832+0000","last_undegraded":"2024-03-12T05:09:55.691832+0000","last_fullsized":"2024-03-12T05:09:55.691832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.961325+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:09:58.961325+0000","last_peered":"2024-03-12T05:09:58.961325+0000","last_clean":"2024-03-12T05:09:58.961325+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:09:58.961325+0000","last_undegraded":"2024-03-12T05:09:58.961325+0000","last_fullsized":"2024-03-12T05:09:58.961325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690492+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:09:55.690492+0000","last_peered":"2024-03-12T05:09:55.690492+0000","last_clean":"2024-03-12T05:09:55.690492+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:09:55.690492+0000","last_undegraded":"2024-03-12T05:09:55.690492+0000","last_fullsized":"2024-03-12T05:09:55.690492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691737+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:09:55.691737+0000","last_peered":"2024-03-12T05:09:55.691737+0000","last_clean":"2024-03-12T05:09:55.691737+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:09:55.691737+0000","last_undegraded":"2024-03-12T05:09:55.691737+0000","last_fullsized":"2024-03-12T05:09:55.691737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738733+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:09:55.738733+0000","last_peered":"2024-03-12T05:09:55.738733+0000","last_clean":"2024-03-12T05:09:55.738733+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:09:55.738733+0000","last_undegraded":"2024-03-12T05:09:55.738733+0000","last_fullsized":"2024-03-12T05:09:55.738733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952664+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:09:57.952664+0000","last_peered":"2024-03-12T05:09:57.952664+0000","last_clean":"2024-03-12T05:09:57.952664+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:09:57.952664+0000","last_undegraded":"2024-03-12T05:09:57.952664+0000","last_fullsized":"2024-03-12T05:09:57.952664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691424+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:09:55.691424+0000","last_peered":"2024-03-12T05:09:55.691424+0000","last_clean":"2024-03-12T05:09:55.691424+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:09:55.691424+0000","last_undegraded":"2024-03-12T05:09:55.691424+0000","last_fullsized":"2024-03-12T05:09:55.691424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.951457+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:09:57.951457+0000","last_peered":"2024-03-12T05:09:57.951457+0000","last_clean":"2024-03-12T05:09:57.951457+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:09:57.951457+0000","last_undegraded":"2024-03-12T05:09:57.951457+0000","last_fullsized":"2024-03-12T05:09:57.951457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568914+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:09:56.568914+0000","last_peered":"2024-03-12T05:09:56.568914+0000","last_clean":"2024-03-12T05:09:56.568914+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:09:56.568914+0000","last_undegraded":"2024-03-12T05:09:56.568914+0000","last_fullsized":"2024-03-12T05:09:56.568914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.692170+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:09:55.692170+0000","last_peered":"2024-03-12T05:09:55.692170+0000","last_clean":"2024-03-12T05:09:55.692170+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:09:55.692170+0000","last_undegraded":"2024-03-12T05:09:55.692170+0000","last_fullsized":"2024-03-12T05:09:55.692170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691408+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:09:55.691408+0000","last_peered":"2024-03-12T05:09:55.691408+0000","last_clean":"2024-03-12T05:09:55.691408+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:09:55.691408+0000","last_undegraded":"2024-03-12T05:09:55.691408+0000","last_fullsized":"2024-03-12T05:09:55.691408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:10:00.661217+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:10:00.661217+0000","last_peered":"2024-03-12T05:10:00.661217+0000","last_clean":"2024-03-12T05:10:00.661217+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:10:00.661217+0000","last_undegraded":"2024-03-12T05:10:00.661217+0000","last_fullsized":"2024-03-12T05:10:00.661217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932675+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:09:55.932675+0000","last_peered":"2024-03-12T05:09:55.932675+0000","last_clean":"2024-03-12T05:09:55.932675+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:09:55.932675+0000","last_undegraded":"2024-03-12T05:09:55.932675+0000","last_fullsized":"2024-03-12T05:09:55.932675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569197+0000","last_change":"2024-03-12T05:09:53.945301+0000","last_active":"2024-03-12T05:09:56.569197+0000","last_peered":"2024-03-12T05:09:56.569197+0000","last_clean":"2024-03-12T05:09:56.569197+0000","last_became_active":"2024-03-12T05:09:53.944944+0000","last_became_peered":"2024-03-12T05:09:53.944944+0000","last_unstale":"2024-03-12T05:09:56.569197+0000","last_undegraded":"2024-03-12T05:09:56.569197+0000","last_fullsized":"2024-03-12T05:09:56.569197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:23:08.438529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568052+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:09:56.568052+0000","last_peered":"2024-03-12T05:09:56.568052+0000","last_clean":"2024-03-12T05:09:56.568052+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:09:56.568052+0000","last_undegraded":"2024-03-12T05:09:56.568052+0000","last_fullsized":"2024-03-12T05:09:56.568052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952888+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:09:57.952888+0000","last_peered":"2024-03-12T05:09:57.952888+0000","last_clean":"2024-03-12T05:09:57.952888+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:09:57.952888+0000","last_undegraded":"2024-03-12T05:09:57.952888+0000","last_fullsized":"2024-03-12T05:09:57.952888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183214+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:09:56.183214+0000","last_peered":"2024-03-12T05:09:56.183214+0000","last_clean":"2024-03-12T05:09:56.183214+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:09:56.183214+0000","last_undegraded":"2024-03-12T05:09:56.183214+0000","last_fullsized":"2024-03-12T05:09:56.183214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:10:00.974118+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:10:00.974118+0000","last_peered":"2024-03-12T05:10:00.974118+0000","last_clean":"2024-03-12T05:10:00.974118+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:10:00.974118+0000","last_undegraded":"2024-03-12T05:10:00.974118+0000","last_fullsized":"2024-03-12T05:10:00.974118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182920+0000","last_change":"2024-03-12T05:09:54.714174+0000","last_active":"2024-03-12T05:09:56.182920+0000","last_peered":"2024-03-12T05:09:56.182920+0000","last_clean":"2024-03-12T05:09:56.182920+0000","last_became_active":"2024-03-12T05:09:54.713866+0000","last_became_peered":"2024-03-12T05:09:54.713866+0000","last_unstale":"2024-03-12T05:09:56.182920+0000","last_undegraded":"2024-03-12T05:09:56.182920+0000","last_fullsized":"2024-03-12T05:09:56.182920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:48:18.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.961841+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:09:58.961841+0000","last_peered":"2024-03-12T05:09:58.961841+0000","last_clean":"2024-03-12T05:09:58.961841+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:09:58.961841+0000","last_undegraded":"2024-03-12T05:09:58.961841+0000","last_fullsized":"2024-03-12T05:09:58.961841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691853+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:09:55.691853+0000","last_peered":"2024-03-12T05:09:55.691853+0000","last_clean":"2024-03-12T05:09:55.691853+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:09:55.691853+0000","last_undegraded":"2024-03-12T05:09:55.691853+0000","last_fullsized":"2024-03-12T05:09:55.691853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932926+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:09:55.932926+0000","last_peered":"2024-03-12T05:09:55.932926+0000","last_clean":"2024-03-12T05:09:55.932926+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:09:55.932926+0000","last_undegraded":"2024-03-12T05:09:55.932926+0000","last_fullsized":"2024-03-12T05:09:55.932926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568768+0000","last_change":"2024-03-12T05:09:54.709959+0000","last_active":"2024-03-12T05:09:56.568768+0000","last_peered":"2024-03-12T05:09:56.568768+0000","last_clean":"2024-03-12T05:09:56.568768+0000","last_became_active":"2024-03-12T05:09:54.709628+0000","last_became_peered":"2024-03-12T05:09:54.709628+0000","last_unstale":"2024-03-12T05:09:56.568768+0000","last_undegraded":"2024-03-12T05:09:56.568768+0000","last_fullsized":"2024-03-12T05:09:56.568768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:39:54.818675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.738771+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:09:55.738771+0000","last_peered":"2024-03-12T05:09:55.738771+0000","last_clean":"2024-03-12T05:09:55.738771+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:09:55.738771+0000","last_undegraded":"2024-03-12T05:09:55.738771+0000","last_fullsized":"2024-03-12T05:09:55.738771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569283+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:09:56.569283+0000","last_peered":"2024-03-12T05:09:56.569283+0000","last_clean":"2024-03-12T05:09:56.569283+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:09:56.569283+0000","last_undegraded":"2024-03-12T05:09:56.569283+0000","last_fullsized":"2024-03-12T05:09:56.569283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182849+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:09:56.182849+0000","last_peered":"2024-03-12T05:09:56.182849+0000","last_clean":"2024-03-12T05:09:56.182849+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:09:56.182849+0000","last_undegraded":"2024-03-12T05:09:56.182849+0000","last_fullsized":"2024-03-12T05:09:56.182849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569316+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:09:56.569316+0000","last_peered":"2024-03-12T05:09:56.569316+0000","last_clean":"2024-03-12T05:09:56.569316+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:09:56.569316+0000","last_undegraded":"2024-03-12T05:09:56.569316+0000","last_fullsized":"2024-03-12T05:09:56.569316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691962+0000","last_change":"2024-03-12T05:09:54.706799+0000","last_active":"2024-03-12T05:09:55.691962+0000","last_peered":"2024-03-12T05:09:55.691962+0000","last_clean":"2024-03-12T05:09:55.691962+0000","last_became_active":"2024-03-12T05:09:54.706491+0000","last_became_peered":"2024-03-12T05:09:54.706491+0000","last_unstale":"2024-03-12T05:09:55.691962+0000","last_undegraded":"2024-03-12T05:09:55.691962+0000","last_fullsized":"2024-03-12T05:09:55.691962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:28:09.404200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:58.962577+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:09:58.962577+0000","last_peered":"2024-03-12T05:09:58.962577+0000","last_clean":"2024-03-12T05:09:58.962577+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:09:58.962577+0000","last_undegraded":"2024-03-12T05:09:58.962577+0000","last_fullsized":"2024-03-12T05:09:58.962577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690742+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:09:55.690742+0000","last_peered":"2024-03-12T05:09:55.690742+0000","last_clean":"2024-03-12T05:09:55.690742+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:09:55.690742+0000","last_undegraded":"2024-03-12T05:09:55.690742+0000","last_fullsized":"2024-03-12T05:09:55.690742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:59.967508+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:09:59.967508+0000","last_peered":"2024-03-12T05:09:59.967508+0000","last_clean":"2024-03-12T05:09:59.967508+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:09:59.967508+0000","last_undegraded":"2024-03-12T05:09:59.967508+0000","last_fullsized":"2024-03-12T05:09:59.967508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.182566+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:09:56.182566+0000","last_peered":"2024-03-12T05:09:56.182566+0000","last_clean":"2024-03-12T05:09:56.182566+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:09:56.182566+0000","last_undegraded":"2024-03-12T05:09:56.182566+0000","last_fullsized":"2024-03-12T05:09:56.182566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.567835+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:09:56.567835+0000","last_peered":"2024-03-12T05:09:56.567835+0000","last_clean":"2024-03-12T05:09:56.567835+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:09:56.567835+0000","last_undegraded":"2024-03-12T05:09:56.567835+0000","last_fullsized":"2024-03-12T05:09:56.567835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183079+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:09:56.183079+0000","last_peered":"2024-03-12T05:09:56.183079+0000","last_clean":"2024-03-12T05:09:56.183079+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:09:56.183079+0000","last_undegraded":"2024-03-12T05:09:56.183079+0000","last_fullsized":"2024-03-12T05:09:56.183079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.950856+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:09:57.950856+0000","last_peered":"2024-03-12T05:09:57.950856+0000","last_clean":"2024-03-12T05:09:57.950856+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:09:57.950856+0000","last_undegraded":"2024-03-12T05:09:57.950856+0000","last_fullsized":"2024-03-12T05:09:57.950856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568333+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:09:56.568333+0000","last_peered":"2024-03-12T05:09:56.568333+0000","last_clean":"2024-03-12T05:09:56.568333+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:09:56.568333+0000","last_undegraded":"2024-03-12T05:09:56.568333+0000","last_fullsized":"2024-03-12T05:09:56.568333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.931568+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:09:55.931568+0000","last_peered":"2024-03-12T05:09:55.931568+0000","last_clean":"2024-03-12T05:09:55.931568+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:09:55.931568+0000","last_undegraded":"2024-03-12T05:09:55.931568+0000","last_fullsized":"2024-03-12T05:09:55.931568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691046+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:09:55.691046+0000","last_peered":"2024-03-12T05:09:55.691046+0000","last_clean":"2024-03-12T05:09:55.691046+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:09:55.691046+0000","last_undegraded":"2024-03-12T05:09:55.691046+0000","last_fullsized":"2024-03-12T05:09:55.691046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.951151+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:09:57.951151+0000","last_peered":"2024-03-12T05:09:57.951151+0000","last_clean":"2024-03-12T05:09:57.951151+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:09:57.951151+0000","last_undegraded":"2024-03-12T05:09:57.951151+0000","last_fullsized":"2024-03-12T05:09:57.951151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569016+0000","last_change":"2024-03-12T05:09:54.709130+0000","last_active":"2024-03-12T05:09:56.569016+0000","last_peered":"2024-03-12T05:09:56.569016+0000","last_clean":"2024-03-12T05:09:56.569016+0000","last_became_active":"2024-03-12T05:09:54.708690+0000","last_became_peered":"2024-03-12T05:09:54.708690+0000","last_unstale":"2024-03-12T05:09:56.569016+0000","last_undegraded":"2024-03-12T05:09:56.569016+0000","last_fullsized":"2024-03-12T05:09:56.569016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:04.765154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952186+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:09:57.952186+0000","last_peered":"2024-03-12T05:09:57.952186+0000","last_clean":"2024-03-12T05:09:57.952186+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:09:57.952186+0000","last_undegraded":"2024-03-12T05:09:57.952186+0000","last_fullsized":"2024-03-12T05:09:57.952186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.932292+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:09:55.932292+0000","last_peered":"2024-03-12T05:09:55.932292+0000","last_clean":"2024-03-12T05:09:55.932292+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:09:55.932292+0000","last_undegraded":"2024-03-12T05:09:55.932292+0000","last_fullsized":"2024-03-12T05:09:55.932292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690526+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:09:55.690526+0000","last_peered":"2024-03-12T05:09:55.690526+0000","last_clean":"2024-03-12T05:09:55.690526+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:09:55.690526+0000","last_undegraded":"2024-03-12T05:09:55.690526+0000","last_fullsized":"2024-03-12T05:09:55.690526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568547+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:09:56.568547+0000","last_peered":"2024-03-12T05:09:56.568547+0000","last_clean":"2024-03-12T05:09:56.568547+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:09:56.568547+0000","last_undegraded":"2024-03-12T05:09:56.568547+0000","last_fullsized":"2024-03-12T05:09:56.568547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569020+0000","last_change":"2024-03-12T05:09:53.942272+0000","last_active":"2024-03-12T05:09:56.569020+0000","last_peered":"2024-03-12T05:09:56.569020+0000","last_clean":"2024-03-12T05:09:56.569020+0000","last_became_active":"2024-03-12T05:09:53.941874+0000","last_became_peered":"2024-03-12T05:09:53.941874+0000","last_unstale":"2024-03-12T05:09:56.569020+0000","last_undegraded":"2024-03-12T05:09:56.569020+0000","last_fullsized":"2024-03-12T05:09:56.569020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:40:21.770248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690805+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:09:55.690805+0000","last_peered":"2024-03-12T05:09:55.690805+0000","last_clean":"2024-03-12T05:09:55.690805+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:09:55.690805+0000","last_undegraded":"2024-03-12T05:09:55.690805+0000","last_fullsized":"2024-03-12T05:09:55.690805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:57.952626+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:09:57.952626+0000","last_peered":"2024-03-12T05:09:57.952626+0000","last_clean":"2024-03-12T05:09:57.952626+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:09:57.952626+0000","last_undegraded":"2024-03-12T05:09:57.952626+0000","last_fullsized":"2024-03-12T05:09:57.952626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183233+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:09:56.183233+0000","last_peered":"2024-03-12T05:09:56.183233+0000","last_clean":"2024-03-12T05:09:56.183233+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:09:56.183233+0000","last_undegraded":"2024-03-12T05:09:56.183233+0000","last_fullsized":"2024-03-12T05:09:56.183233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569374+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:09:56.569374+0000","last_peered":"2024-03-12T05:09:56.569374+0000","last_clean":"2024-03-12T05:09:56.569374+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:09:56.569374+0000","last_undegraded":"2024-03-12T05:09:56.569374+0000","last_fullsized":"2024-03-12T05:09:56.569374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183256+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:09:56.183256+0000","last_peered":"2024-03-12T05:09:56.183256+0000","last_clean":"2024-03-12T05:09:56.183256+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:09:56.183256+0000","last_undegraded":"2024-03-12T05:09:56.183256+0000","last_fullsized":"2024-03-12T05:09:56.183256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691954+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:09:55.691954+0000","last_peered":"2024-03-12T05:09:55.691954+0000","last_clean":"2024-03-12T05:09:55.691954+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:09:55.691954+0000","last_undegraded":"2024-03-12T05:09:55.691954+0000","last_fullsized":"2024-03-12T05:09:55.691954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183198+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:09:56.183198+0000","last_peered":"2024-03-12T05:09:56.183198+0000","last_clean":"2024-03-12T05:09:56.183198+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:09:56.183198+0000","last_undegraded":"2024-03-12T05:09:56.183198+0000","last_fullsized":"2024-03-12T05:09:56.183198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568166+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:09:56.568166+0000","last_peered":"2024-03-12T05:09:56.568166+0000","last_clean":"2024-03-12T05:09:56.568166+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:09:56.568166+0000","last_undegraded":"2024-03-12T05:09:56.568166+0000","last_fullsized":"2024-03-12T05:09:56.568166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.183229+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:09:56.183229+0000","last_peered":"2024-03-12T05:09:56.183229+0000","last_clean":"2024-03-12T05:09:56.183229+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:09:56.183229+0000","last_undegraded":"2024-03-12T05:09:56.183229+0000","last_fullsized":"2024-03-12T05:09:56.183229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568623+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:09:56.568623+0000","last_peered":"2024-03-12T05:09:56.568623+0000","last_clean":"2024-03-12T05:09:56.568623+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:09:56.568623+0000","last_undegraded":"2024-03-12T05:09:56.568623+0000","last_fullsized":"2024-03-12T05:09:56.568623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569212+0000","last_change":"2024-03-12T05:09:54.707255+0000","last_active":"2024-03-12T05:09:56.569212+0000","last_peered":"2024-03-12T05:09:56.569212+0000","last_clean":"2024-03-12T05:09:56.569212+0000","last_became_active":"2024-03-12T05:09:54.706932+0000","last_became_peered":"2024-03-12T05:09:54.706932+0000","last_unstale":"2024-03-12T05:09:56.569212+0000","last_undegraded":"2024-03-12T05:09:56.569212+0000","last_fullsized":"2024-03-12T05:09:56.569212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:27:00.222615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569026+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:09:56.569026+0000","last_peered":"2024-03-12T05:09:56.569026+0000","last_clean":"2024-03-12T05:09:56.569026+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:09:56.569026+0000","last_undegraded":"2024-03-12T05:09:56.569026+0000","last_fullsized":"2024-03-12T05:09:56.569026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568670+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:09:56.568670+0000","last_peered":"2024-03-12T05:09:56.568670+0000","last_clean":"2024-03-12T05:09:56.568670+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:09:56.568670+0000","last_undegraded":"2024-03-12T05:09:56.568670+0000","last_fullsized":"2024-03-12T05:09:56.568670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568114+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:09:56.568114+0000","last_peered":"2024-03-12T05:09:56.568114+0000","last_clean":"2024-03-12T05:09:56.568114+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:09:56.568114+0000","last_undegraded":"2024-03-12T05:09:56.568114+0000","last_fullsized":"2024-03-12T05:09:56.568114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568761+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:09:56.568761+0000","last_peered":"2024-03-12T05:09:56.568761+0000","last_clean":"2024-03-12T05:09:56.568761+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:09:56.568761+0000","last_undegraded":"2024-03-12T05:09:56.568761+0000","last_fullsized":"2024-03-12T05:09:56.568761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.569470+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:09:56.569470+0000","last_peered":"2024-03-12T05:09:56.569470+0000","last_clean":"2024-03-12T05:09:56.569470+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:09:56.569470+0000","last_undegraded":"2024-03-12T05:09:56.569470+0000","last_fullsized":"2024-03-12T05:09:56.569470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691063+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:09:55.691063+0000","last_peered":"2024-03-12T05:09:55.691063+0000","last_clean":"2024-03-12T05:09:55.691063+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:09:55.691063+0000","last_undegraded":"2024-03-12T05:09:55.691063+0000","last_fullsized":"2024-03-12T05:09:55.691063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.692022+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:09:55.692022+0000","last_peered":"2024-03-12T05:09:55.692022+0000","last_clean":"2024-03-12T05:09:55.692022+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:09:55.692022+0000","last_undegraded":"2024-03-12T05:09:55.692022+0000","last_fullsized":"2024-03-12T05:09:55.692022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.690518+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:09:55.690518+0000","last_peered":"2024-03-12T05:09:55.690518+0000","last_clean":"2024-03-12T05:09:55.690518+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:09:55.690518+0000","last_undegraded":"2024-03-12T05:09:55.690518+0000","last_fullsized":"2024-03-12T05:09:55.690518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:55.691151+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:09:55.691151+0000","last_peered":"2024-03-12T05:09:55.691151+0000","last_clean":"2024-03-12T05:09:55.691151+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:09:55.691151+0000","last_undegraded":"2024-03-12T05:09:55.691151+0000","last_fullsized":"2024-03-12T05:09:55.691151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568256+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:09:56.568256+0000","last_peered":"2024-03-12T05:09:56.568256+0000","last_clean":"2024-03-12T05:09:56.568256+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:09:56.568256+0000","last_undegraded":"2024-03-12T05:09:56.568256+0000","last_fullsized":"2024-03-12T05:09:56.568256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-12T05:09:56.568844+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:09:56.568844+0000","last_peered":"2024-03-12T05:09:56.568844+0000","last_clean":"2024-03-12T05:09:56.568844+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:09:56.568844+0000","last_undegraded":"2024-03-12T05:09:56.568844+0000","last_fullsized":"2024-03-12T05:09:56.568844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24224,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":105,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":33411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1035709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1033723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40408,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704744,"statfs":{"total":95995035648,"available":95953657856,"internally_reserved":0,"allocated":933888,"data_stored":443763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":442255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":442795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724587,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1033995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:10:02.990 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-12T05:10:02.990 INFO:tasks.cephadm:Setting up client nodes... 2024-03-12T05:10:02.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-12T05:10:03.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:02 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4072375123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:10:03.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:02 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/102583328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:03.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:02 smithi161 ceph-mon[36043]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:10:03.266 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:03.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:02 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4072375123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:10:03.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:02 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/102583328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:03.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:02 smithi100 ceph-mon[32210]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:10:03.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:02 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4072375123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:10:03.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:02 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/102583328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:03.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:02 smithi156 ceph-mon[36270]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-12T05:10:04.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-12T05:10:04.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-12T05:10:04.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-12T05:10:04.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:03 smithi100 ceph-mon[32210]: pgmap v286: 97 pgs: 97 active+clean; 601 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-12T05:10:04.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-12T05:10:04.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:03 smithi156 ceph-mon[36270]: pgmap v286: 97 pgs: 97 active+clean; 601 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-12T05:10:04.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:10:04.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-03-12T05:10:04.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-12T05:10:04.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-12T05:10:04.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:03 smithi161 ceph-mon[36043]: pgmap v286: 97 pgs: 97 active+clean; 601 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 -- 172.21.15.100:0/1385127875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 msgr2=0x7f8cc4103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/1385127875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4103340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00099c0 tx=0x7f8cb0009cd0 comp rx=0 tx=0).stop 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 -- 172.21.15.100:0/1385127875 shutdown_connections 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/1385127875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc4101ce0 0x7f8cc4102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/1385127875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc4103880 0x7f8cc410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/1385127875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4103340 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 -- 172.21.15.100:0/1385127875 >> 172.21.15.100:0/1385127875 conn(0x7f8cc40fd280 msgr2=0x7f8cc40ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:04.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 -- 172.21.15.100:0/1385127875 shutdown_connections 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.844+0000 7f8ccadaf700 1 -- 172.21.15.100:0/1385127875 wait complete. 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 Processor -- start 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 -- start start 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc4101ce0 0x7f8cc4075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc4103880 0x7f8cc40745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4074d20 con 0x7f8cc4102ee0 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4074e90 con 0x7f8cc4101ce0 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8ccadaf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc41a1c10 con 0x7f8cc4103880 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36126/0 (socket says 172.21.15.100:36126) 2024-03-12T05:10:04.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 -- 172.21.15.100:0/4255151786 learned_addr learned my addr 172.21.15.100:0/4255151786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc8b4b700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc4101ce0 0x7f8cc4075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 -- 172.21.15.100:0/4255151786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc4103880 msgr2=0x7f8cc40745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc4103880 0x7f8cc40745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 -- 172.21.15.100:0/4255151786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc4101ce0 msgr2=0x7f8cc4075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc4101ce0 0x7f8cc4075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 -- 172.21.15.100:0/4255151786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00095f0 con 0x7f8cc4102ee0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.845+0000 7f8cc3fff700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4074060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0003790 tx=0x7f8cb0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.846+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb001d070 con 0x7f8cc4102ee0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.846+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0004110 con 0x7f8cc4102ee0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.846+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0022da0 con 0x7f8cc4102ee0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.846+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc41a1e90 con 0x7f8cc4102ee0 2024-03-12T05:10:04.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.846+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc41a2250 con 0x7f8cc4102ee0 2024-03-12T05:10:04.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.847+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc4108fd0 con 0x7f8cc4102ee0 2024-03-12T05:10:04.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.848+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f8cb0022470 con 0x7f8cc4102ee0 2024-03-12T05:10:04.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.848+0000 7f8cc1ffb700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8cb40a1c00 0x7f8cb40a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:04.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.848+0000 7f8cc8b4b700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8cb40a1c00 0x7f8cb40a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:04.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.848+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8cb00ce7c0 con 0x7f8cc4102ee0 2024-03-12T05:10:04.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.849+0000 7f8cc8b4b700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8cb40a1c00 0x7f8cb40a40c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4102d40 tx=0x7f8cb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:04.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:04.853+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8cb005dd20 con 0x7f8cc4102ee0 2024-03-12T05:10:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.134+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8cc410d6a0 con 0x7f8cc4102ee0 2024-03-12T05:10:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.137+0000 7f8cc1ffb700 1 -- 172.21.15.100:0/4255151786 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8cb0027050 con 0x7f8cc4102ee0 2024-03-12T05:10:05.135 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-03-12T05:10:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCt4+9ljr8fCBAAgEBdQkY5TzrCchksyxu1XQ== 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8cb40a1c00 msgr2=0x7f8cb40a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8cb40a1c00 0x7f8cb40a40c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4102d40 tx=0x7f8cb800b410 comp rx=0 tx=0).stop 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 msgr2=0x7f8cc4074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4074060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0003790 tx=0x7f8cb0005b00 comp rx=0 tx=0).stop 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 shutdown_connections 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f8cb40a1c00 0x7f8cb40a40c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc4101ce0 0x7f8cc4075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc4103880 0x7f8cc40745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.139+0000 7f8ccadaf700 1 --2- 172.21.15.100:0/4255151786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc4102ee0 0x7f8cc4074060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.140+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 >> 172.21.15.100:0/4255151786 conn(0x7f8cc40fd280 msgr2=0x7f8cc4106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:05.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.140+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 shutdown_connections 2024-03-12T05:10:05.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:05.140+0000 7f8ccadaf700 1 -- 172.21.15.100:0/4255151786 wait complete. 2024-03-12T05:10:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-12T05:10:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-12T05:10:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-12T05:10:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-12T05:10:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-12T05:10:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-12T05:10:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-12T05:10:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-12T05:10:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-12T05:10:05.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:04 smithi156 ceph-mon[36270]: osdmap e87: 12 total, 12 up, 12 in 2024-03-12T05:10:05.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-12T05:10:05.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:04 smithi100 ceph-mon[32210]: osdmap e87: 12 total, 12 up, 12 in 2024-03-12T05:10:05.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-12T05:10:05.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-12T05:10:05.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-12T05:10:05.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-12T05:10:05.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-12T05:10:05.404 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-12T05:10:05.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-12T05:10:05.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-12T05:10:05.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-12T05:10:05.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:04 smithi161 ceph-mon[36043]: osdmap e87: 12 total, 12 up, 12 in 2024-03-12T05:10:05.764 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:10:05.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-12T05:10:05.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-12T05:10:05.806 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-12T05:10:05.806 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-12T05:10:05.807 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph mgr dump --format=json 2024-03-12T05:10:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:05 smithi100 ceph-mon[32210]: osdmap e88: 12 total, 12 up, 12 in 2024-03-12T05:10:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:05 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4255151786' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T05:10:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:05 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4255151786' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T05:10:06.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:05 smithi100 ceph-mon[32210]: pgmap v289: 97 pgs: 97 active+clean; 601 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-03-12T05:10:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:06.335 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:05 smithi161 ceph-mon[36043]: osdmap e88: 12 total, 12 up, 12 in 2024-03-12T05:10:06.335 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:05 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4255151786' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T05:10:06.335 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:05 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4255151786' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T05:10:06.335 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:05 smithi161 ceph-mon[36043]: pgmap v289: 97 pgs: 97 active+clean; 601 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-03-12T05:10:06.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:05 smithi156 ceph-mon[36270]: osdmap e88: 12 total, 12 up, 12 in 2024-03-12T05:10:06.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:05 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4255151786' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T05:10:06.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:05 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4255151786' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T05:10:06.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:05 smithi156 ceph-mon[36270]: pgmap v289: 97 pgs: 97 active+clean; 601 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 -- 172.21.15.100:0/693695212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 msgr2=0x7ff1e01068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/693695212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e01068b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009990 tx=0x7ff1d0009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 -- 172.21.15.100:0/693695212 shutdown_connections 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/693695212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 0x7ff1e010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/693695212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0101eb0 0x7ff1e0102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/693695212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e01068b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 -- 172.21.15.100:0/693695212 >> 172.21.15.100:0/693695212 conn(0x7ff1e00fd720 msgr2=0x7ff1e00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:07.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 -- 172.21.15.100:0/693695212 shutdown_connections 2024-03-12T05:10:07.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 -- 172.21.15.100:0/693695212 wait complete. 2024-03-12T05:10:07.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 Processor -- start 2024-03-12T05:10:07.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.745+0000 7ff1e82ed700 1 -- start start 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0101eb0 0x7ff1e0073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e00744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 0x7ff1e00749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a65c0 con 0x7ff1e0102860 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a6730 con 0x7ff1e0106f90 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a68f0 con 0x7ff1e0101eb0 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e00744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e00744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36136/0 (socket says 172.21.15.100:36136) 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 -- 172.21.15.100:0/1363179569 learned_addr learned my addr 172.21.15.100:0/1363179569 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e6089700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0101eb0 0x7ff1e0073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e688a700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 0x7ff1e00749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 -- 172.21.15.100:0/1363179569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0101eb0 msgr2=0x7ff1e0073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0101eb0 0x7ff1e0073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 -- 172.21.15.100:0/1363179569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 msgr2=0x7ff1e00749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 0x7ff1e00749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 -- 172.21.15.100:0/1363179569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d00095f0 con 0x7ff1e0102860 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e688a700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 0x7ff1e00749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e5888700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e00744b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00098e0 tx=0x7ff1d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d001d070 con 0x7ff1e0102860 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d0003680 con 0x7ff1e0102860 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e01a6b70 con 0x7ff1e0102860 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017650 con 0x7ff1e0102860 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.746+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e01a6f80 con 0x7ff1e0102860 2024-03-12T05:10:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.747+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e010a2b0 con 0x7ff1e0102860 2024-03-12T05:10:07.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.748+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7ff1d0022470 con 0x7ff1e0102860 2024-03-12T05:10:07.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.749+0000 7ff1d77fe700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1cc0a1b50 0x7ff1cc0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:07.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.749+0000 7ff1e6089700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1cc0a1b50 0x7ff1cc0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:07.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.749+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff1d00ce810 con 0x7ff1e0102860 2024-03-12T05:10:07.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.750+0000 7ff1e6089700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1cc0a1b50 0x7ff1cc0a4010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009d10 tx=0x7ff1d8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:07.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:07.754+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1d0097950 con 0x7ff1e0102860 2024-03-12T05:10:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.004+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff1e0111d60 con 0x7ff1e0102860 2024-03-12T05:10:08.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.009+0000 7ff1d77fe700 1 -- 172.21.15.100:0/1363179569 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260668 (secure 0 0 0) 0x7ff1d009b010 con 0x7ff1e0102860 2024-03-12T05:10:08.010 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1cc0a1b50 msgr2=0x7ff1cc0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1cc0a1b50 0x7ff1cc0a4010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009d10 tx=0x7ff1d8009040 comp rx=0 tx=0).stop 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 msgr2=0x7ff1e00744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e00744b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00098e0 tx=0x7ff1d000f690 comp rx=0 tx=0).stop 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 shutdown_connections 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff1cc0a1b50 0x7ff1cc0a4010 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0106f90 0x7ff1e00749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0101eb0 0x7ff1e0073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 --2- 172.21.15.100:0/1363179569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0102860 0x7ff1e00744b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 >> 172.21.15.100:0/1363179569 conn(0x7ff1e00fd720 msgr2=0x7ff1e00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 shutdown_connections 2024-03-12T05:10:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:08.011+0000 7ff1e82ed700 1 -- 172.21.15.100:0/1363179569 wait complete. 2024-03-12T05:10:08.638 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":327651532},{"type":"v1","addr":"172.21.15.100:6801","nonce":327651532}]},"active_addr":"172.21.15.100:6801/327651532","active_change":"2024-03-12T05:03:03.632957+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.100:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":576676871}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3916551108}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2828798630}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1404323639}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1020502120}]}]} 2024-03-12T05:10:08.645 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-12T05:10:08.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-12T05:10:08.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T05:10:08.888 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:09.009 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:08 smithi100 ceph-mon[32210]: pgmap v290: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 18 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:09.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:08 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1363179569' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T05:10:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:08 smithi156 ceph-mon[36270]: pgmap v290: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 18 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:08 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1363179569' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T05:10:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:08 smithi161 ceph-mon[36043]: pgmap v290: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 18 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:08 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1363179569' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T05:10:10.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 -- 172.21.15.100:0/393873847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570106b80 msgr2=0x7f5570106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:10.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 --2- 172.21.15.100:0/393873847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570106b80 0x7f5570106f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f556800b0a0 tx=0x7f556800b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 -- 172.21.15.100:0/393873847 shutdown_connections 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 --2- 172.21.15.100:0/393873847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5570101530 0x7f557010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 --2- 172.21.15.100:0/393873847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570106b80 0x7f5570106f60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 --2- 172.21.15.100:0/393873847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5570100b70 0x7f5570100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 -- 172.21.15.100:0/393873847 >> 172.21.15.100:0/393873847 conn(0x7f5570077aa0 msgr2=0x7f5570077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 -- 172.21.15.100:0/393873847 shutdown_connections 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.548+0000 7f5577b89700 1 -- 172.21.15.100:0/393873847 wait complete. 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 Processor -- start 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 -- start start 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 0x7f557010e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5570101530 0x7f557010e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:10.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5570106b80 0x7f557010ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701a65b0 con 0x7f5570100b70 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701a6720 con 0x7f5570101530 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5577b89700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701a68e0 con 0x7f5570106b80 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 0x7f557010e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 0x7f557010e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51802/0 (socket says 172.21.15.100:51802) 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 -- 172.21.15.100:0/3243943710 learned_addr learned my addr 172.21.15.100:0/3243943710 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575124700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5570101530 0x7f557010e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5576126700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5570106b80 0x7f557010ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 -- 172.21.15.100:0/3243943710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5570106b80 msgr2=0x7f557010ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5570106b80 0x7f557010ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 -- 172.21.15.100:0/3243943710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5570101530 msgr2=0x7f557010e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.549+0000 7f5575925700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5570101530 0x7f557010e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f5575925700 1 -- 172.21.15.100:0/3243943710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5568009d00 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f5575925700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 0x7f557010e390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f556800b070 tx=0x7f5568007670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556800e040 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5568007a40 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55701a6b60 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55680059c0 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.550+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55701a6fd0 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.551+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5570109020 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.552+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f5568005040 con 0x7f5570100b70 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.553+0000 7f556effd700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55580a1b50 0x7f55580a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.553+0000 7f5575124700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55580a1b50 0x7f55580a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:10.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.553+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f55680cf260 con 0x7f5570100b70 2024-03-12T05:10:10.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.553+0000 7f5575124700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55580a1b50 0x7f55580a4010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5564005ea0 tx=0x7f5564005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:10.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.557+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55680983a0 con 0x7f5570100b70 2024-03-12T05:10:10.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.756+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5570111ca0 con 0x7f5570100b70 2024-03-12T05:10:10.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.757+0000 7f556effd700 1 -- 172.21.15.100:0/3243943710 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7f55680a2070 con 0x7f5570100b70 2024-03-12T05:10:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:10.756 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:10:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55580a1b50 msgr2=0x7f55580a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55580a1b50 0x7f55580a4010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5564005ea0 tx=0x7f5564005e30 comp rx=0 tx=0).stop 2024-03-12T05:10:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 msgr2=0x7f557010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 0x7f557010e390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f556800b070 tx=0x7f5568007670 comp rx=0 tx=0).stop 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 shutdown_connections 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55580a1b50 0x7f55580a4010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5570106b80 0x7f557010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5570100b70 0x7f557010e390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 --2- 172.21.15.100:0/3243943710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5570101530 0x7f557010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 >> 172.21.15.100:0/3243943710 conn(0x7f5570077aa0 msgr2=0x7f55700ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 shutdown_connections 2024-03-12T05:10:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:10.760+0000 7f5577b89700 1 -- 172.21.15.100:0/3243943710 wait complete. 2024-03-12T05:10:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:10 smithi100 ceph-mon[32210]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:10 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3243943710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:10 smithi156 ceph-mon[36270]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:10 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3243943710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:10 smithi161 ceph-mon[36043]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:10 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3243943710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:11.414 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-12T05:10:11.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T05:10:11.657 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:13.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:12 smithi100 ceph-mon[32210]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 KiB/s wr, 10 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:13.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:12 smithi156 ceph-mon[36270]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 KiB/s wr, 10 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:13.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:12 smithi161 ceph-mon[36043]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 KiB/s wr, 10 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:13.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 -- 172.21.15.100:0/3259500994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 msgr2=0x7f66941068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 --2- 172.21.15.100:0/3259500994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941068c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6688009990 tx=0x7f6688009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 -- 172.21.15.100:0/3259500994 shutdown_connections 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 --2- 172.21.15.100:0/3259500994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669406a630 0x7f669406aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 --2- 172.21.15.100:0/3259500994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694106ff0 0x7f669410eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 --2- 172.21.15.100:0/3259500994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941068c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 -- 172.21.15.100:0/3259500994 >> 172.21.15.100:0/3259500994 conn(0x7f6694076b80 msgr2=0x7f6694076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 -- 172.21.15.100:0/3259500994 shutdown_connections 2024-03-12T05:10:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.280+0000 7f669a06b700 1 -- 172.21.15.100:0/3259500994 wait complete. 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 Processor -- start 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 -- start start 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f669406a630 0x7f6694198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6694106ff0 0x7f669419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669419d140 con 0x7f669406afe0 2024-03-12T05:10:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669419d2b0 con 0x7f6694106ff0 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f669a06b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669419d470 con 0x7f669406a630 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51826/0 (socket says 172.21.15.100:51826) 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f6692ffd700 1 -- 172.21.15.100:0/2599202214 learned_addr learned my addr 172.21.15.100:0/2599202214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f66937fe700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f669406a630 0x7f6694198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.281+0000 7f6693fff700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6694106ff0 0x7f669419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6692ffd700 1 -- 172.21.15.100:0/2599202214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f669406a630 msgr2=0x7f6694198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6692ffd700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f669406a630 0x7f6694198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6692ffd700 1 -- 172.21.15.100:0/2599202214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6694106ff0 msgr2=0x7f669419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6692ffd700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6694106ff0 0x7f669419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6692ffd700 1 -- 172.21.15.100:0/2599202214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880095f0 con 0x7f669406afe0 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6693fff700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6694106ff0 0x7f669419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6692ffd700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941987a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6688006010 tx=0x7f668800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668801d070 con 0x7f669406afe0 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f668800fb80 con 0x7f669406afe0 2024-03-12T05:10:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688017650 con 0x7f669406afe0 2024-03-12T05:10:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669419d6f0 con 0x7f669406afe0 2024-03-12T05:10:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.282+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66941a4fb0 con 0x7f669406afe0 2024-03-12T05:10:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.283+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669410a1e0 con 0x7f669406afe0 2024-03-12T05:10:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.284+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f6688022470 con 0x7f669406afe0 2024-03-12T05:10:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.284+0000 7f6690ff9700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f667c0a1b50 0x7f667c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.284+0000 7f66937fe700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f667c0a1b50 0x7f667c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:13.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.284+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66880ceb50 con 0x7f669406afe0 2024-03-12T05:10:13.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.285+0000 7f66937fe700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f667c0a1b50 0x7f667c0a4010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6684009a10 tx=0x7f6684009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:13.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.289+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6688097c90 con 0x7f669406afe0 2024-03-12T05:10:13.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.482+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66941995c0 con 0x7f669406afe0 2024-03-12T05:10:13.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.484+0000 7f6690ff9700 1 -- 172.21.15.100:0/2599202214 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7f668809b350 con 0x7f669406afe0 2024-03-12T05:10:13.483 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:10:13.483 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f667c0a1b50 msgr2=0x7f667c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f667c0a1b50 0x7f667c0a4010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6684009a10 tx=0x7f6684009190 comp rx=0 tx=0).stop 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 msgr2=0x7f66941987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941987a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6688006010 tx=0x7f668800f690 comp rx=0 tx=0).stop 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 shutdown_connections 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f667c0a1b50 0x7f667c0a4010 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6694106ff0 0x7f669419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f669406a630 0x7f6694198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 --2- 172.21.15.100:0/2599202214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669406afe0 0x7f66941987a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 >> 172.21.15.100:0/2599202214 conn(0x7f6694076b80 msgr2=0x7f66940fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 shutdown_connections 2024-03-12T05:10:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:13.486+0000 7f669a06b700 1 -- 172.21.15.100:0/2599202214 wait complete. 2024-03-12T05:10:14.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:13 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2599202214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:14.194 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-12T05:10:14.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-12T05:10:14.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-12T05:10:14.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-12T05:10:14.196 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-12T05:10:14.196 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-12T05:10:14.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-12T05:10:14.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-12T05:10:14.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-12T05:10:14.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-12T05:10:14.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-12T05:10:14.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-12T05:10:14.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:13 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2599202214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:14.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:13 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2599202214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:10:14.835 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:14.851 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:14.932 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:14 smithi100 ceph-mon[32210]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:15.197 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:15.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:14 smithi156 ceph-mon[36270]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:15.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:14 smithi161 ceph-mon[36043]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-12T05:10:16.027 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:16.028 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:16.881 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:16.991 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:16 smithi100 ceph-mon[32210]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-12T05:10:17.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:16 smithi156 ceph-mon[36270]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-12T05:10:17.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:16 smithi161 ceph-mon[36043]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-12T05:10:19.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:18 smithi100 ceph-mon[32210]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 8 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-12T05:10:19.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:18 smithi156 ceph-mon[36270]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 8 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-12T05:10:19.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:18 smithi161 ceph-mon[36043]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 8 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-12T05:10:19.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 -- 172.21.15.100:0/2856444359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b2806adc0 msgr2=0x7f9b281068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/2856444359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b2806adc0 0x7f9b281068c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009990 tx=0x7f9b14009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 -- 172.21.15.100:0/2856444359 shutdown_connections 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/2856444359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b2806a410 0x7f9b2806a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/2856444359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b28106ff0 0x7f9b2810eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/2856444359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b2806adc0 0x7f9b281068c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 -- 172.21.15.100:0/2856444359 >> 172.21.15.100:0/2856444359 conn(0x7f9b28076bc0 msgr2=0x7f9b28076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 -- 172.21.15.100:0/2856444359 shutdown_connections 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.416+0000 7f9b2f13b700 1 -- 172.21.15.100:0/2856444359 wait complete. 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 Processor -- start 2024-03-12T05:10:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 -- start start 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2806a410 0x7f9b28198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b2806adc0 0x7f9b28198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b28106ff0 0x7f9b2819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2819d0f0 con 0x7f9b28106ff0 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2819d260 con 0x7f9b2806adc0 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2f13b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2819d420 con 0x7f9b2806a410 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2d6d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b28106ff0 0x7f9b2819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2d6d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b28106ff0 0x7f9b2819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51844/0 (socket says 172.21.15.100:51844) 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2d6d8700 1 -- 172.21.15.100:0/3236823089 learned_addr learned my addr 172.21.15.100:0/3236823089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b2ced7700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2806a410 0x7f9b28198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.417+0000 7f9b27fff700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b2806adc0 0x7f9b28198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2d6d8700 1 -- 172.21.15.100:0/3236823089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2806a410 msgr2=0x7f9b28198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2d6d8700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2806a410 0x7f9b28198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2d6d8700 1 -- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b2806adc0 msgr2=0x7f9b28198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2d6d8700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b2806adc0 0x7f9b28198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2d6d8700 1 -- 172.21.15.100:0/3236823089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b140095f0 con 0x7f9b28106ff0 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2d6d8700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b28106ff0 0x7f9b2819cae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b20007ee0 tx=0x7f9b2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20004db0 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2819d700 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b28100d70 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b20016070 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.418+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20012c60 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.419+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b281033a0 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.421+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f9b20018030 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.421+0000 7f9b25ffb700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b180a1b50 0x7f9b180a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.421+0000 7f9b2ced7700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b180a1b50 0x7f9b180a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.421+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9b200d0cd0 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.422+0000 7f9b25ffb700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] conn(0x7f9b180ab710 0x7f9b180adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.422+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 --> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b180ae280 con 0x7f9b180ab710 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.422+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b200d10c0 con 0x7f9b28106ff0 2024-03-12T05:10:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.422+0000 7f9b2ced7700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b180a1b50 0x7f9b180a4010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009a10 tx=0x7f9b1c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.422+0000 7f9b27fff700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] conn(0x7f9b180ab710 0x7f9b180adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.422+0000 7f9b27fff700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] conn(0x7f9b180ab710 0x7f9b180adbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.423+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== osd.1 v2:172.21.15.156:6800/3629499261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9b180ae280 con 0x7f9b180ab710 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.439+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 --> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b28112050 con 0x7f9b180ab710 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b25ffb700 1 -- 172.21.15.100:0/3236823089 <== osd.1 v2:172.21.15.156:6800/3629499261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9b28112050 con 0x7f9b180ab710 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] conn(0x7f9b180ab710 msgr2=0x7f9b180adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] conn(0x7f9b180ab710 0x7f9b180adbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b180a1b50 msgr2=0x7f9b180a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b180a1b50 0x7f9b180a4010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009a10 tx=0x7f9b1c009190 comp rx=0 tx=0).stop 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b28106ff0 msgr2=0x7f9b2819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 --2- 172.21.15.100:0/3236823089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b28106ff0 0x7f9b2819cae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b20007ee0 tx=0x7f9b2000d3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2d6d8700 1 -- 172.21.15.100:0/3236823089 reap_dead start 2024-03-12T05:10:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 shutdown_connections 2024-03-12T05:10:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 >> 172.21.15.100:0/3236823089 conn(0x7f9b28076bc0 msgr2=0x7f9b280fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 shutdown_connections 2024-03-12T05:10:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:18.440+0000 7f9b2f13b700 1 -- 172.21.15.100:0/3236823089 wait complete. 2024-03-12T05:10:21.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:20 smithi100 ceph-mon[32210]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-12T05:10:21.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:20 smithi156 ceph-mon[36270]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-12T05:10:21.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:20 smithi161 ceph-mon[36043]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-12T05:10:23.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:22 smithi100 ceph-mon[32210]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-12T05:10:23.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:22 smithi156 ceph-mon[36270]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-12T05:10:23.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:22 smithi161 ceph-mon[36043]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-12T05:10:23.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- 172.21.15.100:0/2813218539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940106bd0 msgr2=0x7ff940106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- 172.21.15.100:0/2813218539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940106bd0 0x7ff940106fb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009990 tx=0x7ff93c009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- 172.21.15.100:0/2813218539 shutdown_connections 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- 172.21.15.100:0/2813218539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff940101560 0x7ff94010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- 172.21.15.100:0/2813218539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940106bd0 0x7ff940106fb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- 172.21.15.100:0/2813218539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff940100ba0 0x7ff940101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- 172.21.15.100:0/2813218539 >> 172.21.15.100:0/2813218539 conn(0x7ff940076bc0 msgr2=0x7ff940076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- 172.21.15.100:0/2813218539 shutdown_connections 2024-03-12T05:10:23.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- 172.21.15.100:0/2813218539 wait complete. 2024-03-12T05:10:23.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 Processor -- start 2024-03-12T05:10:23.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- start start 2024-03-12T05:10:23.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff940100ba0 0x7ff940198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:23.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940101560 0x7ff9401994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:23.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff940106bd0 0x7ff9401a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:23.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940199d80 con 0x7ff940101560 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401a4560 con 0x7ff940100ba0 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.517+0000 7ff947692700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401a46a0 con 0x7ff940106bd0 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940101560 0x7ff9401994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940101560 0x7ff9401994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39928/0 (socket says 172.21.15.100:39928) 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 -- 172.21.15.100:0/1420403932 learned_addr learned my addr 172.21.15.100:0/1420403932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff945c2f700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff940106bd0 0x7ff9401a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff94542e700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff940100ba0 0x7ff940198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 -- 172.21.15.100:0/1420403932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff940106bd0 msgr2=0x7ff9401a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff940106bd0 0x7ff9401a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 -- 172.21.15.100:0/1420403932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff940100ba0 msgr2=0x7ff940198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff940100ba0 0x7ff940198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 -- 172.21.15.100:0/1420403932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c0095f0 con 0x7ff940101560 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff944c2d700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940101560 0x7ff9401994e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d980 tx=0x7ff93000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930009b40 con 0x7ff940101560 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401a4950 con 0x7ff940101560 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff93000b6e0 con 0x7ff940101560 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000f620 con 0x7ff940101560 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.518+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a4e50 con 0x7ff940101560 2024-03-12T05:10:23.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.519+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff940110e10 con 0x7ff940101560 2024-03-12T05:10:23.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.520+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7ff930010460 con 0x7ff940101560 2024-03-12T05:10:23.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff9367fc700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff92c0a1b50 0x7ff92c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:23.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff94542e700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff92c0a1b50 0x7ff92c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:23.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9300cd900 con 0x7ff940101560 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff9367fc700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] conn(0x7ff92c0ab710 0x7ff92c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 --> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff92c0ae280 con 0x7ff92c0ab710 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9300cdcf0 con 0x7ff940101560 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff945c2f700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] conn(0x7ff92c0ab710 0x7ff92c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.521+0000 7ff94542e700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff92c0a1b50 0x7ff92c0a4010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0095c0 tx=0x7ff93c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.522+0000 7ff945c2f700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] conn(0x7ff92c0ab710 0x7ff92c0adbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:23.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.522+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== osd.9 v2:172.21.15.100:6826/917772860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff92c0ae280 con 0x7ff92c0ab710 2024-03-12T05:10:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.538+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 --> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff940062380 con 0x7ff92c0ab710 2024-03-12T05:10:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff9367fc700 1 -- 172.21.15.100:0/1420403932 <== osd.9 v2:172.21.15.100:6826/917772860 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff940062380 con 0x7ff92c0ab710 2024-03-12T05:10:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] conn(0x7ff92c0ab710 msgr2=0x7ff92c0adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] conn(0x7ff92c0ab710 0x7ff92c0adbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff92c0a1b50 msgr2=0x7ff92c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff92c0a1b50 0x7ff92c0a4010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0095c0 tx=0x7ff93c009550 comp rx=0 tx=0).stop 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940101560 msgr2=0x7ff9401994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 --2- 172.21.15.100:0/1420403932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff940101560 0x7ff9401994e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d980 tx=0x7ff93000dc90 comp rx=0 tx=0).stop 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff945c2f700 1 -- 172.21.15.100:0/1420403932 reap_dead start 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 shutdown_connections 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 >> 172.21.15.100:0/1420403932 conn(0x7ff940076bc0 msgr2=0x7ff9400ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 shutdown_connections 2024-03-12T05:10:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:23.539+0000 7ff947692700 1 -- 172.21.15.100:0/1420403932 wait complete. 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 -- 172.21.15.100:0/2380795171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd70106ff0 msgr2=0x7efd7010eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 --2- 172.21.15.100:0/2380795171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd70106ff0 0x7efd7010eb70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd6800b0a0 tx=0x7efd6800b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 -- 172.21.15.100:0/2380795171 shutdown_connections 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 --2- 172.21.15.100:0/2380795171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd70106ff0 0x7efd7010eb70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 --2- 172.21.15.100:0/2380795171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7006a410 0x7efd7006a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 --2- 172.21.15.100:0/2380795171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd7006adc0 0x7efd701068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 -- 172.21.15.100:0/2380795171 >> 172.21.15.100:0/2380795171 conn(0x7efd70076bc0 msgr2=0x7efd70076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 -- 172.21.15.100:0/2380795171 shutdown_connections 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.069+0000 7efd77566700 1 -- 172.21.15.100:0/2380795171 wait complete. 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 Processor -- start 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 -- start start 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7006a410 0x7efd701982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7006adc0 0x7efd701987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70106ff0 0x7efd7019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7019d130 con 0x7efd7006a410 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7019d2a0 con 0x7efd70106ff0 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd77566700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7019d460 con 0x7efd7006adc0 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7006a410 0x7efd701982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7006a410 0x7efd701982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39950/0 (socket says 172.21.15.100:39950) 2024-03-12T05:10:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 -- 172.21.15.100:0/1568097500 learned_addr learned my addr 172.21.15.100:0/1568097500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 -- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7006adc0 msgr2=0x7efd701987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd74b01700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7006adc0 0x7efd701987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7006adc0 0x7efd701987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 -- 172.21.15.100:0/1568097500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70106ff0 msgr2=0x7efd7019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd70106ff0 0x7efd7019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.070+0000 7efd75302700 1 -- 172.21.15.100:0/1568097500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68009d00 con 0x7efd7006a410 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd75302700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7006a410 0x7efd701982a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e9a0 tx=0x7efd6c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c009790 con 0x7efd7006a410 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7019d740 con 0x7efd7006a410 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd6c00cbd0 con 0x7efd7006a410 2024-03-12T05:10:24.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c018680 con 0x7efd7006a410 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd701a4fa0 con 0x7efd7006a410 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.071+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd70112000 con 0x7efd7006a410 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.073+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7efd6c00cd40 con 0x7efd7006a410 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.073+0000 7efd667fc700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efd5c0a1b50 0x7efd5c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd74b01700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efd5c0a1b50 0x7efd5c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd6c014070 con 0x7efd7006a410 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd667fc700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] conn(0x7efd5c0ab710 0x7efd5c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 --> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd5c0ae280 con 0x7efd5c0ab710 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd6c0ce310 con 0x7efd7006a410 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd74b01700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efd5c0a1b50 0x7efd5c0a4010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd60005fd0 tx=0x7efd60005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd75b03700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] conn(0x7efd5c0ab710 0x7efd5c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.074+0000 7efd75b03700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] conn(0x7efd5c0ab710 0x7efd5c0adbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.075+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== osd.5 v2:172.21.15.161:6808/3085464556 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd5c0ae280 con 0x7efd5c0ab710 2024-03-12T05:10:24.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.092+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 --> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd70062300 con 0x7efd5c0ab710 2024-03-12T05:10:24.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.092+0000 7efd667fc700 1 -- 172.21.15.100:0/1568097500 <== osd.5 v2:172.21.15.161:6808/3085464556 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efd70062300 con 0x7efd5c0ab710 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] conn(0x7efd5c0ab710 msgr2=0x7efd5c0adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] conn(0x7efd5c0ab710 0x7efd5c0adbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efd5c0a1b50 msgr2=0x7efd5c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7efd5c0a1b50 0x7efd5c0a4010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd60005fd0 tx=0x7efd60005ce0 comp rx=0 tx=0).stop 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7006a410 msgr2=0x7efd701982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 --2- 172.21.15.100:0/1568097500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7006a410 0x7efd701982a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e9a0 tx=0x7efd6c00ed60 comp rx=0 tx=0).stop 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd75b03700 1 -- 172.21.15.100:0/1568097500 reap_dead start 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 shutdown_connections 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 >> 172.21.15.100:0/1568097500 conn(0x7efd70076bc0 msgr2=0x7efd700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 shutdown_connections 2024-03-12T05:10:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:24.093+0000 7efd77566700 1 -- 172.21.15.100:0/1568097500 wait complete. 2024-03-12T05:10:25.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:24 smithi100 ceph-mon[32210]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-12T05:10:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:292057776157 2024-03-12T05:10:25.141 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-12T05:10:25.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:24 smithi156 ceph-mon[36270]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-12T05:10:25.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:24 smithi161 ceph-mon[36043]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-12T05:10:26.106 INFO:teuthology.orchestra.run.smithi100.stdout:55834574914 2024-03-12T05:10:26.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.342+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/490887573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 msgr2=0x7f3cc010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.342+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/490887573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 0x7f3cc010eb60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800b0a0 tx=0x7f3cb800b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/490887573 shutdown_connections 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/490887573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 0x7f3cc0102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/490887573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 0x7f3cc010eb60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/490887573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102860 0x7f3cc01068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/490887573 >> 172.21.15.100:0/490887573 conn(0x7f3cc00fd720 msgr2=0x7f3cc00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/490887573 shutdown_connections 2024-03-12T05:10:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/490887573 wait complete. 2024-03-12T05:10:26.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 Processor -- start 2024-03-12T05:10:26.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.343+0000 7f3cc4dfd700 1 -- start start 2024-03-12T05:10:26.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cc4dfd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 0x7f3cc01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cc4dfd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102860 0x7f3cc01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cc4dfd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 0x7f3cc019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cc4dfd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019d140 con 0x7f3cc0106f90 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cc4dfd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019d2b0 con 0x7f3cc0101eb0 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cc4dfd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019d470 con 0x7f3cc0102860 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 0x7f3cc01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 0x7f3cc01982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:54174/0 (socket says 172.21.15.100:54174) 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 -- 172.21.15.100:0/1007096044 learned_addr learned my addr 172.21.15.100:0/1007096044 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbdd9b700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102860 0x7f3cc01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbed9d700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 0x7f3cc019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 -- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102860 msgr2=0x7f3cc01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102860 0x7f3cc01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 -- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 msgr2=0x7f3cc019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 0x7f3cc019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 -- 172.21.15.100:0/1007096044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8009d00 con 0x7f3cc0101eb0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbdd9b700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102860 0x7f3cc01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbed9d700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0106f90 0x7f3cc019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.344+0000 7f3cbe59c700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 0x7f3cc01982b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9a0 tx=0x7f3cb000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.345+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0009790 con 0x7f3cc0101eb0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.345+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cb000cbd0 con 0x7f3cc0101eb0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.345+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc019d750 con 0x7f3cc0101eb0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.345+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc01a4fb0 con 0x7f3cc0101eb0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.345+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0018680 con 0x7f3cc0101eb0 2024-03-12T05:10:26.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.345+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ca0001030 con 0x7f3cc0101eb0 2024-03-12T05:10:26.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.347+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f3cb000cd40 con 0x7f3cc0101eb0 2024-03-12T05:10:26.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.347+0000 7f3caf7fe700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3ca80a1b50 0x7f3ca80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3cb0014070 con 0x7f3cc0101eb0 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3cbdd9b700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3ca80a1b50 0x7f3ca80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3caf7fe700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] conn(0x7f3ca80ab710 0x7f3ca80adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 --> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ca80ae280 con 0x7f3ca80ab710 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb00ce530 con 0x7f3cc0101eb0 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3cbed9d700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] conn(0x7f3ca80ab710 0x7f3ca80adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3cbdd9b700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3ca80a1b50 0x7f3ca80a4010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005e10 tx=0x7f3cb4005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.348+0000 7f3cbed9d700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] conn(0x7f3ca80ab710 0x7f3ca80adbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:26.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.349+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== osd.2 v2:172.21.15.161:6800/623458559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3ca80ae280 con 0x7f3ca80ab710 2024-03-12T05:10:26.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.371+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 --> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ca0002dc0 con 0x7f3ca80ab710 2024-03-12T05:10:26.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.372+0000 7f3caf7fe700 1 -- 172.21.15.100:0/1007096044 <== osd.2 v2:172.21.15.161:6800/623458559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ca0002dc0 con 0x7f3ca80ab710 2024-03-12T05:10:26.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.372+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] conn(0x7f3ca80ab710 msgr2=0x7f3ca80adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.372+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] conn(0x7f3ca80ab710 0x7f3ca80adbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.372+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3ca80a1b50 msgr2=0x7f3ca80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.372+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3ca80a1b50 0x7f3ca80a4010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005e10 tx=0x7f3cb4005d60 comp rx=0 tx=0).stop 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 msgr2=0x7f3cc01982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cc4dfd700 1 --2- 172.21.15.100:0/1007096044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cc0101eb0 0x7f3cc01982b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9a0 tx=0x7f3cb000ed60 comp rx=0 tx=0).stop 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cbed9d700 1 -- 172.21.15.100:0/1007096044 reap_dead start 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 shutdown_connections 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 >> 172.21.15.100:0/1007096044 conn(0x7f3cc00fd720 msgr2=0x7f3cc00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 shutdown_connections 2024-03-12T05:10:26.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.373+0000 7f3cc4dfd700 1 -- 172.21.15.100:0/1007096044 wait complete. 2024-03-12T05:10:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3050640782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0806a2c0 msgr2=0x7faf0810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3050640782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0806a2c0 0x7faf0810eb00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faf0000b0d0 tx=0x7faf0000b3e0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3050640782 shutdown_connections 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3050640782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0806a2c0 0x7faf0810eb00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3050640782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf08107cc0 0x7faf081080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3050640782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf08069920 0x7faf08069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3050640782 >> 172.21.15.100:0/3050640782 conn(0x7faf08077aa0 msgr2=0x7faf08077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3050640782 shutdown_connections 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.785+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3050640782 wait complete. 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 Processor -- start 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 -- start start 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf08069920 0x7faf08100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf0806a2c0 0x7faf08101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf08107cc0 0x7faf081016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a65d0 con 0x7faf08107cc0 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a6740 con 0x7faf08069920 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0fbbf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a6900 con 0x7faf0806a2c0 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf08107cc0 0x7faf081016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf08107cc0 0x7faf081016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39994/0 (socket says 172.21.15.100:39994) 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 -- 172.21.15.100:0/3544074500 learned_addr learned my addr 172.21.15.100:0/3544074500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0d95b700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf08069920 0x7faf08100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0d15a700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf0806a2c0 0x7faf08101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 -- 172.21.15.100:0/3544074500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf0806a2c0 msgr2=0x7faf08101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf0806a2c0 0x7faf08101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 -- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf08069920 msgr2=0x7faf08100c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf08069920 0x7faf08100c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 -- 172.21.15.100:0/3544074500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf00009d00 con 0x7faf08107cc0 2024-03-12T05:10:26.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.786+0000 7faf0e15c700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf08107cc0 0x7faf081016c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf0000d010 tx=0x7faf00007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.787+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0000e040 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.787+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf081a6b80 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.787+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf0001dd70 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.787+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0001be20 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.787+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf081a6fa0 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.787+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf08111f30 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.789+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7faf0001b4e0 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faefeffd700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faef40a1b50 0x7faef40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faf0d95b700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faef40a1b50 0x7faef40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faf000cf8e0 con 0x7faf08107cc0 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faefeffd700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] conn(0x7faef40ab710 0x7faef40adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:26.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 --> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faef40ae280 con 0x7faef40ab710 2024-03-12T05:10:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf000cfc90 con 0x7faf08107cc0 2024-03-12T05:10:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faf0d95b700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faef40a1b50 0x7faef40a4010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faef8005950 tx=0x7faef800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.790+0000 7faf0d15a700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] conn(0x7faef40ab710 0x7faef40adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.791+0000 7faf0d15a700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] conn(0x7faef40ab710 0x7faef40adbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.791+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== osd.10 v2:172.21.15.156:6824/3731703036 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faef40ae280 con 0x7faef40ab710 2024-03-12T05:10:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.808+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 --> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf08062380 con 0x7faef40ab710 2024-03-12T05:10:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faefeffd700 1 -- 172.21.15.100:0/3544074500 <== osd.10 v2:172.21.15.156:6824/3731703036 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faf08062380 con 0x7faef40ab710 2024-03-12T05:10:26.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] conn(0x7faef40ab710 msgr2=0x7faef40adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] conn(0x7faef40ab710 0x7faef40adbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faef40a1b50 msgr2=0x7faef40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faef40a1b50 0x7faef40a4010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faef8005950 tx=0x7faef800b410 comp rx=0 tx=0).stop 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf08107cc0 msgr2=0x7faf081016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 --2- 172.21.15.100:0/3544074500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf08107cc0 0x7faf081016c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf0000d010 tx=0x7faf00007870 comp rx=0 tx=0).stop 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0e15c700 1 -- 172.21.15.100:0/3544074500 reap_dead start 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 shutdown_connections 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.809+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 >> 172.21.15.100:0/3544074500 conn(0x7faf08077aa0 msgr2=0x7faf080ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.810+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 shutdown_connections 2024-03-12T05:10:26.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:26.810+0000 7faf0fbbf700 1 -- 172.21.15.100:0/3544074500 wait complete. 2024-03-12T05:10:27.003 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:26 smithi100 ceph-mon[32210]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-12T05:10:27.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:26 smithi156 ceph-mon[36270]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-12T05:10:27.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:26 smithi161 ceph-mon[36043]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-12T05:10:27.493 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:28.018 INFO:teuthology.orchestra.run.smithi100.stdout:77309411391 2024-03-12T05:10:28.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-12T05:10:28.966 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:28 smithi100 ceph-mon[32210]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-12T05:10:29.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:28 smithi156 ceph-mon[36270]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-12T05:10:29.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:28 smithi161 ceph-mon[36043]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-12T05:10:29.505 INFO:teuthology.orchestra.run.smithi100.stdout:313532612633 2024-03-12T05:10:29.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-12T05:10:30.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 -- 172.21.15.100:0/2603074891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580100800 msgr2=0x7f7580100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:30.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 --2- 172.21.15.100:0/2603074891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580100800 0x7f7580100be0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7574009990 tx=0x7f7574009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 -- 172.21.15.100:0/2603074891 shutdown_connections 2024-03-12T05:10:30.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 --2- 172.21.15.100:0/2603074891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7580107020 0x7f758010eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 --2- 172.21.15.100:0/2603074891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580100800 0x7f7580100be0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 --2- 172.21.15.100:0/2603074891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75801011b0 0x7f75801068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 -- 172.21.15.100:0/2603074891 >> 172.21.15.100:0/2603074891 conn(0x7f7580075ea0 msgr2=0x7f75800762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 -- 172.21.15.100:0/2603074891 shutdown_connections 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 -- 172.21.15.100:0/2603074891 wait complete. 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 Processor -- start 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.903+0000 7f7586c10700 1 -- start start 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7580100800 0x7f7580198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75801011b0 0x7f7580198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580107020 0x7f758019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758019d0f0 con 0x7f7580107020 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758019d260 con 0x7f75801011b0 2024-03-12T05:10:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758019d420 con 0x7f7580100800 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580107020 0x7f758019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580107020 0x7f758019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40012/0 (socket says 172.21.15.100:40012) 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 -- 172.21.15.100:0/1215552369 learned_addr learned my addr 172.21.15.100:0/1215552369 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75849ac700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7580100800 0x7f7580198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 -- 172.21.15.100:0/1215552369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7580100800 msgr2=0x7f7580198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7580100800 0x7f7580198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f757ffff700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75801011b0 0x7f7580198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 -- 172.21.15.100:0/1215552369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75801011b0 msgr2=0x7f7580198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75801011b0 0x7f7580198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 -- 172.21.15.100:0/1215552369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740095f0 con 0x7f7580107020 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f757ffff700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75801011b0 0x7f7580198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f75851ad700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580107020 0x7f758019cae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f757800b970 tx=0x7f757800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757800c9b0 con 0x7f7580107020 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7578014440 con 0x7f7580107020 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758019d700 con 0x7f7580107020 2024-03-12T05:10:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.904+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780075d0 con 0x7f7580107020 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.905+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75801a4f60 con 0x7f7580107020 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.905+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7580112070 con 0x7f7580107020 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.906+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f7578014920 con 0x7f7580107020 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f757dffb700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75700a1b50 0x7f75700a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f75849ac700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75700a1b50 0x7f75700a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7578006e40 con 0x7f7580107020 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f757dffb700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] conn(0x7f75700ab710 0x7f75700adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 --> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75700ae280 con 0x7f75700ab710 2024-03-12T05:10:30.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75780cd5d0 con 0x7f7580107020 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f757ffff700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] conn(0x7f75700ab710 0x7f75700adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.907+0000 7f75849ac700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75700a1b50 0x7f75700a4010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7574006010 tx=0x7f757401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.908+0000 7f757ffff700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] conn(0x7f75700ab710 0x7f75700adbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.908+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== osd.0 v2:172.21.15.100:6802/774764839 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f75700ae280 con 0x7f75700ab710 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.924+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 --> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7580062380 con 0x7f75700ab710 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f757dffb700 1 -- 172.21.15.100:0/1215552369 <== osd.0 v2:172.21.15.100:6802/774764839 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7580062380 con 0x7f75700ab710 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] conn(0x7f75700ab710 msgr2=0x7f75700adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] conn(0x7f75700ab710 0x7f75700adbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75700a1b50 msgr2=0x7f75700a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75700a1b50 0x7f75700a4010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7574006010 tx=0x7f757401a040 comp rx=0 tx=0).stop 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580107020 msgr2=0x7f758019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 --2- 172.21.15.100:0/1215552369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580107020 0x7f758019cae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f757800b970 tx=0x7f757800bd30 comp rx=0 tx=0).stop 2024-03-12T05:10:30.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f75851ad700 1 -- 172.21.15.100:0/1215552369 reap_dead start 2024-03-12T05:10:30.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 shutdown_connections 2024-03-12T05:10:30.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 >> 172.21.15.100:0/1215552369 conn(0x7f7580075ea0 msgr2=0x7f75800fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:30.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 shutdown_connections 2024-03-12T05:10:30.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:29.925+0000 7f7586c10700 1 -- 172.21.15.100:0/1215552369 wait complete. 2024-03-12T05:10:30.988 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:30 smithi100 ceph-mon[32210]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-12T05:10:30.990 INFO:teuthology.orchestra.run.smithi100.stdout:167503724592 2024-03-12T05:10:30.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-12T05:10:31.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:30 smithi156 ceph-mon[36270]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-12T05:10:31.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:30 smithi161 ceph-mon[36043]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-12T05:10:31.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 -- 172.21.15.100:0/286786783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564101510 msgr2=0x7f256410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:31.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 --2- 172.21.15.100:0/286786783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564101510 0x7f256410d9b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f255c00b0a0 tx=0x7f255c00b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 -- 172.21.15.100:0/286786783 shutdown_connections 2024-03-12T05:10:31.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 --2- 172.21.15.100:0/286786783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564101510 0x7f256410d9b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 --2- 172.21.15.100:0/286786783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2564105ab0 0x7f2564105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 --2- 172.21.15.100:0/286786783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564100b50 0x7f2564100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 -- 172.21.15.100:0/286786783 >> 172.21.15.100:0/286786783 conn(0x7f2564077b80 msgr2=0x7f2564077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:31.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 -- 172.21.15.100:0/286786783 shutdown_connections 2024-03-12T05:10:31.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.651+0000 7f256a1e0700 1 -- 172.21.15.100:0/286786783 wait complete. 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 Processor -- start 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 -- start start 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564100b50 0x7f256419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2564101510 0x7f256419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564105ab0 0x7f256419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256419f5e0 con 0x7f2564100b50 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256419f750 con 0x7f2564105ab0 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f256a1e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256419f910 con 0x7f2564101510 2024-03-12T05:10:31.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f25637fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564100b50 0x7f256419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f25637fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564100b50 0x7f256419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43460/0 (socket says 172.21.15.100:43460) 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.652+0000 7f25637fe700 1 -- 172.21.15.100:0/2129267776 learned_addr learned my addr 172.21.15.100:0/2129267776 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f2562ffd700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2564101510 0x7f256419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f2563fff700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564105ab0 0x7f256419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f25637fe700 1 -- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2564101510 msgr2=0x7f256419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f25637fe700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2564101510 0x7f256419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f25637fe700 1 -- 172.21.15.100:0/2129267776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564105ab0 msgr2=0x7f256419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f25637fe700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564105ab0 0x7f256419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f25637fe700 1 -- 172.21.15.100:0/2129267776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009d00 con 0x7f2564100b50 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f25637fe700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564100b50 0x7f256419a700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f255400b530 tx=0x7f255400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:31.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554011830 con 0x7f2564100b50 2024-03-12T05:10:31.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256419fbf0 con 0x7f2564100b50 2024-03-12T05:10:31.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2554011e70 con 0x7f2564100b50 2024-03-12T05:10:31.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400f2e0 con 0x7f2564100b50 2024-03-12T05:10:31.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.653+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25641a7450 con 0x7f2564100b50 2024-03-12T05:10:31.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.654+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2564110f10 con 0x7f2564100b50 2024-03-12T05:10:31.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.655+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f25540105f0 con 0x7f2564100b50 2024-03-12T05:10:31.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2560ff9700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f254c0a1b50 0x7f254c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:31.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2562ffd700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f254c0a1b50 0x7f254c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:31.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25540cd540 con 0x7f2564100b50 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2560ff9700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] conn(0x7f254c0ab710 0x7f254c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 --> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f254c0ae280 con 0x7f254c0ab710 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25540cd930 con 0x7f2564100b50 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2562ffd700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f254c0a1b50 0x7f254c0a4010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2558006fd0 tx=0x7f2558008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2563fff700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] conn(0x7f254c0ab710 0x7f254c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.656+0000 7f2563fff700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] conn(0x7f254c0ab710 0x7f254c0adbb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:31.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.657+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== osd.11 v2:172.21.15.161:6824/2757536197 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f254c0ae280 con 0x7f254c0ab710 2024-03-12T05:10:31.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.673+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 --> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f256406ad20 con 0x7f254c0ab710 2024-03-12T05:10:31.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.674+0000 7f2560ff9700 1 -- 172.21.15.100:0/2129267776 <== osd.11 v2:172.21.15.161:6824/2757536197 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f256406ad20 con 0x7f254c0ab710 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.674+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] conn(0x7f254c0ab710 msgr2=0x7f254c0adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.674+0000 7f256a1e0700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] conn(0x7f254c0ab710 0x7f254c0adbb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f254c0a1b50 msgr2=0x7f254c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f254c0a1b50 0x7f254c0a4010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2558006fd0 tx=0x7f2558008040 comp rx=0 tx=0).stop 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564100b50 msgr2=0x7f256419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 --2- 172.21.15.100:0/2129267776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2564100b50 0x7f256419a700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f255400b530 tx=0x7f255400b8f0 comp rx=0 tx=0).stop 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f2563fff700 1 -- 172.21.15.100:0/2129267776 reap_dead start 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 shutdown_connections 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 >> 172.21.15.100:0/2129267776 conn(0x7f2564077b80 msgr2=0x7f256406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 shutdown_connections 2024-03-12T05:10:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:31.675+0000 7f256a1e0700 1 -- 172.21.15.100:0/2129267776 wait complete. 2024-03-12T05:10:32.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/1403775471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4101560 msgr2=0x7fa9c410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/1403775471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4101560 0x7fa9c410d9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b0a0 tx=0x7fa9bc00b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/1403775471 shutdown_connections 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/1403775471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4101560 0x7fa9c410d9a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/1403775471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c4106bd0 0x7fa9c4106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/1403775471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9c4100ba0 0x7fa9c4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.697+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/1403775471 >> 172.21.15.100:0/1403775471 conn(0x7fa9c4076bc0 msgr2=0x7fa9c4076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.698+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/1403775471 shutdown_connections 2024-03-12T05:10:32.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.698+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/1403775471 wait complete. 2024-03-12T05:10:32.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.698+0000 7fa9c8d2f700 1 Processor -- start 2024-03-12T05:10:32.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.698+0000 7fa9c8d2f700 1 -- start start 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4100ba0 0x7fa9c4102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c4101560 0x7fa9c4102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9c4106bd0 0x7fa9c4103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41a6650 con 0x7fa9c4100ba0 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41a67c0 con 0x7fa9c4106bd0 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41a6980 con 0x7fa9c4101560 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4100ba0 0x7fa9c4102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4100ba0 0x7fa9c4102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43490/0 (socket says 172.21.15.100:43490) 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 -- 172.21.15.100:0/934226787 learned_addr learned my addr 172.21.15.100:0/934226787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c2d9d700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9c4106bd0 0x7fa9c4103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c1d9b700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c4101560 0x7fa9c4102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 -- 172.21.15.100:0/934226787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c4101560 msgr2=0x7fa9c4102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c4101560 0x7fa9c4102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 -- 172.21.15.100:0/934226787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9c4106bd0 msgr2=0x7fa9c4103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9c4106bd0 0x7fa9c4103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 -- 172.21.15.100:0/934226787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc009d00 con 0x7fa9c4100ba0 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c259c700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4100ba0 0x7fa9c4102790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400e9a0 tx=0x7fa9b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4009790 con 0x7fa9c4100ba0 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c41a6c60 con 0x7fa9c4100ba0 2024-03-12T05:10:32.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9b4010430 con 0x7fa9c4100ba0 2024-03-12T05:10:32.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4018680 con 0x7fa9c4100ba0 2024-03-12T05:10:32.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.699+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c41a7160 con 0x7fa9c4100ba0 2024-03-12T05:10:32.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.700+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9c4110e40 con 0x7fa9c4100ba0 2024-03-12T05:10:32.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.701+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fa9b4010910 con 0x7fa9c4100ba0 2024-03-12T05:10:32.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.702+0000 7fa9b37fe700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa9ac0a1b50 0x7fa9ac0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:32.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.702+0000 7fa9c1d9b700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa9ac0a1b50 0x7fa9ac0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.702+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa9b4014070 con 0x7fa9c4100ba0 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.702+0000 7fa9b37fe700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] conn(0x7fa9ac0ab710 0x7fa9ac0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.702+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 --> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9ac0ae280 con 0x7fa9ac0ab710 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.702+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b40ce190 con 0x7fa9c4100ba0 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.703+0000 7fa9c2d9d700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] conn(0x7fa9ac0ab710 0x7fa9ac0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.703+0000 7fa9c1d9b700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa9ac0a1b50 0x7fa9ac0a4010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40771a0 tx=0x7fa9b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.703+0000 7fa9c2d9d700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] conn(0x7fa9ac0ab710 0x7fa9ac0adbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:32.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.703+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== osd.3 v2:172.21.15.100:6810/2837761202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa9ac0ae280 con 0x7fa9ac0ab710 2024-03-12T05:10:32.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.719+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 --> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9c4104b00 con 0x7fa9ac0ab710 2024-03-12T05:10:32.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9b37fe700 1 -- 172.21.15.100:0/934226787 <== osd.3 v2:172.21.15.100:6810/2837761202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa9c4104b00 con 0x7fa9ac0ab710 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] conn(0x7fa9ac0ab710 msgr2=0x7fa9ac0adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] conn(0x7fa9ac0ab710 0x7fa9ac0adbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa9ac0a1b50 msgr2=0x7fa9ac0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa9ac0a1b50 0x7fa9ac0a4010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40771a0 tx=0x7fa9b800b410 comp rx=0 tx=0).stop 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4100ba0 msgr2=0x7fa9c4102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 --2- 172.21.15.100:0/934226787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c4100ba0 0x7fa9c4102790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400e9a0 tx=0x7fa9b400ecb0 comp rx=0 tx=0).stop 2024-03-12T05:10:32.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c2d9d700 1 -- 172.21.15.100:0/934226787 reap_dead start 2024-03-12T05:10:32.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 shutdown_connections 2024-03-12T05:10:32.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 >> 172.21.15.100:0/934226787 conn(0x7fa9c4076bc0 msgr2=0x7fa9c40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:32.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 shutdown_connections 2024-03-12T05:10:32.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:32.720+0000 7fa9c8d2f700 1 -- 172.21.15.100:0/934226787 wait complete. 2024-03-12T05:10:32.907 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:32 smithi100 ceph-mon[32210]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-12T05:10:33.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:32 smithi156 ceph-mon[36270]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-12T05:10:33.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:32 smithi161 ceph-mon[36043]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 -- 172.21.15.100:0/4218277894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8107d70 msgr2=0x7f75a8108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 --2- 172.21.15.100:0/4218277894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8107d70 0x7f75a8108150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7598009990 tx=0x7f7598009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 -- 172.21.15.100:0/4218277894 shutdown_connections 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 --2- 172.21.15.100:0/4218277894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a8102680 0x7f75a810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 --2- 172.21.15.100:0/4218277894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8107d70 0x7f75a8108150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 --2- 172.21.15.100:0/4218277894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a8101cc0 0x7f75a8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 -- 172.21.15.100:0/4218277894 >> 172.21.15.100:0/4218277894 conn(0x7f75a8078da0 msgr2=0x7f75a80791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 -- 172.21.15.100:0/4218277894 shutdown_connections 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.381+0000 7f75ac975700 1 -- 172.21.15.100:0/4218277894 wait complete. 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 Processor -- start 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 -- start start 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a8101cc0 0x7f75a8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8102680 0x7f75a81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a8107d70 0x7f75a819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a819d140 con 0x7f75a8102680 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a819d2b0 con 0x7f75a8101cc0 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75ac975700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a819d470 con 0x7f75a8107d70 2024-03-12T05:10:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8102680 0x7f75a81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8102680 0x7f75a81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43508/0 (socket says 172.21.15.100:43508) 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.382+0000 7f75a5d9b700 1 -- 172.21.15.100:0/3929887223 learned_addr learned my addr 172.21.15.100:0/3929887223 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a659c700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a8101cc0 0x7f75a8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a6d9d700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a8107d70 0x7f75a819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a5d9b700 1 -- 172.21.15.100:0/3929887223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a8107d70 msgr2=0x7f75a819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a5d9b700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a8107d70 0x7f75a819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a5d9b700 1 -- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a8101cc0 msgr2=0x7f75a8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a5d9b700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a8101cc0 0x7f75a8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a5d9b700 1 -- 172.21.15.100:0/3929887223 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980095f0 con 0x7f75a8102680 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a6d9d700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a8107d70 0x7f75a819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a659c700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75a8101cc0 0x7f75a8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75a5d9b700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8102680 0x7f75a81987a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f759c00c920 tx=0x7f759c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c007b00 con 0x7f75a8102680 2024-03-12T05:10:33.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f759c004d10 con 0x7f75a8102680 2024-03-12T05:10:33.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0056c0 con 0x7f75a8102680 2024-03-12T05:10:33.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a819d750 con 0x7f75a8102680 2024-03-12T05:10:33.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.383+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a81a4fb0 con 0x7f75a8102680 2024-03-12T05:10:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.384+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75a8112060 con 0x7f75a8102680 2024-03-12T05:10:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.385+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f759c007c60 con 0x7f75a8102680 2024-03-12T05:10:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75977fe700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75900a1b50 0x7f75900a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75a659c700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75900a1b50 0x7f75900a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f759c0cd740 con 0x7f75a8102680 2024-03-12T05:10:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75977fe700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] conn(0x7f75900ab710 0x7f75900adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 --> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75900ae280 con 0x7f75900ab710 2024-03-12T05:10:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759c0cdb30 con 0x7f75a8102680 2024-03-12T05:10:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75a659c700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75900a1b50 0x7f75900a4010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7598006010 tx=0x7f759800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.386+0000 7f75a6d9d700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] conn(0x7f75900ab710 0x7f75900adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.387+0000 7f75a6d9d700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] conn(0x7f75900ab710 0x7f75900adbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.387+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== osd.7 v2:172.21.15.156:6816/3327014468 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f75900ae280 con 0x7f75900ab710 2024-03-12T05:10:33.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.404+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 --> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75a8062380 con 0x7f75900ab710 2024-03-12T05:10:33.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.404+0000 7f75977fe700 1 -- 172.21.15.100:0/3929887223 <== osd.7 v2:172.21.15.156:6816/3327014468 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f75a8062380 con 0x7f75900ab710 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] conn(0x7f75900ab710 msgr2=0x7f75900adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] conn(0x7f75900ab710 0x7f75900adbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75900a1b50 msgr2=0x7f75900a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75900a1b50 0x7f75900a4010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7598006010 tx=0x7f759800b540 comp rx=0 tx=0).stop 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8102680 msgr2=0x7f75a81987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 --2- 172.21.15.100:0/3929887223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a8102680 0x7f75a81987a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f759c00c920 tx=0x7f759c00cce0 comp rx=0 tx=0).stop 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75a6d9d700 1 -- 172.21.15.100:0/3929887223 reap_dead start 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 shutdown_connections 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 >> 172.21.15.100:0/3929887223 conn(0x7f75a8078da0 msgr2=0x7f75a8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 shutdown_connections 2024-03-12T05:10:33.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:33.405+0000 7f75ac975700 1 -- 172.21.15.100:0/3929887223 wait complete. 2024-03-12T05:10:33.849 INFO:teuthology.orchestra.run.smithi100.stdout:111669149756 2024-03-12T05:10:33.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-12T05:10:34.485 INFO:teuthology.orchestra.run.smithi100.stdout:223338299431 2024-03-12T05:10:34.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-12T05:10:34.502 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:34.503 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:34.503 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:34.921 INFO:teuthology.orchestra.run.smithi100.stdout:347892350997 2024-03-12T05:10:34.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-12T05:10:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:35.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:34 smithi100 ceph-mon[32210]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:35.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:34 smithi156 ceph-mon[36270]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:35.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:34 smithi161 ceph-mon[36043]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:35.438 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:35.474 INFO:teuthology.orchestra.run.smithi100.stdout:34359738441 2024-03-12T05:10:35.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-12T05:10:36.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:36 smithi100 ceph-mon[32210]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:37.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:36 smithi156 ceph-mon[36270]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:37.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:36 smithi161 ceph-mon[36043]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:39.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:38 smithi100 ceph-mon[32210]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:10:39.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:38 smithi156 ceph-mon[36270]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:10:39.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:38 smithi161 ceph-mon[36043]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:10:41.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:40 smithi100 ceph-mon[32210]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:41.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:40 smithi156 ceph-mon[36270]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:41.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:40 smithi161 ceph-mon[36043]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.960+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1924087660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4106b80 msgr2=0x7fb1b4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.960+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1924087660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4106b80 0x7fb1b4106f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009990 tx=0x7fb1a4009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.960+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1924087660 shutdown_connections 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.960+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1924087660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b4101530 0x7fb1b410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.960+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1924087660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4106b80 0x7fb1b4106f60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.960+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1924087660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b4100b70 0x7fb1b4100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.961+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1924087660 >> 172.21.15.100:0/1924087660 conn(0x7fb1b4077aa0 msgr2=0x7fb1b4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.961+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1924087660 shutdown_connections 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.961+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1924087660 wait complete. 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.961+0000 7fb1b89d5700 1 Processor -- start 2024-03-12T05:10:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.961+0000 7fb1b89d5700 1 -- start start 2024-03-12T05:10:41.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b4100b70 0x7fb1b4197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:41.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4101530 0x7fb1b4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:41.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b4106b80 0x7fb1b419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:41.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b419bf30 con 0x7fb1b4101530 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b419c0a0 con 0x7fb1b4100b70 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b419c260 con 0x7fb1b4106b80 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4101530 0x7fb1b4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4101530 0x7fb1b4197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55242/0 (socket says 172.21.15.100:55242) 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 -- 172.21.15.100:0/1572292211 learned_addr learned my addr 172.21.15.100:0/1572292211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b259c700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b4100b70 0x7fb1b4197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b2d9d700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b4106b80 0x7fb1b419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 -- 172.21.15.100:0/1572292211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b4106b80 msgr2=0x7fb1b419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b4106b80 0x7fb1b419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 -- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b4100b70 msgr2=0x7fb1b4197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b4100b70 0x7fb1b4197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 -- 172.21.15.100:0/1572292211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40095f0 con 0x7fb1b4101530 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b2d9d700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b4106b80 0x7fb1b419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b1d9b700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4101530 0x7fb1b4197590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c9b0 tx=0x7fb1a800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8007b00 con 0x7fb1b4101530 2024-03-12T05:10:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1a8007c60 con 0x7fb1b4101530 2024-03-12T05:10:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8018730 con 0x7fb1b4101530 2024-03-12T05:10:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.962+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b419c540 con 0x7fb1b4101530 2024-03-12T05:10:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.963+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b4069af0 con 0x7fb1b4101530 2024-03-12T05:10:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.963+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1b4110ed0 con 0x7fb1b4101530 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fb1a8018890 con 0x7fb1b4101530 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1a37fe700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb19c0a1b50 0x7fb19c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1b259c700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb19c0a1b50 0x7fb19c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb1a80cdb10 con 0x7fb1b4101530 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1a37fe700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] conn(0x7fb19c0ab710 0x7fb19c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 --> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb19c0ae280 con 0x7fb19c0ab710 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.965+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1a80cdf00 con 0x7fb1b4101530 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.966+0000 7fb1b259c700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb19c0a1b50 0x7fb19c0a4010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4006010 tx=0x7fb1a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.966+0000 7fb1b2d9d700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] conn(0x7fb19c0ab710 0x7fb19c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.966+0000 7fb1b2d9d700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] conn(0x7fb19c0ab710 0x7fb19c0adbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.967+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== osd.4 v2:172.21.15.156:6808/551000361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb19c0ae280 con 0x7fb19c0ab710 2024-03-12T05:10:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.983+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 --> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb1b406a040 con 0x7fb19c0ab710 2024-03-12T05:10:41.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.983+0000 7fb1a37fe700 1 -- 172.21.15.100:0/1572292211 <== osd.4 v2:172.21.15.156:6808/551000361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb1b406a040 con 0x7fb19c0ab710 2024-03-12T05:10:41.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] conn(0x7fb19c0ab710 msgr2=0x7fb19c0adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:41.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] conn(0x7fb19c0ab710 0x7fb19c0adbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:41.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb19c0a1b50 msgr2=0x7fb19c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:41.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb19c0a1b50 0x7fb19c0a4010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4006010 tx=0x7fb1a401a040 comp rx=0 tx=0).stop 2024-03-12T05:10:41.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4101530 msgr2=0x7fb1b4197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 --2- 172.21.15.100:0/1572292211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b4101530 0x7fb1b4197590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c9b0 tx=0x7fb1a800cd70 comp rx=0 tx=0).stop 2024-03-12T05:10:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b2d9d700 1 -- 172.21.15.100:0/1572292211 reap_dead start 2024-03-12T05:10:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 shutdown_connections 2024-03-12T05:10:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 >> 172.21.15.100:0/1572292211 conn(0x7fb1b4077aa0 msgr2=0x7fb1b40ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 shutdown_connections 2024-03-12T05:10:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:41.984+0000 7fb1b89d5700 1 -- 172.21.15.100:0/1572292211 wait complete. 2024-03-12T05:10:43.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:42 smithi100 ceph-mon[32210]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 -- 172.21.15.100:0/424559798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 msgr2=0x7fd12c10eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 --2- 172.21.15.100:0/424559798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c10eba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b0a0 tx=0x7fd12400b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 -- 172.21.15.100:0/424559798 shutdown_connections 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 --2- 172.21.15.100:0/424559798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd12c06a630 0x7fd12c06aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 --2- 172.21.15.100:0/424559798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c10eba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 --2- 172.21.15.100:0/424559798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd12c06afe0 0x7fd12c1068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.081+0000 7fd13370c700 1 -- 172.21.15.100:0/424559798 >> 172.21.15.100:0/424559798 conn(0x7fd12c077c70 msgr2=0x7fd12c078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:43.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 -- 172.21.15.100:0/424559798 shutdown_connections 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 -- 172.21.15.100:0/424559798 wait complete. 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 Processor -- start 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 -- start start 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd12c06a630 0x7fd12c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd12c06afe0 0x7fd12c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19f2c0 con 0x7fd12c106fd0 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19f430 con 0x7fd12c06afe0 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd13370c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19f5f0 con 0x7fd12c06a630 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.082+0000 7fd131ca9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55282/0 (socket says 172.21.15.100:55282) 2024-03-12T05:10:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 -- 172.21.15.100:0/961294396 learned_addr learned my addr 172.21.15.100:0/961294396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd1314a8700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd12c06a630 0x7fd12c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd130ca7700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd12c06afe0 0x7fd12c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 -- 172.21.15.100:0/961294396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd12c06a630 msgr2=0x7fd12c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd12c06a630 0x7fd12c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 -- 172.21.15.100:0/961294396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd12c06afe0 msgr2=0x7fd12c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd12c06afe0 0x7fd12c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 -- 172.21.15.100:0/961294396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124009d00 con 0x7fd12c106fd0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd131ca9700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c19ecb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b830 tx=0x7fd12400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400e040 con 0x7fd12c106fd0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c19f870 con 0x7fd12c106fd0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd12401bb30 con 0x7fd12c106fd0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c1a7130 con 0x7fd12c106fd0 2024-03-12T05:10:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.083+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240058b0 con 0x7fd12c106fd0 2024-03-12T05:10:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.084+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c1033d0 con 0x7fd12c106fd0 2024-03-12T05:10:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.085+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fd12401bca0 con 0x7fd12c106fd0 2024-03-12T05:10:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.086+0000 7fd1227fc700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd1180a1bd0 0x7fd1180a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.086+0000 7fd1314a8700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd1180a1bd0 0x7fd1180a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.086+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1240cef00 con 0x7fd12c106fd0 2024-03-12T05:10:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.086+0000 7fd1314a8700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd1180a1bd0 0x7fd1180a4090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd12800ac50 tx=0x7fd12800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.091+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd124098040 con 0x7fd12c106fd0 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/1608200091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf34106ff0 msgr2=0x7fdf3410eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/1608200091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf34106ff0 0x7fdf3410eb70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00b0a0 tx=0x7fdf2c00b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/1608200091 shutdown_connections 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/1608200091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf34106ff0 0x7fdf3410eb70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/1608200091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3406a630 0x7fdf3406aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/1608200091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3406afe0 0x7fdf341068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/1608200091 >> 172.21.15.100:0/1608200091 conn(0x7fdf34076b80 msgr2=0x7fdf34076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/1608200091 shutdown_connections 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/1608200091 wait complete. 2024-03-12T05:10:43.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.680+0000 7fdf3bbfb700 1 Processor -- start 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 -- start start 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3406a630 0x7fdf341983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3406afe0 0x7fdf341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34106ff0 0x7fdf3419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3419d280 con 0x7fdf3406afe0 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3419d3f0 con 0x7fdf3406a630 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3419d5b0 con 0x7fdf34106ff0 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3406afe0 0x7fdf341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3406afe0 0x7fdf341988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55266/0 (socket says 172.21.15.100:55266) 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 -- 172.21.15.100:0/2964611534 learned_addr learned my addr 172.21.15.100:0/2964611534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3a198700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34106ff0 0x7fdf3419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39997700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3406a630 0x7fdf341983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 -- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34106ff0 msgr2=0x7fdf3419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34106ff0 0x7fdf3419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 -- 172.21.15.100:0/2964611534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3406a630 msgr2=0x7fdf341983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3406a630 0x7fdf341983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 -- 172.21.15.100:0/2964611534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c009d00 con 0x7fdf3406afe0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39997700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3406a630 0x7fdf341983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3a198700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34106ff0 0x7fdf3419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf39196700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3406afe0 0x7fdf341988e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c8c0 tx=0x7fdf2400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24004220 con 0x7fdf3406afe0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf24007de0 con 0x7fdf3406afe0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3419d890 con 0x7fdf3406afe0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.681+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24005020 con 0x7fdf3406afe0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.682+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf341a50f0 con 0x7fdf3406afe0 2024-03-12T05:10:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.682+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf34112000 con 0x7fdf3406afe0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.683+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fdf24007900 con 0x7fdf3406afe0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf2affd700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf200a1b50 0x7fdf200a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf39997700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf200a1b50 0x7fdf200a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdf240cd590 con 0x7fdf3406afe0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf2affd700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] conn(0x7fdf200ab710 0x7fdf200adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 --> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf200ae280 con 0x7fdf200ab710 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf240cd980 con 0x7fdf3406afe0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.684+0000 7fdf39997700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf200a1b50 0x7fdf200a4010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007900 tx=0x7fdf30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.685+0000 7fdf3a198700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] conn(0x7fdf200ab710 0x7fdf200adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.685+0000 7fdf3a198700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] conn(0x7fdf200ab710 0x7fdf200adbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.686+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== osd.8 v2:172.21.15.161:6816/2263371293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdf200ae280 con 0x7fdf200ab710 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.703+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 --> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf34062380 con 0x7fdf200ab710 2024-03-12T05:10:43.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.703+0000 7fdf2affd700 1 -- 172.21.15.100:0/2964611534 <== osd.8 v2:172.21.15.161:6816/2263371293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdf34062380 con 0x7fdf200ab710 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.703+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] conn(0x7fdf200ab710 msgr2=0x7fdf200adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.703+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] conn(0x7fdf200ab710 0x7fdf200adbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf200a1b50 msgr2=0x7fdf200a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdf200a1b50 0x7fdf200a4010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007900 tx=0x7fdf30008040 comp rx=0 tx=0).stop 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3406afe0 msgr2=0x7fdf341988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 --2- 172.21.15.100:0/2964611534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3406afe0 0x7fdf341988e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c8c0 tx=0x7fdf2400cc80 comp rx=0 tx=0).stop 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3a198700 1 -- 172.21.15.100:0/2964611534 reap_dead start 2024-03-12T05:10:43.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 shutdown_connections 2024-03-12T05:10:43.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 >> 172.21.15.100:0/2964611534 conn(0x7fdf34076b80 msgr2=0x7fdf34100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:43.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 shutdown_connections 2024-03-12T05:10:43.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:42.704+0000 7fdf3bbfb700 1 -- 172.21.15.100:0/2964611534 wait complete. 2024-03-12T05:10:43.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:42 smithi156 ceph-mon[36270]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:10:43.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:42 smithi161 ceph-mon[36043]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:10:43.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.288+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd12c111d30 con 0x7fd12c106fd0 2024-03-12T05:10:43.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.288+0000 7fd1227fc700 1 -- 172.21.15.100:0/961294396 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd12409b700 con 0x7fd12c106fd0 2024-03-12T05:10:43.287 INFO:teuthology.orchestra.run.smithi100.stdout:313532612636 2024-03-12T05:10:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd1180a1bd0 msgr2=0x7fd1180a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd1180a1bd0 0x7fd1180a4090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd12800ac50 tx=0x7fd12800a380 comp rx=0 tx=0).stop 2024-03-12T05:10:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 msgr2=0x7fd12c19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c19ecb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b830 tx=0x7fd12400cf90 comp rx=0 tx=0).stop 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 shutdown_connections 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd1180a1bd0 0x7fd1180a4090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd12c106fd0 0x7fd12c19ecb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd12c06a630 0x7fd12c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 --2- 172.21.15.100:0/961294396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd12c06afe0 0x7fd12c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.290+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 >> 172.21.15.100:0/961294396 conn(0x7fd12c077c70 msgr2=0x7fd12c10d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.291+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 shutdown_connections 2024-03-12T05:10:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:43.291+0000 7fd13370c700 1 -- 172.21.15.100:0/961294396 wait complete. 2024-03-12T05:10:44.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:43 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/961294396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-12T05:10:44.154 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612636 for osd.10 2024-03-12T05:10:44.154 DEBUG:teuthology.parallel:result is None 2024-03-12T05:10:44.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:43 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/961294396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-12T05:10:44.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:43 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/961294396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-12T05:10:44.780 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:44.781 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:44.783 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:44.786 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:45.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:44 smithi100 ceph-mon[32210]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-12T05:10:45.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:44 smithi156 ceph-mon[36270]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-12T05:10:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:44 smithi161 ceph-mon[36043]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-12T05:10:45.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 -- 172.21.15.100:0/1117641911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8101770 msgr2=0x7fb0b81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 --2- 172.21.15.100:0/1117641911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8101770 0x7fb0b81057a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009990 tx=0x7fb0b4009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 -- 172.21.15.100:0/1117641911 shutdown_connections 2024-03-12T05:10:45.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 --2- 172.21.15.100:0/1117641911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b8100dc0 0x7fb0b81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 --2- 172.21.15.100:0/1117641911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b8105e80 0x7fb0b810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 --2- 172.21.15.100:0/1117641911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8101770 0x7fb0b81057a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 -- 172.21.15.100:0/1117641911 >> 172.21.15.100:0/1117641911 conn(0x7fb0b8077ae0 msgr2=0x7fb0b8077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 -- 172.21.15.100:0/1117641911 shutdown_connections 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.488+0000 7fb0c0213700 1 -- 172.21.15.100:0/1117641911 wait complete. 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 Processor -- start 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 -- start start 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8100dc0 0x7fb0b8102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b8101770 0x7fb0b8102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b8105e80 0x7fb0b81032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81a6620 con 0x7fb0b8100dc0 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81a6790 con 0x7fb0b8101770 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.489+0000 7fb0c0213700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81a6950 con 0x7fb0b8105e80 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8100dc0 0x7fb0b8102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8100dc0 0x7fb0b8102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55306/0 (socket says 172.21.15.100:55306) 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 -- 172.21.15.100:0/3230292863 learned_addr learned my addr 172.21.15.100:0/3230292863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bd7ae700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b8101770 0x7fb0b8102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0be7b0700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b8105e80 0x7fb0b81032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 -- 172.21.15.100:0/3230292863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b8105e80 msgr2=0x7fb0b81032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b8105e80 0x7fb0b81032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 -- 172.21.15.100:0/3230292863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b8101770 msgr2=0x7fb0b8102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b8101770 0x7fb0b8102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 -- 172.21.15.100:0/3230292863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b40095f0 con 0x7fb0b8100dc0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0bdfaf700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8100dc0 0x7fb0b8102840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800d950 tx=0x7fb0a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800f890 con 0x7fb0b8100dc0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b81a6c30 con 0x7fb0b8100dc0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8010460 con 0x7fb0b8100dc0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800e610 con 0x7fb0b8100dc0 2024-03-12T05:10:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.490+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b81a7150 con 0x7fb0b8100dc0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.491+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0b8110e70 con 0x7fb0b8100dc0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.493+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fb0a8010960 con 0x7fb0b8100dc0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.493+0000 7fb0aeffd700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb0a40a1b50 0x7fb0a40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0bd7ae700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb0a40a1b50 0x7fb0a40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0a80cd7d0 con 0x7fb0b8100dc0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0aeffd700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] conn(0x7fb0a40ab710 0x7fb0a40adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 --> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0a40ae280 con 0x7fb0a40ab710 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a80cdbc0 con 0x7fb0b8100dc0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0be7b0700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] conn(0x7fb0a40ab710 0x7fb0a40adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0bd7ae700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb0a40a1b50 0x7fb0a40a4010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b5c0 tx=0x7fb0b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:45.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.494+0000 7fb0be7b0700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] conn(0x7fb0a40ab710 0x7fb0a40adbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:45.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.495+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== osd.6 v2:172.21.15.100:6818/125600956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0a40ae280 con 0x7fb0a40ab710 2024-03-12T05:10:45.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.511+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 --> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0b8062380 con 0x7fb0a40ab710 2024-03-12T05:10:45.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0aeffd700 1 -- 172.21.15.100:0/3230292863 <== osd.6 v2:172.21.15.100:6818/125600956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb0b8062380 con 0x7fb0a40ab710 2024-03-12T05:10:45.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] conn(0x7fb0a40ab710 msgr2=0x7fb0a40adbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] conn(0x7fb0a40ab710 0x7fb0a40adbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb0a40a1b50 msgr2=0x7fb0a40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fb0a40a1b50 0x7fb0a40a4010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b5c0 tx=0x7fb0b4005960 comp rx=0 tx=0).stop 2024-03-12T05:10:45.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8100dc0 msgr2=0x7fb0b8102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 --2- 172.21.15.100:0/3230292863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8100dc0 0x7fb0b8102840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800d950 tx=0x7fb0a800dc60 comp rx=0 tx=0).stop 2024-03-12T05:10:45.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0be7b0700 1 -- 172.21.15.100:0/3230292863 reap_dead start 2024-03-12T05:10:45.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 shutdown_connections 2024-03-12T05:10:45.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 >> 172.21.15.100:0/3230292863 conn(0x7fb0b8077ae0 msgr2=0x7fb0b806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:45.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 shutdown_connections 2024-03-12T05:10:45.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.512+0000 7fb0c0213700 1 -- 172.21.15.100:0/3230292863 wait complete. 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.713+0000 7fe5f408f700 1 -- 172.21.15.100:0/3091990847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec100b70 msgr2=0x7fe5ec100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.713+0000 7fe5f408f700 1 --2- 172.21.15.100:0/3091990847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec100b70 0x7fe5ec100ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009990 tx=0x7fe5dc009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.713+0000 7fe5f408f700 1 -- 172.21.15.100:0/3091990847 shutdown_connections 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.713+0000 7fe5f408f700 1 --2- 172.21.15.100:0/3091990847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ec101530 0x7fe5ec10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 --2- 172.21.15.100:0/3091990847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5ec106b80 0x7fe5ec106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 --2- 172.21.15.100:0/3091990847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec100b70 0x7fe5ec100ff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- 172.21.15.100:0/3091990847 >> 172.21.15.100:0/3091990847 conn(0x7fe5ec077aa0 msgr2=0x7fe5ec077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:45.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- 172.21.15.100:0/3091990847 shutdown_connections 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- 172.21.15.100:0/3091990847 wait complete. 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 Processor -- start 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- start start 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5ec100b70 0x7fe5ec199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ec101530 0x7fe5ec199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 0x7fe5ec1a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec199da0 con 0x7fe5ec106b80 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1a45d0 con 0x7fe5ec101530 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.714+0000 7fe5f408f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1a4710 con 0x7fe5ec100b70 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 0x7fe5ec1a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 0x7fe5ec1a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55320/0 (socket says 172.21.15.100:55320) 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 -- 172.21.15.100:0/1798407730 learned_addr learned my addr 172.21.15.100:0/1798407730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f1e2b700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5ec100b70 0x7fe5ec199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f162a700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ec101530 0x7fe5ec199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 -- 172.21.15.100:0/1798407730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5ec100b70 msgr2=0x7fe5ec199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5ec100b70 0x7fe5ec199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 -- 172.21.15.100:0/1798407730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ec101530 msgr2=0x7fe5ec199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ec101530 0x7fe5ec199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 -- 172.21.15.100:0/1798407730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0095f0 con 0x7fe5ec106b80 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f262c700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 0x7fe5ec1a4090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b9a0 tx=0x7fe5e400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400d850 con 0x7fe5ec106b80 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5e400de90 con 0x7fe5ec106b80 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec1a4970 con 0x7fe5ec106b80 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4017420 con 0x7fe5ec106b80 2024-03-12T05:10:45.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.715+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec1a4df0 con 0x7fe5ec106b80 2024-03-12T05:10:45.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.716+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec109020 con 0x7fe5ec106b80 2024-03-12T05:10:45.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.717+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fe5e400e6b0 con 0x7fe5ec106b80 2024-03-12T05:10:45.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.718+0000 7fe5e2ffd700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5d80a1b50 0x7fe5d80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:45.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.718+0000 7fe5f1e2b700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5d80a1b50 0x7fe5d80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:45.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.718+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5e4006e40 con 0x7fe5ec106b80 2024-03-12T05:10:45.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.719+0000 7fe5f1e2b700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5d80a1b50 0x7fe5d80a4010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ac50 tx=0x7fe5e800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:45.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.722+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5e40963e0 con 0x7fe5ec106b80 2024-03-12T05:10:45.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.926+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe5ec068930 con 0x7fe5ec106b80 2024-03-12T05:10:45.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.926+0000 7fe5e2ffd700 1 -- 172.21.15.100:0/1798407730 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5e4099aa0 con 0x7fe5ec106b80 2024-03-12T05:10:45.924 INFO:teuthology.orchestra.run.smithi100.stdout:292057776161 2024-03-12T05:10:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5d80a1b50 msgr2=0x7fe5d80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5d80a1b50 0x7fe5d80a4010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ac50 tx=0x7fe5e800a380 comp rx=0 tx=0).stop 2024-03-12T05:10:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 msgr2=0x7fe5ec1a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 0x7fe5ec1a4090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b9a0 tx=0x7fe5e400bcb0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 shutdown_connections 2024-03-12T05:10:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe5d80a1b50 0x7fe5d80a4010 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5ec106b80 0x7fe5ec1a4090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5ec100b70 0x7fe5ec199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 --2- 172.21.15.100:0/1798407730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ec101530 0x7fe5ec199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.929+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 >> 172.21.15.100:0/1798407730 conn(0x7fe5ec077aa0 msgr2=0x7fe5ec0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.930+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 shutdown_connections 2024-03-12T05:10:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:45.930+0000 7fe5f408f700 1 -- 172.21.15.100:0/1798407730 wait complete. 2024-03-12T05:10:46.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.619+0000 7f6d94bd9700 1 -- 172.21.15.100:0/1723999878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 msgr2=0x7f6d90102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:46.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.619+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/1723999878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90102140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d780099c0 tx=0x7f6d78009cd0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 -- 172.21.15.100:0/1723999878 shutdown_connections 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/1723999878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 0x7f6d90108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/1723999878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d90102680 0x7f6d9010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/1723999878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90102140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 -- 172.21.15.100:0/1723999878 >> 172.21.15.100:0/1723999878 conn(0x7f6d90078da0 msgr2=0x7f6d900791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 -- 172.21.15.100:0/1723999878 shutdown_connections 2024-03-12T05:10:46.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 -- 172.21.15.100:0/1723999878 wait complete. 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 Processor -- start 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.620+0000 7f6d94bd9700 1 -- start start 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d94bd9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d94bd9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d90102680 0x7f6d90198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d94bd9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 0x7f6d9019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d94bd9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9019d0d0 con 0x7f6d90101cc0 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d94bd9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9019d240 con 0x7f6d90107ca0 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d94bd9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9019d400 con 0x7f6d90102680 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55338/0 (socket says 172.21.15.100:55338) 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 -- 172.21.15.100:0/2860583961 learned_addr learned my addr 172.21.15.100:0/2860583961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d85d9b700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d90102680 0x7f6d90198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8ed9d700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 0x7f6d9019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 -- 172.21.15.100:0/2860583961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d90102680 msgr2=0x7f6d90198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d90102680 0x7f6d90198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 -- 172.21.15.100:0/2860583961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 msgr2=0x7f6d9019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 0x7f6d9019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 -- 172.21.15.100:0/2860583961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d780095f0 con 0x7f6d90101cc0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8ed9d700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 0x7f6d9019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d8e59c700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90198240 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000b530 tx=0x7f6d8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.621+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80011830 con 0x7f6d90101cc0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.622+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9019d6e0 con 0x7f6d90101cc0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.622+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d80011e70 con 0x7f6d90101cc0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.622+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000f300 con 0x7f6d90101cc0 2024-03-12T05:10:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.622+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d901a4f40 con 0x7f6d90101cc0 2024-03-12T05:10:46.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.622+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9010a180 con 0x7f6d90101cc0 2024-03-12T05:10:46.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.624+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f6d800106d0 con 0x7f6d90101cc0 2024-03-12T05:10:46.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.624+0000 7f6d87fff700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6d7c0a1b50 0x7f6d7c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:46.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.624+0000 7f6d85d9b700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6d7c0a1b50 0x7f6d7c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:46.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.624+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6d800cd430 con 0x7f6d90101cc0 2024-03-12T05:10:46.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.625+0000 7f6d85d9b700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6d7c0a1b50 0x7f6d7c0a4010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009990 tx=0x7f6d78005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:46.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.628+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d800965f0 con 0x7f6d90101cc0 2024-03-12T05:10:46.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.829+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6d901995b0 con 0x7f6d90101cc0 2024-03-12T05:10:46.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.831+0000 7f6d87fff700 1 -- 172.21.15.100:0/2860583961 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d80099cb0 con 0x7f6d90101cc0 2024-03-12T05:10:46.828 INFO:teuthology.orchestra.run.smithi100.stdout:77309411395 2024-03-12T05:10:46.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6d7c0a1b50 msgr2=0x7f6d7c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6d7c0a1b50 0x7f6d7c0a4010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009990 tx=0x7f6d78005bc0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 msgr2=0x7f6d90198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90198240 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000b530 tx=0x7f6d8000b840 comp rx=0 tx=0).stop 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 shutdown_connections 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6d7c0a1b50 0x7f6d7c0a4010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d90107ca0 0x7f6d9019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90101cc0 0x7f6d90198240 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 --2- 172.21.15.100:0/2860583961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d90102680 0x7f6d90198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 >> 172.21.15.100:0/2860583961 conn(0x7f6d90078da0 msgr2=0x7f6d901008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 shutdown_connections 2024-03-12T05:10:46.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:46.833+0000 7f6d94bd9700 1 -- 172.21.15.100:0/2860583961 wait complete. 2024-03-12T05:10:46.876 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:46 smithi100 ceph-mon[32210]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-12T05:10:46.876 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:46 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1798407730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-12T05:10:47.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:46 smithi156 ceph-mon[36270]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-12T05:10:47.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:46 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1798407730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-12T05:10:47.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:46 smithi161 ceph-mon[36043]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-12T05:10:47.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:46 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1798407730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-12T05:10:48.041 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:47 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2860583961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T05:10:48.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:47 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2860583961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T05:10:48.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:47 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2860583961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T05:10:48.888 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:48 smithi100 ceph-mon[32210]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-12T05:10:49.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:48 smithi156 ceph-mon[36270]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-12T05:10:49.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:48 smithi161 ceph-mon[36043]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-12T05:10:50.150 INFO:teuthology.orchestra.run.smithi100.stdout:257698037797 2024-03-12T05:10:50.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-12T05:10:50.409 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:50.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:50 smithi100 ceph-mon[32210]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:10:51.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:50 smithi156 ceph-mon[36270]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:10:51.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:50 smithi161 ceph-mon[36043]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:10:51.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776157 got 292057776161 for osd.9 2024-03-12T05:10:51.452 DEBUG:teuthology.parallel:result is None 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 -- 172.21.15.100:0/2088175263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90101810 msgr2=0x7f2a9010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 --2- 172.21.15.100:0/2088175263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90101810 0x7f2a9010da30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a880099c0 tx=0x7f2a88009cd0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 -- 172.21.15.100:0/2088175263 shutdown_connections 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 --2- 172.21.15.100:0/2088175263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a90106ca0 0x7f2a90107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 --2- 172.21.15.100:0/2088175263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90101810 0x7f2a9010da30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 --2- 172.21.15.100:0/2088175263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90100e50 0x7f2a901012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 -- 172.21.15.100:0/2088175263 >> 172.21.15.100:0/2088175263 conn(0x7f2a90077a30 msgr2=0x7f2a90077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 -- 172.21.15.100:0/2088175263 shutdown_connections 2024-03-12T05:10:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.650+0000 7f2a979fb700 1 -- 172.21.15.100:0/2088175263 wait complete. 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 Processor -- start 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 -- start start 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a90100e50 0x7f2a9019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90101810 0x7f2a9019d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 0x7f2a90198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90198b40 con 0x7f2a90106ca0 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90198cb0 con 0x7f2a90100e50 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a979fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90198e70 con 0x7f2a90101810 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 0x7f2a90198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 0x7f2a90198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48690/0 (socket says 172.21.15.100:48690) 2024-03-12T05:10:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 -- 172.21.15.100:0/1857962728 learned_addr learned my addr 172.21.15.100:0/1857962728 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95797700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a90100e50 0x7f2a9019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a94f96700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90101810 0x7f2a9019d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 -- 172.21.15.100:0/1857962728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90101810 msgr2=0x7f2a9019d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90101810 0x7f2a9019d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 -- 172.21.15.100:0/1857962728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a90100e50 msgr2=0x7f2a9019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.651+0000 7f2a95f98700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a90100e50 0x7f2a9019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a95f98700 1 -- 172.21.15.100:0/1857962728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880095f0 con 0x7f2a90106ca0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a94f96700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90101810 0x7f2a9019d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a95f98700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 0x7f2a90198430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a880059d0 tx=0x7f2a8800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8801d070 con 0x7f2a90106ca0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a88003f50 con 0x7f2a90106ca0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a901990f0 con 0x7f2a90106ca0 2024-03-12T05:10:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88022d40 con 0x7f2a90106ca0 2024-03-12T05:10:51.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.652+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901a4d60 con 0x7f2a90106ca0 2024-03-12T05:10:51.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.653+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a90068930 con 0x7f2a90106ca0 2024-03-12T05:10:51.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.654+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f2a88022470 con 0x7f2a90106ca0 2024-03-12T05:10:51.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.654+0000 7f2a867fc700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b50 0x7f2a7c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:51.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.655+0000 7f2a95797700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b50 0x7f2a7c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:51.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.655+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a880cefc0 con 0x7f2a90106ca0 2024-03-12T05:10:51.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.655+0000 7f2a95797700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b50 0x7f2a7c0a4010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c007fd0 tx=0x7f2a8c0079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:51.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.658+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a880981c0 con 0x7f2a90106ca0 2024-03-12T05:10:51.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.867+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2a90110bf0 con 0x7f2a90106ca0 2024-03-12T05:10:51.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.867+0000 7f2a867fc700 1 -- 172.21.15.100:0/1857962728 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2a8809b880 con 0x7f2a90106ca0 2024-03-12T05:10:51.865 INFO:teuthology.orchestra.run.smithi100.stdout:55834574921 2024-03-12T05:10:51.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b50 msgr2=0x7f2a7c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:51.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b50 0x7f2a7c0a4010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c007fd0 tx=0x7f2a8c0079e0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 msgr2=0x7f2a90198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:51.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 0x7f2a90198430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a880059d0 tx=0x7f2a8800bfd0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 shutdown_connections 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b50 0x7f2a7c0a4010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a90100e50 0x7f2a9019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a90106ca0 0x7f2a90198430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.869+0000 7f2a979fb700 1 --2- 172.21.15.100:0/1857962728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90101810 0x7f2a9019d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.870+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 >> 172.21.15.100:0/1857962728 conn(0x7f2a90077a30 msgr2=0x7f2a900fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.870+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 shutdown_connections 2024-03-12T05:10:51.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:51.870+0000 7f2a979fb700 1 -- 172.21.15.100:0/1857962728 wait complete. 2024-03-12T05:10:52.411 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:52.443 INFO:teuthology.orchestra.run.smithi100.stdout:201863462960 2024-03-12T05:10:52.443 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-12T05:10:52.911 INFO:teuthology.orchestra.run.smithi100.stdout:146028888121 2024-03-12T05:10:52.911 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-12T05:10:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:52 smithi100 ceph-mon[32210]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:10:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1857962728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T05:10:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:52 smithi156 ceph-mon[36270]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:10:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1857962728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T05:10:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:52 smithi161 ceph-mon[36043]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:10:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1857962728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T05:10:54.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:53 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:10:54.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:53 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:10:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:53 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:10:54.434 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574914 got 55834574921 for osd.1 2024-03-12T05:10:54.435 DEBUG:teuthology.parallel:result is None 2024-03-12T05:10:54.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:54 smithi161 ceph-mon[36043]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:10:55.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:54 smithi100 ceph-mon[32210]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:10:55.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:54 smithi156 ceph-mon[36270]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:10:55.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411391 got 77309411395 for osd.2 2024-03-12T05:10:55.509 DEBUG:teuthology.parallel:result is None 2024-03-12T05:10:55.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.815+0000 7f57aac6a700 1 -- 172.21.15.100:0/2956822750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 msgr2=0x7f57a4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:55.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.815+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2956822750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4106f60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5798009990 tx=0x7f5798009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:55.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- 172.21.15.100:0/2956822750 shutdown_connections 2024-03-12T05:10:55.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2956822750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57a4101530 0x7f57a410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2956822750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4106f60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2956822750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57a4100b70 0x7f57a4100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- 172.21.15.100:0/2956822750 >> 172.21.15.100:0/2956822750 conn(0x7f57a4077aa0 msgr2=0x7f57a4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- 172.21.15.100:0/2956822750 shutdown_connections 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- 172.21.15.100:0/2956822750 wait complete. 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 Processor -- start 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- start start 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57a4100b70 0x7f57a41027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57a4101530 0x7f57a4102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a64b0 con 0x7f57a4106b80 2024-03-12T05:10:55.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a6620 con 0x7f57a4101530 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57aac6a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a67e0 con 0x7f57a4100b70 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.816+0000 7f57a9207700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a8a06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57a4100b70 0x7f57a41027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48702/0 (socket says 172.21.15.100:48702) 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 -- 172.21.15.100:0/2954283903 learned_addr learned my addr 172.21.15.100:0/2954283903 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a3fff700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57a4101530 0x7f57a4102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 -- 172.21.15.100:0/2954283903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57a4100b70 msgr2=0x7f57a41027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57a4100b70 0x7f57a41027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 -- 172.21.15.100:0/2954283903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57a4101530 msgr2=0x7f57a4102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57a4101530 0x7f57a4102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 -- 172.21.15.100:0/2954283903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980095f0 con 0x7f57a4106b80 2024-03-12T05:10:55.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a9207700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4103230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f579c007ee0 tx=0x7f579c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:55.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c017070 con 0x7f57a4106b80 2024-03-12T05:10:55.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f579c00f040 con 0x7f57a4106b80 2024-03-12T05:10:55.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a41a6ac0 con 0x7f57a4106b80 2024-03-12T05:10:55.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c005650 con 0x7f57a4106b80 2024-03-12T05:10:55.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.817+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a41a6f70 con 0x7f57a4106b80 2024-03-12T05:10:55.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.818+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a4068930 con 0x7f57a4106b80 2024-03-12T05:10:55.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.819+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f579c004cd0 con 0x7f57a4106b80 2024-03-12T05:10:55.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.820+0000 7f57a1ffb700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f57940a1b50 0x7f57940a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:55.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.820+0000 7f57a8a06700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f57940a1b50 0x7f57940a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:55.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.820+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f579c0ccc90 con 0x7f57a4106b80 2024-03-12T05:10:55.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.820+0000 7f57a8a06700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f57940a1b50 0x7f57940a4010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f57980098e0 tx=0x7f5798005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:55.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:55.825+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f579c0d2020 con 0x7f57a4106b80 2024-03-12T05:10:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.032+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f57a4110bc0 con 0x7f57a4106b80 2024-03-12T05:10:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.032+0000 7f57a1ffb700 1 -- 172.21.15.100:0/2954283903 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f579c095d40 con 0x7f57a4106b80 2024-03-12T05:10:56.030 INFO:teuthology.orchestra.run.smithi100.stdout:167503724598 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f57940a1b50 msgr2=0x7f57940a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f57940a1b50 0x7f57940a4010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f57980098e0 tx=0x7f5798005960 comp rx=0 tx=0).stop 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 msgr2=0x7f57a4103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4103230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f579c007ee0 tx=0x7f579c00d3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 shutdown_connections 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f57940a1b50 0x7f57940a4010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4106b80 0x7f57a4103230 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57a4100b70 0x7f57a41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 --2- 172.21.15.100:0/2954283903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57a4101530 0x7f57a4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 >> 172.21.15.100:0/2954283903 conn(0x7f57a4077aa0 msgr2=0x7f57a40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 shutdown_connections 2024-03-12T05:10:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.034+0000 7f57aac6a700 1 -- 172.21.15.100:0/2954283903 wait complete. 2024-03-12T05:10:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 -- 172.21.15.100:0/1174415238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69041017b0 msgr2=0x7f69041057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 --2- 172.21.15.100:0/1174415238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69041017b0 0x7f69041057b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6900009990 tx=0x7f6900009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 -- 172.21.15.100:0/1174415238 shutdown_connections 2024-03-12T05:10:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 --2- 172.21.15.100:0/1174415238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6904100e00 0x7f69041011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 --2- 172.21.15.100:0/1174415238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904105e90 0x7f690410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 --2- 172.21.15.100:0/1174415238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69041017b0 0x7f69041057b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 -- 172.21.15.100:0/1174415238 >> 172.21.15.100:0/1174415238 conn(0x7f6904076b90 msgr2=0x7f6904076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 -- 172.21.15.100:0/1174415238 shutdown_connections 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.463+0000 7f690b740700 1 -- 172.21.15.100:0/1174415238 wait complete. 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 Processor -- start 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 -- start start 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6904100e00 0x7f6904198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 0x7f6904198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904105e90 0x7f690419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690419cff0 con 0x7f6904100e00 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690419d160 con 0x7f69041017b0 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f690b740700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690419d320 con 0x7f6904105e90 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f6908cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 0x7f6904198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f69094dc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6904100e00 0x7f6904198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f6908cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 0x7f6904198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45510/0 (socket says 172.21.15.100:45510) 2024-03-12T05:10:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f6908cdb700 1 -- 172.21.15.100:0/3744553095 learned_addr learned my addr 172.21.15.100:0/3744553095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.464+0000 7f6909cdd700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904105e90 0x7f690419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f6908cdb700 1 -- 172.21.15.100:0/3744553095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904105e90 msgr2=0x7f690419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f6908cdb700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904105e90 0x7f690419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f6908cdb700 1 -- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6904100e00 msgr2=0x7f6904198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f6908cdb700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6904100e00 0x7f6904198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f6908cdb700 1 -- 172.21.15.100:0/3744553095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000095f0 con 0x7f69041017b0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f69094dc700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6904100e00 0x7f6904198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f6908cdb700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 0x7f6904198650 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69000098e0 tx=0x7f690000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690001d070 con 0x7f69041017b0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6900003680 con 0x7f69041017b0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690419d5a0 con 0x7f69041017b0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69000173f0 con 0x7f69041017b0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.465+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69041a4e60 con 0x7f69041017b0 2024-03-12T05:10:56.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.466+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690410a070 con 0x7f69041017b0 2024-03-12T05:10:56.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.467+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f6900022720 con 0x7f69041017b0 2024-03-12T05:10:56.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.467+0000 7f68fa7fc700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f68f00a1b50 0x7f68f00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:56.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.467+0000 7f69094dc700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f68f00a1b50 0x7f68f00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:56.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.467+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f69000ce660 con 0x7f69041017b0 2024-03-12T05:10:56.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.468+0000 7f69094dc700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f68f00a1b50 0x7f68f00a4010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005950 tx=0x7f68f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:56.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.472+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69000978c0 con 0x7f69041017b0 2024-03-12T05:10:56.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.670+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6904102290 con 0x7f69041017b0 2024-03-12T05:10:56.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:56 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:56 smithi100 ceph-mon[32210]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:10:56.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2954283903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T05:10:56.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.670+0000 7f68fa7fc700 1 -- 172.21.15.100:0/3744553095 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f690009af80 con 0x7f69041017b0 2024-03-12T05:10:56.669 INFO:teuthology.orchestra.run.smithi100.stdout:111669149760 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f68f00a1b50 msgr2=0x7f68f00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f68f00a1b50 0x7f68f00a4010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005950 tx=0x7f68f400b410 comp rx=0 tx=0).stop 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 msgr2=0x7f6904198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 0x7f6904198650 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69000098e0 tx=0x7f690000f690 comp rx=0 tx=0).stop 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 shutdown_connections 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f68f00a1b50 0x7f68f00a4010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904105e90 0x7f690419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6904100e00 0x7f6904198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 --2- 172.21.15.100:0/3744553095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69041017b0 0x7f6904198650 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 >> 172.21.15.100:0/3744553095 conn(0x7f6904076b90 msgr2=0x7f690406a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 shutdown_connections 2024-03-12T05:10:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:56.672+0000 7f690b740700 1 -- 172.21.15.100:0/3744553095 wait complete. 2024-03-12T05:10:56.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:56 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:56 smithi156 ceph-mon[36270]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:10:56.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2954283903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T05:10:56.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:56 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:10:56.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:56 smithi161 ceph-mon[36043]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:10:56.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2954283903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T05:10:57.322 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724592 got 167503724598 for osd.5 2024-03-12T05:10:57.323 DEBUG:teuthology.parallel:result is None 2024-03-12T05:10:57.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3744553095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T05:10:57.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:57 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3744553095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T05:10:57.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:57 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3744553095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T05:10:58.386 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149760 for osd.3 2024-03-12T05:10:58.386 DEBUG:teuthology.parallel:result is None 2024-03-12T05:10:58.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:58 smithi156 ceph-mon[36270]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-12T05:10:58.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:58 smithi161 ceph-mon[36043]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-12T05:10:58.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:58 smithi100 ceph-mon[32210]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-12T05:10:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.974+0000 7f109130d700 1 -- 172.21.15.100:0/3756722257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c100bd0 msgr2=0x7f108c101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.974+0000 7f109130d700 1 --2- 172.21.15.100:0/3756722257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c100bd0 0x7f108c101050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1080009990 tx=0x7f1080009ca0 comp rx=0 tx=0).stop 2024-03-12T05:10:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 -- 172.21.15.100:0/3756722257 shutdown_connections 2024-03-12T05:10:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 --2- 172.21.15.100:0/3756722257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f108c106c00 0x7f108c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 --2- 172.21.15.100:0/3756722257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108c101590 0x7f108c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 --2- 172.21.15.100:0/3756722257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c100bd0 0x7f108c101050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 -- 172.21.15.100:0/3756722257 >> 172.21.15.100:0/3756722257 conn(0x7f108c076bc0 msgr2=0x7f108c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 -- 172.21.15.100:0/3756722257 shutdown_connections 2024-03-12T05:10:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 -- 172.21.15.100:0/3756722257 wait complete. 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.975+0000 7f109130d700 1 Processor -- start 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 -- start start 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f108c100bd0 0x7f108c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108c101590 0x7f108c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 0x7f108c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c19bfd0 con 0x7f108c106c00 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c19c140 con 0x7f108c100bd0 2024-03-12T05:10:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f109130d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c19c300 con 0x7f108c101590 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 0x7f108c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 0x7f108c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48748/0 (socket says 172.21.15.100:48748) 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 -- 172.21.15.100:0/2563005303 learned_addr learned my addr 172.21.15.100:0/2563005303 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108a7fc700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108c101590 0x7f108c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108affd700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f108c100bd0 0x7f108c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 -- 172.21.15.100:0/2563005303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108c101590 msgr2=0x7f108c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108c101590 0x7f108c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 -- 172.21.15.100:0/2563005303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f108c100bd0 msgr2=0x7f108c1970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f108c100bd0 0x7f108c1970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 -- 172.21.15.100:0/2563005303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800095f0 con 0x7f108c106c00 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f108b7fe700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 0x7f108c19b9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f107c00e940 tx=0x7f107c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.976+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0097b0 con 0x7f108c106c00 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.977+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c19c5e0 con 0x7f108c106c00 2024-03-12T05:10:58.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.977+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c1a3e40 con 0x7f108c106c00 2024-03-12T05:10:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.977+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f107c00ee90 con 0x7f108c106c00 2024-03-12T05:10:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.977+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0055e0 con 0x7f108c106c00 2024-03-12T05:10:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.977+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c109090 con 0x7f108c106c00 2024-03-12T05:10:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.979+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f107c00cc60 con 0x7f108c106c00 2024-03-12T05:10:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.979+0000 7f1073fff700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f10740a1bd0 0x7f10740a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.979+0000 7f108affd700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f10740a1bd0 0x7f10740a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.979+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f107c014070 con 0x7f108c106c00 2024-03-12T05:10:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.980+0000 7f108affd700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f10740a1bd0 0x7f10740a4090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1084009c90 tx=0x7f1084009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:58.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:58.984+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f107c0967a0 con 0x7f108c106c00 2024-03-12T05:10:59.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.183+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f108c198450 con 0x7f108c106c00 2024-03-12T05:10:59.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.183+0000 7f1073fff700 1 -- 172.21.15.100:0/2563005303 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f107c099e60 con 0x7f108c106c00 2024-03-12T05:10:59.181 INFO:teuthology.orchestra.run.smithi100.stdout:223338299436 2024-03-12T05:10:59.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.186+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f10740a1bd0 msgr2=0x7f10740a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.186+0000 7f109130d700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f10740a1bd0 0x7f10740a4090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1084009c90 tx=0x7f1084009040 comp rx=0 tx=0).stop 2024-03-12T05:10:59.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.186+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 msgr2=0x7f108c19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.186+0000 7f109130d700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 0x7f108c19b9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f107c00e940 tx=0x7f107c00ec50 comp rx=0 tx=0).stop 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 shutdown_connections 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f10740a1bd0 0x7f10740a4090 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f108c100bd0 0x7f108c1970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f108c106c00 0x7f108c19b9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 --2- 172.21.15.100:0/2563005303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108c101590 0x7f108c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 >> 172.21.15.100:0/2563005303 conn(0x7f108c076bc0 msgr2=0x7f108c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 shutdown_connections 2024-03-12T05:10:59.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.187+0000 7f109130d700 1 -- 172.21.15.100:0/2563005303 wait complete. 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- 172.21.15.100:0/1969062372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6558106ff0 msgr2=0x7f655810eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- 172.21.15.100:0/1969062372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6558106ff0 0x7f655810eb70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f655000b0a0 tx=0x7f655000b3b0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- 172.21.15.100:0/1969062372 shutdown_connections 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- 172.21.15.100:0/1969062372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6558106ff0 0x7f655810eb70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- 172.21.15.100:0/1969062372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 0x7f655806a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- 172.21.15.100:0/1969062372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655806adc0 0x7f65581068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- 172.21.15.100:0/1969062372 >> 172.21.15.100:0/1969062372 conn(0x7f6558076bc0 msgr2=0x7f6558076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- 172.21.15.100:0/1969062372 shutdown_connections 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- 172.21.15.100:0/1969062372 wait complete. 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 Processor -- start 2024-03-12T05:10:59.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- start start 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 0x7f6558198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 0x7f6558198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6558106ff0 0x7f655819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655819d090 con 0x7f655806adc0 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655819d200 con 0x7f6558106ff0 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.232+0000 7f655f604700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655819d3c0 con 0x7f655806a410 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 0x7f6558198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 0x7f6558198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48756/0 (socket says 172.21.15.100:48756) 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 -- 172.21.15.100:0/990254436 learned_addr learned my addr 172.21.15.100:0/990254436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655d3a0700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 0x7f6558198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 -- 172.21.15.100:0/990254436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 msgr2=0x7f6558198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 0x7f6558198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 -- 172.21.15.100:0/990254436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6558106ff0 msgr2=0x7f655819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6558106ff0 0x7f655819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 -- 172.21.15.100:0/990254436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550009d00 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655d3a0700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 0x7f6558198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655cb9f700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 0x7f6558198740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b4d0 tx=0x7f654c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c011830 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655819d6a0 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f654c011e70 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00f2e0 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.233+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65581a4f00 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.234+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655810a1e0 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.235+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f654c0105a0 con 0x7f655806adc0 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.236+0000 7f65567fc700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f65400a1b50 0x7f65400a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:10:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.236+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f654c0ccff0 con 0x7f655806adc0 2024-03-12T05:10:59.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.236+0000 7f655d3a0700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f65400a1b50 0x7f65400a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:10:59.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.236+0000 7f655d3a0700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f65400a1b50 0x7f65400a4010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6548005fd0 tx=0x7f6548005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:10:59.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.240+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f654c096170 con 0x7f655806adc0 2024-03-12T05:10:59.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.451+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f6558068930 con 0x7f655806adc0 2024-03-12T05:10:59.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.451+0000 7f65567fc700 1 -- 172.21.15.100:0/990254436 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f654c099830 con 0x7f655806adc0 2024-03-12T05:10:59.449 INFO:teuthology.orchestra.run.smithi100.stdout:347892351002 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f65400a1b50 msgr2=0x7f65400a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f65400a1b50 0x7f65400a4010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6548005fd0 tx=0x7f6548005ee0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 msgr2=0x7f6558198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 0x7f6558198740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b4d0 tx=0x7f654c00b890 comp rx=0 tx=0).stop 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 shutdown_connections 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f65400a1b50 0x7f65400a4010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6558106ff0 0x7f655819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f655806a410 0x7f6558198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 --2- 172.21.15.100:0/990254436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655806adc0 0x7f6558198740 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:10:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 >> 172.21.15.100:0/990254436 conn(0x7f6558076bc0 msgr2=0x7f6558100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:10:59.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 shutdown_connections 2024-03-12T05:10:59.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:10:59.453+0000 7f655f604700 1 -- 172.21.15.100:0/990254436 wait complete. 2024-03-12T05:10:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:10:59.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:10:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2563005303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T05:10:59.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:10:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2563005303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T05:10:59.813 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:10:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2563005303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T05:11:00.101 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:00.115 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350997 got 347892351002 for osd.11 2024-03-12T05:11:00.116 DEBUG:teuthology.parallel:result is None 2024-03-12T05:11:00.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:00 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/990254436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-12T05:11:00.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:00 smithi100 ceph-mon[32210]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-12T05:11:00.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:00 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/990254436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-12T05:11:00.705 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:00 smithi156 ceph-mon[36270]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-12T05:11:00.714 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299431 got 223338299436 for osd.7 2024-03-12T05:11:00.714 DEBUG:teuthology.parallel:result is None 2024-03-12T05:11:00.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:00 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/990254436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-12T05:11:00.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:00 smithi161 ceph-mon[36043]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 -- 172.21.15.100:0/3881528555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80101cc0 msgr2=0x7fec80102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 --2- 172.21.15.100:0/3881528555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80101cc0 0x7fec80102140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec740099c0 tx=0x7fec74009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 -- 172.21.15.100:0/3881528555 shutdown_connections 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 --2- 172.21.15.100:0/3881528555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80102680 0x7fec8010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 --2- 172.21.15.100:0/3881528555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec80107ca0 0x7fec80108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 --2- 172.21.15.100:0/3881528555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80101cc0 0x7fec80102140 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 -- 172.21.15.100:0/3881528555 >> 172.21.15.100:0/3881528555 conn(0x7fec80078da0 msgr2=0x7fec800791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 -- 172.21.15.100:0/3881528555 shutdown_connections 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 -- 172.21.15.100:0/3881528555 wait complete. 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.059+0000 7fec85a16700 1 Processor -- start 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 -- start start 2024-03-12T05:11:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80101cc0 0x7fec801981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 0x7fec80198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec80107ca0 0x7fec8019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8019d070 con 0x7fec80102680 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8019d1e0 con 0x7fec80101cc0 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec85a16700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8019d3a0 con 0x7fec80107ca0 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 0x7fec80198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 0x7fec80198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53312/0 (socket says 172.21.15.100:53312) 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 -- 172.21.15.100:0/950970278 learned_addr learned my addr 172.21.15.100:0/950970278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7effd700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80101cc0 0x7fec801981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 -- 172.21.15.100:0/950970278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec80107ca0 msgr2=0x7fec8019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7f7fe700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec80107ca0 0x7fec8019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec80107ca0 0x7fec8019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 -- 172.21.15.100:0/950970278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80101cc0 msgr2=0x7fec801981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80101cc0 0x7fec801981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 -- 172.21.15.100:0/950970278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740095f0 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7effd700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80101cc0 0x7fec801981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.060+0000 7fec7e7fc700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 0x7fec80198720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fec7400b580 tx=0x7fec74005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.061+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7401d070 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.061+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec74003fd0 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.061+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8019d620 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.061+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74022e50 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.061+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec801a4ee0 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.062+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8010a180 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.063+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fec74022590 con 0x7fec80102680 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.063+0000 7fec84a14700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fec680a1b50 0x7fec680a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.063+0000 7fec7effd700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fec680a1b50 0x7fec680a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.063+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fec740ce650 con 0x7fec80102680 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.064+0000 7fec7effd700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fec680a1b50 0x7fec680a4010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec70005e10 tx=0x7fec70005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.068+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec7405dd20 con 0x7fec80102680 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.268+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fec801994f0 con 0x7fec80102680 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.268+0000 7fec84a14700 1 -- 172.21.15.100:0/950970278 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec74097790 con 0x7fec80102680 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stdout:34359738448 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fec680a1b50 msgr2=0x7fec680a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fec680a1b50 0x7fec680a4010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec70005e10 tx=0x7fec70005d60 comp rx=0 tx=0).stop 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 msgr2=0x7fec80198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 0x7fec80198720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fec7400b580 tx=0x7fec74005e80 comp rx=0 tx=0).stop 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 shutdown_connections 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fec680a1b50 0x7fec680a4010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec80101cc0 0x7fec801981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec80107ca0 0x7fec8019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 --2- 172.21.15.100:0/950970278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80102680 0x7fec80198720 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 >> 172.21.15.100:0/950970278 conn(0x7fec80078da0 msgr2=0x7fec80100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 shutdown_connections 2024-03-12T05:11:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:02.270+0000 7fec85a16700 1 -- 172.21.15.100:0/950970278 wait complete. 2024-03-12T05:11:03.019 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:02 smithi100 ceph-mon[32210]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:03.020 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:02 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/950970278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T05:11:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:02 smithi156 ceph-mon[36270]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:02 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/950970278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T05:11:03.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:02 smithi161 ceph-mon[36043]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:03.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:02 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/950970278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T05:11:03.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738448 for osd.0 2024-03-12T05:11:03.811 DEBUG:teuthology.parallel:result is None 2024-03-12T05:11:05.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:04 smithi100 ceph-mon[32210]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:05.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 -- 172.21.15.100:0/474115035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c102680 msgr2=0x7fe07c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:05.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 --2- 172.21.15.100:0/474115035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c102680 0x7fe07c10eac0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b0d0 tx=0x7fe07400b3e0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 -- 172.21.15.100:0/474115035 shutdown_connections 2024-03-12T05:11:05.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 --2- 172.21.15.100:0/474115035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c102680 0x7fe07c10eac0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 --2- 172.21.15.100:0/474115035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c107ca0 0x7fe07c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 --2- 172.21.15.100:0/474115035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe07c101cc0 0x7fe07c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.909+0000 7fe084526700 1 -- 172.21.15.100:0/474115035 >> 172.21.15.100:0/474115035 conn(0x7fe07c078da0 msgr2=0x7fe07c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 -- 172.21.15.100:0/474115035 shutdown_connections 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 -- 172.21.15.100:0/474115035 wait complete. 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 Processor -- start 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 -- start start 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe07c101cc0 0x7fe07c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c102680 0x7fe07c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 0x7fe07c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19d190 con 0x7fe07c107ca0 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19d300 con 0x7fe07c101cc0 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe084526700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19d4c0 con 0x7fe07c102680 2024-03-12T05:11:05.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe082ac3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 0x7fe07c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe082ac3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 0x7fe07c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53332/0 (socket says 172.21.15.100:53332) 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe082ac3700 1 -- 172.21.15.100:0/1412719 learned_addr learned my addr 172.21.15.100:0/1412719 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.910+0000 7fe081ac1700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c102680 0x7fe07c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe0822c2700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe07c101cc0 0x7fe07c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe082ac3700 1 -- 172.21.15.100:0/1412719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c102680 msgr2=0x7fe07c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe082ac3700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c102680 0x7fe07c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe082ac3700 1 -- 172.21.15.100:0/1412719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe07c101cc0 msgr2=0x7fe07c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe082ac3700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe07c101cc0 0x7fe07c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe082ac3700 1 -- 172.21.15.100:0/1412719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074009d00 con 0x7fe07c107ca0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe082ac3700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 0x7fe07c19cb80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe0740077f0 tx=0x7fe074007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400e040 con 0x7fe07c107ca0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe074003e10 con 0x7fe07c107ca0 2024-03-12T05:11:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c19d740 con 0x7fe07c107ca0 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07401ba20 con 0x7fe07c107ca0 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stdout:257698037801 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.911+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c1a5000 con 0x7fe07c107ca0 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.912+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c10a180 con 0x7fe07c107ca0 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.913+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe074024410 con 0x7fe07c107ca0 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.913+0000 7fe0737fe700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe0680a1ac0 0x7fe0680a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.914+0000 7fe0822c2700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe0680a1ac0 0x7fe0680a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.914+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe0740cea60 con 0x7fe07c107ca0 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.914+0000 7fe0822c2700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe0680a1ac0 0x7fe0680a3f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe078005950 tx=0x7fe0780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:04.918+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe074097d00 con 0x7fe07c107ca0 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.117+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fe07c199610 con 0x7fe07c107ca0 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.117+0000 7fe0737fe700 1 -- 172.21.15.100:0/1412719 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe07409b3c0 con 0x7fe07c107ca0 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe0680a1ac0 msgr2=0x7fe0680a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe0680a1ac0 0x7fe0680a3f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe078005950 tx=0x7fe0780058e0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 msgr2=0x7fe07c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 0x7fe07c19cb80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe0740077f0 tx=0x7fe074007820 comp rx=0 tx=0).stop 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 shutdown_connections 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe0680a1ac0 0x7fe0680a3f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.119+0000 7fe084526700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe07c101cc0 0x7fe07c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.120+0000 7fe084526700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c107ca0 0x7fe07c19cb80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.120+0000 7fe084526700 1 --2- 172.21.15.100:0/1412719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c102680 0x7fe07c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.120+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 >> 172.21.15.100:0/1412719 conn(0x7fe07c078da0 msgr2=0x7fe07c1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.120+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 shutdown_connections 2024-03-12T05:11:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:05.120+0000 7fe084526700 1 -- 172.21.15.100:0/1412719 wait complete. 2024-03-12T05:11:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:04 smithi156 ceph-mon[36270]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:05.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:04 smithi161 ceph-mon[36043]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:05.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:05 smithi100 ceph-mon[32210]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-12T05:11:05.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:05 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1412719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-12T05:11:06.042 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:05 smithi161 ceph-mon[36043]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-12T05:11:06.042 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:05 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1412719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-12T05:11:06.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:05 smithi156 ceph-mon[36270]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-12T05:11:06.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:05 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1412719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-12T05:11:06.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 -- 172.21.15.100:0/985641559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc107e50 msgr2=0x7fe6fc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:06.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 --2- 172.21.15.100:0/985641559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc107e50 0x7fe6fc108230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009990 tx=0x7fe6f8009ca0 comp rx=0 tx=0).stop 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 -- 172.21.15.100:0/985641559 shutdown_connections 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 --2- 172.21.15.100:0/985641559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6fc06ad70 0x7fe6fc10eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 --2- 172.21.15.100:0/985641559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc107e50 0x7fe6fc108230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 --2- 172.21.15.100:0/985641559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 -- 172.21.15.100:0/985641559 >> 172.21.15.100:0/985641559 conn(0x7fe6fc0779d0 msgr2=0x7fe6fc077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 -- 172.21.15.100:0/985641559 shutdown_connections 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.814+0000 7fe7039c4700 1 -- 172.21.15.100:0/985641559 wait complete. 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 Processor -- start 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 -- start start 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6fc06ad70 0x7fe6fc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6fc107e50 0x7fe6fc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:06.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc19d110 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc19d280 con 0x7fe6fc107e50 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe7039c4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc19d440 con 0x7fe6fc06ad70 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe701760700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe701760700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53344/0 (socket says 172.21.15.100:53344) 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe701760700 1 -- 172.21.15.100:0/4212816267 learned_addr learned my addr 172.21.15.100:0/4212816267 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe700f5f700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6fc06ad70 0x7fe6fc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.815+0000 7fe701f61700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6fc107e50 0x7fe6fc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe701760700 1 -- 172.21.15.100:0/4212816267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6fc06ad70 msgr2=0x7fe6fc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe701760700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6fc06ad70 0x7fe6fc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe701760700 1 -- 172.21.15.100:0/4212816267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6fc107e50 msgr2=0x7fe6fc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe701760700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6fc107e50 0x7fe6fc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe701760700 1 -- 172.21.15.100:0/4212816267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80095f0 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe701760700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc198230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:06.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f801d070 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc19d6c0 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f8022470 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8017630 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.816+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc1a4f80 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.817+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc10a250 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.818+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe6f80225e0 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.818+0000 7fe6f27fc700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe6e80a1ac0 0x7fe6e80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:06.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.819+0000 7fe700f5f700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe6e80a1ac0 0x7fe6e80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:06.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.819+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6f80cea30 con 0x7fe6fc06a3b0 2024-03-12T05:11:06.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.819+0000 7fe700f5f700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe6e80a1ac0 0x7fe6e80a3f80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc078060 tx=0x7fe6ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:06.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:06.822+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6f8097cd0 con 0x7fe6fc06a3b0 2024-03-12T05:11:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.018+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe6fc199590 con 0x7fe6fc06a3b0 2024-03-12T05:11:07.016 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:06 smithi100 ceph-mon[32210]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:11:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.019+0000 7fe6f27fc700 1 -- 172.21.15.100:0/4212816267 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6f8027020 con 0x7fe6fc06a3b0 2024-03-12T05:11:07.017 INFO:teuthology.orchestra.run.smithi100.stdout:201863462964 2024-03-12T05:11:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe6e80a1ac0 msgr2=0x7fe6e80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe6e80a1ac0 0x7fe6e80a3f80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc078060 tx=0x7fe6ec009260 comp rx=0 tx=0).stop 2024-03-12T05:11:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 msgr2=0x7fe6fc198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc198230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8004730 comp rx=0 tx=0).stop 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 shutdown_connections 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe6e80a1ac0 0x7fe6e80a3f80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6fc107e50 0x7fe6fc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6fc06a3b0 0x7fe6fc198230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 --2- 172.21.15.100:0/4212816267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6fc06ad70 0x7fe6fc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 >> 172.21.15.100:0/4212816267 conn(0x7fe6fc0779d0 msgr2=0x7fe6fc0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 shutdown_connections 2024-03-12T05:11:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:07.021+0000 7fe7039c4700 1 -- 172.21.15.100:0/4212816267 wait complete. 2024-03-12T05:11:07.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:06 smithi156 ceph-mon[36270]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:11:07.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:06 smithi161 ceph-mon[36043]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-12T05:11:07.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462960 got 201863462964 for osd.6 2024-03-12T05:11:07.862 DEBUG:teuthology.parallel:result is None 2024-03-12T05:11:08.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:07 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4212816267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T05:11:08.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:07 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4212816267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T05:11:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:07 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4212816267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T05:11:08.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037801 for osd.8 2024-03-12T05:11:08.279 DEBUG:teuthology.parallel:result is None 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 -- 172.21.15.100:0/1619862950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 msgr2=0x7f417410bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 --2- 172.21.15.100:0/1619862950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f417410bd90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4170009990 tx=0x7f4170009ca0 comp rx=0 tx=0).stop 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 -- 172.21.15.100:0/1619862950 shutdown_connections 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 --2- 172.21.15.100:0/1619862950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 0x7f417410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 --2- 172.21.15.100:0/1619862950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 0x7f4174103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 --2- 172.21.15.100:0/1619862950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f417410bd90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:08.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 -- 172.21.15.100:0/1619862950 >> 172.21.15.100:0/1619862950 conn(0x7f4174077ca0 msgr2=0x7f41740780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:08.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 -- 172.21.15.100:0/1619862950 shutdown_connections 2024-03-12T05:11:08.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.768+0000 7f417b8ff700 1 -- 172.21.15.100:0/1619862950 wait complete. 2024-03-12T05:11:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 Processor -- start 2024-03-12T05:11:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 -- start start 2024-03-12T05:11:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 0x7f41741982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f41741987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 0x7f417419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417419d130 con 0x7f41741038a0 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417419d2a0 con 0x7f417410c360 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417b8ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417419d460 con 0x7f4174102f80 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f41741987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f41741987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53368/0 (socket says 172.21.15.100:53368) 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 -- 172.21.15.100:0/3630307752 learned_addr learned my addr 172.21.15.100:0/3630307752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417969b700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 0x7f41741982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4179e9c700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 0x7f417419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 -- 172.21.15.100:0/3630307752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 msgr2=0x7f41741982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 0x7f41741982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 -- 172.21.15.100:0/3630307752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 msgr2=0x7f417419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 0x7f417419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4178e9a700 1 -- 172.21.15.100:0/3630307752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700095f0 con 0x7f41741038a0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f417969b700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 0x7f41741982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.769+0000 7f4179e9c700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 0x7f417419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.770+0000 7f4178e9a700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f41741987e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41700098e0 tx=0x7f417000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.770+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417001d070 con 0x7f41741038a0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.770+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417419d6e0 con 0x7f41741038a0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.770+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4170003680 con 0x7f41741038a0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.770+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700175f0 con 0x7f41741038a0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.770+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41741a4fa0 con 0x7f41741038a0 2024-03-12T05:11:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.771+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41741074e0 con 0x7f41741038a0 2024-03-12T05:11:08.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.772+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4170022470 con 0x7f41741038a0 2024-03-12T05:11:08.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.772+0000 7f416a7fc700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41600a1ac0 0x7f41600a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:08.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.772+0000 7f417969b700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41600a1ac0 0x7f41600a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:08.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.772+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f41700ce450 con 0x7f41741038a0 2024-03-12T05:11:08.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.773+0000 7f417969b700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41600a1ac0 0x7f41600a3f80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4164005950 tx=0x7f41640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:08.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:08.777+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4170097810 con 0x7f41741038a0 2024-03-12T05:11:08.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:08 smithi100 ceph-mon[32210]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-12T05:11:09.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.032+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f41741995b0 con 0x7f41741038a0 2024-03-12T05:11:09.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.033+0000 7f416a7fc700 1 -- 172.21.15.100:0/3630307752 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f417009aed0 con 0x7f41741038a0 2024-03-12T05:11:09.030 INFO:teuthology.orchestra.run.smithi100.stdout:146028888126 2024-03-12T05:11:09.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41600a1ac0 msgr2=0x7f41600a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:09.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41600a1ac0 0x7f41600a3f80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4164005950 tx=0x7f41640058e0 comp rx=0 tx=0).stop 2024-03-12T05:11:09.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 msgr2=0x7f41741987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f41741987e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41700098e0 tx=0x7f417000f690 comp rx=0 tx=0).stop 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 shutdown_connections 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41600a1ac0 0x7f41600a3f80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417410c360 0x7f417419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174102f80 0x7f41741982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 --2- 172.21.15.100:0/3630307752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41741038a0 0x7f41741987e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 >> 172.21.15.100:0/3630307752 conn(0x7f4174077ca0 msgr2=0x7f41740fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 shutdown_connections 2024-03-12T05:11:09.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:09.035+0000 7f417b8ff700 1 -- 172.21.15.100:0/3630307752 wait complete. 2024-03-12T05:11:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:08 smithi156 ceph-mon[36270]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-12T05:11:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:08 smithi161 ceph-mon[36043]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-12T05:11:09.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888121 got 146028888126 for osd.4 2024-03-12T05:11:09.671 DEBUG:teuthology.parallel:result is None 2024-03-12T05:11:09.671 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-12T05:11:09.672 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-12T05:11:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:10.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:09 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3630307752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T05:11:10.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:10.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:09 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:09 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3630307752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T05:11:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:09 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:09 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3630307752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T05:11:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:09 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:11.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:10 smithi100 ceph-mon[32210]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-12T05:11:11.035 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:11:11.035 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:11:11.035 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:10 smithi100 ceph-mon[32210]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:10 smithi156 ceph-mon[36270]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-12T05:11:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:11:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:11:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:10 smithi156 ceph-mon[36270]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:11.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:10 smithi161 ceph-mon[36043]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-12T05:11:11.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:11:11.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:11:11.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:10 smithi161 ceph-mon[36043]: from='mgr.14152 172.21.15.100:0/3839392723' entity='mgr.x' 2024-03-12T05:11:11.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 -- 172.21.15.100:0/2711721093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 msgr2=0x7f582810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:11.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 --2- 172.21.15.100:0/2711721093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f582810eac0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58180099c0 tx=0x7f5818009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 -- 172.21.15.100:0/2711721093 shutdown_connections 2024-03-12T05:11:11.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 --2- 172.21.15.100:0/2711721093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f582810eac0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 --2- 172.21.15.100:0/2711721093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5828107ca0 0x7f5828108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 --2- 172.21.15.100:0/2711721093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582806a3c0 0x7f582806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 -- 172.21.15.100:0/2711721093 >> 172.21.15.100:0/2711721093 conn(0x7f5828077b80 msgr2=0x7f5828077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:11.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 -- 172.21.15.100:0/2711721093 shutdown_connections 2024-03-12T05:11:11.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.538+0000 7f582ed8c700 1 -- 172.21.15.100:0/2711721093 wait complete. 2024-03-12T05:11:11.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 Processor -- start 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 -- start start 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582806a3c0 0x7f5828197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f58281984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828107ca0 0x7f582819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582819ce30 con 0x7f582806ad80 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582819cfa0 con 0x7f5828107ca0 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.539+0000 7f582ed8c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582819d160 con 0x7f582806a3c0 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f58281984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f58281984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46284/0 (socket says 172.21.15.100:46284) 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 -- 172.21.15.100:0/4061271012 learned_addr learned my addr 172.21.15.100:0/4061271012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f582cb28700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582806a3c0 0x7f5828197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:11.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f582d329700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828107ca0 0x7f582819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 -- 172.21.15.100:0/4061271012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582806a3c0 msgr2=0x7f5828197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582806a3c0 0x7f5828197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 -- 172.21.15.100:0/4061271012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828107ca0 msgr2=0x7f582819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828107ca0 0x7f582819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 -- 172.21.15.100:0/4061271012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180095f0 con 0x7f582806ad80 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5827fff700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f58281984e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c9b0 tx=0x7f581c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c007b00 con 0x7f582806ad80 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f581c007c60 con 0x7f582806ad80 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582819d440 con 0x7f582806ad80 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c018660 con 0x7f582806ad80 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.540+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58281a4ca0 con 0x7f582806ad80 2024-03-12T05:11:11.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.541+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5828068930 con 0x7f582806ad80 2024-03-12T05:11:11.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.542+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f581c00f450 con 0x7f582806ad80 2024-03-12T05:11:11.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.543+0000 7f5825ffb700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f58100a1ac0 0x7f58100a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:11.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.543+0000 7f582cb28700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f58100a1ac0 0x7f58100a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:11.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.543+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f581c0cd3d0 con 0x7f582806ad80 2024-03-12T05:11:11.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.543+0000 7f582cb28700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f58100a1ac0 0x7f58100a3f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f582000f440 tx=0x7f58200079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.547+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f581c0965e0 con 0x7f582806ad80 2024-03-12T05:11:11.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.735+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5828111cb0 con 0x7f58100a1ac0 2024-03-12T05:11:11.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.744+0000 7f5825ffb700 1 -- 172.21.15.100:0/4061271012 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258888 (secure 0 0 0) 0x7f5828111cb0 con 0x7f58100a1ac0 2024-03-12T05:11:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:11.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.746+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f58100a1ac0 msgr2=0x7f58100a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:11.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.746+0000 7f582ed8c700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f58100a1ac0 0x7f58100a3f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f582000f440 tx=0x7f58200079e0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.746+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 msgr2=0x7f58281984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:11.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.746+0000 7f582ed8c700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f58281984e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c9b0 tx=0x7f581c00ccc0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 shutdown_connections 2024-03-12T05:11:11.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f58100a1ac0 0x7f58100a3f80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828107ca0 0x7f582819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582806a3c0 0x7f5828197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 --2- 172.21.15.100:0/4061271012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f582806ad80 0x7f58281984e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 >> 172.21.15.100:0/4061271012 conn(0x7f5828077b80 msgr2=0x7f58280ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 shutdown_connections 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:11.747+0000 7f582ed8c700 1 -- 172.21.15.100:0/4061271012 wait complete. 2024-03-12T05:11:11.748 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:11:12.418 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-03-12T05:11:11.728600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656571,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":591,"num_read_kb":417,"num_write":349,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374988,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566836,"statfs":{"total":1151940427776,"available":1151556440064,"internally_reserved":0,"allocated":13676544,"data_stored":7481697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":43,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003099"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744383+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:11:03.744383+0000","last_peered":"2024-03-12T05:11:03.744383+0000","last_clean":"2024-03-12T05:11:03.744383+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:11:03.744383+0000","last_undegraded":"2024-03-12T05:11:03.744383+0000","last_fullsized":"2024-03-12T05:11:03.744383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.739101+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:11:03.739101+0000","last_peered":"2024-03-12T05:11:03.739101+0000","last_clean":"2024-03-12T05:11:03.739101+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:11:03.739101+0000","last_undegraded":"2024-03-12T05:11:03.739101+0000","last_fullsized":"2024-03-12T05:11:03.739101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757927+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:11:03.757927+0000","last_peered":"2024-03-12T05:11:03.757927+0000","last_clean":"2024-03-12T05:11:03.757927+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:11:03.757927+0000","last_undegraded":"2024-03-12T05:11:03.757927+0000","last_fullsized":"2024-03-12T05:11:03.757927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733109+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:11:03.733109+0000","last_peered":"2024-03-12T05:11:03.733109+0000","last_clean":"2024-03-12T05:11:03.733109+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:11:03.733109+0000","last_undegraded":"2024-03-12T05:11:03.733109+0000","last_fullsized":"2024-03-12T05:11:03.733109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743525+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:11:03.743525+0000","last_peered":"2024-03-12T05:11:03.743525+0000","last_clean":"2024-03-12T05:11:03.743525+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:11:03.743525+0000","last_undegraded":"2024-03-12T05:11:03.743525+0000","last_fullsized":"2024-03-12T05:11:03.743525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745830+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:11:03.745830+0000","last_peered":"2024-03-12T05:11:03.745830+0000","last_clean":"2024-03-12T05:11:03.745830+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:11:03.745830+0000","last_undegraded":"2024-03-12T05:11:03.745830+0000","last_fullsized":"2024-03-12T05:11:03.745830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750437+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:11:03.750437+0000","last_peered":"2024-03-12T05:11:03.750437+0000","last_clean":"2024-03-12T05:11:03.750437+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:11:03.750437+0000","last_undegraded":"2024-03-12T05:11:03.750437+0000","last_fullsized":"2024-03-12T05:11:03.750437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747959+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:11:03.747959+0000","last_peered":"2024-03-12T05:11:03.747959+0000","last_clean":"2024-03-12T05:11:03.747959+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:11:03.747959+0000","last_undegraded":"2024-03-12T05:11:03.747959+0000","last_fullsized":"2024-03-12T05:11:03.747959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.752826+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:11:03.752826+0000","last_peered":"2024-03-12T05:11:03.752826+0000","last_clean":"2024-03-12T05:11:03.752826+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:11:03.752826+0000","last_undegraded":"2024-03-12T05:11:03.752826+0000","last_fullsized":"2024-03-12T05:11:03.752826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744896+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:11:03.744896+0000","last_peered":"2024-03-12T05:11:03.744896+0000","last_clean":"2024-03-12T05:11:03.744896+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:11:03.744896+0000","last_undegraded":"2024-03-12T05:11:03.744896+0000","last_fullsized":"2024-03-12T05:11:03.744896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758668+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:11:03.758668+0000","last_peered":"2024-03-12T05:11:03.758668+0000","last_clean":"2024-03-12T05:11:03.758668+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:11:03.758668+0000","last_undegraded":"2024-03-12T05:11:03.758668+0000","last_fullsized":"2024-03-12T05:11:03.758668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748754+0000","last_change":"2024-03-12T05:10:04.986947+0000","last_active":"2024-03-12T05:11:03.748754+0000","last_peered":"2024-03-12T05:11:03.748754+0000","last_clean":"2024-03-12T05:11:03.748754+0000","last_became_active":"2024-03-12T05:10:04.986526+0000","last_became_peered":"2024-03-12T05:10:04.986526+0000","last_unstale":"2024-03-12T05:11:03.748754+0000","last_undegraded":"2024-03-12T05:11:03.748754+0000","last_fullsized":"2024-03-12T05:11:03.748754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:30:47.611206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743287+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:11:03.743287+0000","last_peered":"2024-03-12T05:11:03.743287+0000","last_clean":"2024-03-12T05:11:03.743287+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:11:03.743287+0000","last_undegraded":"2024-03-12T05:11:03.743287+0000","last_fullsized":"2024-03-12T05:11:03.743287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749425+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:11:03.749425+0000","last_peered":"2024-03-12T05:11:03.749425+0000","last_clean":"2024-03-12T05:11:03.749425+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:11:03.749425+0000","last_undegraded":"2024-03-12T05:11:03.749425+0000","last_fullsized":"2024-03-12T05:11:03.749425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.740596+0000","last_change":"2024-03-12T05:10:04.989251+0000","last_active":"2024-03-12T05:11:03.740596+0000","last_peered":"2024-03-12T05:11:03.740596+0000","last_clean":"2024-03-12T05:11:03.740596+0000","last_became_active":"2024-03-12T05:10:04.988845+0000","last_became_peered":"2024-03-12T05:10:04.988845+0000","last_unstale":"2024-03-12T05:11:03.740596+0000","last_undegraded":"2024-03-12T05:11:03.740596+0000","last_fullsized":"2024-03-12T05:11:03.740596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:21:42.513931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760056+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:11:03.760056+0000","last_peered":"2024-03-12T05:11:03.760056+0000","last_clean":"2024-03-12T05:11:03.760056+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:11:03.760056+0000","last_undegraded":"2024-03-12T05:11:03.760056+0000","last_fullsized":"2024-03-12T05:11:03.760056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744673+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:11:03.744673+0000","last_peered":"2024-03-12T05:11:03.744673+0000","last_clean":"2024-03-12T05:11:03.744673+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:11:03.744673+0000","last_undegraded":"2024-03-12T05:11:03.744673+0000","last_fullsized":"2024-03-12T05:11:03.744673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749801+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:11:03.749801+0000","last_peered":"2024-03-12T05:11:03.749801+0000","last_clean":"2024-03-12T05:11:03.749801+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:11:03.749801+0000","last_undegraded":"2024-03-12T05:11:03.749801+0000","last_fullsized":"2024-03-12T05:11:03.749801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748000+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:11:03.748000+0000","last_peered":"2024-03-12T05:11:03.748000+0000","last_clean":"2024-03-12T05:11:03.748000+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:11:03.748000+0000","last_undegraded":"2024-03-12T05:11:03.748000+0000","last_fullsized":"2024-03-12T05:11:03.748000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983601+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:10:04.983601+0000","last_peered":"2024-03-12T05:10:04.983601+0000","last_clean":"2024-03-12T05:10:04.983601+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:10:04.983601+0000","last_undegraded":"2024-03-12T05:10:04.983601+0000","last_fullsized":"2024-03-12T05:10:04.983601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745669+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:11:03.745669+0000","last_peered":"2024-03-12T05:11:03.745669+0000","last_clean":"2024-03-12T05:11:03.745669+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:11:03.745669+0000","last_undegraded":"2024-03-12T05:11:03.745669+0000","last_fullsized":"2024-03-12T05:11:03.745669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985285+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:10:04.985285+0000","last_peered":"2024-03-12T05:10:04.985285+0000","last_clean":"2024-03-12T05:10:04.985285+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:10:04.985285+0000","last_undegraded":"2024-03-12T05:10:04.985285+0000","last_fullsized":"2024-03-12T05:10:04.985285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757822+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:11:03.757822+0000","last_peered":"2024-03-12T05:11:03.757822+0000","last_clean":"2024-03-12T05:11:03.757822+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:11:03.757822+0000","last_undegraded":"2024-03-12T05:11:03.757822+0000","last_fullsized":"2024-03-12T05:11:03.757822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985028+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:10:04.985028+0000","last_peered":"2024-03-12T05:10:04.985028+0000","last_clean":"2024-03-12T05:10:04.985028+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:10:04.985028+0000","last_undegraded":"2024-03-12T05:10:04.985028+0000","last_fullsized":"2024-03-12T05:10:04.985028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758666+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:11:03.758666+0000","last_peered":"2024-03-12T05:11:03.758666+0000","last_clean":"2024-03-12T05:11:03.758666+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:11:03.758666+0000","last_undegraded":"2024-03-12T05:11:03.758666+0000","last_fullsized":"2024-03-12T05:11:03.758666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985370+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:10:04.985370+0000","last_peered":"2024-03-12T05:10:04.985370+0000","last_clean":"2024-03-12T05:10:04.985370+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:10:04.985370+0000","last_undegraded":"2024-03-12T05:10:04.985370+0000","last_fullsized":"2024-03-12T05:10:04.985370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748647+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:11:03.748647+0000","last_peered":"2024-03-12T05:11:03.748647+0000","last_clean":"2024-03-12T05:11:03.748647+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:11:03.748647+0000","last_undegraded":"2024-03-12T05:11:03.748647+0000","last_fullsized":"2024-03-12T05:11:03.748647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:08.087872+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:11:08.087872+0000","last_peered":"2024-03-12T05:11:08.087872+0000","last_clean":"2024-03-12T05:11:08.087872+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:11:08.087872+0000","last_undegraded":"2024-03-12T05:11:08.087872+0000","last_fullsized":"2024-03-12T05:11:08.087872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748799+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:11:03.748799+0000","last_peered":"2024-03-12T05:11:03.748799+0000","last_clean":"2024-03-12T05:11:03.748799+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:11:03.748799+0000","last_undegraded":"2024-03-12T05:11:03.748799+0000","last_fullsized":"2024-03-12T05:11:03.748799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983832+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:10:04.983832+0000","last_peered":"2024-03-12T05:10:04.983832+0000","last_clean":"2024-03-12T05:10:04.983832+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:10:04.983832+0000","last_undegraded":"2024-03-12T05:10:04.983832+0000","last_fullsized":"2024-03-12T05:10:04.983832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748879+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:11:03.748879+0000","last_peered":"2024-03-12T05:11:03.748879+0000","last_clean":"2024-03-12T05:11:03.748879+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:11:03.748879+0000","last_undegraded":"2024-03-12T05:11:03.748879+0000","last_fullsized":"2024-03-12T05:11:03.748879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.993890+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:10:04.993890+0000","last_peered":"2024-03-12T05:10:04.993890+0000","last_clean":"2024-03-12T05:10:04.993890+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:10:04.993890+0000","last_undegraded":"2024-03-12T05:10:04.993890+0000","last_fullsized":"2024-03-12T05:10:04.993890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745430+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:11:03.745430+0000","last_peered":"2024-03-12T05:11:03.745430+0000","last_clean":"2024-03-12T05:11:03.745430+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:11:03.745430+0000","last_undegraded":"2024-03-12T05:11:03.745430+0000","last_fullsized":"2024-03-12T05:11:03.745430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990178+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:10:04.990178+0000","last_peered":"2024-03-12T05:10:04.990178+0000","last_clean":"2024-03-12T05:10:04.990178+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:10:04.990178+0000","last_undegraded":"2024-03-12T05:10:04.990178+0000","last_fullsized":"2024-03-12T05:10:04.990178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748530+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:11:03.748530+0000","last_peered":"2024-03-12T05:11:03.748530+0000","last_clean":"2024-03-12T05:11:03.748530+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:11:03.748530+0000","last_undegraded":"2024-03-12T05:11:03.748530+0000","last_fullsized":"2024-03-12T05:11:03.748530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:09.088847+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:11:09.088847+0000","last_peered":"2024-03-12T05:11:09.088847+0000","last_clean":"2024-03-12T05:11:09.088847+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:11:09.088847+0000","last_undegraded":"2024-03-12T05:11:09.088847+0000","last_fullsized":"2024-03-12T05:11:09.088847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.756261+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:11:03.756261+0000","last_peered":"2024-03-12T05:11:03.756261+0000","last_clean":"2024-03-12T05:11:03.756261+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:11:03.756261+0000","last_undegraded":"2024-03-12T05:11:03.756261+0000","last_fullsized":"2024-03-12T05:11:03.756261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981550+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:10:04.981550+0000","last_peered":"2024-03-12T05:10:04.981550+0000","last_clean":"2024-03-12T05:10:04.981550+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:10:04.981550+0000","last_undegraded":"2024-03-12T05:10:04.981550+0000","last_fullsized":"2024-03-12T05:10:04.981550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748739+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:11:03.748739+0000","last_peered":"2024-03-12T05:11:03.748739+0000","last_clean":"2024-03-12T05:11:03.748739+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:11:03.748739+0000","last_undegraded":"2024-03-12T05:11:03.748739+0000","last_fullsized":"2024-03-12T05:11:03.748739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990717+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:10:04.990717+0000","last_peered":"2024-03-12T05:10:04.990717+0000","last_clean":"2024-03-12T05:10:04.990717+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:10:04.990717+0000","last_undegraded":"2024-03-12T05:10:04.990717+0000","last_fullsized":"2024-03-12T05:10:04.990717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.759675+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:11:03.759675+0000","last_peered":"2024-03-12T05:11:03.759675+0000","last_clean":"2024-03-12T05:11:03.759675+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:11:03.759675+0000","last_undegraded":"2024-03-12T05:11:03.759675+0000","last_fullsized":"2024-03-12T05:11:03.759675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:10.090580+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:11:10.090580+0000","last_peered":"2024-03-12T05:11:10.090580+0000","last_clean":"2024-03-12T05:11:10.090580+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:11:10.090580+0000","last_undegraded":"2024-03-12T05:11:10.090580+0000","last_fullsized":"2024-03-12T05:11:10.090580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751355+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:11:03.751355+0000","last_peered":"2024-03-12T05:11:03.751355+0000","last_clean":"2024-03-12T05:11:03.751355+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:11:03.751355+0000","last_undegraded":"2024-03-12T05:11:03.751355+0000","last_fullsized":"2024-03-12T05:11:03.751355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.998175+0000","last_change":"2024-03-12T05:10:04.997655+0000","last_active":"2024-03-12T05:10:04.998175+0000","last_peered":"2024-03-12T05:10:04.998175+0000","last_clean":"2024-03-12T05:10:04.998175+0000","last_became_active":"2024-03-12T05:10:04.985974+0000","last_became_peered":"2024-03-12T05:10:04.985974+0000","last_unstale":"2024-03-12T05:10:04.998175+0000","last_undegraded":"2024-03-12T05:10:04.998175+0000","last_fullsized":"2024-03-12T05:10:04.998175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:50:52.646109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751063+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:11:03.751063+0000","last_peered":"2024-03-12T05:11:03.751063+0000","last_clean":"2024-03-12T05:11:03.751063+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:11:03.751063+0000","last_undegraded":"2024-03-12T05:11:03.751063+0000","last_fullsized":"2024-03-12T05:11:03.751063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.989829+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:10:04.989829+0000","last_peered":"2024-03-12T05:10:04.989829+0000","last_clean":"2024-03-12T05:10:04.989829+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:10:04.989829+0000","last_undegraded":"2024-03-12T05:10:04.989829+0000","last_fullsized":"2024-03-12T05:10:04.989829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750601+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:11:03.750601+0000","last_peered":"2024-03-12T05:11:03.750601+0000","last_clean":"2024-03-12T05:11:03.750601+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:11:03.750601+0000","last_undegraded":"2024-03-12T05:11:03.750601+0000","last_fullsized":"2024-03-12T05:11:03.750601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:09.100981+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:11:09.100981+0000","last_peered":"2024-03-12T05:11:09.100981+0000","last_clean":"2024-03-12T05:11:09.100981+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:11:09.100981+0000","last_undegraded":"2024-03-12T05:11:09.100981+0000","last_fullsized":"2024-03-12T05:11:09.100981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751248+0000","last_change":"2024-03-12T05:10:04.985396+0000","last_active":"2024-03-12T05:11:03.751248+0000","last_peered":"2024-03-12T05:11:03.751248+0000","last_clean":"2024-03-12T05:11:03.751248+0000","last_became_active":"2024-03-12T05:10:04.984994+0000","last_became_peered":"2024-03-12T05:10:04.984994+0000","last_unstale":"2024-03-12T05:11:03.751248+0000","last_undegraded":"2024-03-12T05:11:03.751248+0000","last_fullsized":"2024-03-12T05:11:03.751248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:26:42.993458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984101+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:10:04.984101+0000","last_peered":"2024-03-12T05:10:04.984101+0000","last_clean":"2024-03-12T05:10:04.984101+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:10:04.984101+0000","last_undegraded":"2024-03-12T05:10:04.984101+0000","last_fullsized":"2024-03-12T05:10:04.984101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.738306+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:11:03.738306+0000","last_peered":"2024-03-12T05:11:03.738306+0000","last_clean":"2024-03-12T05:11:03.738306+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:11:03.738306+0000","last_undegraded":"2024-03-12T05:11:03.738306+0000","last_fullsized":"2024-03-12T05:11:03.738306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984702+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:10:04.984702+0000","last_peered":"2024-03-12T05:10:04.984702+0000","last_clean":"2024-03-12T05:10:04.984702+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:10:04.984702+0000","last_undegraded":"2024-03-12T05:10:04.984702+0000","last_fullsized":"2024-03-12T05:10:04.984702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.742201+0000","last_change":"2024-03-12T05:10:04.995448+0000","last_active":"2024-03-12T05:11:03.742201+0000","last_peered":"2024-03-12T05:11:03.742201+0000","last_clean":"2024-03-12T05:11:03.742201+0000","last_became_active":"2024-03-12T05:10:04.995056+0000","last_became_peered":"2024-03-12T05:10:04.995056+0000","last_unstale":"2024-03-12T05:11:03.742201+0000","last_undegraded":"2024-03-12T05:11:03.742201+0000","last_fullsized":"2024-03-12T05:11:03.742201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:23:27.643478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983622+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:10:04.983622+0000","last_peered":"2024-03-12T05:10:04.983622+0000","last_clean":"2024-03-12T05:10:04.983622+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:10:04.983622+0000","last_undegraded":"2024-03-12T05:10:04.983622+0000","last_fullsized":"2024-03-12T05:10:04.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747337+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:11:03.747337+0000","last_peered":"2024-03-12T05:11:03.747337+0000","last_clean":"2024-03-12T05:11:03.747337+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:11:03.747337+0000","last_undegraded":"2024-03-12T05:11:03.747337+0000","last_fullsized":"2024-03-12T05:11:03.747337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981281+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:10:04.981281+0000","last_peered":"2024-03-12T05:10:04.981281+0000","last_clean":"2024-03-12T05:10:04.981281+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:10:04.981281+0000","last_undegraded":"2024-03-12T05:10:04.981281+0000","last_fullsized":"2024-03-12T05:10:04.981281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737885+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:11:03.737885+0000","last_peered":"2024-03-12T05:11:03.737885+0000","last_clean":"2024-03-12T05:11:03.737885+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:11:03.737885+0000","last_undegraded":"2024-03-12T05:11:03.737885+0000","last_fullsized":"2024-03-12T05:11:03.737885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745048+0000","last_change":"2024-03-12T05:10:04.997291+0000","last_active":"2024-03-12T05:11:03.745048+0000","last_peered":"2024-03-12T05:11:03.745048+0000","last_clean":"2024-03-12T05:11:03.745048+0000","last_became_active":"2024-03-12T05:10:04.996814+0000","last_became_peered":"2024-03-12T05:10:04.996814+0000","last_unstale":"2024-03-12T05:11:03.745048+0000","last_undegraded":"2024-03-12T05:11:03.745048+0000","last_fullsized":"2024-03-12T05:11:03.745048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:42.585416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:08.098516+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:11:08.098516+0000","last_peered":"2024-03-12T05:11:08.098516+0000","last_clean":"2024-03-12T05:11:08.098516+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:11:08.098516+0000","last_undegraded":"2024-03-12T05:11:08.098516+0000","last_fullsized":"2024-03-12T05:11:08.098516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746635+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:11:03.746635+0000","last_peered":"2024-03-12T05:11:03.746635+0000","last_clean":"2024-03-12T05:11:03.746635+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:11:03.746635+0000","last_undegraded":"2024-03-12T05:11:03.746635+0000","last_fullsized":"2024-03-12T05:11:03.746635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996155+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:10:04.996155+0000","last_peered":"2024-03-12T05:10:04.996155+0000","last_clean":"2024-03-12T05:10:04.996155+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:10:04.996155+0000","last_undegraded":"2024-03-12T05:10:04.996155+0000","last_fullsized":"2024-03-12T05:10:04.996155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981148+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:10:04.981148+0000","last_peered":"2024-03-12T05:10:04.981148+0000","last_clean":"2024-03-12T05:10:04.981148+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:10:04.981148+0000","last_undegraded":"2024-03-12T05:10:04.981148+0000","last_fullsized":"2024-03-12T05:10:04.981148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748237+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:11:03.748237+0000","last_peered":"2024-03-12T05:11:03.748237+0000","last_clean":"2024-03-12T05:11:03.748237+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:11:03.748237+0000","last_undegraded":"2024-03-12T05:11:03.748237+0000","last_fullsized":"2024-03-12T05:11:03.748237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981395+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:10:04.981395+0000","last_peered":"2024-03-12T05:10:04.981395+0000","last_clean":"2024-03-12T05:10:04.981395+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:10:04.981395+0000","last_undegraded":"2024-03-12T05:10:04.981395+0000","last_fullsized":"2024-03-12T05:10:04.981395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984811+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:10:04.984811+0000","last_peered":"2024-03-12T05:10:04.984811+0000","last_clean":"2024-03-12T05:10:04.984811+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:10:04.984811+0000","last_undegraded":"2024-03-12T05:10:04.984811+0000","last_fullsized":"2024-03-12T05:10:04.984811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744189+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:11:03.744189+0000","last_peered":"2024-03-12T05:11:03.744189+0000","last_clean":"2024-03-12T05:11:03.744189+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:11:03.744189+0000","last_undegraded":"2024-03-12T05:11:03.744189+0000","last_fullsized":"2024-03-12T05:11:03.744189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:07.933870+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:10:07.933870+0000","last_peered":"2024-03-12T05:10:07.933870+0000","last_clean":"2024-03-12T05:10:07.933870+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:10:07.933870+0000","last_undegraded":"2024-03-12T05:10:07.933870+0000","last_fullsized":"2024-03-12T05:10:07.933870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750882+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:11:03.750882+0000","last_peered":"2024-03-12T05:11:03.750882+0000","last_clean":"2024-03-12T05:11:03.750882+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:11:03.750882+0000","last_undegraded":"2024-03-12T05:11:03.750882+0000","last_fullsized":"2024-03-12T05:11:03.750882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996107+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:10:04.996107+0000","last_peered":"2024-03-12T05:10:04.996107+0000","last_clean":"2024-03-12T05:10:04.996107+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:10:04.996107+0000","last_undegraded":"2024-03-12T05:10:04.996107+0000","last_fullsized":"2024-03-12T05:10:04.996107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748414+0000","last_change":"2024-03-12T05:10:04.997304+0000","last_active":"2024-03-12T05:11:03.748414+0000","last_peered":"2024-03-12T05:11:03.748414+0000","last_clean":"2024-03-12T05:11:03.748414+0000","last_became_active":"2024-03-12T05:10:04.996794+0000","last_became_peered":"2024-03-12T05:10:04.996794+0000","last_unstale":"2024-03-12T05:11:03.748414+0000","last_undegraded":"2024-03-12T05:11:03.748414+0000","last_fullsized":"2024-03-12T05:11:03.748414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:33.408355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.139977+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:10:05.139977+0000","last_peered":"2024-03-12T05:10:05.139977+0000","last_clean":"2024-03-12T05:10:05.139977+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:10:05.139977+0000","last_undegraded":"2024-03-12T05:10:05.139977+0000","last_fullsized":"2024-03-12T05:10:05.139977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733684+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:11:03.733684+0000","last_peered":"2024-03-12T05:11:03.733684+0000","last_clean":"2024-03-12T05:11:03.733684+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:11:03.733684+0000","last_undegraded":"2024-03-12T05:11:03.733684+0000","last_fullsized":"2024-03-12T05:11:03.733684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984298+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:10:04.984298+0000","last_peered":"2024-03-12T05:10:04.984298+0000","last_clean":"2024-03-12T05:10:04.984298+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:10:04.984298+0000","last_undegraded":"2024-03-12T05:10:04.984298+0000","last_fullsized":"2024-03-12T05:10:04.984298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749472+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:11:03.749472+0000","last_peered":"2024-03-12T05:11:03.749472+0000","last_clean":"2024-03-12T05:11:03.749472+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:11:03.749472+0000","last_undegraded":"2024-03-12T05:11:03.749472+0000","last_fullsized":"2024-03-12T05:11:03.749472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.000638+0000","last_change":"2024-03-12T05:10:05.000161+0000","last_active":"2024-03-12T05:10:05.000638+0000","last_peered":"2024-03-12T05:10:05.000638+0000","last_clean":"2024-03-12T05:10:05.000638+0000","last_became_active":"2024-03-12T05:10:04.985941+0000","last_became_peered":"2024-03-12T05:10:04.985941+0000","last_unstale":"2024-03-12T05:10:05.000638+0000","last_undegraded":"2024-03-12T05:10:05.000638+0000","last_fullsized":"2024-03-12T05:10:05.000638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:34.170134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745090+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:11:03.745090+0000","last_peered":"2024-03-12T05:11:03.745090+0000","last_clean":"2024-03-12T05:11:03.745090+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:11:03.745090+0000","last_undegraded":"2024-03-12T05:11:03.745090+0000","last_fullsized":"2024-03-12T05:11:03.745090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984347+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:10:04.984347+0000","last_peered":"2024-03-12T05:10:04.984347+0000","last_clean":"2024-03-12T05:10:04.984347+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:10:04.984347+0000","last_undegraded":"2024-03-12T05:10:04.984347+0000","last_fullsized":"2024-03-12T05:10:04.984347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758283+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:11:03.758283+0000","last_peered":"2024-03-12T05:11:03.758283+0000","last_clean":"2024-03-12T05:11:03.758283+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:11:03.758283+0000","last_undegraded":"2024-03-12T05:11:03.758283+0000","last_fullsized":"2024-03-12T05:11:03.758283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985425+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:10:04.985425+0000","last_peered":"2024-03-12T05:10:04.985425+0000","last_clean":"2024-03-12T05:10:04.985425+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:10:04.985425+0000","last_undegraded":"2024-03-12T05:10:04.985425+0000","last_fullsized":"2024-03-12T05:10:04.985425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758796+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:11:03.758796+0000","last_peered":"2024-03-12T05:11:03.758796+0000","last_clean":"2024-03-12T05:11:03.758796+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:11:03.758796+0000","last_undegraded":"2024-03-12T05:11:03.758796+0000","last_fullsized":"2024-03-12T05:11:03.758796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990547+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:10:04.990547+0000","last_peered":"2024-03-12T05:10:04.990547+0000","last_clean":"2024-03-12T05:10:04.990547+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:10:04.990547+0000","last_undegraded":"2024-03-12T05:10:04.990547+0000","last_fullsized":"2024-03-12T05:10:04.990547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749176+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:11:03.749176+0000","last_peered":"2024-03-12T05:11:03.749176+0000","last_clean":"2024-03-12T05:11:03.749176+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:11:03.749176+0000","last_undegraded":"2024-03-12T05:11:03.749176+0000","last_fullsized":"2024-03-12T05:11:03.749176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.994225+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:10:04.994225+0000","last_peered":"2024-03-12T05:10:04.994225+0000","last_clean":"2024-03-12T05:10:04.994225+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:10:04.994225+0000","last_undegraded":"2024-03-12T05:10:04.994225+0000","last_fullsized":"2024-03-12T05:10:04.994225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751718+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:11:03.751718+0000","last_peered":"2024-03-12T05:11:03.751718+0000","last_clean":"2024-03-12T05:11:03.751718+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:11:03.751718+0000","last_undegraded":"2024-03-12T05:11:03.751718+0000","last_fullsized":"2024-03-12T05:11:03.751718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749780+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:11:03.749780+0000","last_peered":"2024-03-12T05:11:03.749780+0000","last_clean":"2024-03-12T05:11:03.749780+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:11:03.749780+0000","last_undegraded":"2024-03-12T05:11:03.749780+0000","last_fullsized":"2024-03-12T05:11:03.749780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743933+0000","last_change":"2024-03-12T05:10:04.992158+0000","last_active":"2024-03-12T05:11:03.743933+0000","last_peered":"2024-03-12T05:11:03.743933+0000","last_clean":"2024-03-12T05:11:03.743933+0000","last_became_active":"2024-03-12T05:10:04.991721+0000","last_became_peered":"2024-03-12T05:10:04.991721+0000","last_unstale":"2024-03-12T05:11:03.743933+0000","last_undegraded":"2024-03-12T05:11:03.743933+0000","last_fullsized":"2024-03-12T05:11:03.743933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:24.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748140+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:11:03.748140+0000","last_peered":"2024-03-12T05:11:03.748140+0000","last_clean":"2024-03-12T05:11:03.748140+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:11:03.748140+0000","last_undegraded":"2024-03-12T05:11:03.748140+0000","last_fullsized":"2024-03-12T05:11:03.748140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748105+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:11:03.748105+0000","last_peered":"2024-03-12T05:11:03.748105+0000","last_clean":"2024-03-12T05:11:03.748105+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:11:03.748105+0000","last_undegraded":"2024-03-12T05:11:03.748105+0000","last_fullsized":"2024-03-12T05:11:03.748105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748285+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:11:03.748285+0000","last_peered":"2024-03-12T05:11:03.748285+0000","last_clean":"2024-03-12T05:11:03.748285+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:11:03.748285+0000","last_undegraded":"2024-03-12T05:11:03.748285+0000","last_fullsized":"2024-03-12T05:11:03.748285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737606+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:11:03.737606+0000","last_peered":"2024-03-12T05:11:03.737606+0000","last_clean":"2024-03-12T05:11:03.737606+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:11:03.737606+0000","last_undegraded":"2024-03-12T05:11:03.737606+0000","last_fullsized":"2024-03-12T05:11:03.737606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743523+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:11:03.743523+0000","last_peered":"2024-03-12T05:11:03.743523+0000","last_clean":"2024-03-12T05:11:03.743523+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:11:03.743523+0000","last_undegraded":"2024-03-12T05:11:03.743523+0000","last_fullsized":"2024-03-12T05:11:03.743523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745476+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:11:03.745476+0000","last_peered":"2024-03-12T05:11:03.745476+0000","last_clean":"2024-03-12T05:11:03.745476+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:11:03.745476+0000","last_undegraded":"2024-03-12T05:11:03.745476+0000","last_fullsized":"2024-03-12T05:11:03.745476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744920+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:11:03.744920+0000","last_peered":"2024-03-12T05:11:03.744920+0000","last_clean":"2024-03-12T05:11:03.744920+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:11:03.744920+0000","last_undegraded":"2024-03-12T05:11:03.744920+0000","last_fullsized":"2024-03-12T05:11:03.744920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746188+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:11:03.746188+0000","last_peered":"2024-03-12T05:11:03.746188+0000","last_clean":"2024-03-12T05:11:03.746188+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:11:03.746188+0000","last_undegraded":"2024-03-12T05:11:03.746188+0000","last_fullsized":"2024-03-12T05:11:03.746188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760230+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:11:03.760230+0000","last_peered":"2024-03-12T05:11:03.760230+0000","last_clean":"2024-03-12T05:11:03.760230+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:11:03.760230+0000","last_undegraded":"2024-03-12T05:11:03.760230+0000","last_fullsized":"2024-03-12T05:11:03.760230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751293+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:11:03.751293+0000","last_peered":"2024-03-12T05:11:03.751293+0000","last_clean":"2024-03-12T05:11:03.751293+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:11:03.751293+0000","last_undegraded":"2024-03-12T05:11:03.751293+0000","last_fullsized":"2024-03-12T05:11:03.751293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747355+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:11:03.747355+0000","last_peered":"2024-03-12T05:11:03.747355+0000","last_clean":"2024-03-12T05:11:03.747355+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:11:03.747355+0000","last_undegraded":"2024-03-12T05:11:03.747355+0000","last_fullsized":"2024-03-12T05:11:03.747355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65969,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":455,"num_read_kb":298,"num_write":166,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1113222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":460150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1052673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1024000,"data_stored":520627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":460810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":459850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":460508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1051825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:11:12.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-12T05:11:12.663 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:13.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:12 smithi100 ceph-mon[32210]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-12T05:11:13.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:12 smithi100 ceph-mon[32210]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:13.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:12 smithi156 ceph-mon[36270]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-12T05:11:13.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:12 smithi156 ceph-mon[36270]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:13.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:12 smithi161 ceph-mon[36043]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-12T05:11:13.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:12 smithi161 ceph-mon[36043]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 -- 172.21.15.100:0/1841216424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc101cc0 msgr2=0x7faedc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 --2- 172.21.15.100:0/1841216424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc101cc0 0x7faedc102140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faecc0099c0 tx=0x7faecc009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 -- 172.21.15.100:0/1841216424 shutdown_connections 2024-03-12T05:11:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 --2- 172.21.15.100:0/1841216424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 0x7faedc108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 --2- 172.21.15.100:0/1841216424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faedc102680 0x7faedc10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 --2- 172.21.15.100:0/1841216424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc101cc0 0x7faedc102140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 -- 172.21.15.100:0/1841216424 >> 172.21.15.100:0/1841216424 conn(0x7faedc078da0 msgr2=0x7faedc0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 -- 172.21.15.100:0/1841216424 shutdown_connections 2024-03-12T05:11:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.246+0000 7faee0e9e700 1 -- 172.21.15.100:0/1841216424 wait complete. 2024-03-12T05:11:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 Processor -- start 2024-03-12T05:11:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 -- start start 2024-03-12T05:11:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faedc101cc0 0x7faedc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 0x7faedc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 0x7faedc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc19d0d0 con 0x7faedc102680 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc19d240 con 0x7faedc107ca0 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faee0e9e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc19d400 con 0x7faedc101cc0 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faed9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 0x7faedc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faed9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 0x7faedc198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46304/0 (socket says 172.21.15.100:46304) 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faed9d9b700 1 -- 172.21.15.100:0/2930715720 learned_addr learned my addr 172.21.15.100:0/2930715720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faeda59c700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faedc101cc0 0x7faedc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.247+0000 7faedad9d700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 0x7faedc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faed9d9b700 1 -- 172.21.15.100:0/2930715720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faedc101cc0 msgr2=0x7faedc198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faed9d9b700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faedc101cc0 0x7faedc198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faed9d9b700 1 -- 172.21.15.100:0/2930715720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 msgr2=0x7faedc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faed9d9b700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 0x7faedc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faed9d9b700 1 -- 172.21.15.100:0/2930715720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0095f0 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faedad9d700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 0x7faedc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faed9d9b700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 0x7faedc198780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faecc00b580 tx=0x7faecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc01d070 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faecc003f50 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc19d680 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc022d60 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.248+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc1a4f40 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.249+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc10a180 con 0x7faedc102680 2024-03-12T05:11:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.250+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7faecc022470 con 0x7faedc102680 2024-03-12T05:11:14.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.250+0000 7faecb7fe700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faec40a1ac0 0x7faec40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:14.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.251+0000 7faeda59c700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faec40a1ac0 0x7faec40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:14.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.251+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faecc0ce680 con 0x7faedc102680 2024-03-12T05:11:14.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.251+0000 7faeda59c700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faec40a1ac0 0x7faec40a3f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faed0009a10 tx=0x7faed0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:14.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.254+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faecc097920 con 0x7faedc102680 2024-03-12T05:11:14.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.442+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faedc199550 con 0x7faec40a1ac0 2024-03-12T05:11:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.449+0000 7faecb7fe700 1 -- 172.21.15.100:0/2930715720 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258828 (secure 0 0 0) 0x7faedc199550 con 0x7faec40a1ac0 2024-03-12T05:11:14.454 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:14.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faec40a1ac0 msgr2=0x7faec40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:14.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faec40a1ac0 0x7faec40a3f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faed0009a10 tx=0x7faed0009190 comp rx=0 tx=0).stop 2024-03-12T05:11:14.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 msgr2=0x7faedc198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:14.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 0x7faedc198780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faecc00b580 tx=0x7faecc004730 comp rx=0 tx=0).stop 2024-03-12T05:11:14.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 shutdown_connections 2024-03-12T05:11:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7faec40a1ac0 0x7faec40a3f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faedc107ca0 0x7faedc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faedc101cc0 0x7faedc198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 --2- 172.21.15.100:0/2930715720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faedc102680 0x7faedc198780 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 >> 172.21.15.100:0/2930715720 conn(0x7faedc078da0 msgr2=0x7faedc100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 shutdown_connections 2024-03-12T05:11:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:14.452+0000 7faee0e9e700 1 -- 172.21.15.100:0/2930715720 wait complete. 2024-03-12T05:11:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:11:15.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:14 smithi100 ceph-mon[32210]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-12T05:11:15.132 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-03-12T05:11:13.728982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656571,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":591,"num_read_kb":417,"num_write":349,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374988,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566836,"statfs":{"total":1151940427776,"available":1151556440064,"internally_reserved":0,"allocated":13676544,"data_stored":7481697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":33,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003080"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744383+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:11:03.744383+0000","last_peered":"2024-03-12T05:11:03.744383+0000","last_clean":"2024-03-12T05:11:03.744383+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:11:03.744383+0000","last_undegraded":"2024-03-12T05:11:03.744383+0000","last_fullsized":"2024-03-12T05:11:03.744383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.739101+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:11:03.739101+0000","last_peered":"2024-03-12T05:11:03.739101+0000","last_clean":"2024-03-12T05:11:03.739101+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:11:03.739101+0000","last_undegraded":"2024-03-12T05:11:03.739101+0000","last_fullsized":"2024-03-12T05:11:03.739101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757927+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:11:03.757927+0000","last_peered":"2024-03-12T05:11:03.757927+0000","last_clean":"2024-03-12T05:11:03.757927+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:11:03.757927+0000","last_undegraded":"2024-03-12T05:11:03.757927+0000","last_fullsized":"2024-03-12T05:11:03.757927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733109+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:11:03.733109+0000","last_peered":"2024-03-12T05:11:03.733109+0000","last_clean":"2024-03-12T05:11:03.733109+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:11:03.733109+0000","last_undegraded":"2024-03-12T05:11:03.733109+0000","last_fullsized":"2024-03-12T05:11:03.733109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743525+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:11:03.743525+0000","last_peered":"2024-03-12T05:11:03.743525+0000","last_clean":"2024-03-12T05:11:03.743525+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:11:03.743525+0000","last_undegraded":"2024-03-12T05:11:03.743525+0000","last_fullsized":"2024-03-12T05:11:03.743525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745830+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:11:03.745830+0000","last_peered":"2024-03-12T05:11:03.745830+0000","last_clean":"2024-03-12T05:11:03.745830+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:11:03.745830+0000","last_undegraded":"2024-03-12T05:11:03.745830+0000","last_fullsized":"2024-03-12T05:11:03.745830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750437+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:11:03.750437+0000","last_peered":"2024-03-12T05:11:03.750437+0000","last_clean":"2024-03-12T05:11:03.750437+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:11:03.750437+0000","last_undegraded":"2024-03-12T05:11:03.750437+0000","last_fullsized":"2024-03-12T05:11:03.750437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747959+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:11:03.747959+0000","last_peered":"2024-03-12T05:11:03.747959+0000","last_clean":"2024-03-12T05:11:03.747959+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:11:03.747959+0000","last_undegraded":"2024-03-12T05:11:03.747959+0000","last_fullsized":"2024-03-12T05:11:03.747959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.752826+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:11:03.752826+0000","last_peered":"2024-03-12T05:11:03.752826+0000","last_clean":"2024-03-12T05:11:03.752826+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:11:03.752826+0000","last_undegraded":"2024-03-12T05:11:03.752826+0000","last_fullsized":"2024-03-12T05:11:03.752826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744896+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:11:03.744896+0000","last_peered":"2024-03-12T05:11:03.744896+0000","last_clean":"2024-03-12T05:11:03.744896+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:11:03.744896+0000","last_undegraded":"2024-03-12T05:11:03.744896+0000","last_fullsized":"2024-03-12T05:11:03.744896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758668+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:11:03.758668+0000","last_peered":"2024-03-12T05:11:03.758668+0000","last_clean":"2024-03-12T05:11:03.758668+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:11:03.758668+0000","last_undegraded":"2024-03-12T05:11:03.758668+0000","last_fullsized":"2024-03-12T05:11:03.758668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748754+0000","last_change":"2024-03-12T05:10:04.986947+0000","last_active":"2024-03-12T05:11:03.748754+0000","last_peered":"2024-03-12T05:11:03.748754+0000","last_clean":"2024-03-12T05:11:03.748754+0000","last_became_active":"2024-03-12T05:10:04.986526+0000","last_became_peered":"2024-03-12T05:10:04.986526+0000","last_unstale":"2024-03-12T05:11:03.748754+0000","last_undegraded":"2024-03-12T05:11:03.748754+0000","last_fullsized":"2024-03-12T05:11:03.748754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:30:47.611206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743287+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:11:03.743287+0000","last_peered":"2024-03-12T05:11:03.743287+0000","last_clean":"2024-03-12T05:11:03.743287+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:11:03.743287+0000","last_undegraded":"2024-03-12T05:11:03.743287+0000","last_fullsized":"2024-03-12T05:11:03.743287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749425+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:11:03.749425+0000","last_peered":"2024-03-12T05:11:03.749425+0000","last_clean":"2024-03-12T05:11:03.749425+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:11:03.749425+0000","last_undegraded":"2024-03-12T05:11:03.749425+0000","last_fullsized":"2024-03-12T05:11:03.749425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.740596+0000","last_change":"2024-03-12T05:10:04.989251+0000","last_active":"2024-03-12T05:11:03.740596+0000","last_peered":"2024-03-12T05:11:03.740596+0000","last_clean":"2024-03-12T05:11:03.740596+0000","last_became_active":"2024-03-12T05:10:04.988845+0000","last_became_peered":"2024-03-12T05:10:04.988845+0000","last_unstale":"2024-03-12T05:11:03.740596+0000","last_undegraded":"2024-03-12T05:11:03.740596+0000","last_fullsized":"2024-03-12T05:11:03.740596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:21:42.513931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760056+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:11:03.760056+0000","last_peered":"2024-03-12T05:11:03.760056+0000","last_clean":"2024-03-12T05:11:03.760056+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:11:03.760056+0000","last_undegraded":"2024-03-12T05:11:03.760056+0000","last_fullsized":"2024-03-12T05:11:03.760056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744673+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:11:03.744673+0000","last_peered":"2024-03-12T05:11:03.744673+0000","last_clean":"2024-03-12T05:11:03.744673+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:11:03.744673+0000","last_undegraded":"2024-03-12T05:11:03.744673+0000","last_fullsized":"2024-03-12T05:11:03.744673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749801+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:11:03.749801+0000","last_peered":"2024-03-12T05:11:03.749801+0000","last_clean":"2024-03-12T05:11:03.749801+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:11:03.749801+0000","last_undegraded":"2024-03-12T05:11:03.749801+0000","last_fullsized":"2024-03-12T05:11:03.749801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748000+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:11:03.748000+0000","last_peered":"2024-03-12T05:11:03.748000+0000","last_clean":"2024-03-12T05:11:03.748000+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:11:03.748000+0000","last_undegraded":"2024-03-12T05:11:03.748000+0000","last_fullsized":"2024-03-12T05:11:03.748000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983601+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:10:04.983601+0000","last_peered":"2024-03-12T05:10:04.983601+0000","last_clean":"2024-03-12T05:10:04.983601+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:10:04.983601+0000","last_undegraded":"2024-03-12T05:10:04.983601+0000","last_fullsized":"2024-03-12T05:10:04.983601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745669+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:11:03.745669+0000","last_peered":"2024-03-12T05:11:03.745669+0000","last_clean":"2024-03-12T05:11:03.745669+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:11:03.745669+0000","last_undegraded":"2024-03-12T05:11:03.745669+0000","last_fullsized":"2024-03-12T05:11:03.745669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985285+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:10:04.985285+0000","last_peered":"2024-03-12T05:10:04.985285+0000","last_clean":"2024-03-12T05:10:04.985285+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:10:04.985285+0000","last_undegraded":"2024-03-12T05:10:04.985285+0000","last_fullsized":"2024-03-12T05:10:04.985285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757822+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:11:03.757822+0000","last_peered":"2024-03-12T05:11:03.757822+0000","last_clean":"2024-03-12T05:11:03.757822+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:11:03.757822+0000","last_undegraded":"2024-03-12T05:11:03.757822+0000","last_fullsized":"2024-03-12T05:11:03.757822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985028+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:10:04.985028+0000","last_peered":"2024-03-12T05:10:04.985028+0000","last_clean":"2024-03-12T05:10:04.985028+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:10:04.985028+0000","last_undegraded":"2024-03-12T05:10:04.985028+0000","last_fullsized":"2024-03-12T05:10:04.985028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758666+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:11:03.758666+0000","last_peered":"2024-03-12T05:11:03.758666+0000","last_clean":"2024-03-12T05:11:03.758666+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:11:03.758666+0000","last_undegraded":"2024-03-12T05:11:03.758666+0000","last_fullsized":"2024-03-12T05:11:03.758666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985370+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:10:04.985370+0000","last_peered":"2024-03-12T05:10:04.985370+0000","last_clean":"2024-03-12T05:10:04.985370+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:10:04.985370+0000","last_undegraded":"2024-03-12T05:10:04.985370+0000","last_fullsized":"2024-03-12T05:10:04.985370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748647+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:11:03.748647+0000","last_peered":"2024-03-12T05:11:03.748647+0000","last_clean":"2024-03-12T05:11:03.748647+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:11:03.748647+0000","last_undegraded":"2024-03-12T05:11:03.748647+0000","last_fullsized":"2024-03-12T05:11:03.748647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:08.087872+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:11:08.087872+0000","last_peered":"2024-03-12T05:11:08.087872+0000","last_clean":"2024-03-12T05:11:08.087872+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:11:08.087872+0000","last_undegraded":"2024-03-12T05:11:08.087872+0000","last_fullsized":"2024-03-12T05:11:08.087872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748799+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:11:03.748799+0000","last_peered":"2024-03-12T05:11:03.748799+0000","last_clean":"2024-03-12T05:11:03.748799+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:11:03.748799+0000","last_undegraded":"2024-03-12T05:11:03.748799+0000","last_fullsized":"2024-03-12T05:11:03.748799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983832+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:10:04.983832+0000","last_peered":"2024-03-12T05:10:04.983832+0000","last_clean":"2024-03-12T05:10:04.983832+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:10:04.983832+0000","last_undegraded":"2024-03-12T05:10:04.983832+0000","last_fullsized":"2024-03-12T05:10:04.983832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748879+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:11:03.748879+0000","last_peered":"2024-03-12T05:11:03.748879+0000","last_clean":"2024-03-12T05:11:03.748879+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:11:03.748879+0000","last_undegraded":"2024-03-12T05:11:03.748879+0000","last_fullsized":"2024-03-12T05:11:03.748879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.993890+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:10:04.993890+0000","last_peered":"2024-03-12T05:10:04.993890+0000","last_clean":"2024-03-12T05:10:04.993890+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:10:04.993890+0000","last_undegraded":"2024-03-12T05:10:04.993890+0000","last_fullsized":"2024-03-12T05:10:04.993890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745430+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:11:03.745430+0000","last_peered":"2024-03-12T05:11:03.745430+0000","last_clean":"2024-03-12T05:11:03.745430+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:11:03.745430+0000","last_undegraded":"2024-03-12T05:11:03.745430+0000","last_fullsized":"2024-03-12T05:11:03.745430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990178+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:10:04.990178+0000","last_peered":"2024-03-12T05:10:04.990178+0000","last_clean":"2024-03-12T05:10:04.990178+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:10:04.990178+0000","last_undegraded":"2024-03-12T05:10:04.990178+0000","last_fullsized":"2024-03-12T05:10:04.990178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748530+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:11:03.748530+0000","last_peered":"2024-03-12T05:11:03.748530+0000","last_clean":"2024-03-12T05:11:03.748530+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:11:03.748530+0000","last_undegraded":"2024-03-12T05:11:03.748530+0000","last_fullsized":"2024-03-12T05:11:03.748530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:09.088847+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:11:09.088847+0000","last_peered":"2024-03-12T05:11:09.088847+0000","last_clean":"2024-03-12T05:11:09.088847+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:11:09.088847+0000","last_undegraded":"2024-03-12T05:11:09.088847+0000","last_fullsized":"2024-03-12T05:11:09.088847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.756261+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:11:03.756261+0000","last_peered":"2024-03-12T05:11:03.756261+0000","last_clean":"2024-03-12T05:11:03.756261+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:11:03.756261+0000","last_undegraded":"2024-03-12T05:11:03.756261+0000","last_fullsized":"2024-03-12T05:11:03.756261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981550+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:10:04.981550+0000","last_peered":"2024-03-12T05:10:04.981550+0000","last_clean":"2024-03-12T05:10:04.981550+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:10:04.981550+0000","last_undegraded":"2024-03-12T05:10:04.981550+0000","last_fullsized":"2024-03-12T05:10:04.981550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748739+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:11:03.748739+0000","last_peered":"2024-03-12T05:11:03.748739+0000","last_clean":"2024-03-12T05:11:03.748739+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:11:03.748739+0000","last_undegraded":"2024-03-12T05:11:03.748739+0000","last_fullsized":"2024-03-12T05:11:03.748739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990717+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:10:04.990717+0000","last_peered":"2024-03-12T05:10:04.990717+0000","last_clean":"2024-03-12T05:10:04.990717+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:10:04.990717+0000","last_undegraded":"2024-03-12T05:10:04.990717+0000","last_fullsized":"2024-03-12T05:10:04.990717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.759675+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:11:03.759675+0000","last_peered":"2024-03-12T05:11:03.759675+0000","last_clean":"2024-03-12T05:11:03.759675+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:11:03.759675+0000","last_undegraded":"2024-03-12T05:11:03.759675+0000","last_fullsized":"2024-03-12T05:11:03.759675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:10.090580+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:11:10.090580+0000","last_peered":"2024-03-12T05:11:10.090580+0000","last_clean":"2024-03-12T05:11:10.090580+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:11:10.090580+0000","last_undegraded":"2024-03-12T05:11:10.090580+0000","last_fullsized":"2024-03-12T05:11:10.090580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751355+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:11:03.751355+0000","last_peered":"2024-03-12T05:11:03.751355+0000","last_clean":"2024-03-12T05:11:03.751355+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:11:03.751355+0000","last_undegraded":"2024-03-12T05:11:03.751355+0000","last_fullsized":"2024-03-12T05:11:03.751355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.998175+0000","last_change":"2024-03-12T05:10:04.997655+0000","last_active":"2024-03-12T05:10:04.998175+0000","last_peered":"2024-03-12T05:10:04.998175+0000","last_clean":"2024-03-12T05:10:04.998175+0000","last_became_active":"2024-03-12T05:10:04.985974+0000","last_became_peered":"2024-03-12T05:10:04.985974+0000","last_unstale":"2024-03-12T05:10:04.998175+0000","last_undegraded":"2024-03-12T05:10:04.998175+0000","last_fullsized":"2024-03-12T05:10:04.998175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:50:52.646109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751063+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:11:03.751063+0000","last_peered":"2024-03-12T05:11:03.751063+0000","last_clean":"2024-03-12T05:11:03.751063+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:11:03.751063+0000","last_undegraded":"2024-03-12T05:11:03.751063+0000","last_fullsized":"2024-03-12T05:11:03.751063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.989829+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:10:04.989829+0000","last_peered":"2024-03-12T05:10:04.989829+0000","last_clean":"2024-03-12T05:10:04.989829+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:10:04.989829+0000","last_undegraded":"2024-03-12T05:10:04.989829+0000","last_fullsized":"2024-03-12T05:10:04.989829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750601+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:11:03.750601+0000","last_peered":"2024-03-12T05:11:03.750601+0000","last_clean":"2024-03-12T05:11:03.750601+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:11:03.750601+0000","last_undegraded":"2024-03-12T05:11:03.750601+0000","last_fullsized":"2024-03-12T05:11:03.750601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:09.100981+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:11:09.100981+0000","last_peered":"2024-03-12T05:11:09.100981+0000","last_clean":"2024-03-12T05:11:09.100981+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:11:09.100981+0000","last_undegraded":"2024-03-12T05:11:09.100981+0000","last_fullsized":"2024-03-12T05:11:09.100981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751248+0000","last_change":"2024-03-12T05:10:04.985396+0000","last_active":"2024-03-12T05:11:03.751248+0000","last_peered":"2024-03-12T05:11:03.751248+0000","last_clean":"2024-03-12T05:11:03.751248+0000","last_became_active":"2024-03-12T05:10:04.984994+0000","last_became_peered":"2024-03-12T05:10:04.984994+0000","last_unstale":"2024-03-12T05:11:03.751248+0000","last_undegraded":"2024-03-12T05:11:03.751248+0000","last_fullsized":"2024-03-12T05:11:03.751248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:26:42.993458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984101+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:10:04.984101+0000","last_peered":"2024-03-12T05:10:04.984101+0000","last_clean":"2024-03-12T05:10:04.984101+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:10:04.984101+0000","last_undegraded":"2024-03-12T05:10:04.984101+0000","last_fullsized":"2024-03-12T05:10:04.984101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.738306+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:11:03.738306+0000","last_peered":"2024-03-12T05:11:03.738306+0000","last_clean":"2024-03-12T05:11:03.738306+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:11:03.738306+0000","last_undegraded":"2024-03-12T05:11:03.738306+0000","last_fullsized":"2024-03-12T05:11:03.738306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984702+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:10:04.984702+0000","last_peered":"2024-03-12T05:10:04.984702+0000","last_clean":"2024-03-12T05:10:04.984702+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:10:04.984702+0000","last_undegraded":"2024-03-12T05:10:04.984702+0000","last_fullsized":"2024-03-12T05:10:04.984702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.742201+0000","last_change":"2024-03-12T05:10:04.995448+0000","last_active":"2024-03-12T05:11:03.742201+0000","last_peered":"2024-03-12T05:11:03.742201+0000","last_clean":"2024-03-12T05:11:03.742201+0000","last_became_active":"2024-03-12T05:10:04.995056+0000","last_became_peered":"2024-03-12T05:10:04.995056+0000","last_unstale":"2024-03-12T05:11:03.742201+0000","last_undegraded":"2024-03-12T05:11:03.742201+0000","last_fullsized":"2024-03-12T05:11:03.742201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:23:27.643478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983622+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:10:04.983622+0000","last_peered":"2024-03-12T05:10:04.983622+0000","last_clean":"2024-03-12T05:10:04.983622+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:10:04.983622+0000","last_undegraded":"2024-03-12T05:10:04.983622+0000","last_fullsized":"2024-03-12T05:10:04.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747337+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:11:03.747337+0000","last_peered":"2024-03-12T05:11:03.747337+0000","last_clean":"2024-03-12T05:11:03.747337+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:11:03.747337+0000","last_undegraded":"2024-03-12T05:11:03.747337+0000","last_fullsized":"2024-03-12T05:11:03.747337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981281+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:10:04.981281+0000","last_peered":"2024-03-12T05:10:04.981281+0000","last_clean":"2024-03-12T05:10:04.981281+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:10:04.981281+0000","last_undegraded":"2024-03-12T05:10:04.981281+0000","last_fullsized":"2024-03-12T05:10:04.981281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737885+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:11:03.737885+0000","last_peered":"2024-03-12T05:11:03.737885+0000","last_clean":"2024-03-12T05:11:03.737885+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:11:03.737885+0000","last_undegraded":"2024-03-12T05:11:03.737885+0000","last_fullsized":"2024-03-12T05:11:03.737885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745048+0000","last_change":"2024-03-12T05:10:04.997291+0000","last_active":"2024-03-12T05:11:03.745048+0000","last_peered":"2024-03-12T05:11:03.745048+0000","last_clean":"2024-03-12T05:11:03.745048+0000","last_became_active":"2024-03-12T05:10:04.996814+0000","last_became_peered":"2024-03-12T05:10:04.996814+0000","last_unstale":"2024-03-12T05:11:03.745048+0000","last_undegraded":"2024-03-12T05:11:03.745048+0000","last_fullsized":"2024-03-12T05:11:03.745048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:42.585416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:08.098516+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:11:08.098516+0000","last_peered":"2024-03-12T05:11:08.098516+0000","last_clean":"2024-03-12T05:11:08.098516+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:11:08.098516+0000","last_undegraded":"2024-03-12T05:11:08.098516+0000","last_fullsized":"2024-03-12T05:11:08.098516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746635+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:11:03.746635+0000","last_peered":"2024-03-12T05:11:03.746635+0000","last_clean":"2024-03-12T05:11:03.746635+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:11:03.746635+0000","last_undegraded":"2024-03-12T05:11:03.746635+0000","last_fullsized":"2024-03-12T05:11:03.746635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996155+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:10:04.996155+0000","last_peered":"2024-03-12T05:10:04.996155+0000","last_clean":"2024-03-12T05:10:04.996155+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:10:04.996155+0000","last_undegraded":"2024-03-12T05:10:04.996155+0000","last_fullsized":"2024-03-12T05:10:04.996155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981148+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:10:04.981148+0000","last_peered":"2024-03-12T05:10:04.981148+0000","last_clean":"2024-03-12T05:10:04.981148+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:10:04.981148+0000","last_undegraded":"2024-03-12T05:10:04.981148+0000","last_fullsized":"2024-03-12T05:10:04.981148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748237+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:11:03.748237+0000","last_peered":"2024-03-12T05:11:03.748237+0000","last_clean":"2024-03-12T05:11:03.748237+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:11:03.748237+0000","last_undegraded":"2024-03-12T05:11:03.748237+0000","last_fullsized":"2024-03-12T05:11:03.748237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981395+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:10:04.981395+0000","last_peered":"2024-03-12T05:10:04.981395+0000","last_clean":"2024-03-12T05:10:04.981395+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:10:04.981395+0000","last_undegraded":"2024-03-12T05:10:04.981395+0000","last_fullsized":"2024-03-12T05:10:04.981395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984811+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:10:04.984811+0000","last_peered":"2024-03-12T05:10:04.984811+0000","last_clean":"2024-03-12T05:10:04.984811+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:10:04.984811+0000","last_undegraded":"2024-03-12T05:10:04.984811+0000","last_fullsized":"2024-03-12T05:10:04.984811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744189+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:11:03.744189+0000","last_peered":"2024-03-12T05:11:03.744189+0000","last_clean":"2024-03-12T05:11:03.744189+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:11:03.744189+0000","last_undegraded":"2024-03-12T05:11:03.744189+0000","last_fullsized":"2024-03-12T05:11:03.744189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:07.933870+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:10:07.933870+0000","last_peered":"2024-03-12T05:10:07.933870+0000","last_clean":"2024-03-12T05:10:07.933870+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:10:07.933870+0000","last_undegraded":"2024-03-12T05:10:07.933870+0000","last_fullsized":"2024-03-12T05:10:07.933870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750882+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:11:03.750882+0000","last_peered":"2024-03-12T05:11:03.750882+0000","last_clean":"2024-03-12T05:11:03.750882+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:11:03.750882+0000","last_undegraded":"2024-03-12T05:11:03.750882+0000","last_fullsized":"2024-03-12T05:11:03.750882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996107+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:10:04.996107+0000","last_peered":"2024-03-12T05:10:04.996107+0000","last_clean":"2024-03-12T05:10:04.996107+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:10:04.996107+0000","last_undegraded":"2024-03-12T05:10:04.996107+0000","last_fullsized":"2024-03-12T05:10:04.996107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748414+0000","last_change":"2024-03-12T05:10:04.997304+0000","last_active":"2024-03-12T05:11:03.748414+0000","last_peered":"2024-03-12T05:11:03.748414+0000","last_clean":"2024-03-12T05:11:03.748414+0000","last_became_active":"2024-03-12T05:10:04.996794+0000","last_became_peered":"2024-03-12T05:10:04.996794+0000","last_unstale":"2024-03-12T05:11:03.748414+0000","last_undegraded":"2024-03-12T05:11:03.748414+0000","last_fullsized":"2024-03-12T05:11:03.748414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:33.408355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.139977+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:10:05.139977+0000","last_peered":"2024-03-12T05:10:05.139977+0000","last_clean":"2024-03-12T05:10:05.139977+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:10:05.139977+0000","last_undegraded":"2024-03-12T05:10:05.139977+0000","last_fullsized":"2024-03-12T05:10:05.139977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733684+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:11:03.733684+0000","last_peered":"2024-03-12T05:11:03.733684+0000","last_clean":"2024-03-12T05:11:03.733684+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:11:03.733684+0000","last_undegraded":"2024-03-12T05:11:03.733684+0000","last_fullsized":"2024-03-12T05:11:03.733684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984298+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:10:04.984298+0000","last_peered":"2024-03-12T05:10:04.984298+0000","last_clean":"2024-03-12T05:10:04.984298+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:10:04.984298+0000","last_undegraded":"2024-03-12T05:10:04.984298+0000","last_fullsized":"2024-03-12T05:10:04.984298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749472+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:11:03.749472+0000","last_peered":"2024-03-12T05:11:03.749472+0000","last_clean":"2024-03-12T05:11:03.749472+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:11:03.749472+0000","last_undegraded":"2024-03-12T05:11:03.749472+0000","last_fullsized":"2024-03-12T05:11:03.749472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.000638+0000","last_change":"2024-03-12T05:10:05.000161+0000","last_active":"2024-03-12T05:10:05.000638+0000","last_peered":"2024-03-12T05:10:05.000638+0000","last_clean":"2024-03-12T05:10:05.000638+0000","last_became_active":"2024-03-12T05:10:04.985941+0000","last_became_peered":"2024-03-12T05:10:04.985941+0000","last_unstale":"2024-03-12T05:10:05.000638+0000","last_undegraded":"2024-03-12T05:10:05.000638+0000","last_fullsized":"2024-03-12T05:10:05.000638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:34.170134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745090+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:11:03.745090+0000","last_peered":"2024-03-12T05:11:03.745090+0000","last_clean":"2024-03-12T05:11:03.745090+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:11:03.745090+0000","last_undegraded":"2024-03-12T05:11:03.745090+0000","last_fullsized":"2024-03-12T05:11:03.745090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984347+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:10:04.984347+0000","last_peered":"2024-03-12T05:10:04.984347+0000","last_clean":"2024-03-12T05:10:04.984347+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:10:04.984347+0000","last_undegraded":"2024-03-12T05:10:04.984347+0000","last_fullsized":"2024-03-12T05:10:04.984347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758283+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:11:03.758283+0000","last_peered":"2024-03-12T05:11:03.758283+0000","last_clean":"2024-03-12T05:11:03.758283+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:11:03.758283+0000","last_undegraded":"2024-03-12T05:11:03.758283+0000","last_fullsized":"2024-03-12T05:11:03.758283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985425+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:10:04.985425+0000","last_peered":"2024-03-12T05:10:04.985425+0000","last_clean":"2024-03-12T05:10:04.985425+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:10:04.985425+0000","last_undegraded":"2024-03-12T05:10:04.985425+0000","last_fullsized":"2024-03-12T05:10:04.985425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758796+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:11:03.758796+0000","last_peered":"2024-03-12T05:11:03.758796+0000","last_clean":"2024-03-12T05:11:03.758796+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:11:03.758796+0000","last_undegraded":"2024-03-12T05:11:03.758796+0000","last_fullsized":"2024-03-12T05:11:03.758796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990547+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:10:04.990547+0000","last_peered":"2024-03-12T05:10:04.990547+0000","last_clean":"2024-03-12T05:10:04.990547+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:10:04.990547+0000","last_undegraded":"2024-03-12T05:10:04.990547+0000","last_fullsized":"2024-03-12T05:10:04.990547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749176+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:11:03.749176+0000","last_peered":"2024-03-12T05:11:03.749176+0000","last_clean":"2024-03-12T05:11:03.749176+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:11:03.749176+0000","last_undegraded":"2024-03-12T05:11:03.749176+0000","last_fullsized":"2024-03-12T05:11:03.749176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.994225+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:10:04.994225+0000","last_peered":"2024-03-12T05:10:04.994225+0000","last_clean":"2024-03-12T05:10:04.994225+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:10:04.994225+0000","last_undegraded":"2024-03-12T05:10:04.994225+0000","last_fullsized":"2024-03-12T05:10:04.994225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751718+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:11:03.751718+0000","last_peered":"2024-03-12T05:11:03.751718+0000","last_clean":"2024-03-12T05:11:03.751718+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:11:03.751718+0000","last_undegraded":"2024-03-12T05:11:03.751718+0000","last_fullsized":"2024-03-12T05:11:03.751718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749780+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:11:03.749780+0000","last_peered":"2024-03-12T05:11:03.749780+0000","last_clean":"2024-03-12T05:11:03.749780+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:11:03.749780+0000","last_undegraded":"2024-03-12T05:11:03.749780+0000","last_fullsized":"2024-03-12T05:11:03.749780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743933+0000","last_change":"2024-03-12T05:10:04.992158+0000","last_active":"2024-03-12T05:11:03.743933+0000","last_peered":"2024-03-12T05:11:03.743933+0000","last_clean":"2024-03-12T05:11:03.743933+0000","last_became_active":"2024-03-12T05:10:04.991721+0000","last_became_peered":"2024-03-12T05:10:04.991721+0000","last_unstale":"2024-03-12T05:11:03.743933+0000","last_undegraded":"2024-03-12T05:11:03.743933+0000","last_fullsized":"2024-03-12T05:11:03.743933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:24.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748140+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:11:03.748140+0000","last_peered":"2024-03-12T05:11:03.748140+0000","last_clean":"2024-03-12T05:11:03.748140+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:11:03.748140+0000","last_undegraded":"2024-03-12T05:11:03.748140+0000","last_fullsized":"2024-03-12T05:11:03.748140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748105+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:11:03.748105+0000","last_peered":"2024-03-12T05:11:03.748105+0000","last_clean":"2024-03-12T05:11:03.748105+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:11:03.748105+0000","last_undegraded":"2024-03-12T05:11:03.748105+0000","last_fullsized":"2024-03-12T05:11:03.748105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748285+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:11:03.748285+0000","last_peered":"2024-03-12T05:11:03.748285+0000","last_clean":"2024-03-12T05:11:03.748285+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:11:03.748285+0000","last_undegraded":"2024-03-12T05:11:03.748285+0000","last_fullsized":"2024-03-12T05:11:03.748285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737606+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:11:03.737606+0000","last_peered":"2024-03-12T05:11:03.737606+0000","last_clean":"2024-03-12T05:11:03.737606+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:11:03.737606+0000","last_undegraded":"2024-03-12T05:11:03.737606+0000","last_fullsized":"2024-03-12T05:11:03.737606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743523+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:11:03.743523+0000","last_peered":"2024-03-12T05:11:03.743523+0000","last_clean":"2024-03-12T05:11:03.743523+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:11:03.743523+0000","last_undegraded":"2024-03-12T05:11:03.743523+0000","last_fullsized":"2024-03-12T05:11:03.743523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745476+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:11:03.745476+0000","last_peered":"2024-03-12T05:11:03.745476+0000","last_clean":"2024-03-12T05:11:03.745476+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:11:03.745476+0000","last_undegraded":"2024-03-12T05:11:03.745476+0000","last_fullsized":"2024-03-12T05:11:03.745476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744920+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:11:03.744920+0000","last_peered":"2024-03-12T05:11:03.744920+0000","last_clean":"2024-03-12T05:11:03.744920+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:11:03.744920+0000","last_undegraded":"2024-03-12T05:11:03.744920+0000","last_fullsized":"2024-03-12T05:11:03.744920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746188+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:11:03.746188+0000","last_peered":"2024-03-12T05:11:03.746188+0000","last_clean":"2024-03-12T05:11:03.746188+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:11:03.746188+0000","last_undegraded":"2024-03-12T05:11:03.746188+0000","last_fullsized":"2024-03-12T05:11:03.746188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760230+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:11:03.760230+0000","last_peered":"2024-03-12T05:11:03.760230+0000","last_clean":"2024-03-12T05:11:03.760230+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:11:03.760230+0000","last_undegraded":"2024-03-12T05:11:03.760230+0000","last_fullsized":"2024-03-12T05:11:03.760230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751293+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:11:03.751293+0000","last_peered":"2024-03-12T05:11:03.751293+0000","last_clean":"2024-03-12T05:11:03.751293+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:11:03.751293+0000","last_undegraded":"2024-03-12T05:11:03.751293+0000","last_fullsized":"2024-03-12T05:11:03.751293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747355+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:11:03.747355+0000","last_peered":"2024-03-12T05:11:03.747355+0000","last_clean":"2024-03-12T05:11:03.747355+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:11:03.747355+0000","last_undegraded":"2024-03-12T05:11:03.747355+0000","last_fullsized":"2024-03-12T05:11:03.747355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65969,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":455,"num_read_kb":298,"num_write":166,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1113222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":460150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1052673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1024000,"data_stored":520627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":460810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":459850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":460508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1051825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:11:15.139 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-12T05:11:15.139 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-12T05:11:15.139 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-12T05:11:15.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph health --format=json 2024-03-12T05:11:15.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:14 smithi156 ceph-mon[36270]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-12T05:11:15.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:14 smithi161 ceph-mon[36043]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-12T05:11:15.376 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:16.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:15 smithi100 ceph-mon[32210]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:16.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:15 smithi156 ceph-mon[36270]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:16.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:15 smithi161 ceph-mon[36043]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 -- 172.21.15.100:0/2479653872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac100b70 msgr2=0x7ff6ac100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2479653872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac100b70 0x7ff6ac100ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009990 tx=0x7ff69c009ca0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 -- 172.21.15.100:0/2479653872 shutdown_connections 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2479653872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac101530 0x7ff6ac10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2479653872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac106b80 0x7ff6ac106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2479653872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac100b70 0x7ff6ac100ff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 -- 172.21.15.100:0/2479653872 >> 172.21.15.100:0/2479653872 conn(0x7ff6ac077aa0 msgr2=0x7ff6ac077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.046+0000 7ff6b448a700 1 -- 172.21.15.100:0/2479653872 shutdown_connections 2024-03-12T05:11:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.047+0000 7ff6b448a700 1 -- 172.21.15.100:0/2479653872 wait complete. 2024-03-12T05:11:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.047+0000 7ff6b448a700 1 Processor -- start 2024-03-12T05:11:17.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:16 smithi100 ceph-mon[32210]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-12T05:11:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.047+0000 7ff6b448a700 1 -- start start 2024-03-12T05:11:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b448a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac100b70 0x7ff6ac198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b448a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 0x7ff6ac198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b448a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac106b80 0x7ff6ac19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b448a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19ced0 con 0x7ff6ac101530 2024-03-12T05:11:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b448a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19d040 con 0x7ff6ac100b70 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b448a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19d200 con 0x7ff6ac106b80 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 0x7ff6ac198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 0x7ff6ac198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46328/0 (socket says 172.21.15.100:46328) 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 -- 172.21.15.100:0/2901443146 learned_addr learned my addr 172.21.15.100:0/2901443146 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b2a27700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac106b80 0x7ff6ac19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b2226700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac100b70 0x7ff6ac198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 -- 172.21.15.100:0/2901443146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac106b80 msgr2=0x7ff6ac19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac106b80 0x7ff6ac19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 -- 172.21.15.100:0/2901443146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac100b70 msgr2=0x7ff6ac198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac100b70 0x7ff6ac198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.048+0000 7ff6b1a25700 1 -- 172.21.15.100:0/2901443146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0095f0 con 0x7ff6ac101530 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.049+0000 7ff6b1a25700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 0x7ff6ac198580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0098e0 tx=0x7ff69c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.049+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c01d070 con 0x7ff6ac101530 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.049+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff69c003680 con 0x7ff6ac101530 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.049+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac19d480 con 0x7ff6ac101530 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.049+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0173f0 con 0x7ff6ac101530 2024-03-12T05:11:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.049+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac102b40 con 0x7ff6ac101530 2024-03-12T05:11:17.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.050+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac1021f0 con 0x7ff6ac101530 2024-03-12T05:11:17.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.051+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff69c022720 con 0x7ff6ac101530 2024-03-12T05:11:17.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.052+0000 7ff6a37fe700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff6980a1b40 0x7ff6980a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:17.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.052+0000 7ff6b2226700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff6980a1b40 0x7ff6980a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:17.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.052+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff69c0ce8b0 con 0x7ff6ac101530 2024-03-12T05:11:17.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.052+0000 7ff6b2226700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff6980a1b40 0x7ff6980a4000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8005950 tx=0x7ff6a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:17.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.056+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff69c097b10 con 0x7ff6ac101530 2024-03-12T05:11:17.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:16 smithi156 ceph-mon[36270]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-12T05:11:17.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:16 smithi161 ceph-mon[36043]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-12T05:11:17.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.297+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff6ac199350 con 0x7ff6ac101530 2024-03-12T05:11:17.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.297+0000 7ff6a37fe700 1 -- 172.21.15.100:0/2901443146 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff69c09b1d0 con 0x7ff6ac101530 2024-03-12T05:11:17.295 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:17.295 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff6980a1b40 msgr2=0x7ff6980a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff6980a1b40 0x7ff6980a4000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8005950 tx=0x7ff6a800b410 comp rx=0 tx=0).stop 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 msgr2=0x7ff6ac198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 0x7ff6ac198580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0098e0 tx=0x7ff69c00f690 comp rx=0 tx=0).stop 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 shutdown_connections 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff6980a1b40 0x7ff6980a4000 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac100b70 0x7ff6ac198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac106b80 0x7ff6ac19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 --2- 172.21.15.100:0/2901443146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6ac101530 0x7ff6ac198580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 >> 172.21.15.100:0/2901443146 conn(0x7ff6ac077aa0 msgr2=0x7ff6ac104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:17.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 shutdown_connections 2024-03-12T05:11:17.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.301+0000 7ff6b448a700 1 -- 172.21.15.100:0/2901443146 wait complete. 2024-03-12T05:11:17.863 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:17 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2901443146' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T05:11:17.889 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-12T05:11:17.889 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:17.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 -- 172.21.15.100:0/1798092805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 msgr2=0x7fa544101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:17.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 --2- 172.21.15.100:0/1798092805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544101710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:17.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 -- 172.21.15.100:0/1798092805 shutdown_connections 2024-03-12T05:11:17.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 --2- 172.21.15.100:0/1798092805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa544101c50 0x7fa5441084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 --2- 172.21.15.100:0/1798092805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544101710 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:17.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 --2- 172.21.15.100:0/1798092805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5441000a0 0x7fa5441004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 -- 172.21.15.100:0/1798092805 >> 172.21.15.100:0/1798092805 conn(0x7fa5440fb610 msgr2=0x7fa5440fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 -- 172.21.15.100:0/1798092805 shutdown_connections 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.998+0000 7fa54a815640 1 -- 172.21.15.100:0/1798092805 wait complete. 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 Processor -- start 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 -- start start 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5441000a0 0x7fa5441971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa544101c50 0x7fa54419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419ecb0 con 0x7fa544101290 2024-03-12T05:11:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419ee20 con 0x7fa5441000a0 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa54a815640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419efe0 con 0x7fa544101c50 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:17.999+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46348/0 (socket says 172.21.15.100:46348) 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 -- 172.21.15.100:0/4132867114 learned_addr learned my addr 172.21.15.100:0/4132867114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa548db2640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa544101c50 0x7fa54419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa543fff640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5441000a0 0x7fa5441971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 -- 172.21.15.100:0/4132867114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa544101c50 msgr2=0x7fa54419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa544101c50 0x7fa54419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 -- 172.21.15.100:0/4132867114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5441000a0 msgr2=0x7fa5441971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5441000a0 0x7fa5441971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 -- 172.21.15.100:0/4132867114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa538009560 con 0x7fa544101290 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa543fff640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5441000a0 0x7fa5441971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5437fe640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544197700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa538006010 tx=0x7fa5380040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380044c0 con 0x7fa544101290 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5380052d0 con 0x7fa544101290 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54419f260 con 0x7fa544101290 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538003e10 con 0x7fa544101290 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.000+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54419f770 con 0x7fa544101290 2024-03-12T05:11:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.001+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5441073c0 con 0x7fa544101290 2024-03-12T05:11:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.002+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa538005440 con 0x7fa544101290 2024-03-12T05:11:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.003+0000 7fa5417fa640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa52c0a1a30 0x7fa52c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.003+0000 7fa543fff640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa52c0a1a30 0x7fa52c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.003+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa5380efda0 con 0x7fa544101290 2024-03-12T05:11:18.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.003+0000 7fa543fff640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa52c0a1a30 0x7fa52c0a3ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5441010f0 tx=0x7fa534005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.006+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5380b8fb0 con 0x7fa544101290 2024-03-12T05:11:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.182+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa54410ce00 con 0x7fa544101290 2024-03-12T05:11:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.184+0000 7fa5417fa640 1 -- 172.21.15.100:0/4132867114 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7fa5380bc670 con 0x7fa544101290 2024-03-12T05:11:18.182 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:18.182 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:18.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa52c0a1a30 msgr2=0x7fa52c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa52c0a1a30 0x7fa52c0a3ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5441010f0 tx=0x7fa534005f90 comp rx=0 tx=0).stop 2024-03-12T05:11:18.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 msgr2=0x7fa544197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544197700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa538006010 tx=0x7fa5380040c0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 shutdown_connections 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fa52c0a1a30 0x7fa52c0a3ef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa544101c50 0x7fa54419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa544101290 0x7fa544197700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 --2- 172.21.15.100:0/4132867114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5441000a0 0x7fa5441971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 >> 172.21.15.100:0/4132867114 conn(0x7fa5440fb610 msgr2=0x7fa544104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 shutdown_connections 2024-03-12T05:11:18.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.186+0000 7fa54a815640 1 -- 172.21.15.100:0/4132867114 wait complete. 2024-03-12T05:11:18.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:18.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:18.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:17 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2901443146' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T05:11:18.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:17 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2901443146' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T05:11:18.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 -- 172.21.15.100:0/466760196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101c30 msgr2=0x7f0f5c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 --2- 172.21.15.100:0/466760196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101c30 0x7f0f5c1084c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009ca0 tx=0x7f0f5402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 -- 172.21.15.100:0/466760196 shutdown_connections 2024-03-12T05:11:18.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 --2- 172.21.15.100:0/466760196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101c30 0x7f0f5c1084c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 --2- 172.21.15.100:0/466760196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5c101270 0x7f0f5c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 --2- 172.21.15.100:0/466760196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c100080 0x7f0f5c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 -- 172.21.15.100:0/466760196 >> 172.21.15.100:0/466760196 conn(0x7f0f5c0fb610 msgr2=0x7f0f5c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 -- 172.21.15.100:0/466760196 shutdown_connections 2024-03-12T05:11:18.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.306+0000 7f0f63623640 1 -- 172.21.15.100:0/466760196 wait complete. 2024-03-12T05:11:18.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 Processor -- start 2024-03-12T05:11:18.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 -- start start 2024-03-12T05:11:18.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c100080 0x7f0f5c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 0x7f0f5c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5c101c30 0x7f0f5c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c19ecd0 con 0x7f0f5c101270 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c19ee40 con 0x7f0f5c101c30 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.307+0000 7f0f63623640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c19f000 con 0x7f0f5c100080 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 0x7f0f5c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 0x7f0f5c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46368/0 (socket says 172.21.15.100:46368) 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 -- 172.21.15.100:0/3764034019 learned_addr learned my addr 172.21.15.100:0/3764034019 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f613bf640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c100080 0x7f0f5c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 -- 172.21.15.100:0/3764034019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c100080 msgr2=0x7f0f5c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c100080 0x7f0f5c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 -- 172.21.15.100:0/3764034019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5c101c30 msgr2=0x7f0f5c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5c101c30 0x7f0f5c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 -- 172.21.15.100:0/3764034019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009900 con 0x7f0f5c101270 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f60bbe640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 0x7f0f5c197720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c890 tx=0x7f0f4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c004220 con 0x7f0f5c101270 2024-03-12T05:11:18.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c007e90 con 0x7f0f5c101270 2024-03-12T05:11:18.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c19f2e0 con 0x7f0f5c101270 2024-03-12T05:11:18.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c004a00 con 0x7f0f5c101270 2024-03-12T05:11:18.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.308+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c19f7f0 con 0x7f0f5c101270 2024-03-12T05:11:18.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.309+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c10ba90 con 0x7f0f5c101270 2024-03-12T05:11:18.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.310+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f0f4c0079b0 con 0x7f0f5c101270 2024-03-12T05:11:18.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.310+0000 7f0f527fc640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f480a1a30 0x7f0f480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.311+0000 7f0f613bf640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f480a1a30 0x7f0f480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.311+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0f4c0cd100 con 0x7f0f5c101270 2024-03-12T05:11:18.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.311+0000 7f0f613bf640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f480a1a30 0x7f0f480a3ef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005fd0 tx=0x7f0f5800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.315+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f4c096310 con 0x7f0f5c101270 2024-03-12T05:11:18.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.487+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f5c19fa70 con 0x7f0f5c101270 2024-03-12T05:11:18.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.487+0000 7f0f527fc640 1 -- 172.21.15.100:0/3764034019 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f4c0999d0 con 0x7f0f5c101270 2024-03-12T05:11:18.484 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:18.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f480a1a30 msgr2=0x7f0f480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f480a1a30 0x7f0f480a3ef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005fd0 tx=0x7f0f5800b390 comp rx=0 tx=0).stop 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 msgr2=0x7f0f5c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 0x7f0f5c197720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c890 tx=0x7f0f4c00cd50 comp rx=0 tx=0).stop 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 shutdown_connections 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0f480a1a30 0x7f0f480a3ef0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5c101c30 0x7f0f5c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c101270 0x7f0f5c197720 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 --2- 172.21.15.100:0/3764034019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c100080 0x7f0f5c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 >> 172.21.15.100:0/3764034019 conn(0x7f0f5c0fb610 msgr2=0x7f0f5c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 shutdown_connections 2024-03-12T05:11:18.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.489+0000 7f0f63623640 1 -- 172.21.15.100:0/3764034019 wait complete. 2024-03-12T05:11:18.497 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:18.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 -- 172.21.15.100:0/3942780756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 msgr2=0x7ffb1c106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 --2- 172.21.15.100:0/3942780756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c106990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009900 tx=0x7ffb0c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 -- 172.21.15.100:0/3942780756 shutdown_connections 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 --2- 172.21.15.100:0/3942780756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb1c106ed0 0x7ffb1c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 --2- 172.21.15.100:0/3942780756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c106990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 --2- 172.21.15.100:0/3942780756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb1c101c70 0x7ffb1c104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 -- 172.21.15.100:0/3942780756 >> 172.21.15.100:0/3942780756 conn(0x7ffb1c0fb620 msgr2=0x7ffb1c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 -- 172.21.15.100:0/3942780756 shutdown_connections 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.605+0000 7ffb2238a640 1 -- 172.21.15.100:0/3942780756 wait complete. 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 Processor -- start 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 -- start start 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb1c101c70 0x7ffb1c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb1c106ed0 0x7ffb1c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c19edd0 con 0x7ffb1c1045a0 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c19ef40 con 0x7ffb1c101c70 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb2238a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c19f100 con 0x7ffb1c106ed0 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46388/0 (socket says 172.21.15.100:46388) 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 -- 172.21.15.100:0/2653311473 learned_addr learned my addr 172.21.15.100:0/2653311473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1bfff640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb1c101c70 0x7ffb1c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb20927640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb1c106ed0 0x7ffb1c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 -- 172.21.15.100:0/2653311473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb1c106ed0 msgr2=0x7ffb1c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb1c106ed0 0x7ffb1c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 -- 172.21.15.100:0/2653311473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb1c101c70 msgr2=0x7ffb1c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb1c101c70 0x7ffb1c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.606+0000 7ffb1b7fe640 1 -- 172.21.15.100:0/2653311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009560 con 0x7ffb1c1045a0 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.607+0000 7ffb1b7fe640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c197820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c005ab0 tx=0x7ffb0c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.607+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0044c0 con 0x7ffb1c1045a0 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.607+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb0c0054e0 con 0x7ffb1c1045a0 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.607+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c19f380 con 0x7ffb1c1045a0 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.607+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c03e450 con 0x7ffb1c1045a0 2024-03-12T05:11:18.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.607+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c19f910 con 0x7ffb1c1045a0 2024-03-12T05:11:18.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.608+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1c0fd110 con 0x7ffb1c1045a0 2024-03-12T05:11:18.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.609+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ffb0c003780 con 0x7ffb1c1045a0 2024-03-12T05:11:18.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.609+0000 7ffb197fa640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ffb040a1a30 0x7ffb040a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.609+0000 7ffb1bfff640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ffb040a1a30 0x7ffb040a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.609+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb0c0f0280 con 0x7ffb1c1045a0 2024-03-12T05:11:18.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.610+0000 7ffb1bfff640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ffb040a1a30 0x7ffb040a3ef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb100058c0 tx=0x7ffb1000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.613+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb0c0b9490 con 0x7ffb1c1045a0 2024-03-12T05:11:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.787+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb1c10ce80 con 0x7ffb1c1045a0 2024-03-12T05:11:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.787+0000 7ffb197fa640 1 -- 172.21.15.100:0/2653311473 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb0c0430a0 con 0x7ffb1c1045a0 2024-03-12T05:11:18.785 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.789+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ffb040a1a30 msgr2=0x7ffb040a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.789+0000 7ffb2238a640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ffb040a1a30 0x7ffb040a3ef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb100058c0 tx=0x7ffb1000a2d0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.789+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 msgr2=0x7ffb1c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.789+0000 7ffb2238a640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c197820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c005ab0 tx=0x7ffb0c0042d0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 shutdown_connections 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ffb040a1a30 0x7ffb040a3ef0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb1c106ed0 0x7ffb1c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c1045a0 0x7ffb1c197820 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 --2- 172.21.15.100:0/2653311473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb1c101c70 0x7ffb1c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 >> 172.21.15.100:0/2653311473 conn(0x7ffb1c0fb620 msgr2=0x7ffb1c102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 shutdown_connections 2024-03-12T05:11:18.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.790+0000 7ffb2238a640 1 -- 172.21.15.100:0/2653311473 wait complete. 2024-03-12T05:11:18.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 -- 172.21.15.100:0/3460306893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101cd0 msgr2=0x7f83ac108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/3460306893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101cd0 0x7f83ac108560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009ca0 tx=0x7f83a402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 -- 172.21.15.100:0/3460306893 shutdown_connections 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/3460306893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101cd0 0x7f83ac108560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/3460306893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ac101310 0x7f83ac101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/3460306893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ac100120 0x7f83ac100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 -- 172.21.15.100:0/3460306893 >> 172.21.15.100:0/3460306893 conn(0x7f83ac0fb690 msgr2=0x7f83ac0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.908+0000 7f83b3d4b640 1 -- 172.21.15.100:0/3460306893 shutdown_connections 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 -- 172.21.15.100:0/3460306893 wait complete. 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 Processor -- start 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 -- start start 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ac100120 0x7f83ac06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 0x7f83ac19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ac101cd0 0x7f83ac19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac19edc0 con 0x7f83ac101310 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac19ef30 con 0x7f83ac100120 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b3d4b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac19f0f0 con 0x7f83ac101cd0 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b12e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 0x7f83ac19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b12e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 0x7f83ac19bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46392/0 (socket says 172.21.15.100:46392) 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.909+0000 7f83b12e6640 1 -- 172.21.15.100:0/1480205099 learned_addr learned my addr 172.21.15.100:0/1480205099 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b12e6640 1 -- 172.21.15.100:0/1480205099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ac101cd0 msgr2=0x7f83ac19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b12e6640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ac101cd0 0x7f83ac19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b12e6640 1 -- 172.21.15.100:0/1480205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ac100120 msgr2=0x7f83ac06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b12e6640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ac100120 0x7f83ac06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b12e6640 1 -- 172.21.15.100:0/1480205099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009900 con 0x7f83ac101310 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b12e6640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 0x7f83ac19bfc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f839c00b4a0 tx=0x7f839c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c010610 con 0x7f83ac101310 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac19f370 con 0x7f83ac101310 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f839c010c50 con 0x7f83ac101310 2024-03-12T05:11:18.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c017e20 con 0x7f83ac101310 2024-03-12T05:11:18.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.910+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac19f880 con 0x7f83ac101310 2024-03-12T05:11:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.911+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ac107440 con 0x7f83ac101310 2024-03-12T05:11:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.912+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f839c017480 con 0x7f83ac101310 2024-03-12T05:11:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.912+0000 7f83a2ffd640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f83980a1a30 0x7f83980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.913+0000 7f83b1ae7640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f83980a1a30 0x7f83980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.913+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f839c0cced0 con 0x7f83ac101310 2024-03-12T05:11:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.913+0000 7f83b1ae7640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f83980a1a30 0x7f83980a3ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83ac101170 tx=0x7f83a8005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:18.916+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f839c0960e0 con 0x7f83ac101310 2024-03-12T05:11:19.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:18 smithi100 ceph-mon[32210]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-12T05:11:19.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:18 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4132867114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:19.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:18 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3764034019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:19.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:18 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2653311473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:19.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.091+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83ac02cee0 con 0x7f83ac101310 2024-03-12T05:11:19.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.091+0000 7f83a2ffd640 1 -- 172.21.15.100:0/1480205099 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f839c0092b0 con 0x7f83ac101310 2024-03-12T05:11:19.089 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:19.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f83980a1a30 msgr2=0x7f83980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f83980a1a30 0x7f83980a3ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83ac101170 tx=0x7f83a8005ed0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 msgr2=0x7f83ac19bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 0x7f83ac19bfc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f839c00b4a0 tx=0x7f839c00b960 comp rx=0 tx=0).stop 2024-03-12T05:11:19.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 shutdown_connections 2024-03-12T05:11:19.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f83980a1a30 0x7f83980a3ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ac101cd0 0x7f83ac19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ac101310 0x7f83ac19bfc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 --2- 172.21.15.100:0/1480205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ac100120 0x7f83ac06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 >> 172.21.15.100:0/1480205099 conn(0x7f83ac0fb690 msgr2=0x7f83ac104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:19.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 shutdown_connections 2024-03-12T05:11:19.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.093+0000 7f83b3d4b640 1 -- 172.21.15.100:0/1480205099 wait complete. 2024-03-12T05:11:19.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:19.209 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:18 smithi156 ceph-mon[36270]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-12T05:11:19.210 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:18 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4132867114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:19.210 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:18 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3764034019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:19.210 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:18 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2653311473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:19.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 -- 172.21.15.100:0/522816425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1012a0 msgr2=0x7f7fac101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/522816425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1012a0 0x7f7fac101720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009900 tx=0x7f7f9402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:19.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 -- 172.21.15.100:0/522816425 shutdown_connections 2024-03-12T05:11:19.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/522816425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fac101c60 0x7f7fac1084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/522816425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1012a0 0x7f7fac101720 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/522816425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fac1000b0 0x7f7fac1004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 -- 172.21.15.100:0/522816425 >> 172.21.15.100:0/522816425 conn(0x7f7fac0fb660 msgr2=0x7f7fac0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:19.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 -- 172.21.15.100:0/522816425 shutdown_connections 2024-03-12T05:11:19.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.205+0000 7f7fb08ae640 1 -- 172.21.15.100:0/522816425 wait complete. 2024-03-12T05:11:19.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 Processor -- start 2024-03-12T05:11:19.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 -- start start 2024-03-12T05:11:19.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 0x7f7fac10a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fac1012a0 0x7f7fac108fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fac101c60 0x7f7fac1094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac109c50 con 0x7f7fac1000b0 2024-03-12T05:11:19.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac109dc0 con 0x7f7fac101c60 2024-03-12T05:11:19.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7fb08ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1a12b0 con 0x7f7fac1012a0 2024-03-12T05:11:19.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 0x7f7fac10a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 0x7f7fac10a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46408/0 (socket says 172.21.15.100:46408) 2024-03-12T05:11:19.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 -- 172.21.15.100:0/1465670299 learned_addr learned my addr 172.21.15.100:0/1465670299 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:19.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 -- 172.21.15.100:0/1465670299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fac1012a0 msgr2=0x7f7fac108fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fac1012a0 0x7f7fac108fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 -- 172.21.15.100:0/1465670299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fac101c60 msgr2=0x7f7fac1094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fac101c60 0x7f7fac1094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.206+0000 7f7faa59c640 1 -- 172.21.15.100:0/1465670299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94009560 con 0x7f7fac1000b0 2024-03-12T05:11:19.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.207+0000 7f7faa59c640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 0x7f7fac10a970 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00e9f0 tx=0x7f7f9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:19.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.207+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0096e0 con 0x7f7fac1000b0 2024-03-12T05:11:19.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.207+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c00cd80 con 0x7f7fac1000b0 2024-03-12T05:11:19.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.207+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac1a1560 con 0x7f7fac1000b0 2024-03-12T05:11:19.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.207+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c010430 con 0x7f7fac1000b0 2024-03-12T05:11:19.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.207+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac1a1a20 con 0x7f7fac1000b0 2024-03-12T05:11:19.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.208+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac1073d0 con 0x7f7fac1000b0 2024-03-12T05:11:19.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.209+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7f9c003680 con 0x7f7fac1000b0 2024-03-12T05:11:19.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.209+0000 7f7fa37fe640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7f980a1a30 0x7f7f980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.209+0000 7f7fa3fff640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7f980a1a30 0x7f7f980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.209+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f9c014070 con 0x7f7fac1000b0 2024-03-12T05:11:19.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.210+0000 7f7fa3fff640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7f980a1a30 0x7f7f980a3ef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f9402c6a0 tx=0x7f7f94000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:19.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.214+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f9c096af0 con 0x7f7fac1000b0 2024-03-12T05:11:19.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:18 smithi161 ceph-mon[36043]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-12T05:11:19.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:18 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4132867114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:19.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:18 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3764034019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:19.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:18 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2653311473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.424+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fac10cbc0 con 0x7f7fac1000b0 2024-03-12T05:11:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.425+0000 7f7fa37fe640 1 -- 172.21.15.100:0/1465670299 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f7f9c09a1b0 con 0x7f7fac1000b0 2024-03-12T05:11:19.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:09:59.961590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:11:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.426+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7f980a1a30 msgr2=0x7f7f980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.426+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7f980a1a30 0x7f7f980a3ef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f9402c6a0 tx=0x7f7f94000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.426+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 msgr2=0x7f7fac10a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.426+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 0x7f7fac10a970 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00e9f0 tx=0x7f7f9c00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 shutdown_connections 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7f980a1a30 0x7f7f980a3ef0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fac101c60 0x7f7fac1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fac1012a0 0x7f7fac108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 --2- 172.21.15.100:0/1465670299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1000b0 0x7f7fac10a970 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 >> 172.21.15.100:0/1465670299 conn(0x7f7fac0fb660 msgr2=0x7f7fac1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 shutdown_connections 2024-03-12T05:11:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.427+0000 7f7fb08ae640 1 -- 172.21.15.100:0/1465670299 wait complete. 2024-03-12T05:11:19.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:19.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 -- 172.21.15.100:0/3069561543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 msgr2=0x7ff420101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 --2- 172.21.15.100:0/3069561543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff420101710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff410009900 tx=0x7ff41002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 -- 172.21.15.100:0/3069561543 shutdown_connections 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 --2- 172.21.15.100:0/3069561543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff420101c50 0x7ff4201084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 --2- 172.21.15.100:0/3069561543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff420101710 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 --2- 172.21.15.100:0/3069561543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4201000a0 0x7ff4201004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 -- 172.21.15.100:0/3069561543 >> 172.21.15.100:0/3069561543 conn(0x7ff4200fb610 msgr2=0x7ff4200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 -- 172.21.15.100:0/3069561543 shutdown_connections 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 -- 172.21.15.100:0/3069561543 wait complete. 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 Processor -- start 2024-03-12T05:11:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.540+0000 7ff427301640 1 -- start start 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4201000a0 0x7ff420199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff4201999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff420101c50 0x7ff4201a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201a0f70 con 0x7ff420101290 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201a10e0 con 0x7ff4201000a0 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201a12a0 con 0x7ff420101c50 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff4201999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff4201999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46418/0 (socket says 172.21.15.100:46418) 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 -- 172.21.15.100:0/2924697737 learned_addr learned my addr 172.21.15.100:0/2924697737 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42589e640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff420101c50 0x7ff4201a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 -- 172.21.15.100:0/2924697737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff420101c50 msgr2=0x7ff4201a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42509d640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4201000a0 0x7ff420199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff420101c50 0x7ff4201a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 -- 172.21.15.100:0/2924697737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4201000a0 msgr2=0x7ff420199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4201000a0 0x7ff420199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 -- 172.21.15.100:0/2924697737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009560 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42589e640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff420101c50 0x7ff4201a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff42489c640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff4201999c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff410006010 tx=0x7ff4100040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4100044c0 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4100052d0 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4201a1520 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410003e10 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.541+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4201a1a30 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.542+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42010bab0 con 0x7ff420101290 2024-03-12T05:11:19.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.543+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff410005440 con 0x7ff420101290 2024-03-12T05:11:19.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.544+0000 7ff4167fc640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff40c0a1a30 0x7ff40c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.544+0000 7ff42509d640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff40c0a1a30 0x7ff40c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.544+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4100ef880 con 0x7ff420101290 2024-03-12T05:11:19.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.544+0000 7ff42509d640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff40c0a1a30 0x7ff40c0a3ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4201010f0 tx=0x7ff418009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:19.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.548+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4100b8a90 con 0x7ff420101290 2024-03-12T05:11:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.721+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff42010bc40 con 0x7ff420101290 2024-03-12T05:11:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.722+0000 7ff4167fc640 1 -- 172.21.15.100:0/2924697737 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7ff4100bc150 con 0x7ff420101290 2024-03-12T05:11:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:19.721 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.724+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff40c0a1a30 msgr2=0x7ff40c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.724+0000 7ff427301640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff40c0a1a30 0x7ff40c0a3ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4201010f0 tx=0x7ff418009500 comp rx=0 tx=0).stop 2024-03-12T05:11:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.724+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 msgr2=0x7ff4201999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.724+0000 7ff427301640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff4201999c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff410006010 tx=0x7ff4100040c0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 shutdown_connections 2024-03-12T05:11:19.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff40c0a1a30 0x7ff40c0a3ef0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff420101c50 0x7ff4201a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff420101290 0x7ff4201999c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 --2- 172.21.15.100:0/2924697737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4201000a0 0x7ff420199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 >> 172.21.15.100:0/2924697737 conn(0x7ff4200fb610 msgr2=0x7ff4200fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 shutdown_connections 2024-03-12T05:11:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.725+0000 7ff427301640 1 -- 172.21.15.100:0/2924697737 wait complete. 2024-03-12T05:11:19.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:19.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:19.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 -- 172.21.15.100:0/364550311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0100090 msgr2=0x7f21e01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 --2- 172.21.15.100:0/364550311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0100090 0x7f21e01004b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009900 tx=0x7f21d802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 -- 172.21.15.100:0/364550311 shutdown_connections 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 --2- 172.21.15.100:0/364550311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e0101c40 0x7f21e01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 --2- 172.21.15.100:0/364550311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101280 0x7f21e0101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 --2- 172.21.15.100:0/364550311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0100090 0x7f21e01004b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.840+0000 7f21e53b6640 1 -- 172.21.15.100:0/364550311 >> 172.21.15.100:0/364550311 conn(0x7f21e00fb620 msgr2=0x7f21e00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 -- 172.21.15.100:0/364550311 shutdown_connections 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 -- 172.21.15.100:0/364550311 wait complete. 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 Processor -- start 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 -- start start 2024-03-12T05:11:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e0100090 0x7f21e0104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 0x7f21e01027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101c40 0x7f21e0102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0103440 con 0x7f21e0101280 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e01035b0 con 0x7f21e0101c40 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21e53b6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e01a12b0 con 0x7f21e0100090 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21de7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 0x7f21e01027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21de7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 0x7f21e01027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46442/0 (socket says 172.21.15.100:46442) 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.841+0000 7f21de7fc640 1 -- 172.21.15.100:0/1916873337 learned_addr learned my addr 172.21.15.100:0/1916873337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21deffd640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e0100090 0x7f21e0104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21de7fc640 1 -- 172.21.15.100:0/1916873337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e0100090 msgr2=0x7f21e0104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21de7fc640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e0100090 0x7f21e0104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21de7fc640 1 -- 172.21.15.100:0/1916873337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101c40 msgr2=0x7f21e0102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21df7fe640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101c40 0x7f21e0102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21de7fc640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101c40 0x7f21e0102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21de7fc640 1 -- 172.21.15.100:0/1916873337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009560 con 0x7f21e0101280 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21df7fe640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101c40 0x7f21e0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21de7fc640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 0x7f21e01027a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21c800d980 tx=0x7f21c800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8009c30 con 0x7f21e0101280 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21c800b660 con 0x7f21e0101280 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e01a1560 con 0x7f21e0101280 2024-03-12T05:11:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800f6b0 con 0x7f21e0101280 2024-03-12T05:11:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.842+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e01a1a70 con 0x7f21e0101280 2024-03-12T05:11:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.843+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e010baa0 con 0x7f21e0101280 2024-03-12T05:11:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.844+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f21c8010460 con 0x7f21e0101280 2024-03-12T05:11:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.845+0000 7f21d7fff640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f21cc0a1a30 0x7f21cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.845+0000 7f21deffd640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f21cc0a1a30 0x7f21cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.845+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f21c80cd160 con 0x7f21e0101280 2024-03-12T05:11:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.845+0000 7f21deffd640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f21cc0a1a30 0x7f21cc0a3ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006010 tx=0x7f21d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:19.849+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21c8096370 con 0x7f21e0101280 2024-03-12T05:11:20.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.023+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21e01a1c90 con 0x7f21e0101280 2024-03-12T05:11:20.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.023+0000 7f21d7fff640 1 -- 172.21.15.100:0/1916873337 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21c8099a30 con 0x7f21e0101280 2024-03-12T05:11:20.022 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:20.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f21cc0a1a30 msgr2=0x7f21cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f21cc0a1a30 0x7f21cc0a3ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006010 tx=0x7f21d8000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 msgr2=0x7f21e01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 0x7f21e01027a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21c800d980 tx=0x7f21c800de40 comp rx=0 tx=0).stop 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 shutdown_connections 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f21cc0a1a30 0x7f21cc0a3ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21e0101c40 0x7f21e0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e0101280 0x7f21e01027a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 --2- 172.21.15.100:0/1916873337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e0100090 0x7f21e0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 >> 172.21.15.100:0/1916873337 conn(0x7f21e00fb620 msgr2=0x7f21e00696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 shutdown_connections 2024-03-12T05:11:20.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.025+0000 7f21e53b6640 1 -- 172.21.15.100:0/1916873337 wait complete. 2024-03-12T05:11:20.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:19 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1480205099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:20.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:19 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1465670299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:20.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:19 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2924697737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:20.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:20.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 -- 172.21.15.100:0/141250690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55381045e0 msgr2=0x7f55381069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 --2- 172.21.15.100:0/141250690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55381045e0 0x7f55381069d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5520009900 tx=0x7f552002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 -- 172.21.15.100:0/141250690 shutdown_connections 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 --2- 172.21.15.100:0/141250690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 0x7f5538109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 --2- 172.21.15.100:0/141250690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55381045e0 0x7f55381069d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 --2- 172.21.15.100:0/141250690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5538101cb0 0x7f55381040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 -- 172.21.15.100:0/141250690 >> 172.21.15.100:0/141250690 conn(0x7f55380fb620 msgr2=0x7f55380fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 -- 172.21.15.100:0/141250690 shutdown_connections 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.141+0000 7f553f081640 1 -- 172.21.15.100:0/141250690 wait complete. 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 Processor -- start 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 -- start start 2024-03-12T05:11:20.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 0x7f5538197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55381045e0 0x7f5538197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 0x7f553819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553819ed40 con 0x7f5538101cb0 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553819eeb0 con 0x7f5538106f10 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553f081640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553819f070 con 0x7f55381045e0 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 0x7f5538197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 0x7f5538197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46472/0 (socket says 172.21.15.100:46472) 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 -- 172.21.15.100:0/3423279697 learned_addr learned my addr 172.21.15.100:0/3423279697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f5537fff640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55381045e0 0x7f5538197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553d61e640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 0x7f553819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 -- 172.21.15.100:0/3423279697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55381045e0 msgr2=0x7f5538197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55381045e0 0x7f5538197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 -- 172.21.15.100:0/3423279697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 msgr2=0x7f553819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 0x7f553819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553ce1d640 1 -- 172.21.15.100:0/3423279697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520009560 con 0x7f5538101cb0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f553d61e640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 0x7f553819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.142+0000 7f5537fff640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55381045e0 0x7f5538197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.143+0000 7f553ce1d640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 0x7f5538197250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f552800d770 tx=0x7f552800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.143+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528009c30 con 0x7f5538101cb0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.143+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553819f350 con 0x7f5538101cb0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.143+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f552800de80 con 0x7f5538101cb0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.143+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004e80 con 0x7f5538101cb0 2024-03-12T05:11:20.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.143+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553819f7e0 con 0x7f5538101cb0 2024-03-12T05:11:20.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.144+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380fd110 con 0x7f5538101cb0 2024-03-12T05:11:20.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.145+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5528005020 con 0x7f5538101cb0 2024-03-12T05:11:20.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.145+0000 7f5535ffb640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55240a1a30 0x7f55240a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.145+0000 7f5537fff640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55240a1a30 0x7f55240a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.145+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f55280ccef0 con 0x7f5538101cb0 2024-03-12T05:11:20.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.146+0000 7f5537fff640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55240a1a30 0x7f55240a3ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5520006010 tx=0x7f5520005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:20.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.150+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5528096100 con 0x7f5538101cb0 2024-03-12T05:11:20.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:19 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1480205099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:20.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:19 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1465670299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:20.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:19 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2924697737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:20.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:19 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1480205099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:20.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:19 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1465670299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:20.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:19 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2924697737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.322+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f553802ce40 con 0x7f5538101cb0 2024-03-12T05:11:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.323+0000 7f5535ffb640 1 -- 172.21.15.100:0/3423279697 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5528017020 con 0x7f5538101cb0 2024-03-12T05:11:20.321 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:20.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55240a1a30 msgr2=0x7f55240a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55240a1a30 0x7f55240a3ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5520006010 tx=0x7f5520005be0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 msgr2=0x7f5538197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 0x7f5538197250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f552800d770 tx=0x7f552800dc30 comp rx=0 tx=0).stop 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 shutdown_connections 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f55240a1a30 0x7f55240a3ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5538106f10 0x7f553819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55381045e0 0x7f5538197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 --2- 172.21.15.100:0/3423279697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5538101cb0 0x7f5538197250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 >> 172.21.15.100:0/3423279697 conn(0x7f55380fb620 msgr2=0x7f55381026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 shutdown_connections 2024-03-12T05:11:20.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.325+0000 7f553f081640 1 -- 172.21.15.100:0/3423279697 wait complete. 2024-03-12T05:11:20.334 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:20.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 -- 172.21.15.100:0/3783110909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64101190 msgr2=0x7f9e641015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/3783110909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64101190 0x7f9e641015b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 -- 172.21.15.100:0/3783110909 shutdown_connections 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/3783110909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64102d40 0x7f9e641095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/3783110909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e64102380 0x7f9e64102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/3783110909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64101190 0x7f9e641015b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 -- 172.21.15.100:0/3783110909 >> 172.21.15.100:0/3783110909 conn(0x7f9e640fc720 msgr2=0x7f9e640feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 -- 172.21.15.100:0/3783110909 shutdown_connections 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.440+0000 7f9e68eb5640 1 -- 172.21.15.100:0/3783110909 wait complete. 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 Processor -- start 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 -- start start 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64101190 0x7f9e64074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 0x7f9e64073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e64102d40 0x7f9e64073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64074210 con 0x7f9e64102380 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64074380 con 0x7f9e64101190 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e68eb5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641a1310 con 0x7f9e64102d40 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 0x7f9e64073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 0x7f9e64073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46498/0 (socket says 172.21.15.100:46498) 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 -- 172.21.15.100:0/2635448123 learned_addr learned my addr 172.21.15.100:0/2635448123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e6259c640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64101190 0x7f9e64074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 -- 172.21.15.100:0/2635448123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e64102d40 msgr2=0x7f9e64073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e64102d40 0x7f9e64073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 -- 172.21.15.100:0/2635448123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64101190 msgr2=0x7f9e64074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64101190 0x7f9e64074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 -- 172.21.15.100:0/2635448123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54009560 con 0x7f9e64102380 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e6259c640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64101190 0x7f9e64074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.441+0000 7f9e61d9b640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 0x7f9e64073570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800d7a0 tx=0x7f9e5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.442+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58009c30 con 0x7f9e64102380 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.442+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e5800deb0 con 0x7f9e64102380 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.442+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e641a15c0 con 0x7f9e64102380 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.442+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004ea0 con 0x7f9e64102380 2024-03-12T05:11:20.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.442+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e641a1a80 con 0x7f9e64102380 2024-03-12T05:11:20.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.443+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e64108520 con 0x7f9e64102380 2024-03-12T05:11:20.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.444+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9e58005020 con 0x7f9e64102380 2024-03-12T05:11:20.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.444+0000 7f9e537fe640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9e4c0a1a30 0x7f9e4c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.445+0000 7f9e6259c640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9e4c0a1a30 0x7f9e4c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.445+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e580ccf10 con 0x7f9e64102380 2024-03-12T05:11:20.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.445+0000 7f9e6259c640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9e4c0a1a30 0x7f9e4c0a3ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e54036040 tx=0x7f9e54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:20.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.449+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e58096120 con 0x7f9e64102380 2024-03-12T05:11:20.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.622+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e6402cea0 con 0x7f9e64102380 2024-03-12T05:11:20.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.623+0000 7f9e537fe640 1 -- 172.21.15.100:0/2635448123 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e580170a0 con 0x7f9e64102380 2024-03-12T05:11:20.621 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9e4c0a1a30 msgr2=0x7f9e4c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9e4c0a1a30 0x7f9e4c0a3ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e54036040 tx=0x7f9e54000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 msgr2=0x7f9e64073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 0x7f9e64073570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800d7a0 tx=0x7f9e5800dc60 comp rx=0 tx=0).stop 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 shutdown_connections 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9e4c0a1a30 0x7f9e4c0a3ef0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e64102d40 0x7f9e64073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64102380 0x7f9e64073570 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 --2- 172.21.15.100:0/2635448123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64101190 0x7f9e64074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 >> 172.21.15.100:0/2635448123 conn(0x7f9e640fc720 msgr2=0x7f9e641056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 shutdown_connections 2024-03-12T05:11:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.625+0000 7f9e68eb5640 1 -- 172.21.15.100:0/2635448123 wait complete. 2024-03-12T05:11:20.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 -- 172.21.15.100:0/568995323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8106f20 msgr2=0x7f04b8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/568995323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8106f20 0x7f04b8109310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009900 tx=0x7f04b002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 -- 172.21.15.100:0/568995323 shutdown_connections 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/568995323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8106f20 0x7f04b8109310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/568995323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04b81045f0 0x7f04b81069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/568995323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04b8101cc0 0x7f04b81040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 -- 172.21.15.100:0/568995323 >> 172.21.15.100:0/568995323 conn(0x7f04b80fb610 msgr2=0x7f04b80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 -- 172.21.15.100:0/568995323 shutdown_connections 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.737+0000 7f04bcdd0640 1 -- 172.21.15.100:0/568995323 wait complete. 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 Processor -- start 2024-03-12T05:11:20.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 -- start start 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 0x7f04b8101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04b81045f0 0x7f04b80ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04b8106f20 0x7f04b81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8100910 con 0x7f04b8101cc0 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8100a80 con 0x7f04b81045f0 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04bcdd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b81a12b0 con 0x7f04b8106f20 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04b659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 0x7f04b8101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04b659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 0x7f04b8101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37578/0 (socket says 172.21.15.100:37578) 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.738+0000 7f04b659c640 1 -- 172.21.15.100:0/579005489 learned_addr learned my addr 172.21.15.100:0/579005489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04b659c640 1 -- 172.21.15.100:0/579005489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04b8106f20 msgr2=0x7f04b81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04b659c640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04b8106f20 0x7f04b81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04b659c640 1 -- 172.21.15.100:0/579005489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04b81045f0 msgr2=0x7f04b80ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:20.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04b659c640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04b81045f0 0x7f04b80ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04b659c640 1 -- 172.21.15.100:0/579005489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0009560 con 0x7f04b8101cc0 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04b659c640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 0x7f04b8101630 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04a800ba80 tx=0x7f04a800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a800cc00 con 0x7f04b8101cc0 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b81a1560 con 0x7f04b8101cc0 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04a800cd60 con 0x7f04b8101cc0 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80077e0 con 0x7f04b8101cc0 2024-03-12T05:11:20.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.739+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b81a1b40 con 0x7f04b8101cc0 2024-03-12T05:11:20.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.740+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b80fd100 con 0x7f04b8101cc0 2024-03-12T05:11:20.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.741+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f04a8014a90 con 0x7f04b8101cc0 2024-03-12T05:11:20.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.741+0000 7f04a77fe640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04a00a1a30 0x7f04a00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:20.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.742+0000 7f04b5d9b640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04a00a1a30 0x7f04a00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:20.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.742+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04a80ccda0 con 0x7f04b8101cc0 2024-03-12T05:11:20.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.742+0000 7f04b5d9b640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04a00a1a30 0x7f04a00a3ef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005fd0 tx=0x7f04ac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:20.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.746+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04a8095fb0 con 0x7f04b8101cc0 2024-03-12T05:11:20.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.955+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04b810cc30 con 0x7f04b8101cc0 2024-03-12T05:11:21.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:20 smithi161 ceph-mon[36043]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:21.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1916873337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3423279697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:20 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2635448123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:20 smithi156 ceph-mon[36270]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:21.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1916873337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.303 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3423279697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.303 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:20 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2635448123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.956+0000 7f04a77fe640 1 -- 172.21.15.100:0/579005489 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f04b810cc30 con 0x7f04b8101cc0 2024-03-12T05:11:21.303 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:21.303 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:09:59.961590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:11:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04a00a1a30 msgr2=0x7f04a00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04a00a1a30 0x7f04a00a3ef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005fd0 tx=0x7f04ac005850 comp rx=0 tx=0).stop 2024-03-12T05:11:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 msgr2=0x7f04b8101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 0x7f04b8101630 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04a800ba80 tx=0x7f04a800bf40 comp rx=0 tx=0).stop 2024-03-12T05:11:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 shutdown_connections 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04a00a1a30 0x7f04a00a3ef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04b8106f20 0x7f04b81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04b81045f0 0x7f04b80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 --2- 172.21.15.100:0/579005489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04b8101cc0 0x7f04b8101630 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 >> 172.21.15.100:0/579005489 conn(0x7f04b80fb610 msgr2=0x7f04b8102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 shutdown_connections 2024-03-12T05:11:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:20.958+0000 7f04bcdd0640 1 -- 172.21.15.100:0/579005489 wait complete. 2024-03-12T05:11:21.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:20 smithi100 ceph-mon[32210]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:21.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1916873337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3423279697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:20 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2635448123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:21.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 -- 172.21.15.100:0/1561055575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 msgr2=0x7f6fd8108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/1561055575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd8108500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009900 tx=0x7f6fd402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 -- 172.21.15.100:0/1561055575 shutdown_connections 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/1561055575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd8108500 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/1561055575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 0x7f6fd8101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/1561055575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 0x7f6fd81004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 -- 172.21.15.100:0/1561055575 >> 172.21.15.100:0/1561055575 conn(0x7f6fd80fb630 msgr2=0x7f6fd80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 -- 172.21.15.100:0/1561055575 shutdown_connections 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.411+0000 7f6fdf98c640 1 -- 172.21.15.100:0/1561055575 wait complete. 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 Processor -- start 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 -- start start 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 0x7f6fd8197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 0x7f6fd8197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd819ecf0 con 0x7f6fd8101c70 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd819ee60 con 0x7f6fd81000c0 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd819f020 con 0x7f6fd81012b0 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37582/0 (socket says 172.21.15.100:37582) 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 -- 172.21.15.100:0/431681182 learned_addr learned my addr 172.21.15.100:0/431681182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdcf27640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 0x7f6fd8197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdd728640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 0x7f6fd8197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 -- 172.21.15.100:0/431681182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 msgr2=0x7f6fd8197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 0x7f6fd8197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 -- 172.21.15.100:0/431681182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 msgr2=0x7f6fd8197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 0x7f6fd8197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 -- 172.21.15.100:0/431681182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4009560 con 0x7f6fd8101c70 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdcf27640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 0x7f6fd8197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdd728640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 0x7f6fd8197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fddf29640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd819e7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009230 tx=0x7f6fd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4004df0 con 0x7f6fd8101c70 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4005430 con 0x7f6fd8101c70 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd819f2a0 con 0x7f6fd8101c70 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.412+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4003de0 con 0x7f6fd8101c70 2024-03-12T05:11:21.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.413+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd819f7b0 con 0x7f6fd8101c70 2024-03-12T05:11:21.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.414+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd81073e0 con 0x7f6fd8101c70 2024-03-12T05:11:21.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.414+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6fd40055a0 con 0x7f6fd8101c70 2024-03-12T05:11:21.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.415+0000 7f6fce7fc640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6fc40a1a30 0x7f6fc40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.415+0000 7f6fdd728640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6fc40a1a30 0x7f6fc40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.415+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6fd40efaa0 con 0x7f6fd8101c70 2024-03-12T05:11:21.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.415+0000 7f6fdd728640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6fc40a1a30 0x7f6fc40a3ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8101110 tx=0x7f6fd0009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:21.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.420+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6fd40b8cb0 con 0x7f6fd8101c70 2024-03-12T05:11:21.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.593+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fd802ce40 con 0x7f6fd8101c70 2024-03-12T05:11:21.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.595+0000 7f6fce7fc640 1 -- 172.21.15.100:0/431681182 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7f6fd40bc370 con 0x7f6fd8101c70 2024-03-12T05:11:21.594 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:21.594 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:21.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6fc40a1a30 msgr2=0x7f6fc40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6fc40a1a30 0x7f6fc40a3ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8101110 tx=0x7f6fd0009540 comp rx=0 tx=0).stop 2024-03-12T05:11:21.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 msgr2=0x7f6fd819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd819e7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009230 tx=0x7f6fd4004270 comp rx=0 tx=0).stop 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 shutdown_connections 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6fc40a1a30 0x7f6fc40a3ef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8101c70 0x7f6fd819e7b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fd81012b0 0x7f6fd8197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 --2- 172.21.15.100:0/431681182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd81000c0 0x7f6fd8197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 >> 172.21.15.100:0/431681182 conn(0x7f6fd80fb630 msgr2=0x7f6fd81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 shutdown_connections 2024-03-12T05:11:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.597+0000 7f6fdf98c640 1 -- 172.21.15.100:0/431681182 wait complete. 2024-03-12T05:11:21.607 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2175566669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d881000b0 msgr2=0x7f5d881004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2175566669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d881000b0 0x7f5d881004d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009900 tx=0x7f5d7c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2175566669 shutdown_connections 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2175566669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d88101c60 0x7f5d881084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2175566669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 0x7f5d88101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2175566669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d881000b0 0x7f5d881004d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2175566669 >> 172.21.15.100:0/2175566669 conn(0x7f5d880fb660 msgr2=0x7f5d880fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2175566669 shutdown_connections 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2175566669 wait complete. 2024-03-12T05:11:21.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 Processor -- start 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.710+0000 7f5d8efc2640 1 -- start start 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d881000b0 0x7f5d8806a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 0x7f5d8819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 0x7f5d8819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819ed60 con 0x7f5d88101c60 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819eed0 con 0x7f5d881012a0 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819f090 con 0x7f5d881000b0 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 0x7f5d8819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 0x7f5d8819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37586/0 (socket says 172.21.15.100:37586) 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 -- 172.21.15.100:0/2809006755 learned_addr learned my addr 172.21.15.100:0/2809006755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d87fff640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 0x7f5d8819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 -- 172.21.15.100:0/2809006755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d881000b0 msgr2=0x7f5d8806a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8cd5e640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d881000b0 0x7f5d8806a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d881000b0 0x7f5d8806a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 -- 172.21.15.100:0/2809006755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 msgr2=0x7f5d8819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 0x7f5d8819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 -- 172.21.15.100:0/2809006755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009560 con 0x7f5d88101c60 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d87fff640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 0x7f5d8819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8d55f640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 0x7f5d8819e5d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009fd0 tx=0x7f5d8000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000bea0 con 0x7f5d88101c60 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d80009940 con 0x7f5d88101c60 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8819f370 con 0x7f5d88101c60 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d800106c0 con 0x7f5d88101c60 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.711+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8819f830 con 0x7f5d88101c60 2024-03-12T05:11:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.712+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88107450 con 0x7f5d88101c60 2024-03-12T05:11:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.713+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5d80004240 con 0x7f5d88101c60 2024-03-12T05:11:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.714+0000 7f5d85ffb640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5d780a1a30 0x7f5d780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.714+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5d800cd1d0 con 0x7f5d88101c60 2024-03-12T05:11:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.714+0000 7f5d8cd5e640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5d780a1a30 0x7f5d780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.714+0000 7f5d8cd5e640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5d780a1a30 0x7f5d780a3ef0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c02c6a0 tx=0x7f5d7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.719+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d800963e0 con 0x7f5d88101c60 2024-03-12T05:11:21.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.930+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d8802ce40 con 0x7f5d88101c60 2024-03-12T05:11:21.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.931+0000 7f5d85ffb640 1 -- 172.21.15.100:0/2809006755 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f5d80099aa0 con 0x7f5d88101c60 2024-03-12T05:11:21.928 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:21.928 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:09:59.961590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:21.928 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:11:21.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5d780a1a30 msgr2=0x7f5d780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5d780a1a30 0x7f5d780a3ef0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c02c6a0 tx=0x7f5d7c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 msgr2=0x7f5d8819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:21.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 0x7f5d8819e5d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009fd0 tx=0x7f5d8000b260 comp rx=0 tx=0).stop 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 shutdown_connections 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5d780a1a30 0x7f5d780a3ef0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d88101c60 0x7f5d8819e5d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d881012a0 0x7f5d8819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 --2- 172.21.15.100:0/2809006755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d881000b0 0x7f5d8806a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 >> 172.21.15.100:0/2809006755 conn(0x7f5d880fb660 msgr2=0x7f5d88104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 shutdown_connections 2024-03-12T05:11:21.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:21.933+0000 7f5d8efc2640 1 -- 172.21.15.100:0/2809006755 wait complete. 2024-03-12T05:11:21.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/579005489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:21.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/431681182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:21.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-12T05:09:54.274771+0000', 'modified': '2024-03-12T05:09:59.961590+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24461, 'mds_1': 24451, 'mds_2': 24439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2592644588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2592644588}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2592644588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/692407308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 692407308}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 692407308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/854337146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 854337146}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 854337146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24439': {'gid': 24439, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6837/2463295988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 2463295988}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 2463295988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.161:6839/1712807460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 1712807460}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 1712807460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/1209044969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1209044969}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1209044969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-12T05:11:21.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-12T05:11:21.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-12T05:11:22.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 -- 172.21.15.100:0/4050758986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 msgr2=0x7f6c041015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 --2- 172.21.15.100:0/4050758986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c041015b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009900 tx=0x7f6bf402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 -- 172.21.15.100:0/4050758986 shutdown_connections 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 --2- 172.21.15.100:0/4050758986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 0x7f6c041095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 --2- 172.21.15.100:0/4050758986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 0x7f6c04102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 --2- 172.21.15.100:0/4050758986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c041015b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 -- 172.21.15.100:0/4050758986 >> 172.21.15.100:0/4050758986 conn(0x7f6c040fc720 msgr2=0x7f6c040feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 -- 172.21.15.100:0/4050758986 shutdown_connections 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 -- 172.21.15.100:0/4050758986 wait complete. 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.047+0000 7f6c09040640 1 Processor -- start 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 -- start start 2024-03-12T05:11:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c04197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 0x7f6c041977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 0x7f6c0419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419ed80 con 0x7f6c04101190 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419eef0 con 0x7f6c04102380 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c09040640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419f0b0 con 0x7f6c04102d40 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c04197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c04197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37596/0 (socket says 172.21.15.100:37596) 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 -- 172.21.15.100:0/1426721989 learned_addr learned my addr 172.21.15.100:0/1426721989 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c0359e640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 0x7f6c0419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c0259c640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 0x7f6c041977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 -- 172.21.15.100:0/1426721989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 msgr2=0x7f6c0419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 0x7f6c0419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 -- 172.21.15.100:0/1426721989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 msgr2=0x7f6c041977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 0x7f6c041977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 -- 172.21.15.100:0/1426721989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4009560 con 0x7f6c04101190 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c0259c640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 0x7f6c041977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c0359e640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 0x7f6c0419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c04197290 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4000c00 tx=0x7f6bf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.048+0000 7f6bebfff640 1 -- 172.21.15.100:0/1426721989 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4004df0 con 0x7f6c04101190 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.049+0000 7f6bebfff640 1 -- 172.21.15.100:0/1426721989 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bf4005430 con 0x7f6c04101190 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.049+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0419f330 con 0x7f6c04101190 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.049+0000 7f6bebfff640 1 -- 172.21.15.100:0/1426721989 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf403e450 con 0x7f6c04101190 2024-03-12T05:11:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.049+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0419f840 con 0x7f6c04101190 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6bebfff640 1 -- 172.21.15.100:0/1426721989 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6bf4003680 con 0x7f6c04101190 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 Processor -- start 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 -- start start 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a83682c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a836831470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a836833f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8367e7cd0 con 0x55a83682bf80 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a83679af20 con 0x55a836831ae0 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a836798450 con 0x55a83682c970 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6bebfff640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6bec0a1a30 0x7f6bec0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a83682c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a83682c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37604/0 (socket says 172.21.15.100:37604) 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0359e640 1 -- 172.21.15.100:0/4035811182 learned_addr learned my addr 172.21.15.100:0/4035811182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0259c640 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a836833f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6bebfff640 1 -- 172.21.15.100:0/1426721989 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6bf40ef9e0 con 0x7f6c04101190 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a836831470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0359e640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6bec0a1a30 0x7f6bec0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0359e640 1 -- 172.21.15.100:0/4035811182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 msgr2=0x55a836831470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.051+0000 7f6c0359e640 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a836831470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6c0359e640 1 -- 172.21.15.100:0/4035811182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 msgr2=0x55a836833f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6c0359e640 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a836833f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6c0359e640 1 -- 172.21.15.100:0/4035811182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a836748f20 con 0x55a83682bf80 2024-03-12T05:11:22.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6c0359e640 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a83682c3a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc014a90 tx=0x7f6bfc014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909615f0189e873a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6be97fa640 1 -- 172.21.15.100:0/4035811182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc014530 con 0x55a83682bf80 2024-03-12T05:11:22.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6be97fa640 1 -- 172.21.15.100:0/4035811182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bfc01a070 con 0x55a83682bf80 2024-03-12T05:11:22.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.052+0000 7f6be97fa640 1 -- 172.21.15.100:0/4035811182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00cd20 con 0x55a83682bf80 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4035811182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 msgr2=0x55a83682c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0359e640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6bec0a1a30 0x7f6bec0a3ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc022000 tx=0x7f6bfc020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a83682c3a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc014a90 tx=0x7f6bfc014f50 comp rx=0 tx=0).stop 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4035811182 shutdown_connections 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a836833f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a836831470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4035811182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a83682c3a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4035811182 >> 172.21.15.100:0/4035811182 conn(0x55a8368274f0 msgr2=0x55a8368278f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4035811182 shutdown_connections 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4035811182 wait complete. 2024-03-12T05:11:22.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- start start 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 Processor -- start 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836836170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a8368366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a83683d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a836793c30 con 0x55a83682bf80 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a836738d30 con 0x55a836831ae0 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a836467310 con 0x55a83682c970 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0bc10740 10 client.0 _ll_register_callbacks cb 0x55a836740d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836836170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836836170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37620/0 (socket says 172.21.15.100:37620) 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c02d9d640 1 -- 172.21.15.100:0/4056904536 learned_addr learned my addr 172.21.15.100:0/4056904536 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.053+0000 7f6c0359e640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a83683d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0259c640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a8368366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c02d9d640 1 -- 172.21.15.100:0/4056904536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 msgr2=0x55a8368366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a8368366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c02d9d640 1 -- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 msgr2=0x55a83683d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a83683d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c02d9d640 1 -- 172.21.15.100:0/4056904536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc0146d0 con 0x55a83682bf80 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0359e640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a83683d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0259c640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a8368366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836836170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4005ab0 tx=0x7f6bf40bc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d753289acbd47b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6be6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6be6ff5640 1 -- 172.21.15.100:0/4056904536 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40d4070 con 0x55a83682bf80 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6be6ff5640 1 -- 172.21.15.100:0/4056904536 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6bf40c50f0 con 0x55a83682bf80 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8364936c0 con 0x55a83682bf80 2024-03-12T05:11:22.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6be6ff5640 1 -- 172.21.15.100:0/4056904536 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bf40055a0 con 0x55a83682bf80 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 10 client.15033 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a8367922b0 con 0x55a83682bf80 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6be6ff5640 1 -- 172.21.15.100:0/4056904536 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bf4086020 con 0x55a83682bf80 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6be6ff5640 1 -- 172.21.15.100:0/4056904536 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6bf403c070 con 0x55a83682bf80 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 10 client.15033 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 10 client.15033 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 20 client.15033 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a836842920 0x55a836841be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 4 client.15033 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.054+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8364e4270 con 0x55a836842920 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.055+0000 7f6c0259c640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a836842920 0x55a836841be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.055+0000 7f6be6ff5640 10 client.15033 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:22.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.055+0000 7f6c0259c640 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a836842920 0x55a836841be0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a7d8084a10cfb56 server_cookie=a32ed267a1367cfe in_seq=0 out_seq=0 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6be6ff5640 1 -- 172.21.15.100:0/4056904536 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8364e4270 con 0x55a836842920 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6be6ff5640 10 client.15033 handle_command_reply: tid=0 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 client.15033 shutdown 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 msgr2=0x55a836836170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836836170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4005ab0 tx=0x7f6bf40bc370 comp rx=0 tx=0).stop 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 shutdown_connections 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a836842920 0x55a836841be0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a836831ae0 0x55a83683d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682c970 0x55a8368366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6be6ff5640 0 client.15033 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/4056904536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836836170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 >> 172.21.15.100:0/4056904536 conn(0x55a8368274f0 msgr2=0x55a8368267d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 shutdown_connections 2024-03-12T05:11:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 1 -- 172.21.15.100:0/4056904536 wait complete. 2024-03-12T05:11:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.056+0000 7f6c0bc10740 20 client.15033 trim_cache size 0 max 16384 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.071+0000 7f6c0bc10740 1 Processor -- start 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 -- start start 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836841be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a83682de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a836831ae0 0x55a83682e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a836719490 con 0x55a83682bf80 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a836844900 con 0x55a83682c970 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8364e4270 con 0x55a836831ae0 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836841be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836841be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37622/0 (socket says 172.21.15.100:37622) 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 -- 172.21.15.100:0/3643015665 learned_addr learned my addr 172.21.15.100:0/3643015665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 -- 172.21.15.100:0/3643015665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a836831ae0 msgr2=0x55a83682e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a836831ae0 0x55a83682e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 -- 172.21.15.100:0/3643015665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 msgr2=0x55a83682de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a83682de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 -- 172.21.15.100:0/3643015665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a83672ea20 con 0x55a83682bf80 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836841be0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40bc8e0 tx=0x7f6bf40c0b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c75b5d21611b59b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6be8ff9640 1 -- 172.21.15.100:0/3643015665 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40d4070 con 0x55a83682bf80 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6be8ff9640 1 -- 172.21.15.100:0/3643015665 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bf4086070 con 0x55a83682bf80 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.072+0000 7f6be8ff9640 1 -- 172.21.15.100:0/3643015665 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b86a0 con 0x55a83682bf80 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- 172.21.15.100:0/3643015665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 msgr2=0x55a836841be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836841be0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40bc8e0 tx=0x7f6bf40c0b60 comp rx=0 tx=0).stop 2024-03-12T05:11:22.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- 172.21.15.100:0/3643015665 shutdown_connections 2024-03-12T05:11:22.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a836831ae0 0x55a83682e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a83682de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/3643015665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a83682bf80 0x55a836841be0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- 172.21.15.100:0/3643015665 >> 172.21.15.100:0/3643015665 conn(0x55a8368274f0 msgr2=0x55a8368278d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- 172.21.15.100:0/3643015665 shutdown_connections 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- 172.21.15.100:0/3643015665 wait complete. 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- start start 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 Processor -- start 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682bf80 0x55a836895490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a8368959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 0x55a83689c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a83689cee0 con 0x55a836831ae0 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a83689d020 con 0x55a83682c970 2024-03-12T05:11:22.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a83689d190 con 0x55a83682bf80 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0bc10740 10 client.0 _ll_register_callbacks cb 0x7f6bfc009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.073+0000 7f6c0359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 0x55a83689c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 0x55a83689c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37634/0 (socket says 172.21.15.100:37634) 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 -- 172.21.15.100:0/199726336 learned_addr learned my addr 172.21.15.100:0/199726336 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0259c640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a8368959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682bf80 0x55a836895490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 -- 172.21.15.100:0/199726336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682bf80 msgr2=0x55a836895490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682bf80 0x55a836895490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 -- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 msgr2=0x55a8368959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a8368959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 -- 172.21.15.100:0/199726336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8367922b0 con 0x55a836831ae0 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0359e640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 0x55a83689c9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc004a30 tx=0x7f6bfc004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8950142fab7c7c9f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6be6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6be6ff5640 1 -- 172.21.15.100:0/199726336 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc01ee50 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6be6ff5640 1 -- 172.21.15.100:0/199726336 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6bfc009900 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a836842630 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6be6ff5640 1 -- 172.21.15.100:0/199726336 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bfc014530 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0bc10740 10 client.15039 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a836754780 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6be6ff5640 1 -- 172.21.15.100:0/199726336 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a836842630 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6be6ff5640 1 -- 172.21.15.100:0/199726336 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6bfc01d070 con 0x55a836831ae0 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.074+0000 7f6c0bc10740 10 client.15039 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 10 client.15039 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 20 client.15039 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a8368a23d0 0x55a83689e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 4 client.15039 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a836843950 con 0x55a8368a23d0 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a8368a23d0 0x55a83689e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6be6ff5640 10 client.15039 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c02d9d640 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a8368a23d0 0x55a83689e8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29d2f78a0ba7da17 server_cookie=fe365e35cdf0b481 in_seq=0 out_seq=0 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6be6ff5640 1 -- 172.21.15.100:0/199726336 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a836843950 con 0x55a8368a23d0 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6be6ff5640 10 client.15039 handle_command_reply: tid=0 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 1 client.15039 shutdown 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 msgr2=0x55a83689c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.075+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 0x55a83689c9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc004a30 tx=0x7f6bfc004a60 comp rx=0 tx=0).stop 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 shutdown_connections 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a8368a23d0 0x55a83689e8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a836831ae0 0x55a83689c9a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a83682c970 0x55a8368959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 --2- 172.21.15.100:0/199726336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a83682bf80 0x55a836895490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 >> 172.21.15.100:0/199726336 conn(0x55a8368274f0 msgr2=0x55a83682de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6be6ff5640 0 client.15039 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 shutdown_connections 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 1 -- 172.21.15.100:0/199726336 wait complete. 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c0bc10740 20 client.15039 trim_cache size 0 max 16384 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:11:22.082 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24439, 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 83.123776991, 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 22, 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 88, 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 86, 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 103.63615961000001 2024-03-12T05:11:22.083 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:22.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6bec0a1a30 msgr2=0x7f6bec0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6bec0a1a30 0x7f6bec0a3ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc022000 tx=0x7f6bfc020000 comp rx=0 tx=0).stop 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 msgr2=0x7f6c04197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c04197290 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4000c00 tx=0x7f6bf4004270 comp rx=0 tx=0).stop 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 shutdown_connections 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6bec0a1a30 0x7f6bec0a3ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c04102d40 0x7f6c0419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c04102380 0x7f6c041977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 --2- 172.21.15.100:0/1426721989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c04101190 0x7f6c04197290 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 >> 172.21.15.100:0/1426721989 conn(0x7f6c040fc720 msgr2=0x7f6c041056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 shutdown_connections 2024-03-12T05:11:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.076+0000 7f6c09040640 1 -- 172.21.15.100:0/1426721989 wait complete. 2024-03-12T05:11:22.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-12T05:11:22.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 -- 172.21.15.100:0/2315554589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f801000a0 msgr2=0x7f6f801004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 --2- 172.21.15.100:0/2315554589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f801000a0 0x7f6f801004c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009900 tx=0x7f6f7c02bc50 comp rx=0 tx=0).stop 2024-03-12T05:11:22.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 -- 172.21.15.100:0/2315554589 shutdown_connections 2024-03-12T05:11:22.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 --2- 172.21.15.100:0/2315554589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101c50 0x7f6f801084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 --2- 172.21.15.100:0/2315554589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80101290 0x7f6f80101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 --2- 172.21.15.100:0/2315554589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f801000a0 0x7f6f801004c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 -- 172.21.15.100:0/2315554589 >> 172.21.15.100:0/2315554589 conn(0x7f6f800fb630 msgr2=0x7f6f800fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 -- 172.21.15.100:0/2315554589 shutdown_connections 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.191+0000 7f6f8738a640 1 -- 172.21.15.100:0/2315554589 wait complete. 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 Processor -- start 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 -- start start 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f801000a0 0x7f6f8006a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101290 0x7f6f8019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 0x7f6f8019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8019edb0 con 0x7f6f80101c50 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8019ef20 con 0x7f6f80101290 2024-03-12T05:11:22.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8019f0e0 con 0x7f6f801000a0 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 0x7f6f8019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 0x7f6f8019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37646/0 (socket says 172.21.15.100:37646) 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 -- 172.21.15.100:0/281166932 learned_addr learned my addr 172.21.15.100:0/281166932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85126640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f801000a0 0x7f6f8006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f84925640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101290 0x7f6f8019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 -- 172.21.15.100:0/281166932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f801000a0 msgr2=0x7f6f8006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f801000a0 0x7f6f8006a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 -- 172.21.15.100:0/281166932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101290 msgr2=0x7f6f8019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101290 0x7f6f8019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 -- 172.21.15.100:0/281166932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c009560 con 0x7f6f80101c50 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f84925640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101290 0x7f6f8019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f85927640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 0x7f6f8019e620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800cce0 tx=0x7f6f7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f767fc640 1 -- 172.21.15.100:0/281166932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7800f450 con 0x7f6f80101c50 2024-03-12T05:11:22.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f767fc640 1 -- 172.21.15.100:0/281166932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f7800fa90 con 0x7f6f80101c50 2024-03-12T05:11:22.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8019f3c0 con 0x7f6f80101c50 2024-03-12T05:11:22.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f767fc640 1 -- 172.21.15.100:0/281166932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78018b90 con 0x7f6f80101c50 2024-03-12T05:11:22.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.192+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8019f900 con 0x7f6f80101c50 2024-03-12T05:11:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.194+0000 7f6f767fc640 1 -- 172.21.15.100:0/281166932 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6f7800fc00 con 0x7f6f80101c50 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 Processor -- start 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 -- start start 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f767fc640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6f6c0a1a30 0x7f6f6c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 0x5626cf7d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7de470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7deae0 0x5626cf7e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf794cd0 con 0x5626cf7d8f80 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f767fc640 1 -- 172.21.15.100:0/281166932 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6f780cdc80 con 0x7f6f80101c50 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf747f20 con 0x5626cf7d9970 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf745450 con 0x5626cf7deae0 2024-03-12T05:11:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f84925640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6f6c0a1a30 0x7f6f6c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f85126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 0x5626cf7d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f85126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 0x5626cf7d93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37652/0 (socket says 172.21.15.100:37652) 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f85126640 1 -- 172.21.15.100:0/1104021273 learned_addr learned my addr 172.21.15.100:0/1104021273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.195+0000 7f6f84925640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85126640 1 -- 172.21.15.100:0/1104021273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7deae0 msgr2=0x5626cf7e0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85927640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7deae0 0x5626cf7e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85126640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7deae0 0x5626cf7e0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85126640 1 -- 172.21.15.100:0/1104021273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 msgr2=0x5626cf7de470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85126640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7de470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85126640 1 -- 172.21.15.100:0/1104021273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626cf6f5f20 con 0x5626cf7d8f80 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f84925640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85927640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7deae0 0x5626cf7e0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f84925640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6f6c0a1a30 0x7f6f6c0a3ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006a070 tx=0x7f6f70013650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f85126640 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 0x5626cf7d93a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f801010f0 tx=0x7f6f7c02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c7040aae9a3920 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f6b7fe640 1 -- 172.21.15.100:0/1104021273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0046c0 con 0x5626cf7d8f80 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f6b7fe640 1 -- 172.21.15.100:0/1104021273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c004510 con 0x5626cf7d8f80 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.196+0000 7f6f6b7fe640 1 -- 172.21.15.100:0/1104021273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c005460 con 0x5626cf7d8f80 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1104021273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 msgr2=0x5626cf7d93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 0x5626cf7d93a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f801010f0 tx=0x7f6f7c02c900 comp rx=0 tx=0).stop 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1104021273 shutdown_connections 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7deae0 0x5626cf7e0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1104021273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d8f80 0x5626cf7d93a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1104021273 >> 172.21.15.100:0/1104021273 conn(0x5626cf7d44f0 msgr2=0x5626cf7d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1104021273 shutdown_connections 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1104021273 wait complete. 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 -- start start 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.197+0000 7f6f8a03a740 1 Processor -- start 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d8f80 0x5626cf7e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d9970 0x5626cf7e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf740c30 con 0x5626cf7deae0 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf6e5d30 con 0x5626cf7d8f80 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf414310 con 0x5626cf7d9970 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 10 client.0 _ll_register_callbacks cb 0x5626cf6edd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37658/0 (socket says 172.21.15.100:37658) 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 -- 172.21.15.100:0/1921757012 learned_addr learned my addr 172.21.15.100:0/1921757012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85126640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d8f80 0x5626cf7e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f84925640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d9970 0x5626cf7e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 -- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d9970 msgr2=0x5626cf7e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d9970 0x5626cf7e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 -- 172.21.15.100:0/1921757012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d8f80 msgr2=0x5626cf7e30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d8f80 0x5626cf7e30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 -- 172.21.15.100:0/1921757012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f80108de0 con 0x5626cf7deae0 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f84925640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d9970 0x5626cf7e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f85927640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7ea650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f780a2e60 tx=0x7f6f7809b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82cec0479c5f508 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f68ff9640 1 -- 172.21.15.100:0/1921757012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780a3390 con 0x5626cf7deae0 2024-03-12T05:11:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f68ff9640 1 -- 172.21.15.100:0/1921757012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6f780a2f00 con 0x5626cf7deae0 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626cf4406c0 con 0x5626cf7deae0 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f68ff9640 1 -- 172.21.15.100:0/1921757012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f780a30c0 con 0x5626cf7deae0 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 10 client.15051 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5626cf73f2b0 con 0x5626cf7deae0 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.198+0000 7f6f68ff9640 1 -- 172.21.15.100:0/1921757012 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f78064020 con 0x5626cf7deae0 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f68ff9640 1 -- 172.21.15.100:0/1921757012 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f78016020 con 0x5626cf7deae0 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f8a03a740 10 client.15051 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f8a03a740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f8a03a740 20 client.15051 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf7ef9a0 0x5626cf7f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f8a03a740 4 client.15051 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626cf491270 con 0x5626cf7ef9a0 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f85126640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf7ef9a0 0x5626cf7f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f68ff9640 10 client.15051 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.199+0000 7f6f85126640 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf7ef9a0 0x5626cf7f3e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87151ce5d9e620f1 server_cookie=814361bfd8818ffd in_seq=0 out_seq=0 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f68ff9640 1 -- 172.21.15.100:0/1921757012 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626cf491270 con 0x5626cf7ef9a0 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f68ff9640 10 client.15051 handle_command_reply: tid=0 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 client.15051 shutdown 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 msgr2=0x5626cf7ea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7ea650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f780a2e60 tx=0x7f6f7809b3b0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 shutdown_connections 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7ea650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d9970 0x5626cf7e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf7ef9a0 0x5626cf7f3e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1921757012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d8f80 0x5626cf7e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f68ff9640 0 client.15051 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 >> 172.21.15.100:0/1921757012 conn(0x5626cf7d44f0 msgr2=0x5626cf7d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 shutdown_connections 2024-03-12T05:11:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1921757012 wait complete. 2024-03-12T05:11:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.200+0000 7f6f8a03a740 20 client.15051 trim_cache size 0 max 16384 2024-03-12T05:11:22.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:21 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/579005489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:22.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:21 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/431681182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 Processor -- start 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 -- start start 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf6c6490 con 0x5626cf7deae0 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf7f1980 con 0x5626cf7d9970 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf7d4fe0 con 0x5626cf7d8f80 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7e30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37660/0 (socket says 172.21.15.100:37660) 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 -- 172.21.15.100:0/1405205560 learned_addr learned my addr 172.21.15.100:0/1405205560 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f84925640 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 -- 172.21.15.100:0/1405205560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 msgr2=0x5626cf7e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 -- 172.21.15.100:0/1405205560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 msgr2=0x5626cf7ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 -- 172.21.15.100:0/1405205560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626cf6dba20 con 0x5626cf7deae0 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f85927640 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7e30d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f7809ab80 tx=0x7f6f78099100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d7e91ad8b4a669 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.216+0000 7f6f6affd640 1 -- 172.21.15.100:0/1405205560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780a3390 con 0x5626cf7deae0 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f6affd640 1 -- 172.21.15.100:0/1405205560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f78064030 con 0x5626cf7deae0 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f6affd640 1 -- 172.21.15.100:0/1405205560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7809a2a0 con 0x5626cf7deae0 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1405205560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 msgr2=0x5626cf7e30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7e30d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f7809ab80 tx=0x7f6f78099100 comp rx=0 tx=0).stop 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1405205560 shutdown_connections 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7deae0 0x5626cf7e30d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7d9970 0x5626cf7ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/1405205560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1405205560 >> 172.21.15.100:0/1405205560 conn(0x5626cf7d44f0 msgr2=0x5626cf7d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1405205560 shutdown_connections 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 -- 172.21.15.100:0/1405205560 wait complete. 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 -- start start 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.217+0000 7f6f8a03a740 1 Processor -- start 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 0x5626cf7e7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7deae0 0x5626cf845ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf7ea6c0 con 0x5626cf7d9970 2024-03-12T05:11:22.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf7ea830 con 0x5626cf7deae0 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cf7ea9f0 con 0x5626cf7d8f80 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f8a03a740 10 client.0 _ll_register_callbacks cb 0x7f6f7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 0x5626cf7e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 0x5626cf7e7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37670/0 (socket says 172.21.15.100:37670) 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 -- 172.21.15.100:0/3441113961 learned_addr learned my addr 172.21.15.100:0/3441113961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f85927640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7deae0 0x5626cf845ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f85126640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 -- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 msgr2=0x5626cf7e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 -- 172.21.15.100:0/3441113961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7deae0 msgr2=0x5626cf845ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7deae0 0x5626cf845ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 -- 172.21.15.100:0/3441113961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626cf73f2b0 con 0x5626cf7d9970 2024-03-12T05:11:22.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f85126640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f84925640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 0x5626cf7e7790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f70013b50 tx=0x7f6f700046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec01a0bb0dbbeb46 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.218+0000 7f6f68ff9640 1 -- 172.21.15.100:0/3441113961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000a610 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f68ff9640 1 -- 172.21.15.100:0/3441113961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6f70004020 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626cf7f0370 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f68ff9640 1 -- 172.21.15.100:0/3441113961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f700041e0 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 10 client.15057 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5626cf701780 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f68ff9640 1 -- 172.21.15.100:0/3441113961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f70032020 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f68ff9640 1 -- 172.21.15.100:0/3441113961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f70003930 con 0x5626cf7d9970 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 10 client.15057 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T05:11:22.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 20 client.15057 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf84b240 0x5626cf8476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 4 client.15057 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f6f7c004510 con 0x5626cf84b240 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.219+0000 7f6f85126640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf84b240 0x5626cf8476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f68ff9640 10 client.15057 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f85126640 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf84b240 0x5626cf8476c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4f3ec67807e02cf server_cookie=fbdde879dc1f75c1 in_seq=0 out_seq=0 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f68ff9640 1 -- 172.21.15.100:0/3441113961 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f6f7c004510 con 0x5626cf84b240 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f68ff9640 10 client.15057 handle_command_reply: tid=0 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 client.15057 shutdown 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 msgr2=0x5626cf7e7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 0x5626cf7e7790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f70013b50 tx=0x7f6f700046e0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 shutdown_connections 2024-03-12T05:11:22.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5626cf84b240 0x5626cf8476c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626cf7deae0 0x5626cf845ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626cf7d9970 0x5626cf7e7790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f68ff9640 0 client.15057 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 --2- 172.21.15.100:0/3441113961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626cf7d8f80 0x5626cf7e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 >> 172.21.15.100:0/3441113961 conn(0x5626cf7d44f0 msgr2=0x5626cf7dad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 shutdown_connections 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 1 -- 172.21.15.100:0/3441113961 wait complete. 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.220+0000 7f6f8a03a740 20 client.15057 trim_cache size 0 max 16384 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:11:22.225 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24451, 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 84.278097191000001, 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 22, 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 88, 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 86, 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 100.99749117499999 2024-03-12T05:11:22.226 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:22.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6f6c0a1a30 msgr2=0x7f6f6c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6f6c0a1a30 0x7f6f6c0a3ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006a070 tx=0x7f6f70013650 comp rx=0 tx=0).stop 2024-03-12T05:11:22.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 msgr2=0x7f6f8019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 0x7f6f8019e620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800cce0 tx=0x7f6f7800a620 comp rx=0 tx=0).stop 2024-03-12T05:11:22.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 shutdown_connections 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6f6c0a1a30 0x7f6f6c0a3ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f80101c50 0x7f6f8019e620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f80101290 0x7f6f8019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 --2- 172.21.15.100:0/281166932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f801000a0 0x7f6f8006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 >> 172.21.15.100:0/281166932 conn(0x7f6f800fb630 msgr2=0x7f6f80104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 shutdown_connections 2024-03-12T05:11:22.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.221+0000 7f6f8738a640 1 -- 172.21.15.100:0/281166932 wait complete. 2024-03-12T05:11:22.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:21 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/579005489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:22.229 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:21 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/431681182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:22.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 -- 172.21.15.100:0/1998216959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9081016b0 msgr2=0x7fc908101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/1998216959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9081016b0 0x7fc908101ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 -- 172.21.15.100:0/1998216959 shutdown_connections 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/1998216959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc908100cc0 0x7fc908108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/1998216959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc908100300 0x7fc908100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/1998216959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9081016b0 0x7fc908101ad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 -- 172.21.15.100:0/1998216959 >> 172.21.15.100:0/1998216959 conn(0x7fc9080fadb0 msgr2=0x7fc9080fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 -- 172.21.15.100:0/1998216959 shutdown_connections 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.335+0000 7fc90cf6c640 1 -- 172.21.15.100:0/1998216959 wait complete. 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 Processor -- start 2024-03-12T05:11:22.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 -- start start 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 0x7fc908197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc908100cc0 0x7fc908197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9081016b0 0x7fc90819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819edf0 con 0x7fc908100300 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819ef60 con 0x7fc908100cc0 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90cf6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819f120 con 0x7fc9081016b0 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 0x7fc908197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 0x7fc908197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37696/0 (socket says 172.21.15.100:37696) 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 -- 172.21.15.100:0/3360213368 learned_addr learned my addr 172.21.15.100:0/3360213368 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc905d9b640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc908100cc0 0x7fc908197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc906d9d640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9081016b0 0x7fc90819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 -- 172.21.15.100:0/3360213368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9081016b0 msgr2=0x7fc90819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9081016b0 0x7fc90819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 -- 172.21.15.100:0/3360213368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc908100cc0 msgr2=0x7fc908197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc908100cc0 0x7fc908197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 -- 172.21.15.100:0/3360213368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009560 con 0x7fc908100300 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.336+0000 7fc90659c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 0x7fc908197300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc900036040 tx=0x7fc9000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.337+0000 7fc8ff7fe640 1 -- 172.21.15.100:0/3360213368 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900004510 con 0x7fc908100300 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.337+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90819f3a0 con 0x7fc908100300 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.337+0000 7fc8ff7fe640 1 -- 172.21.15.100:0/3360213368 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9000054e0 con 0x7fc908100300 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.337+0000 7fc8ff7fe640 1 -- 172.21.15.100:0/3360213368 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90003e450 con 0x7fc908100300 2024-03-12T05:11:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.337+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90819f980 con 0x7fc908100300 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc8ff7fe640 1 -- 172.21.15.100:0/3360213368 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc900003680 con 0x7fc908100300 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 Processor -- start 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 -- start start 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc8ff7fe640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8f40a1a30 0x7fc8f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c3d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c42470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c44f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365bf8cd0 con 0x55f365c3d970 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365babf20 con 0x55f365c42ae0 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.339+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365ba9450 con 0x55f365c3cf80 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc906d9d640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8f40a1a30 0x7fc8f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc8ff7fe640 1 -- 172.21.15.100:0/3360213368 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9000ef860 con 0x7fc908100300 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c42470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c42470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37702/0 (socket says 172.21.15.100:37702) 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 -- 172.21.15.100:0/2873222145 learned_addr learned my addr 172.21.15.100:0/2873222145 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc90659c640 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c3d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc906d9d640 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c44f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 -- 172.21.15.100:0/2873222145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 msgr2=0x55f365c3d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c3d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 -- 172.21.15.100:0/2873222145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 msgr2=0x55f365c44f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c44f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 -- 172.21.15.100:0/2873222145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f365b59f20 con 0x55f365c3d970 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc906d9d640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8f40a1a30 0x7fc8f40a3ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8014530 tx=0x7fc8f8014930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc905d9b640 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c42470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009a00 tx=0x7fc8f0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f3a495e5a868a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc8fcff9640 1 -- 172.21.15.100:0/2873222145 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000ea60 con 0x55f365c3d970 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc8fcff9640 1 -- 172.21.15.100:0/2873222145 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8f000ebc0 con 0x55f365c3d970 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.340+0000 7fc8fcff9640 1 -- 172.21.15.100:0/2873222145 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0012630 con 0x55f365c3d970 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 -- 172.21.15.100:0/2873222145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 msgr2=0x55f365c42470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c42470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009a00 tx=0x7fc8f0009ec0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 -- 172.21.15.100:0/2873222145 shutdown_connections 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c44f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c42470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/2873222145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c3d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 -- 172.21.15.100:0/2873222145 >> 172.21.15.100:0/2873222145 conn(0x55f365c384f0 msgr2=0x55f365c388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 -- 172.21.15.100:0/2873222145 shutdown_connections 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 -- 172.21.15.100:0/2873222145 wait complete. 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 -- start start 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.341+0000 7fc90fc1b740 1 Processor -- start 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c47160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c4e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365ba4c30 con 0x55f365c3d970 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365b49d30 con 0x55f365c42ae0 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365878310 con 0x55f365c3cf80 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 10 client.0 _ll_register_callbacks cb 0x55f365b51d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37710/0 (socket says 172.21.15.100:37710) 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 -- 172.21.15.100:0/593885852 learned_addr learned my addr 172.21.15.100:0/593885852 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc906d9d640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c4e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90659c640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c47160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 -- 172.21.15.100:0/593885852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 msgr2=0x55f365c47160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c47160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 -- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 msgr2=0x55f365c4e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c4e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 -- 172.21.15.100:0/593885852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f0009630 con 0x55f365c3d970 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc905d9b640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c476a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000e5a0 tx=0x7fc8f0013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956d59e5269e9003 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc8edffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc8edffb640 1 -- 172.21.15.100:0/593885852 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f001c9a0 con 0x55f365c3d970 2024-03-12T05:11:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc8edffb640 1 -- 172.21.15.100:0/593885852 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8f001cb00 con 0x55f365c3d970 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3658a46c0 con 0x55f365c3d970 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc8edffb640 1 -- 172.21.15.100:0/593885852 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f001ccc0 con 0x55f365c3d970 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 10 client.15069 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.342+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f365ba32b0 con 0x55f365c3d970 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc8edffb640 1 -- 172.21.15.100:0/593885852 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f3658a46c0 con 0x55f365c3d970 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc8edffb640 1 -- 172.21.15.100:0/593885852 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc8f0011070 con 0x55f365c3d970 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90fc1b740 10 client.15069 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90fc1b740 10 client.15069 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90fc1b740 20 client.15069 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365c53ad0 0x55f365c52d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90fc1b740 4 client.15069 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3658f5270 con 0x55f365c53ad0 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90659c640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365c53ad0 0x55f365c52d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc8edffb640 10 client.15069 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.343+0000 7fc90659c640 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365c53ad0 0x55f365c52d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ce5e4805e6d661 server_cookie=7992a12bacdddfab in_seq=0 out_seq=0 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc8edffb640 1 -- 172.21.15.100:0/593885852 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3658f5270 con 0x55f365c53ad0 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc8edffb640 10 client.15069 handle_command_reply: tid=0 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 client.15069 shutdown 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 msgr2=0x55f365c476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c476a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000e5a0 tx=0x7fc8f0013c50 comp rx=0 tx=0).stop 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 shutdown_connections 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365c53ad0 0x55f365c52d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c4e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc8edffb640 0 client.15069 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c476a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/593885852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c47160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 >> 172.21.15.100:0/593885852 conn(0x55f365c384f0 msgr2=0x55f365c377d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 shutdown_connections 2024-03-12T05:11:22.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 1 -- 172.21.15.100:0/593885852 wait complete. 2024-03-12T05:11:22.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.344+0000 7fc90fc1b740 20 client.15069 trim_cache size 0 max 16384 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 Processor -- start 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 -- start start 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c44ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c5bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365b2a490 con 0x55f365c3d970 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365c4f2f0 con 0x55f365c42ae0 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365c55ab0 con 0x55f365c3cf80 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c4e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37720/0 (socket says 172.21.15.100:37720) 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 -- 172.21.15.100:0/3827327629 learned_addr learned my addr 172.21.15.100:0/3827327629 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90659c640 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c44ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 -- 172.21.15.100:0/3827327629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 msgr2=0x55f365c44ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c44ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 -- 172.21.15.100:0/3827327629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 msgr2=0x55f365c5bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c5bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 -- 172.21.15.100:0/3827327629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f365b3fa20 con 0x55f365c3d970 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc90659c640 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c44ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc905d9b640 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c4e630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009600 tx=0x7fc8f00069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f68cfa5368e1836 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc8effff640 1 -- 172.21.15.100:0/3827327629 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00098d0 con 0x55f365c3d970 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc8effff640 1 -- 172.21.15.100:0/3827327629 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8f001cc80 con 0x55f365c3d970 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.360+0000 7fc8effff640 1 -- 172.21.15.100:0/3827327629 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0025440 con 0x55f365c3d970 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- 172.21.15.100:0/3827327629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 msgr2=0x55f365c4e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c4e630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009600 tx=0x7fc8f00069f0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- 172.21.15.100:0/3827327629 shutdown_connections 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c42ae0 0x55f365c5bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c3d970 0x55f365c4e630 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/3827327629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c44ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- 172.21.15.100:0/3827327629 >> 172.21.15.100:0/3827327629 conn(0x55f365c384f0 msgr2=0x55f365c3e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- 172.21.15.100:0/3827327629 shutdown_connections 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- 172.21.15.100:0/3827327629 wait complete. 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- start start 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 Processor -- start 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c4b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c3d970 0x55f365c4bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 0x55f365ca96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365c4e690 con 0x55f365c42ae0 2024-03-12T05:11:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365c4e800 con 0x55f365c3d970 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.361+0000 7fc90fc1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f365c4e9c0 con 0x55f365c3cf80 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc90fc1b740 10 client.0 _ll_register_callbacks cb 0x7fc8f0009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 0x55f365ca96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 0x55f365ca96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37724/0 (socket says 172.21.15.100:37724) 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 -- 172.21.15.100:0/4176510065 learned_addr learned my addr 172.21.15.100:0/4176510065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc90659c640 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c4b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 -- 172.21.15.100:0/4176510065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 msgr2=0x55f365c4b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c4b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 -- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c3d970 msgr2=0x55f365c4bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c3d970 0x55f365c4bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 -- 172.21.15.100:0/4176510065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f365ba32b0 con 0x55f365c42ae0 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc906d9d640 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 0x55f365ca96d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8008610 tx=0x7fc8f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662f7f77d7f9c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc8edffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc8edffb640 1 -- 172.21.15.100:0/4176510065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8012ae0 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc8edffb640 1 -- 172.21.15.100:0/4176510065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8f8012c40 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f365c537e0 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc8edffb640 1 -- 172.21.15.100:0/4176510065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f8008cd0 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc90fc1b740 10 client.15075 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f365b65780 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc8edffb640 1 -- 172.21.15.100:0/4176510065 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f365c537e0 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.362+0000 7fc8edffb640 1 -- 172.21.15.100:0/4176510065 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc8f800b050 con 0x55f365c42ae0 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 10 client.15075 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 20 client.15075 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365caedd0 0x55f365cab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 4 client.15075 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T05:11:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fc8f000ebc0 con 0x55f365caedd0 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90659c640 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365caedd0 0x55f365cab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc8edffb640 10 client.15075 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90659c640 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365caedd0 0x55f365cab1f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6242529a5a767f server_cookie=95afcb16b9bfdebc in_seq=0 out_seq=0 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc8edffb640 1 -- 172.21.15.100:0/4176510065 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fc8f000ebc0 con 0x55f365caedd0 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc8edffb640 10 client.15075 handle_command_reply: tid=0 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 1 client.15075 shutdown 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 msgr2=0x55f365ca96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.363+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 0x55f365ca96d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8008610 tx=0x7fc8f8004a40 comp rx=0 tx=0).stop 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 shutdown_connections 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f365caedd0 0x55f365cab1f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f365c42ae0 0x55f365ca96d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f365c3d970 0x55f365c4bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc8edffb640 0 client.15075 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 --2- 172.21.15.100:0/4176510065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365c3cf80 0x55f365c4b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 >> 172.21.15.100:0/4176510065 conn(0x55f365c384f0 msgr2=0x55f365c3e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 shutdown_connections 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 1 -- 172.21.15.100:0/4176510065 wait complete. 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90fc1b740 20 client.15075 trim_cache size 0 max 16384 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24461, 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-03-12T05:11:22.369 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 87.437306406000005, 2024-03-12T05:11:22.370 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 22, 2024-03-12T05:11:22.370 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 88, 2024-03-12T05:11:22.370 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 86, 2024-03-12T05:11:22.370 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 92.670196176999994 2024-03-12T05:11:22.370 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8f40a1a30 msgr2=0x7fc8f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8f40a1a30 0x7fc8f40a3ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8014530 tx=0x7fc8f8014930 comp rx=0 tx=0).stop 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 msgr2=0x7fc908197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 0x7fc908197300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc900036040 tx=0x7fc9000042f0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 shutdown_connections 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8f40a1a30 0x7fc8f40a3ef0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9081016b0 0x7fc90819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc908100cc0 0x7fc908197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 --2- 172.21.15.100:0/3360213368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc908100300 0x7fc908197300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 >> 172.21.15.100:0/3360213368 conn(0x7fc9080fadb0 msgr2=0x7fc908104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 shutdown_connections 2024-03-12T05:11:22.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:22.364+0000 7fc90cf6c640 1 -- 172.21.15.100:0/3360213368 wait complete. 2024-03-12T05:11:22.372 INFO:tasks.cephadm:Setup complete, yielding 2024-03-12T05:11:22.373 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-12T05:11:22.383 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:11:22.383 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph orch status' 2024-03-12T05:11:22.619 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:23.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:22 smithi100 ceph-mon[32210]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:23.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:22 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2809006755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:23.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:22 smithi156 ceph-mon[36270]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:23.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:22 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2809006755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:23.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:22 smithi161 ceph-mon[36043]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:23.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:22 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2809006755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 -- 172.21.15.100:0/2656510233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184103840 msgr2=0x7fc18410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 --2- 172.21.15.100:0/2656510233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184103840 0x7fc18410a0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b0d0 tx=0x7fc17c00b3e0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 -- 172.21.15.100:0/2656510233 shutdown_connections 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 --2- 172.21.15.100:0/2656510233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc184101ca0 0x7fc1841020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 --2- 172.21.15.100:0/2656510233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184103840 0x7fc18410a0f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 --2- 172.21.15.100:0/2656510233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184102ea0 0x7fc184103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.211+0000 7fc189814700 1 -- 172.21.15.100:0/2656510233 >> 172.21.15.100:0/2656510233 conn(0x7fc1840fd280 msgr2=0x7fc1840ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:24.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.212+0000 7fc189814700 1 -- 172.21.15.100:0/2656510233 shutdown_connections 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.212+0000 7fc189814700 1 -- 172.21.15.100:0/2656510233 wait complete. 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.212+0000 7fc189814700 1 Processor -- start 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.212+0000 7fc189814700 1 -- start start 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc184101ca0 0x7fc184197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 0x7fc184198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103840 0x7fc18419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419fa60 con 0x7fc184102ea0 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419fba0 con 0x7fc184101ca0 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419fd30 con 0x7fc184103840 2024-03-12T05:11:24.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 0x7fc184198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 0x7fc184198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37746/0 (socket says 172.21.15.100:37746) 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 -- 172.21.15.100:0/544498400 learned_addr learned my addr 172.21.15.100:0/544498400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc182ffd700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc184101ca0 0x7fc184197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1837fe700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103840 0x7fc18419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 -- 172.21.15.100:0/544498400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103840 msgr2=0x7fc18419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103840 0x7fc18419f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 -- 172.21.15.100:0/544498400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc184101ca0 msgr2=0x7fc184197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc184101ca0 0x7fc184197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 -- 172.21.15.100:0/544498400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009d00 con 0x7fc184102ea0 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc1827fc700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 0x7fc184198410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc17800b560 tx=0x7fc17800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:24.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178011830 con 0x7fc184102ea0 2024-03-12T05:11:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc178011e70 con 0x7fc184102ea0 2024-03-12T05:11:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800f2e0 con 0x7fc184102ea0 2024-03-12T05:11:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.213+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18419ffb0 con 0x7fc184102ea0 2024-03-12T05:11:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.214+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a04b0 con 0x7fc184102ea0 2024-03-12T05:11:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.214+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1841090e0 con 0x7fc184102ea0 2024-03-12T05:11:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.216+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc178011990 con 0x7fc184102ea0 2024-03-12T05:11:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.216+0000 7fc188812700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc16c0a1ac0 0x7fc16c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.216+0000 7fc182ffd700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc16c0a1ac0 0x7fc16c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.216+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc1780cd350 con 0x7fc184102ea0 2024-03-12T05:11:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.216+0000 7fc182ffd700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc16c0a1ac0 0x7fc16c0a3f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.220+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc178096560 con 0x7fc184102ea0 2024-03-12T05:11:24.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.442+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc18402d8d0 con 0x7fc16c0a1ac0 2024-03-12T05:11:24.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.443+0000 7fc188812700 1 -- 172.21.15.100:0/544498400 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc18402d8d0 con 0x7fc16c0a1ac0 2024-03-12T05:11:24.440 INFO:teuthology.orchestra.run.smithi100.stdout:Backend: cephadm 2024-03-12T05:11:24.440 INFO:teuthology.orchestra.run.smithi100.stdout:Available: Yes 2024-03-12T05:11:24.440 INFO:teuthology.orchestra.run.smithi100.stdout:Paused: No 2024-03-12T05:11:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc16c0a1ac0 msgr2=0x7fc16c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc16c0a1ac0 0x7fc16c0a3f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174008040 comp rx=0 tx=0).stop 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 msgr2=0x7fc184198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 0x7fc184198410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc17800b560 tx=0x7fc17800b920 comp rx=0 tx=0).stop 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 shutdown_connections 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc16c0a1ac0 0x7fc16c0a3f80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc184101ca0 0x7fc184197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103840 0x7fc18419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 --2- 172.21.15.100:0/544498400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184102ea0 0x7fc184198410 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 >> 172.21.15.100:0/544498400 conn(0x7fc1840fd280 msgr2=0x7fc184106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 shutdown_connections 2024-03-12T05:11:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:24.445+0000 7fc189814700 1 -- 172.21.15.100:0/544498400 wait complete. 2024-03-12T05:11:25.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:24 smithi100 ceph-mon[32210]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:25.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-12T05:11:25.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:24 smithi156 ceph-mon[36270]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:25.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:24 smithi161 ceph-mon[36043]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:26.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:25 smithi100 ceph-mon[32210]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:26.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:25 smithi156 ceph-mon[36270]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:26.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:25 smithi161 ceph-mon[36043]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 -- 172.21.15.100:0/1393812672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c101ca0 msgr2=0x7f420c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 --2- 172.21.15.100:0/1393812672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c101ca0 0x7f420c1020c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0099c0 tx=0x7f41fc009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 -- 172.21.15.100:0/1393812672 shutdown_connections 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 --2- 172.21.15.100:0/1393812672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c103840 0x7f420c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 --2- 172.21.15.100:0/1393812672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c101ca0 0x7f420c1020c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 --2- 172.21.15.100:0/1393812672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102ea0 0x7f420c103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.018+0000 7f42117e8700 1 -- 172.21.15.100:0/1393812672 >> 172.21.15.100:0/1393812672 conn(0x7f420c0fd280 msgr2=0x7f420c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:27.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 -- 172.21.15.100:0/1393812672 shutdown_connections 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 -- 172.21.15.100:0/1393812672 wait complete. 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 Processor -- start 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 -- start start 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c101ca0 0x7f420c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 0x7f420c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c103840 0x7f420c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19f770 con 0x7f420c102ea0 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19f8b0 con 0x7f420c101ca0 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.019+0000 7f42117e8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19fa40 con 0x7f420c103840 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 0x7f420c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 0x7f420c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37756/0 (socket says 172.21.15.100:37756) 2024-03-12T05:11:27.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 -- 172.21.15.100:0/3280508182 learned_addr learned my addr 172.21.15.100:0/3280508182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420b7fe700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c103840 0x7f420c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420affd700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c101ca0 0x7f420c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 -- 172.21.15.100:0/3280508182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c103840 msgr2=0x7f420c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c103840 0x7f420c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 -- 172.21.15.100:0/3280508182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c101ca0 msgr2=0x7f420c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c101ca0 0x7f420c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 -- 172.21.15.100:0/3280508182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0095f0 con 0x7f420c102ea0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420affd700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c101ca0 0x7f420c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420b7fe700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c103840 0x7f420c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f420a7fc700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 0x7f420c198120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f420000c9e0 tx=0x7f420000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200007ac0 con 0x7f420c102ea0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c19fd20 con 0x7f420c102ea0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4200007c20 con 0x7f420c102ea0 2024-03-12T05:11:27.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200018660 con 0x7f420c102ea0 2024-03-12T05:11:27.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.020+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c1a0210 con 0x7f420c102ea0 2024-03-12T05:11:27.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.021+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c1090e0 con 0x7f420c102ea0 2024-03-12T05:11:27.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.022+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f42000188c0 con 0x7f420c102ea0 2024-03-12T05:11:27.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.023+0000 7f41f3fff700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41f40a1ac0 0x7f41f40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:27.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.023+0000 7f420affd700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41f40a1ac0 0x7f41f40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:27.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.023+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42000cd8b0 con 0x7f420c102ea0 2024-03-12T05:11:27.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.024+0000 7f420affd700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41f40a1ac0 0x7f41f40a3f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0038c0 tx=0x7f41fc009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.027+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4200096a80 con 0x7f420c102ea0 2024-03-12T05:11:27.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:26 smithi100 ceph-mon[32210]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:27.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:26 smithi156 ceph-mon[36270]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:27.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:26 smithi161 ceph-mon[36043]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:27.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.233+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f420c02d8e0 con 0x7f41f40a1ac0 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.246+0000 7f41f3fff700 1 -- 172.21.15.100:0/3280508182 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f420c02d8e0 con 0x7f41f40a1ac0 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (100s) 18s ago 100s 12.7M - 19.0.0-1607-g94a53970 a6477a474d71 45a474f3c504 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi156 running (97s) 31s ago 97s 25.1M - 19.0.0-1607-g94a53970 a6477a474d71 8f30baa94e2b 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:mds.c smithi161 running (111s) 31s ago 111s 20.3M - 19.0.0-1607-g94a53970 a6477a474d71 7f4691dd8286 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:mds.d smithi100 running (119s) 18s ago 119s 16.2M - 19.0.0-1607-g94a53970 a6477a474d71 929a7f2a1b63 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:mds.e smithi156 running (117s) 31s ago 116s 16.8M - 19.0.0-1607-g94a53970 a6477a474d71 06b18cfc8214 2024-03-12T05:11:27.244 INFO:teuthology.orchestra.run.smithi100.stdout:mds.f smithi161 running (114s) 31s ago 114s 15.9M - 19.0.0-1607-g94a53970 a6477a474d71 9434d62e6714 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mds.g smithi100 running (103s) 18s ago 103s 13.8M - 19.0.0-1607-g94a53970 a6477a474d71 5dc8a96ab310 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mds.h smithi156 running (109s) 31s ago 108s 17.2M - 19.0.0-1607-g94a53970 a6477a474d71 9c66f71a5666 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mds.i smithi161 running (106s) 31s ago 105s 20.0M - 19.0.0-1607-g94a53970 a6477a474d71 f73d71f5907b 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mds.j smithi100 running (2m) 18s ago 2m 13.5M - 19.0.0-1607-g94a53970 a6477a474d71 d5ecc4f550e5 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mds.k smithi156 running (2m) 31s ago 2m 13.6M - 19.0.0-1607-g94a53970 a6477a474d71 475daf3e5801 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mds.l smithi161 running (2m) 31s ago 2m 15.3M - 19.0.0-1607-g94a53970 a6477a474d71 7acebf3e2fe9 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi100 *:9283,8765 running (9m) 18s ago 9m 512M - 19.0.0-1607-g94a53970 a6477a474d71 481c1b521344 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi156 *:8443,8765 running (7m) 31s ago 7m 451M - 19.0.0-1607-g94a53970 a6477a474d71 8c9a0e555244 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.z smithi161 *:8443,8765 running (7m) 31s ago 7m 451M - 19.0.0-1607-g94a53970 a6477a474d71 a29f183d81ad 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (9m) 18s ago 9m 53.3M 2048M 19.0.0-1607-g94a53970 a6477a474d71 3a78a7de34af 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi156 running (7m) 31s ago 7m 37.9M 2048M 19.0.0-1607-g94a53970 a6477a474d71 4d7eda35f161 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi161 running (7m) 31s ago 7m 41.9M 2048M 19.0.0-1607-g94a53970 a6477a474d71 72c9153137b7 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (6m) 18s ago 6m 46.8M 4272M 19.0.0-1607-g94a53970 a6477a474d71 a0dda6e7fb63 2024-03-12T05:11:27.245 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi156 running (6m) 31s ago 6m 56.9M 4270M 19.0.0-1607-g94a53970 a6477a474d71 bce56cfd8fe3 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi161 running (6m) 31s ago 6m 49.3M 4270M 19.0.0-1607-g94a53970 a6477a474d71 5298349a5efc 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (5m) 18s ago 5m 54.0M 4272M 19.0.0-1607-g94a53970 a6477a474d71 bda9dc92dea3 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi156 running (5m) 31s ago 5m 54.2M 4270M 19.0.0-1607-g94a53970 a6477a474d71 4d39381bde0a 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi161 running (4m) 31s ago 4m 48.9M 4270M 19.0.0-1607-g94a53970 a6477a474d71 b5132ae8050c 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi100 running (4m) 18s ago 4m 50.9M 4272M 19.0.0-1607-g94a53970 a6477a474d71 c9c95753f409 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi156 running (4m) 31s ago 4m 47.8M 4270M 19.0.0-1607-g94a53970 a6477a474d71 d8de97200528 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 smithi161 running (3m) 31s ago 3m 47.3M 4270M 19.0.0-1607-g94a53970 a6477a474d71 93368c0643e1 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 smithi100 running (3m) 18s ago 3m 52.4M 4272M 19.0.0-1607-g94a53970 a6477a474d71 271d39443757 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 smithi156 running (2m) 31s ago 2m 46.2M 4270M 19.0.0-1607-g94a53970 a6477a474d71 18f4d3a68178 2024-03-12T05:11:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 smithi161 running (2m) 31s ago 2m 52.0M 4270M 19.0.0-1607-g94a53970 a6477a474d71 fdc6ad5e949b 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.248+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41f40a1ac0 msgr2=0x7f41f40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.248+0000 7f42117e8700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41f40a1ac0 0x7f41f40a3f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0038c0 tx=0x7f41fc009510 comp rx=0 tx=0).stop 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.248+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 msgr2=0x7f420c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.248+0000 7f42117e8700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 0x7f420c198120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f420000c9e0 tx=0x7f420000cda0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 shutdown_connections 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f41f40a1ac0 0x7f41f40a3f80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f420c101ca0 0x7f420c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c103840 0x7f420c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 --2- 172.21.15.100:0/3280508182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f420c102ea0 0x7f420c198120 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 >> 172.21.15.100:0/3280508182 conn(0x7f420c0fd280 msgr2=0x7f420c106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 shutdown_connections 2024-03-12T05:11:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:27.249+0000 7f42117e8700 1 -- 172.21.15.100:0/3280508182 wait complete. 2024-03-12T05:11:27.964 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-12T05:11:28.204 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:29.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:28 smithi100 ceph-mon[32210]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:29.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:28 smithi100 ceph-mon[32210]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:29.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:28 smithi156 ceph-mon[36270]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:29.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:28 smithi156 ceph-mon[36270]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:29.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:28 smithi161 ceph-mon[36043]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:29.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:28 smithi161 ceph-mon[36043]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:29.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 -- 172.21.15.100:0/2937181336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228100bf0 msgr2=0x7f6228101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:29.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 --2- 172.21.15.100:0/2937181336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228100bf0 0x7f6228101010 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f62180099c0 tx=0x7f6218009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 -- 172.21.15.100:0/2937181336 shutdown_connections 2024-03-12T05:11:29.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 --2- 172.21.15.100:0/2937181336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228102790 0x7f6228109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 --2- 172.21.15.100:0/2937181336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228100bf0 0x7f6228101010 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 --2- 172.21.15.100:0/2937181336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228101df0 0x7f6228102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 -- 172.21.15.100:0/2937181336 >> 172.21.15.100:0/2937181336 conn(0x7f62280fc170 msgr2=0x7f62280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 -- 172.21.15.100:0/2937181336 shutdown_connections 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.773+0000 7f623039a700 1 -- 172.21.15.100:0/2937181336 wait complete. 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 Processor -- start 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 -- start start 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228100bf0 0x7f6228197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228101df0 0x7f6228198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 0x7f622819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622819f780 con 0x7f6228102790 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622819f8c0 con 0x7f6228100bf0 2024-03-12T05:11:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f623039a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622819fa80 con 0x7f6228101df0 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 0x7f622819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 0x7f622819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37780/0 (socket says 172.21.15.100:37780) 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 -- 172.21.15.100:0/954085954 learned_addr learned my addr 172.21.15.100:0/954085954 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e136700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228100bf0 0x7f6228197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622d935700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228101df0 0x7f6228198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 -- 172.21.15.100:0/954085954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228101df0 msgr2=0x7f6228198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228101df0 0x7f6228198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 -- 172.21.15.100:0/954085954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228100bf0 msgr2=0x7f6228197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228100bf0 0x7f6228197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 -- 172.21.15.100:0/954085954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180095f0 con 0x7f6228102790 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.774+0000 7f622e937700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 0x7f622819f1b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f622000ada0 tx=0x7f622000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.775+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000f070 con 0x7f6228102790 2024-03-12T05:11:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.775+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6220007b30 con 0x7f6228102790 2024-03-12T05:11:29.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.775+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622819fd60 con 0x7f6228102790 2024-03-12T05:11:29.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.775+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220005500 con 0x7f6228102790 2024-03-12T05:11:29.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.775+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281a01b0 con 0x7f6228102790 2024-03-12T05:11:29.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.776+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622810c530 con 0x7f6228102790 2024-03-12T05:11:29.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.777+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6220004ad0 con 0x7f6228102790 2024-03-12T05:11:29.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.777+0000 7f621f7fe700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f62140a1ac0 0x7f62140a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:29.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.777+0000 7f622e136700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f62140a1ac0 0x7f62140a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:29.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.777+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f62200ccb80 con 0x7f6228102790 2024-03-12T05:11:29.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.778+0000 7f622e136700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f62140a1ac0 0x7f62140a3f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6218009990 tx=0x7f6218005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:29.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.782+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6220095e20 con 0x7f6228102790 2024-03-12T05:11:29.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.990+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f622806aa80 con 0x7f62140a1ac0 2024-03-12T05:11:29.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.996+0000 7f621f7fe700 1 -- 172.21.15.100:0/954085954 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f622806aa80 con 0x7f62140a1ac0 2024-03-12T05:11:29.994 INFO:teuthology.orchestra.run.smithi100.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-12T05:11:29.994 INFO:teuthology.orchestra.run.smithi100.stdout:mds.all 12/12 34s ago 2m smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-12T05:11:29.994 INFO:teuthology.orchestra.run.smithi100.stdout:mgr 3/3 34s ago 7m smithi100=x;smithi156=y;smithi161=z;count:3 2024-03-12T05:11:29.994 INFO:teuthology.orchestra.run.smithi100.stdout:mon 3/3 34s ago 7m smithi100:172.21.15.100=a;smithi156:172.21.15.156=b;smithi161:172.21.15.161=c;count:3 2024-03-12T05:11:29.994 INFO:teuthology.orchestra.run.smithi100.stdout:osd 12 34s ago - 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f62140a1ac0 msgr2=0x7f62140a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f62140a1ac0 0x7f62140a3f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6218009990 tx=0x7f6218005fb0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 msgr2=0x7f622819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 0x7f622819f1b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f622000ada0 tx=0x7f622000c330 comp rx=0 tx=0).stop 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 shutdown_connections 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f62140a1ac0 0x7f62140a3f80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228100bf0 0x7f6228197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228102790 0x7f622819f1b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 --2- 172.21.15.100:0/954085954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228101df0 0x7f6228198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 >> 172.21.15.100:0/954085954 conn(0x7f62280fc170 msgr2=0x7f622806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:29.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 shutdown_connections 2024-03-12T05:11:29.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:29.998+0000 7f623039a700 1 -- 172.21.15.100:0/954085954 wait complete. 2024-03-12T05:11:30.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-12T05:11:30.889 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:31.011 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:30 smithi100 ceph-mon[32210]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:31.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:30 smithi156 ceph-mon[36270]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:31.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:30 smithi161 ceph-mon[36043]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:31.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:31 smithi100 ceph-mon[32210]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:32.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:31 smithi156 ceph-mon[36270]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:32.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:31 smithi161 ceph-mon[36043]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:32.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 -- 172.21.15.100:0/222437626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8101ca0 msgr2=0x7fbbd81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 --2- 172.21.15.100:0/222437626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8101ca0 0x7fbbd81020c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40099c0 tx=0x7fbbd4009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 -- 172.21.15.100:0/222437626 shutdown_connections 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 --2- 172.21.15.100:0/222437626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8103840 0x7fbbd810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 --2- 172.21.15.100:0/222437626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8101ca0 0x7fbbd81020c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 --2- 172.21.15.100:0/222437626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbd8102ea0 0x7fbbd8103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 -- 172.21.15.100:0/222437626 >> 172.21.15.100:0/222437626 conn(0x7fbbd80fd280 msgr2=0x7fbbd80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 -- 172.21.15.100:0/222437626 shutdown_connections 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.576+0000 7fbbdf577700 1 -- 172.21.15.100:0/222437626 wait complete. 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 Processor -- start 2024-03-12T05:11:32.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 -- start start 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbd8101ca0 0x7fbbd8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8102ea0 0x7fbbd8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 0x7fbbd819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd819f8c0 con 0x7fbbd8103840 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd819fa00 con 0x7fbbd8101ca0 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdf577700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd819fb90 con 0x7fbbd8102ea0 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbddb14700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 0x7fbbd819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbddb14700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 0x7fbbd819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55746/0 (socket says 172.21.15.100:55746) 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbddb14700 1 -- 172.21.15.100:0/1611406781 learned_addr learned my addr 172.21.15.100:0/1611406781 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdd313700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbd8101ca0 0x7fbbd8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.577+0000 7fbbdcb12700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8102ea0 0x7fbbd8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbddb14700 1 -- 172.21.15.100:0/1611406781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8102ea0 msgr2=0x7fbbd8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbddb14700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8102ea0 0x7fbbd8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbddb14700 1 -- 172.21.15.100:0/1611406781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbd8101ca0 msgr2=0x7fbbd8197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbddb14700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbd8101ca0 0x7fbbd8197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbddb14700 1 -- 172.21.15.100:0/1611406781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd40095f0 con 0x7fbbd8103840 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbdcb12700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8102ea0 0x7fbbd8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbddb14700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 0x7fbbd819f2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000b9a0 tx=0x7fbbd000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000c930 con 0x7fbbd8103840 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbd0014440 con 0x7fbbd8103840 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd819fe70 con 0x7fbbd8103840 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00075d0 con 0x7fbbd8103840 2024-03-12T05:11:32.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.578+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd81a0310 con 0x7fbbd8103840 2024-03-12T05:11:32.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.579+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd81090b0 con 0x7fbbd8103840 2024-03-12T05:11:32.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.580+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbbd0014920 con 0x7fbbd8103840 2024-03-12T05:11:32.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.580+0000 7fbbce7fc700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbbc40a1ac0 0x7fbbc40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:32.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.581+0000 7fbbdd313700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbbc40a1ac0 0x7fbbc40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:32.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.581+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbbd0006e40 con 0x7fbbd8103840 2024-03-12T05:11:32.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.581+0000 7fbbdd313700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbbc40a1ac0 0x7fbbc40a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40038c0 tx=0x7fbbd400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:32.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.585+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbd0095cb0 con 0x7fbbd8103840 2024-03-12T05:11:32.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.788+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd802d870 con 0x7fbbc40a1ac0 2024-03-12T05:11:32.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.790+0000 7fbbce7fc700 1 -- 172.21.15.100:0/1611406781 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fbbd802d870 con 0x7fbbc40a1ac0 2024-03-12T05:11:32.787 INFO:teuthology.orchestra.run.smithi100.stdout:HOST ADDR LABELS STATUS 2024-03-12T05:11:32.787 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100 172.21.15.100 2024-03-12T05:11:32.787 INFO:teuthology.orchestra.run.smithi100.stdout:smithi156 172.21.15.156 2024-03-12T05:11:32.787 INFO:teuthology.orchestra.run.smithi100.stdout:smithi161 172.21.15.161 2024-03-12T05:11:32.787 INFO:teuthology.orchestra.run.smithi100.stdout:3 hosts in cluster 2024-03-12T05:11:32.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbbc40a1ac0 msgr2=0x7fbbc40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:32.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbbc40a1ac0 0x7fbbc40a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40038c0 tx=0x7fbbd400b540 comp rx=0 tx=0).stop 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 msgr2=0x7fbbd819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 0x7fbbd819f2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000b9a0 tx=0x7fbbd000bcb0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 shutdown_connections 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbbc40a1ac0 0x7fbbc40a3f80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbd8101ca0 0x7fbbd8197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8103840 0x7fbbd819f2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 --2- 172.21.15.100:0/1611406781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbd8102ea0 0x7fbbd8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 >> 172.21.15.100:0/1611406781 conn(0x7fbbd80fd280 msgr2=0x7fbbd81061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 shutdown_connections 2024-03-12T05:11:32.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:32.793+0000 7fbbdf577700 1 -- 172.21.15.100:0/1611406781 wait complete. 2024-03-12T05:11:33.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:32 smithi100 ceph-mon[32210]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:33.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:32 smithi156 ceph-mon[36270]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:33.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:32 smithi161 ceph-mon[36043]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:33.440 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-12T05:11:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:34.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:33 smithi100 ceph-mon[32210]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:34.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:33 smithi156 ceph-mon[36270]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:34.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:33 smithi161 ceph-mon[36043]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:35.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:34 smithi100 ceph-mon[32210]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:35.124 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:34 smithi161 ceph-mon[36043]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:35.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:34 smithi156 ceph-mon[36270]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.344+0000 7f75d6098700 1 -- 172.21.15.100:0/2513129999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0103860 msgr2=0x7f75d010a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.344+0000 7f75d6098700 1 --2- 172.21.15.100:0/2513129999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0103860 0x7f75d010a110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b0d0 tx=0x7f75c800b3e0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- 172.21.15.100:0/2513129999 shutdown_connections 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 --2- 172.21.15.100:0/2513129999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0103860 0x7f75d010a110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 --2- 172.21.15.100:0/2513129999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d0101cc0 0x7f75d01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 --2- 172.21.15.100:0/2513129999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0102ec0 0x7f75d0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- 172.21.15.100:0/2513129999 >> 172.21.15.100:0/2513129999 conn(0x7f75d00fd280 msgr2=0x7f75d00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- 172.21.15.100:0/2513129999 shutdown_connections 2024-03-12T05:11:35.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- 172.21.15.100:0/2513129999 wait complete. 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 Processor -- start 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- start start 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d0101cc0 0x7f75d0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 0x7f75d01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0103860 0x7f75d019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d019f900 con 0x7f75d0102ec0 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d019fa40 con 0x7f75d0103860 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75d6098700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d019fc00 con 0x7f75d0101cc0 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75c7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 0x7f75d01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75c7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 0x7f75d01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55768/0 (socket says 172.21.15.100:55768) 2024-03-12T05:11:35.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.345+0000 7f75c7fff700 1 -- 172.21.15.100:0/169149997 learned_addr learned my addr 172.21.15.100:0/169149997 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75cf7fe700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d0101cc0 0x7f75d0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75c7fff700 1 -- 172.21.15.100:0/169149997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d0101cc0 msgr2=0x7f75d0197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75c7fff700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d0101cc0 0x7f75d0197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75c7fff700 1 -- 172.21.15.100:0/169149997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0103860 msgr2=0x7f75d019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75c7fff700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0103860 0x7f75d019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75c7fff700 1 -- 172.21.15.100:0/169149997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8009d00 con 0x7f75d0102ec0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75c7fff700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 0x7f75d01982f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c920 tx=0x7f75b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8007b00 con 0x7f75d0102ec0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d019fee0 con 0x7f75d0102ec0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75b8004d10 con 0x7f75d0102ec0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b80055c0 con 0x7f75d0102ec0 2024-03-12T05:11:35.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.346+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d01a0320 con 0x7f75d0102ec0 2024-03-12T05:11:35.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.347+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d0109080 con 0x7f75d0102ec0 2024-03-12T05:11:35.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.348+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f75b8004750 con 0x7f75d0102ec0 2024-03-12T05:11:35.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.349+0000 7f75cd7fa700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75bc0a1ac0 0x7f75bc0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:35.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.349+0000 7f75cf7fe700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75bc0a1ac0 0x7f75bc0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:35.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.349+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f75b80cd400 con 0x7f75d0102ec0 2024-03-12T05:11:35.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.349+0000 7f75cf7fe700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75bc0a1ac0 0x7f75bc0a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75c0005950 tx=0x7f75c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:35.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.353+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75b8096610 con 0x7f75d0102ec0 2024-03-12T05:11:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.554+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f75d002da40 con 0x7f75bc0a1ac0 2024-03-12T05:11:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.556+0000 7f75cd7fa700 1 -- 172.21.15.100:0/169149997 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f75d002da40 con 0x7f75bc0a1ac0 2024-03-12T05:11:35.553 INFO:teuthology.orchestra.run.smithi100.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-12T05:11:35.553 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-12T05:11:35.553 INFO:teuthology.orchestra.run.smithi100.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-12T05:11:35.553 INFO:teuthology.orchestra.run.smithi100.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75bc0a1ac0 msgr2=0x7f75bc0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75bc0a1ac0 0x7f75bc0a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75c0005950 tx=0x7f75c00058e0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 msgr2=0x7f75d01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 0x7f75d01982f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c920 tx=0x7f75b800cce0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 shutdown_connections 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f75bc0a1ac0 0x7f75bc0a3f80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0103860 0x7f75d019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d0101cc0 0x7f75d0197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 --2- 172.21.15.100:0/169149997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d0102ec0 0x7f75d01982f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 >> 172.21.15.100:0/169149997 conn(0x7f75d00fd280 msgr2=0x7f75d01061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 shutdown_connections 2024-03-12T05:11:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:35.558+0000 7f75d6098700 1 -- 172.21.15.100:0/169149997 wait complete. 2024-03-12T05:11:36.255 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-12T05:11:36.267 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:11:36.267 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-12T05:11:36.503 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:36.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:36 smithi100 ceph-mon[32210]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:36.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:36 smithi100 ceph-mon[32210]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:37.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:36 smithi156 ceph-mon[36270]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:37.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:36 smithi156 ceph-mon[36270]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:37.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:36 smithi161 ceph-mon[36043]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:37.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:36 smithi161 ceph-mon[36043]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:38.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 -- 172.21.15.100:0/1289522956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50103930 msgr2=0x7fee5010a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:38.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 --2- 172.21.15.100:0/1289522956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50103930 0x7fee5010a1e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fee4800b0a0 tx=0x7fee4800b3b0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 -- 172.21.15.100:0/1289522956 shutdown_connections 2024-03-12T05:11:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 --2- 172.21.15.100:0/1289522956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50103930 0x7fee5010a1e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 --2- 172.21.15.100:0/1289522956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500fffb0 0x7fee501003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 --2- 172.21.15.100:0/1289522956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee50102f90 0x7fee501033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 -- 172.21.15.100:0/1289522956 >> 172.21.15.100:0/1289522956 conn(0x7fee500fb910 msgr2=0x7fee500fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 -- 172.21.15.100:0/1289522956 shutdown_connections 2024-03-12T05:11:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.252+0000 7fee57536700 1 -- 172.21.15.100:0/1289522956 wait complete. 2024-03-12T05:11:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 Processor -- start 2024-03-12T05:11:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 -- start start 2024-03-12T05:11:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500fffb0 0x7fee50197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 0x7fee50198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee50103930 0x7fee5019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019f760 con 0x7fee50102f90 2024-03-12T05:11:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019f8a0 con 0x7fee50103930 2024-03-12T05:11:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee57536700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019fa60 con 0x7fee500fffb0 2024-03-12T05:11:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 0x7fee50198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 0x7fee50198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55786/0 (socket says 172.21.15.100:55786) 2024-03-12T05:11:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 -- 172.21.15.100:0/1116738475 learned_addr learned my addr 172.21.15.100:0/1116738475 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee552d2700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500fffb0 0x7fee50197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee55ad3700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee50103930 0x7fee5019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 -- 172.21.15.100:0/1116738475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500fffb0 msgr2=0x7fee50197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500fffb0 0x7fee50197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 -- 172.21.15.100:0/1116738475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee50103930 msgr2=0x7fee5019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee50103930 0x7fee5019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 -- 172.21.15.100:0/1116738475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009d00 con 0x7fee50102f90 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.253+0000 7fee54ad1700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 0x7fee50198150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00c9b0 tx=0x7fee4c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.254+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c007b00 con 0x7fee50102f90 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.254+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5019fd40 con 0x7fee50102f90 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.254+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee4c007c60 con 0x7fee50102f90 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.254+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c018660 con 0x7fee50102f90 2024-03-12T05:11:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.254+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee501a0180 con 0x7fee50102f90 2024-03-12T05:11:38.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.255+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee50109080 con 0x7fee50102f90 2024-03-12T05:11:38.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.256+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fee4c00f450 con 0x7fee50102f90 2024-03-12T05:11:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.257+0000 7fee467fc700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee3c0a1ac0 0x7fee3c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.257+0000 7fee552d2700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee3c0a1ac0 0x7fee3c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.257+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fee4c0cd620 con 0x7fee50102f90 2024-03-12T05:11:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.257+0000 7fee552d2700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee3c0a1ac0 0x7fee3c0a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee501990f0 tx=0x7fee4000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.260+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee4c0968c0 con 0x7fee50102f90 2024-03-12T05:11:38.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.495+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee5010d880 con 0x7fee50102f90 2024-03-12T05:11:38.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.495+0000 7fee467fc700 1 -- 172.21.15.100:0/1116738475 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2951 (secure 0 0 0) 0x7fee5010d880 con 0x7fee50102f90 2024-03-12T05:11:38.493 INFO:teuthology.orchestra.run.smithi100.stdout:e22 2024-03-12T05:11:38.493 INFO:teuthology.orchestra.run.smithi100.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T05:11:38.493 INFO:teuthology.orchestra.run.smithi100.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T05:11:38.493 INFO:teuthology.orchestra.run.smithi100.stdout:legacy client fscid: 1 2024-03-12T05:11:38.493 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:Filesystem 'cephfs' (1) 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:fs_name cephfs 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 22 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-03-12T05:09:54.274771+0000 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-03-12T05:09:59.961590+0000 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:tableserver 0 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:root 0 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:session_timeout 300 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:session_autoclose 300 2024-03-12T05:11:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:max_file_size 1099511627776 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:max_xattr_size 65536 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:required_client_features {} 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:last_failure 0 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:last_failure_osd_epoch 0 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:max_mds 3 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:in 0,1,2 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:up {0=24461,1=24451,2=24439} 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:failed 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:damaged 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:stopped 2024-03-12T05:11:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:data_pools [3] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:metadata_pool 2 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:inline_data disabled 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:balancer 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:bal_rank_mask -1 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:standby_count_wanted 0 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b{0:24461} state up:active seq 3 addr [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c{0:24427} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/854337146,v1:172.21.15.161:6837/854337146] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.i{1:24451} state up:active seq 6 addr [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.f{1:24415} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/692407308,v1:172.21.15.161:6835/692407308] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.h{2:24439} state up:active seq 7 addr [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.e{2:24401} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/2592644588,v1:172.21.15.156:6835/2592644588] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:Standby daemons: 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.100:6834/3900408401,v1:172.21.15.100:6835/3900408401] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.100:6836/1446338988,v1:172.21.15.100:6837/1446338988] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.g{-1:14595} state up:standby seq 1 addr [v2:172.21.15.100:6838/2856725609,v1:172.21.15.100:6839/2856725609] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a{-1:14613} state up:standby seq 1 addr [v2:172.21.15.100:6840/3942756808,v1:172.21.15.100:6841/3942756808] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.k{-1:24365} state up:standby seq 1 addr [v2:172.21.15.156:6832/692677489,v1:172.21.15.156:6833/692677489] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.l{-1:24382} state up:standby seq 1 addr [v2:172.21.15.161:6832/2628600051,v1:172.21.15.161:6833/2628600051] compat {c=[1],r=[1],i=[fff]}] 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee3c0a1ac0 msgr2=0x7fee3c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee3c0a1ac0 0x7fee3c0a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee501990f0 tx=0x7fee4000a3b0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 msgr2=0x7fee50198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 0x7fee50198150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00c9b0 tx=0x7fee4c00ccc0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 shutdown_connections 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fee3c0a1ac0 0x7fee3c0a3f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee50103930 0x7fee5019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500fffb0 0x7fee50197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 --2- 172.21.15.100:0/1116738475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50102f90 0x7fee50198150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.497+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 >> 172.21.15.100:0/1116738475 conn(0x7fee500fb910 msgr2=0x7fee501061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.498+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 shutdown_connections 2024-03-12T05:11:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:38.498+0000 7fee57536700 1 -- 172.21.15.100:0/1116738475 wait complete. 2024-03-12T05:11:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:11:39.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:38 smithi156 ceph-mon[36270]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:39.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:38 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1116738475' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-12T05:11:39.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:38 smithi161 ceph-mon[36043]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:39.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:38 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1116738475' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-12T05:11:39.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:38 smithi100 ceph-mon[32210]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:39.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:38 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1116738475' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-12T05:11:39.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-12T05:11:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:40.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:40 smithi100 ceph-mon[32210]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:41.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:40 smithi156 ceph-mon[36270]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:41.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:40 smithi161 ceph-mon[36043]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 -- 172.21.15.100:0/3180324704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 msgr2=0x7f4360108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 --2- 172.21.15.100:0/3180324704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f4360108fe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43580099c0 tx=0x7f4358009cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 -- 172.21.15.100:0/3180324704 shutdown_connections 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 --2- 172.21.15.100:0/3180324704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4360100b90 0x7f4360100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 --2- 172.21.15.100:0/3180324704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f4360108fe0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 --2- 172.21.15.100:0/3180324704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4360101d90 0x7f43601021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 -- 172.21.15.100:0/3180324704 >> 172.21.15.100:0/3180324704 conn(0x7f43600fc130 msgr2=0x7f43600fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 -- 172.21.15.100:0/3180324704 shutdown_connections 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.240+0000 7f4366c61700 1 -- 172.21.15.100:0/3180324704 wait complete. 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 Processor -- start 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 -- start start 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4360100b90 0x7f4360196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4360101d90 0x7f4360196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f436019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019e590 con 0x7f4360102730 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019e6d0 con 0x7f4360100b90 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f4366c61700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019e860 con 0x7f4360101d90 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f43651fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f436019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f43651fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f436019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51980/0 (socket says 172.21.15.100:51980) 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f43651fe700 1 -- 172.21.15.100:0/1301162712 learned_addr learned my addr 172.21.15.100:0/1301162712 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.241+0000 7f43649fd700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4360100b90 0x7f4360196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f435ffff700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4360101d90 0x7f4360196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f43651fe700 1 -- 172.21.15.100:0/1301162712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4360101d90 msgr2=0x7f4360196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f43651fe700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4360101d90 0x7f4360196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f43651fe700 1 -- 172.21.15.100:0/1301162712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4360100b90 msgr2=0x7f4360196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f43651fe700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4360100b90 0x7f4360196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f43651fe700 1 -- 172.21.15.100:0/1301162712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580095f0 con 0x7f4360102730 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f43651fe700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f436019dfc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4358005910 tx=0x7f4358005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435801f070 con 0x7f4360102730 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f435801ac30 con 0x7f4360102730 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436006a1b0 con 0x7f4360102730 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800f460 con 0x7f4360102730 2024-03-12T05:11:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.242+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436006a650 con 0x7f4360102730 2024-03-12T05:11:41.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.243+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4360068930 con 0x7f4360102730 2024-03-12T05:11:41.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.245+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4358023070 con 0x7f4360102730 2024-03-12T05:11:41.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.245+0000 7f435dffb700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f43500a1ac0 0x7f43500a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:41.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.245+0000 7f43649fd700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f43500a1ac0 0x7f43500a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.246+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43580d0d30 con 0x7f4360102730 2024-03-12T05:11:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.246+0000 7f43649fd700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f43500a1ac0 0x7f43500a3f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4360101bf0 tx=0x7f435400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:41.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.249+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4358099fd0 con 0x7f4360102730 2024-03-12T05:11:41.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.442+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f436010c600 con 0x7f4360102730 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.443+0000 7f435dffb700 1 -- 172.21.15.100:0/1301162712 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5322 (secure 0 0 0) 0x7f435809d690 con 0x7f4360102730 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 88 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-03-12T05:02:08.927382+0000 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-03-12T05:10:04.977359+0000 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:crush_version 26 2024-03-12T05:11:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:full_ratio 0.95 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:backfillfull_ratio 0.9 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:nearfull_ratio 0.85 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:require_min_compat_client luminous 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:min_compat_client luminous 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:require_osd_release squid 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:stretch_mode_enabled false 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:max_osd 12 2024-03-12T05:11:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6802/774764839,v1:172.21.15.100:6803/774764839] [v2:172.21.15.100:6804/774764839,v1:172.21.15.100:6805/774764839] exists,up df3ee95d-798a-4fd9-82ef-81f981a5461d 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/3629499261,v1:172.21.15.156:6801/3629499261] [v2:172.21.15.156:6802/3629499261,v1:172.21.15.156:6803/3629499261] exists,up 95723563-e515-4042-ac3c-bb298e215da0 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/623458559,v1:172.21.15.161:6801/623458559] [v2:172.21.15.161:6802/623458559,v1:172.21.15.161:6803/623458559] exists,up 79302fd1-469d-440f-b520-378f0f3a459b 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6810/2837761202,v1:172.21.15.100:6811/2837761202] [v2:172.21.15.100:6812/2837761202,v1:172.21.15.100:6813/2837761202] exists,up 037d713e-354a-437d-9f75-3b506f6e4ed0 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/551000361,v1:172.21.15.156:6809/551000361] [v2:172.21.15.156:6810/551000361,v1:172.21.15.156:6811/551000361] exists,up 206a171a-ec8a-474b-9f06-40508adf5df8 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/3085464556,v1:172.21.15.161:6809/3085464556] [v2:172.21.15.161:6810/3085464556,v1:172.21.15.161:6811/3085464556] exists,up a41244b3-befa-47fa-809e-fdc7b1a23f2c 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6818/125600956,v1:172.21.15.100:6819/125600956] [v2:172.21.15.100:6820/125600956,v1:172.21.15.100:6821/125600956] exists,up 4349c6aa-c535-40fb-a462-cf066f57ea26 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/3327014468,v1:172.21.15.156:6817/3327014468] [v2:172.21.15.156:6818/3327014468,v1:172.21.15.156:6819/3327014468] exists,up 1fd43ecf-765c-412a-b351-a8b7fd3b7381 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/2263371293,v1:172.21.15.161:6817/2263371293] [v2:172.21.15.161:6818/2263371293,v1:172.21.15.161:6819/2263371293] exists,up 2fc82b38-b2e7-47c3-9130-8c921510b06c 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6826/917772860,v1:172.21.15.100:6827/917772860] [v2:172.21.15.100:6828/917772860,v1:172.21.15.100:6829/917772860] exists,up 2f17e94a-a270-4e9e-837e-e0c5dfb2e68c 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/3731703036,v1:172.21.15.156:6825/3731703036] [v2:172.21.15.156:6826/3731703036,v1:172.21.15.156:6827/3731703036] exists,up 359d96d0-445b-4e93-8811-5e0407d42957 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2757536197,v1:172.21.15.161:6825/2757536197] [v2:172.21.15.161:6826/2757536197,v1:172.21.15.161:6827/2757536197] exists,up bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.a [2,11] 2024-03-12T05:11:41.442 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.13 [2,11] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.0 [5,9] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.6 [3,7] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.e [4,6] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.10 [5,9] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.21 [4,8] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.31 [4,8] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.34 [4,9] 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/16620000 expires 2024-03-13T05:03:03.632804+0000 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/16620000 expires 2024-03-13T05:03:03.632804+0000 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/4124706748 expires 2024-03-13T05:03:03.632804+0000 2024-03-12T05:11:41.443 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/1254888591 expires 2024-03-13T05:03:03.632804+0000 2024-03-12T05:11:41.444 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/981252995 expires 2024-03-13T05:02:37.449022+0000 2024-03-12T05:11:41.444 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/2732470606 expires 2024-03-13T05:02:37.449022+0000 2024-03-12T05:11:41.444 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/2462375416 expires 2024-03-13T05:02:37.449022+0000 2024-03-12T05:11:41.444 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/3056681718 expires 2024-03-13T05:02:37.449022+0000 2024-03-12T05:11:41.444 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/981252995 expires 2024-03-13T05:02:37.449022+0000 2024-03-12T05:11:41.444 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/3130631042 expires 2024-03-13T05:03:03.632804+0000 2024-03-12T05:11:41.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f43500a1ac0 msgr2=0x7f43500a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:41.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f43500a1ac0 0x7f43500a3f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4360101bf0 tx=0x7f435400c040 comp rx=0 tx=0).stop 2024-03-12T05:11:41.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 msgr2=0x7f436019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:41.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f436019dfc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4358005910 tx=0x7f4358005740 comp rx=0 tx=0).stop 2024-03-12T05:11:41.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 shutdown_connections 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f43500a1ac0 0x7f43500a3f80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4360100b90 0x7f4360196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4360102730 0x7f436019dfc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 --2- 172.21.15.100:0/1301162712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4360101d90 0x7f4360196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 >> 172.21.15.100:0/1301162712 conn(0x7f43600fc130 msgr2=0x7f4360104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 shutdown_connections 2024-03-12T05:11:41.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:41.445+0000 7f4366c61700 1 -- 172.21.15.100:0/1301162712 wait complete. 2024-03-12T05:11:42.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:41 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1301162712' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-12T05:11:42.172 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-12T05:11:42.186 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:42.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:41 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1301162712' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-12T05:11:42.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:41 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1301162712' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-12T05:11:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 -- 172.21.15.100:0/2182822881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a0101c50 msgr2=0x7ff0a01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2182822881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a0101c50 0x7ff0a01084e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff098009ca0 tx=0x7ff09802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 -- 172.21.15.100:0/2182822881 shutdown_connections 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2182822881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a0101c50 0x7ff0a01084e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2182822881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a0101290 0x7ff0a0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2182822881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0a01000a0 0x7ff0a01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 -- 172.21.15.100:0/2182822881 >> 172.21.15.100:0/2182822881 conn(0x7ff0a00fb610 msgr2=0x7ff0a00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 -- 172.21.15.100:0/2182822881 shutdown_connections 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.297+0000 7ff0a7799640 1 -- 172.21.15.100:0/2182822881 wait complete. 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 Processor -- start 2024-03-12T05:11:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 -- start start 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 0x7ff0a0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a0101290 0x7ff0a0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0a0101c50 0x7ff0a019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a019ecf0 con 0x7ff0a01000a0 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a019ee60 con 0x7ff0a0101c50 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a019f020 con 0x7ff0a0101290 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 0x7ff0a0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 0x7ff0a0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52000/0 (socket says 172.21.15.100:52000) 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 -- 172.21.15.100:0/2106761193 learned_addr learned my addr 172.21.15.100:0/2106761193 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5d36640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0a0101c50 0x7ff0a019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 -- 172.21.15.100:0/2106761193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a0101290 msgr2=0x7ff0a0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a0101290 0x7ff0a0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 -- 172.21.15.100:0/2106761193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0a0101c50 msgr2=0x7ff0a019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0a0101c50 0x7ff0a019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 -- 172.21.15.100:0/2106761193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009900 con 0x7ff0a01000a0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a5535640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 0x7ff0a0197200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00c970 tx=0x7ff09c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c004220 con 0x7ff0a01000a0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff09c007e90 con 0x7ff0a01000a0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a019f300 con 0x7ff0a01000a0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00f450 con 0x7ff0a01000a0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.298+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a019f810 con 0x7ff0a01000a0 2024-03-12T05:11:42.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.299+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a01073c0 con 0x7ff0a01000a0 2024-03-12T05:11:42.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.300+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff09c003610 con 0x7ff0a01000a0 2024-03-12T05:11:42.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.301+0000 7ff0967fc640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff08c0a1a30 0x7ff08c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.301+0000 7ff0a4d34640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff08c0a1a30 0x7ff08c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.301+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff09c0cd1a0 con 0x7ff0a01000a0 2024-03-12T05:11:42.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.301+0000 7ff0a4d34640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff08c0a1a30 0x7ff08c0a3ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff090005fd0 tx=0x7ff090005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.305+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff09c0963b0 con 0x7ff0a01000a0 2024-03-12T05:11:42.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.477+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0a002ceb0 con 0x7ff0a01000a0 2024-03-12T05:11:42.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.479+0000 7ff0967fc640 1 -- 172.21.15.100:0/2106761193 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7ff09c099a70 con 0x7ff0a01000a0 2024-03-12T05:11:42.477 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:42.477 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:42.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff08c0a1a30 msgr2=0x7ff08c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff08c0a1a30 0x7ff08c0a3ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff090005fd0 tx=0x7ff090005850 comp rx=0 tx=0).stop 2024-03-12T05:11:42.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 msgr2=0x7ff0a0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 0x7ff0a0197200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00c970 tx=0x7ff09c00ce30 comp rx=0 tx=0).stop 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 shutdown_connections 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7ff08c0a1a30 0x7ff08c0a3ef0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0a0101c50 0x7ff0a019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a0101290 0x7ff0a0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 --2- 172.21.15.100:0/2106761193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a01000a0 0x7ff0a0197200 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 >> 172.21.15.100:0/2106761193 conn(0x7ff0a00fb610 msgr2=0x7ff0a0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 shutdown_connections 2024-03-12T05:11:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.481+0000 7ff0a7799640 1 -- 172.21.15.100:0/2106761193 wait complete. 2024-03-12T05:11:42.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:42.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:42.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 -- 172.21.15.100:0/3290542761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101c40 msgr2=0x7f09501084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 --2- 172.21.15.100:0/3290542761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101c40 0x7f09501084d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0948009ca0 tx=0x7f094802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 -- 172.21.15.100:0/3290542761 shutdown_connections 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 --2- 172.21.15.100:0/3290542761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101c40 0x7f09501084d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 --2- 172.21.15.100:0/3290542761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0950101280 0x7f0950101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 --2- 172.21.15.100:0/3290542761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0950100090 0x7f09501004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 -- 172.21.15.100:0/3290542761 >> 172.21.15.100:0/3290542761 conn(0x7f09500fb620 msgr2=0x7f09500fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 -- 172.21.15.100:0/3290542761 shutdown_connections 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.597+0000 7f0955815640 1 -- 172.21.15.100:0/3290542761 wait complete. 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 Processor -- start 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 -- start start 2024-03-12T05:11:42.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0950100090 0x7f095006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 0x7f095019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0950101c40 0x7f095019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095019ed40 con 0x7f0950101280 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095019eeb0 con 0x7f0950100090 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095019f070 con 0x7f0950101c40 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 0x7f095019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 0x7f095019bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52008/0 (socket says 172.21.15.100:52008) 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 -- 172.21.15.100:0/1788213849 learned_addr learned my addr 172.21.15.100:0/1788213849 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f094effd640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0950100090 0x7f095006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f094f7fe640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0950101c40 0x7f095019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 -- 172.21.15.100:0/1788213849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0950101c40 msgr2=0x7f095019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0950101c40 0x7f095019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 -- 172.21.15.100:0/1788213849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0950100090 msgr2=0x7f095006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0950100090 0x7f095006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 -- 172.21.15.100:0/1788213849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0948009900 con 0x7f0950101280 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0947fff640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 0x7f095019bf40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f093800e9f0 tx=0x7f093800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938009720 con 0x7f0950101280 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f093800cd80 con 0x7f0950101280 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095019f2f0 con 0x7f0950101280 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.598+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938010690 con 0x7f0950101280 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.599+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095019f7b0 con 0x7f0950101280 2024-03-12T05:11:42.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.600+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09501073b0 con 0x7f0950101280 2024-03-12T05:11:42.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.601+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f09380046c0 con 0x7f0950101280 2024-03-12T05:11:42.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.601+0000 7f0954813640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f093c0a1a30 0x7f093c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.601+0000 7f094effd640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f093c0a1a30 0x7f093c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.601+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0938014070 con 0x7f0950101280 2024-03-12T05:11:42.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.602+0000 7f094effd640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f093c0a1a30 0x7f093c0a3ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f09501010e0 tx=0x7f0940005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:42.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.605+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0938096c50 con 0x7f0950101280 2024-03-12T05:11:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.776+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f095010bc30 con 0x7f0950101280 2024-03-12T05:11:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.776+0000 7f0954813640 1 -- 172.21.15.100:0/1788213849 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f093809a310 con 0x7f0950101280 2024-03-12T05:11:42.774 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f093c0a1a30 msgr2=0x7f093c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f093c0a1a30 0x7f093c0a3ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f09501010e0 tx=0x7f0940005f90 comp rx=0 tx=0).stop 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 msgr2=0x7f095019bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 0x7f095019bf40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f093800e9f0 tx=0x7f093800edf0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 shutdown_connections 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f093c0a1a30 0x7f093c0a3ef0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0950101c40 0x7f095019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0950101280 0x7f095019bf40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 --2- 172.21.15.100:0/1788213849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0950100090 0x7f095006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 >> 172.21.15.100:0/1788213849 conn(0x7f09500fb620 msgr2=0x7f0950104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:42.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 shutdown_connections 2024-03-12T05:11:42.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.778+0000 7f0955815640 1 -- 172.21.15.100:0/1788213849 wait complete. 2024-03-12T05:11:42.787 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:42.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 -- 172.21.15.100:0/1515809755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102d40 msgr2=0x7f1ae01095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 --2- 172.21.15.100:0/1515809755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102d40 0x7f1ae01095d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009ca0 tx=0x7f1ad802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 -- 172.21.15.100:0/1515809755 shutdown_connections 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 --2- 172.21.15.100:0/1515809755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102d40 0x7f1ae01095d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 --2- 172.21.15.100:0/1515809755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0102380 0x7f1ae0102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 --2- 172.21.15.100:0/1515809755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ae0101190 0x7f1ae01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 -- 172.21.15.100:0/1515809755 >> 172.21.15.100:0/1515809755 conn(0x7f1ae00fc6e0 msgr2=0x7f1ae00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 -- 172.21.15.100:0/1515809755 shutdown_connections 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.893+0000 7f1ae637a640 1 -- 172.21.15.100:0/1515809755 wait complete. 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 Processor -- start 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 -- start start 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ae0101190 0x7f1ae0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 0x7f1ae01976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0102d40 0x7f1ae019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019ec70 con 0x7f1ae0102380 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019ede0 con 0x7f1ae0102d40 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1ae637a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019efa0 con 0x7f1ae0101190 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 0x7f1ae01976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 0x7f1ae01976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52018/0 (socket says 172.21.15.100:52018) 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 -- 172.21.15.100:0/99500250 learned_addr learned my addr 172.21.15.100:0/99500250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adffff640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ae0101190 0x7f1ae0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 -- 172.21.15.100:0/99500250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ae0101190 msgr2=0x7f1ae0197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ae0101190 0x7f1ae0197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 -- 172.21.15.100:0/99500250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0102d40 msgr2=0x7f1ae019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0102d40 0x7f1ae019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.894+0000 7f1adf7fe640 1 -- 172.21.15.100:0/99500250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8009900 con 0x7f1ae0102380 2024-03-12T05:11:42.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.895+0000 7f1adf7fe640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 0x7f1ae01976c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400e9f0 tx=0x7f1ad400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:42.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.895+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4009700 con 0x7f1ae0102380 2024-03-12T05:11:42.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.895+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ad400cd80 con 0x7f1ae0102380 2024-03-12T05:11:42.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.895+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae019f280 con 0x7f1ae0102380 2024-03-12T05:11:42.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.895+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4018920 con 0x7f1ae0102380 2024-03-12T05:11:42.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.895+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae019f7e0 con 0x7f1ae0102380 2024-03-12T05:11:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.896+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae0108550 con 0x7f1ae0102380 2024-03-12T05:11:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.897+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1ad4018a80 con 0x7f1ae0102380 2024-03-12T05:11:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.897+0000 7f1add7fa640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ac80a1ab0 0x7f1ac80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.897+0000 7f1adffff640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ac80a1ab0 0x7f1ac80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.897+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ad4014070 con 0x7f1ae0102380 2024-03-12T05:11:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.898+0000 7f1adffff640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ac80a1ab0 0x7f1ac80a3f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005fd0 tx=0x7f1ad0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:42.902+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ad4052d20 con 0x7f1ae0102380 2024-03-12T05:11:43.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:42 smithi100 ceph-mon[32210]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:43.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:42 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2106761193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:43.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:42 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1788213849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.077+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ae010cba0 con 0x7f1ae0102380 2024-03-12T05:11:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.077+0000 7f1add7fa640 1 -- 172.21.15.100:0/99500250 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ad4096a60 con 0x7f1ae0102380 2024-03-12T05:11:43.074 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ac80a1ab0 msgr2=0x7f1ac80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ac80a1ab0 0x7f1ac80a3f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005fd0 tx=0x7f1ad0005850 comp rx=0 tx=0).stop 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 msgr2=0x7f1ae01976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 0x7f1ae01976c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400e9f0 tx=0x7f1ad400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 shutdown_connections 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1ac80a1ab0 0x7f1ac80a3f70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0102d40 0x7f1ae019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ae0102380 0x7f1ae01976c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 --2- 172.21.15.100:0/99500250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ae0101190 0x7f1ae0197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 >> 172.21.15.100:0/99500250 conn(0x7f1ae00fc6e0 msgr2=0x7f1ae0105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 shutdown_connections 2024-03-12T05:11:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.079+0000 7f1ae637a640 1 -- 172.21.15.100:0/99500250 wait complete. 2024-03-12T05:11:43.088 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:43.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.195+0000 7f04742d4640 1 -- 172.21.15.100:0/3451023381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101c30 msgr2=0x7f046c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.195+0000 7f04742d4640 1 --2- 172.21.15.100:0/3451023381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101c30 0x7f046c1084c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0468009900 tx=0x7f046802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:43.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- 172.21.15.100:0/3451023381 shutdown_connections 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 --2- 172.21.15.100:0/3451023381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101c30 0x7f046c1084c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 --2- 172.21.15.100:0/3451023381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f046c101270 0x7f046c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 --2- 172.21.15.100:0/3451023381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f046c100080 0x7f046c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- 172.21.15.100:0/3451023381 >> 172.21.15.100:0/3451023381 conn(0x7f046c0fb610 msgr2=0x7f046c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- 172.21.15.100:0/3451023381 shutdown_connections 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- 172.21.15.100:0/3451023381 wait complete. 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 Processor -- start 2024-03-12T05:11:43.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- start start 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f046c100080 0x7f046c068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 0x7f046c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f046c101c30 0x7f046c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c19ec40 con 0x7f046c101270 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c19edb0 con 0x7f046c100080 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f04742d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c19ef70 con 0x7f046c101c30 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f047186f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 0x7f046c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f047186f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 0x7f046c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52046/0 (socket says 172.21.15.100:52046) 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f047186f640 1 -- 172.21.15.100:0/2224615188 learned_addr learned my addr 172.21.15.100:0/2224615188 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.196+0000 7f0472070640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f046c100080 0x7f046c068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f0472871640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f046c101c30 0x7f046c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f047186f640 1 -- 172.21.15.100:0/2224615188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f046c101c30 msgr2=0x7f046c19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f047186f640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f046c101c30 0x7f046c19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f047186f640 1 -- 172.21.15.100:0/2224615188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f046c100080 msgr2=0x7f046c068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f047186f640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f046c100080 0x7f046c068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f047186f640 1 -- 172.21.15.100:0/2224615188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0468009560 con 0x7f046c101270 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f047186f640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 0x7f046c19be70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c920 tx=0x7f045c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c007c70 con 0x7f046c101270 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f045c007dd0 con 0x7f046c101270 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c018860 con 0x7f046c101270 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c19f250 con 0x7f046c101270 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.197+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c19f7b0 con 0x7f046c101270 2024-03-12T05:11:43.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.198+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c107340 con 0x7f046c101270 2024-03-12T05:11:43.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.199+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f045c00fac0 con 0x7f046c101270 2024-03-12T05:11:43.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.199+0000 7f04637fe640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04580a1ab0 0x7f04580a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.200+0000 7f0472070640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04580a1ab0 0x7f04580a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.200+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f045c0cd4d0 con 0x7f046c101270 2024-03-12T05:11:43.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.200+0000 7f0472070640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04580a1ab0 0x7f04580a3f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f046c1010d0 tx=0x7f0464009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:43.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.204+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f045c0966e0 con 0x7f046c101270 2024-03-12T05:11:43.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:42 smithi156 ceph-mon[36270]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:43.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:42 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2106761193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:43.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:42 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1788213849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:43.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:42 smithi161 ceph-mon[36043]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-12T05:11:43.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:42 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2106761193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:43.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:42 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1788213849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:43.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.376+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f046c02ce40 con 0x7f046c101270 2024-03-12T05:11:43.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.376+0000 7f04637fe640 1 -- 172.21.15.100:0/2224615188 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f045c099da0 con 0x7f046c101270 2024-03-12T05:11:43.374 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:43.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04580a1ab0 msgr2=0x7f04580a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04580a1ab0 0x7f04580a3f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f046c1010d0 tx=0x7f0464009500 comp rx=0 tx=0).stop 2024-03-12T05:11:43.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 msgr2=0x7f046c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 0x7f046c19be70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c920 tx=0x7f045c00cde0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 shutdown_connections 2024-03-12T05:11:43.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04580a1ab0 0x7f04580a3f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f046c101c30 0x7f046c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f046c101270 0x7f046c19be70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 --2- 172.21.15.100:0/2224615188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f046c100080 0x7f046c068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.378+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 >> 172.21.15.100:0/2224615188 conn(0x7f046c0fb610 msgr2=0x7f046c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:43.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.379+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 shutdown_connections 2024-03-12T05:11:43.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.379+0000 7f04742d4640 1 -- 172.21.15.100:0/2224615188 wait complete. 2024-03-12T05:11:43.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 -- 172.21.15.100:0/2976512587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 msgr2=0x7f28ac108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 --2- 172.21.15.100:0/2976512587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac108340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009ca0 tx=0x7f28a402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 -- 172.21.15.100:0/2976512587 shutdown_connections 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 --2- 172.21.15.100:0/2976512587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac108340 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 --2- 172.21.15.100:0/2976512587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 0x7f28ac1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 --2- 172.21.15.100:0/2976512587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28ac0ffe80 0x7f28ac1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 -- 172.21.15.100:0/2976512587 >> 172.21.15.100:0/2976512587 conn(0x7f28ac0fb460 msgr2=0x7f28ac0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 -- 172.21.15.100:0/2976512587 shutdown_connections 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.490+0000 7f28b199f640 1 -- 172.21.15.100:0/2976512587 wait complete. 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 Processor -- start 2024-03-12T05:11:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 -- start start 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28ac0ffe80 0x7f28ac199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 0x7f28ac199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1a0cf0 con 0x7f28ac101a20 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1a0e60 con 0x7f28ac101060 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b199f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1a1020 con 0x7f28ac0ffe80 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac1a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52062/0 (socket says 172.21.15.100:52062) 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 -- 172.21.15.100:0/2208621688 learned_addr learned my addr 172.21.15.100:0/2208621688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b099d640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28ac0ffe80 0x7f28ac199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28abfff640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 0x7f28ac199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 -- 172.21.15.100:0/2208621688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28ac0ffe80 msgr2=0x7f28ac199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28ac0ffe80 0x7f28ac199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 -- 172.21.15.100:0/2208621688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 msgr2=0x7f28ac199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 0x7f28ac199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28b119e640 1 -- 172.21.15.100:0/2208621688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4009900 con 0x7f28ac101a20 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.491+0000 7f28abfff640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 0x7f28ac199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.492+0000 7f28b119e640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac1a07b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f28a4037040 tx=0x7f28a4034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.492+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4032eb0 con 0x7f28ac101a20 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.492+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28a4003e50 con 0x7f28ac101a20 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.492+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac1a12a0 con 0x7f28ac101a20 2024-03-12T05:11:43.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.492+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4040450 con 0x7f28ac101a20 2024-03-12T05:11:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.492+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac1a1750 con 0x7f28ac101a20 2024-03-12T05:11:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.493+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac192e50 con 0x7f28ac101a20 2024-03-12T05:11:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.494+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f28a400a040 con 0x7f28ac101a20 2024-03-12T05:11:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.495+0000 7f28a9ffb640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f289c0a1a10 0x7f289c0a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.495+0000 7f28b099d640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f289c0a1a10 0x7f289c0a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.495+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28a40f07c0 con 0x7f28ac101a20 2024-03-12T05:11:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.495+0000 7f28b099d640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f289c0a1a10 0x7f289c0a3ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f28a0005fd0 tx=0x7f28a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.498+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28a40b99d0 con 0x7f28ac101a20 2024-03-12T05:11:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.707+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28ac10a7c0 con 0x7f28ac101a20 2024-03-12T05:11:43.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.707+0000 7f28a9ffb640 1 -- 172.21.15.100:0/2208621688 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f28a40bd090 con 0x7f28ac101a20 2024-03-12T05:11:43.705 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:43.705 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:09:59.961590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:43.705 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f289c0a1a10 msgr2=0x7f289c0a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f289c0a1a10 0x7f289c0a3ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f28a0005fd0 tx=0x7f28a000b410 comp rx=0 tx=0).stop 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 msgr2=0x7f28ac1a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac1a07b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f28a4037040 tx=0x7f28a4034d40 comp rx=0 tx=0).stop 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 shutdown_connections 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f289c0a1a10 0x7f289c0a3ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ac101a20 0x7f28ac1a07b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ac101060 0x7f28ac199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 --2- 172.21.15.100:0/2208621688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28ac0ffe80 0x7f28ac199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.709+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 >> 172.21.15.100:0/2208621688 conn(0x7f28ac0fb460 msgr2=0x7f28ac190020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.710+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 shutdown_connections 2024-03-12T05:11:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.710+0000 7f28b199f640 1 -- 172.21.15.100:0/2208621688 wait complete. 2024-03-12T05:11:43.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- 172.21.15.100:0/2949208011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 msgr2=0x7f0714101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- 172.21.15.100:0/2949208011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f0714101ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0708009900 tx=0x7f070802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- 172.21.15.100:0/2949208011 shutdown_connections 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- 172.21.15.100:0/2949208011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0714100cd0 0x7f0714108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- 172.21.15.100:0/2949208011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0714100310 0x7f0714100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- 172.21.15.100:0/2949208011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f0714101ae0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- 172.21.15.100:0/2949208011 >> 172.21.15.100:0/2949208011 conn(0x7f07140fada0 msgr2=0x7f07140fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- 172.21.15.100:0/2949208011 shutdown_connections 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- 172.21.15.100:0/2949208011 wait complete. 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 Processor -- start 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- start start 2024-03-12T05:11:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0714100310 0x7f07140ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0714100cd0 0x7f071419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f071419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071419ed90 con 0x7f07141016c0 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071419ef00 con 0x7f0714100cd0 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f071ad6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071419f0c0 con 0x7f0714100310 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f0719308640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f071419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f0719308640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f071419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52086/0 (socket says 172.21.15.100:52086) 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.823+0000 7f0719308640 1 -- 172.21.15.100:0/225511841 learned_addr learned my addr 172.21.15.100:0/225511841 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0718b07640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0714100310 0x7f07140ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0719308640 1 -- 172.21.15.100:0/225511841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0714100310 msgr2=0x7f07140ffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0719308640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0714100310 0x7f07140ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0719308640 1 -- 172.21.15.100:0/225511841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0714100cd0 msgr2=0x7f071419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0719308640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0714100cd0 0x7f071419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0719308640 1 -- 172.21.15.100:0/225511841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708009560 con 0x7f07141016c0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0719308640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f071419e600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f070c00b970 tx=0x7f070c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00cb40 con 0x7f07141016c0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f070c00cca0 con 0x7f07141016c0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071419f3a0 con 0x7f07141016c0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0077d0 con 0x7f07141016c0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.824+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071419f930 con 0x7f07141016c0 2024-03-12T05:11:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.825+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0714107450 con 0x7f07141016c0 2024-03-12T05:11:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.826+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f070c014440 con 0x7f07141016c0 2024-03-12T05:11:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.826+0000 7f0711ffb640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f07040a1ab0 0x7f07040a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.827+0000 7f0718b07640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f07040a1ab0 0x7f07040a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.827+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f070c0cccc0 con 0x7f07141016c0 2024-03-12T05:11:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.827+0000 7f0718b07640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f07040a1ab0 0x7f07040a3f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f070802c6a0 tx=0x7f0708005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:43.831+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f070c095ed0 con 0x7f07141016c0 2024-03-12T05:11:44.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.009+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f071402ce40 con 0x7f07141016c0 2024-03-12T05:11:44.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.010+0000 7f0711ffb640 1 -- 172.21.15.100:0/225511841 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7f070c019090 con 0x7f07141016c0 2024-03-12T05:11:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:44.009 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f07040a1ab0 msgr2=0x7f07040a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f07040a1ab0 0x7f07040a3f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f070802c6a0 tx=0x7f0708005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 msgr2=0x7f071419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f071419e600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f070c00b970 tx=0x7f070c00be30 comp rx=0 tx=0).stop 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 shutdown_connections 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f07040a1ab0 0x7f07040a3f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07141016c0 0x7f071419e600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0714100cd0 0x7f071419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 --2- 172.21.15.100:0/225511841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0714100310 0x7f07140ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.012+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 >> 172.21.15.100:0/225511841 conn(0x7f07140fada0 msgr2=0x7f0714104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.013+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 shutdown_connections 2024-03-12T05:11:44.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.013+0000 7f071ad6b640 1 -- 172.21.15.100:0/225511841 wait complete. 2024-03-12T05:11:44.020 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:43 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/99500250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:44.020 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:43 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2224615188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:44.020 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:43 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2208621688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:44.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:44.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 -- 172.21.15.100:0/133577096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0104550 msgr2=0x7f19c0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 --2- 172.21.15.100:0/133577096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0104550 0x7f19c0106940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009900 tx=0x7f19b402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 -- 172.21.15.100:0/133577096 shutdown_connections 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 --2- 172.21.15.100:0/133577096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c0106e80 0x7f19c01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 --2- 172.21.15.100:0/133577096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0104550 0x7f19c0106940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 --2- 172.21.15.100:0/133577096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0101c20 0x7f19c0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 -- 172.21.15.100:0/133577096 >> 172.21.15.100:0/133577096 conn(0x7f19c00fb630 msgr2=0x7f19c00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 -- 172.21.15.100:0/133577096 shutdown_connections 2024-03-12T05:11:44.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.128+0000 7f19c6283640 1 -- 172.21.15.100:0/133577096 wait complete. 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 Processor -- start 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 -- start start 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 0x7f19c01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0104550 0x7f19c01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c0106e80 0x7f19c019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c019ed90 con 0x7f19c0101c20 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c019ef00 con 0x7f19c0106e80 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19c6283640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c019f0c0 con 0x7f19c0104550 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19bffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 0x7f19c01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19bffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 0x7f19c01972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52098/0 (socket says 172.21.15.100:52098) 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.129+0000 7f19bffff640 1 -- 172.21.15.100:0/2469061264 learned_addr learned my addr 172.21.15.100:0/2469061264 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bf7fe640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0104550 0x7f19c01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bffff640 1 -- 172.21.15.100:0/2469061264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0104550 msgr2=0x7f19c01977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bffff640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0104550 0x7f19c01977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bffff640 1 -- 172.21.15.100:0/2469061264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c0106e80 msgr2=0x7f19c019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bffff640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c0106e80 0x7f19c019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bffff640 1 -- 172.21.15.100:0/2469061264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4009560 con 0x7f19c0101c20 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bf7fe640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0104550 0x7f19c01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bffff640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 0x7f19c01972a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c970 tx=0x7f19b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0007bb0 con 0x7f19c0101c20 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c019f3a0 con 0x7f19c0101c20 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b0007d10 con 0x7f19c0101c20 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000f450 con 0x7f19c0101c20 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.130+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c019f8b0 con 0x7f19c0101c20 2024-03-12T05:11:44.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.131+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c00fd120 con 0x7f19c0101c20 2024-03-12T05:11:44.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.132+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f19b0004f70 con 0x7f19c0101c20 2024-03-12T05:11:44.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.132+0000 7f19bd7fa640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f19a80a1a30 0x7f19a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.133+0000 7f19bf7fe640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f19a80a1a30 0x7f19a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.133+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f19b00cd1f0 con 0x7f19c0101c20 2024-03-12T05:11:44.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.133+0000 7f19bf7fe640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f19a80a1a30 0x7f19a80a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f19b4036040 tx=0x7f19b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:44.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.137+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19b0096400 con 0x7f19c0101c20 2024-03-12T05:11:44.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:43 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/99500250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:44.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:43 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2224615188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:44.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:43 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2208621688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:44.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:43 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/99500250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:44.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:43 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2224615188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:44.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:43 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2208621688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:44.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.311+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19c010cbb0 con 0x7f19c0101c20 2024-03-12T05:11:44.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.312+0000 7f19bd7fa640 1 -- 172.21.15.100:0/2469061264 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19b001d020 con 0x7f19c0101c20 2024-03-12T05:11:44.309 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:44.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f19a80a1a30 msgr2=0x7f19a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f19a80a1a30 0x7f19a80a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f19b4036040 tx=0x7f19b4000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 msgr2=0x7f19c01972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 0x7f19c01972a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c970 tx=0x7f19b000ce30 comp rx=0 tx=0).stop 2024-03-12T05:11:44.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 shutdown_connections 2024-03-12T05:11:44.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f19a80a1a30 0x7f19a80a3ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c0106e80 0x7f19c019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19c0104550 0x7f19c01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 --2- 172.21.15.100:0/2469061264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c0101c20 0x7f19c01972a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 >> 172.21.15.100:0/2469061264 conn(0x7f19c00fb630 msgr2=0x7f19c0102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 shutdown_connections 2024-03-12T05:11:44.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.314+0000 7f19c6283640 1 -- 172.21.15.100:0/2469061264 wait complete. 2024-03-12T05:11:44.322 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.430+0000 7f5cd089d640 1 -- 172.21.15.100:0/502901022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101270 msgr2=0x7f5ccc1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.430+0000 7f5cd089d640 1 --2- 172.21.15.100:0/502901022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101270 0x7f5ccc1016f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- 172.21.15.100:0/502901022 shutdown_connections 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 --2- 172.21.15.100:0/502901022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101c30 0x7f5ccc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 --2- 172.21.15.100:0/502901022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101270 0x7f5ccc1016f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 --2- 172.21.15.100:0/502901022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ccc100080 0x7f5ccc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- 172.21.15.100:0/502901022 >> 172.21.15.100:0/502901022 conn(0x7f5ccc0fb610 msgr2=0x7f5ccc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- 172.21.15.100:0/502901022 shutdown_connections 2024-03-12T05:11:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- 172.21.15.100:0/502901022 wait complete. 2024-03-12T05:11:44.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 Processor -- start 2024-03-12T05:11:44.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- start start 2024-03-12T05:11:44.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ccc100080 0x7f5ccc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101270 0x7f5ccc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 0x7f5ccc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc19edb0 con 0x7f5ccc101c30 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc19ef20 con 0x7f5ccc101270 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.431+0000 7f5cd089d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc19f0e0 con 0x7f5ccc100080 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 0x7f5ccc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 0x7f5ccc19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52118/0 (socket says 172.21.15.100:52118) 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 -- 172.21.15.100:0/2013593001 learned_addr learned my addr 172.21.15.100:0/2013593001 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cca59c640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ccc100080 0x7f5ccc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cc9d9b640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101270 0x7f5ccc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 -- 172.21.15.100:0/2013593001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ccc100080 msgr2=0x7f5ccc1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ccc100080 0x7f5ccc1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 -- 172.21.15.100:0/2013593001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101270 msgr2=0x7f5ccc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101270 0x7f5ccc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 -- 172.21.15.100:0/2013593001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc009560 con 0x7f5ccc101c30 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cc9d9b640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101270 0x7f5ccc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:44.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5ccad9d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 0x7f5ccc19e870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400b970 tx=0x7f5cc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400cb40 con 0x7f5ccc101c30 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cc400cca0 con 0x7f5ccc101c30 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40077d0 con 0x7f5ccc101c30 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc19f3c0 con 0x7f5ccc101c30 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.432+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc19f900 con 0x7f5ccc101c30 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.433+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ccc107420 con 0x7f5ccc101c30 2024-03-12T05:11:44.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.434+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5cc4014440 con 0x7f5ccc101c30 2024-03-12T05:11:44.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.435+0000 7f5cbb7fe640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1a30 0x7f5cb40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.435+0000 7f5cca59c640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1a30 0x7f5cb40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.435+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5cc40cccc0 con 0x7f5ccc101c30 2024-03-12T05:11:44.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.435+0000 7f5cca59c640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1a30 0x7f5cb40a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc1010d0 tx=0x7f5cc000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:44.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.439+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cc4095f80 con 0x7f5ccc101c30 2024-03-12T05:11:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.612+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ccc02ce40 con 0x7f5ccc101c30 2024-03-12T05:11:44.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.613+0000 7f5cbb7fe640 1 -- 172.21.15.100:0/2013593001 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cc4019020 con 0x7f5ccc101c30 2024-03-12T05:11:44.610 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1a30 msgr2=0x7f5cb40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1a30 0x7f5cb40a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc1010d0 tx=0x7f5cc000b390 comp rx=0 tx=0).stop 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 msgr2=0x7f5ccc19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 0x7f5ccc19e870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400b970 tx=0x7f5cc400be30 comp rx=0 tx=0).stop 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 shutdown_connections 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f5cb40a1a30 0x7f5cb40a3ef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ccc101c30 0x7f5ccc19e870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ccc101270 0x7f5ccc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 --2- 172.21.15.100:0/2013593001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ccc100080 0x7f5ccc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 >> 172.21.15.100:0/2013593001 conn(0x7f5ccc0fb610 msgr2=0x7f5ccc1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 shutdown_connections 2024-03-12T05:11:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.615+0000 7f5cd089d640 1 -- 172.21.15.100:0/2013593001 wait complete. 2024-03-12T05:11:44.623 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 -- 172.21.15.100:0/993970157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 msgr2=0x7f9b501015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/993970157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b501015d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 -- 172.21.15.100:0/993970157 shutdown_connections 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/993970157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b50102d60 0x7f9b501095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/993970157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b501023a0 0x7f9b50102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/993970157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b501015d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.730+0000 7f9b55cc3640 1 -- 172.21.15.100:0/993970157 >> 172.21.15.100:0/993970157 conn(0x7f9b500fc720 msgr2=0x7f9b500feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 -- 172.21.15.100:0/993970157 shutdown_connections 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 -- 172.21.15.100:0/993970157 wait complete. 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 Processor -- start 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 -- start start 2024-03-12T05:11:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b50197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b501023a0 0x7f9b50197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b50102d60 0x7f9b5019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5019ed30 con 0x7f9b501011b0 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5019eea0 con 0x7f9b50102d60 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b55cc3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5019f060 con 0x7f9b501023a0 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b50197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b50197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52134/0 (socket says 172.21.15.100:52134) 2024-03-12T05:11:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.731+0000 7f9b4f7fe640 1 -- 172.21.15.100:0/378342690 learned_addr learned my addr 172.21.15.100:0/378342690 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4effd640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b501023a0 0x7f9b50197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4f7fe640 1 -- 172.21.15.100:0/378342690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b501023a0 msgr2=0x7f9b50197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4f7fe640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b501023a0 0x7f9b50197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4f7fe640 1 -- 172.21.15.100:0/378342690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b50102d60 msgr2=0x7f9b5019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4ffff640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b50102d60 0x7f9b5019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4f7fe640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b50102d60 0x7f9b5019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4f7fe640 1 -- 172.21.15.100:0/378342690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40009900 con 0x7f9b501011b0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4f7fe640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b50197240 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b40037040 tx=0x7f9b40034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40032eb0 con 0x7f9b501011b0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b40003e50 con 0x7f9b501011b0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40040450 con 0x7f9b501011b0 2024-03-12T05:11:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5019f2e0 con 0x7f9b501011b0 2024-03-12T05:11:44.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.732+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5019f790 con 0x7f9b501011b0 2024-03-12T05:11:44.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.733+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b50108520 con 0x7f9b501011b0 2024-03-12T05:11:44.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.734+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9b4000a040 con 0x7f9b501011b0 2024-03-12T05:11:44.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.734+0000 7f9b4cff9640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b380a1a30 0x7f9b380a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:44.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.735+0000 7f9b4effd640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b380a1a30 0x7f9b380a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:44.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.735+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9b400f02a0 con 0x7f9b501011b0 2024-03-12T05:11:44.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.735+0000 7f9b4effd640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b380a1a30 0x7f9b380a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9b50198750 tx=0x7f9b4400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:44.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.739+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b400b94b0 con 0x7f9b501011b0 2024-03-12T05:11:44.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.913+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b5002ce40 con 0x7f9b501011b0 2024-03-12T05:11:44.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.913+0000 7f9b4cff9640 1 -- 172.21.15.100:0/378342690 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b400bcb70 con 0x7f9b501011b0 2024-03-12T05:11:44.911 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b380a1a30 msgr2=0x7f9b380a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b380a1a30 0x7f9b380a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9b50198750 tx=0x7f9b4400a2d0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 msgr2=0x7f9b50197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b50197240 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b40037040 tx=0x7f9b40034d40 comp rx=0 tx=0).stop 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 shutdown_connections 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f9b380a1a30 0x7f9b380a3ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b50102d60 0x7f9b5019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b501023a0 0x7f9b50197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 --2- 172.21.15.100:0/378342690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b501011b0 0x7f9b50197240 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:44.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.915+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 >> 172.21.15.100:0/378342690 conn(0x7f9b500fc720 msgr2=0x7f9b501056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:44.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.916+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 shutdown_connections 2024-03-12T05:11:44.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:44.916+0000 7f9b55cc3640 1 -- 172.21.15.100:0/378342690 wait complete. 2024-03-12T05:11:44.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:45.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 -- 172.21.15.100:0/1664693991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1180ffbe0 msgr2=0x7fc118109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 --2- 172.21.15.100:0/1664693991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1180ffbe0 0x7fc118109790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc108009900 tx=0x7fc10802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:45.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 -- 172.21.15.100:0/1664693991 shutdown_connections 2024-03-12T05:11:45.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 --2- 172.21.15.100:0/1664693991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1180ffbe0 0x7fc118109790 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 --2- 172.21.15.100:0/1664693991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1180ff220 0x7fc1180ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 --2- 172.21.15.100:0/1664693991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1181027b0 0x7fc118102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 -- 172.21.15.100:0/1664693991 >> 172.21.15.100:0/1664693991 conn(0x7fc1180fadc0 msgr2=0x7fc1180fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 -- 172.21.15.100:0/1664693991 shutdown_connections 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 -- 172.21.15.100:0/1664693991 wait complete. 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.029+0000 7fc11dd48640 1 Processor -- start 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 -- start start 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1180ff220 0x7fc118101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1180ffbe0 0x7fc1181004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 0x7fc118100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118101190 con 0x7fc1181027b0 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118101300 con 0x7fc1180ff220 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181a1250 con 0x7fc1180ffbe0 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 0x7fc118100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 0x7fc118100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52140/0 (socket says 172.21.15.100:52140) 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 -- 172.21.15.100:0/244882289 learned_addr learned my addr 172.21.15.100:0/244882289 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc116ffd640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1180ffbe0 0x7fc1181004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc1177fe640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1180ff220 0x7fc118101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 -- 172.21.15.100:0/244882289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1180ffbe0 msgr2=0x7fc1181004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1180ffbe0 0x7fc1181004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 -- 172.21.15.100:0/244882289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1180ff220 msgr2=0x7fc118101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1180ff220 0x7fc118101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 -- 172.21.15.100:0/244882289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108009560 con 0x7fc1181027b0 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc1177fe640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1180ff220 0x7fc118101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc117fff640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 0x7fc118100a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc108009430 tx=0x7fc1080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1080044e0 con 0x7fc1181027b0 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1080054e0 con 0x7fc1181027b0 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1181a14a0 con 0x7fc1181027b0 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10803e450 con 0x7fc1181027b0 2024-03-12T05:11:45.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.030+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1181a1a00 con 0x7fc1181027b0 2024-03-12T05:11:45.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.031+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc118108610 con 0x7fc1181027b0 2024-03-12T05:11:45.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.032+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc108005000 con 0x7fc1181027b0 2024-03-12T05:11:45.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.033+0000 7fc114ff9640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc1040a1ab0 0x7fc1040a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.033+0000 7fc1177fe640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc1040a1ab0 0x7fc1040a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.033+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc1080efdb0 con 0x7fc1181027b0 2024-03-12T05:11:45.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.034+0000 7fc1177fe640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc1040a1ab0 0x7fc1040a3f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc1180694f0 tx=0x7fc1100094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.037+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1080b8f80 con 0x7fc1181027b0 2024-03-12T05:11:45.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:44 smithi156 ceph-mon[36270]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:45.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/225511841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:45.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2469061264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:45.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:44 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2013593001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:44 smithi161 ceph-mon[36043]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/225511841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2469061264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:45.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:44 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2013593001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:45.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.247+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc11810cc10 con 0x7fc1181027b0 2024-03-12T05:11:45.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.247+0000 7fc114ff9640 1 -- 172.21.15.100:0/244882289 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fc1080bc640 con 0x7fc1181027b0 2024-03-12T05:11:45.245 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:45.245 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:09:59.961590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:45.245 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 22 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc1040a1ab0 msgr2=0x7fc1040a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc1040a1ab0 0x7fc1040a3f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc1180694f0 tx=0x7fc1100094f0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 msgr2=0x7fc118100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 0x7fc118100a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc108009430 tx=0x7fc1080042f0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 shutdown_connections 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc1040a1ab0 0x7fc1040a3f70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1181027b0 0x7fc118100a30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1180ffbe0 0x7fc1181004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 --2- 172.21.15.100:0/244882289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1180ff220 0x7fc118101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 >> 172.21.15.100:0/244882289 conn(0x7fc1180fadc0 msgr2=0x7fc1181057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 shutdown_connections 2024-03-12T05:11:45.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.249+0000 7fc11dd48640 1 -- 172.21.15.100:0/244882289 wait complete. 2024-03-12T05:11:45.257 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:44 smithi100 ceph-mon[32210]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:11:45.257 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/225511841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:45.257 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2469061264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:45.257 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:44 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2013593001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:45.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:45.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.362+0000 7f6785e9d640 1 -- 172.21.15.100:0/3742928138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67801011a0 msgr2=0x7f67801015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.362+0000 7f6785e9d640 1 --2- 172.21.15.100:0/3742928138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67801011a0 0x7f67801015c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6770009900 tx=0x7f677002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:45.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- 172.21.15.100:0/3742928138 shutdown_connections 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 --2- 172.21.15.100:0/3742928138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780102d50 0x7f67801095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 --2- 172.21.15.100:0/3742928138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6780102390 0x7f6780102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 --2- 172.21.15.100:0/3742928138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67801011a0 0x7f67801015c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- 172.21.15.100:0/3742928138 >> 172.21.15.100:0/3742928138 conn(0x7f67800fc730 msgr2=0x7f67800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- 172.21.15.100:0/3742928138 shutdown_connections 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- 172.21.15.100:0/3742928138 wait complete. 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 Processor -- start 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- start start 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67801011a0 0x7f67801971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780102390 0x7f6780197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 0x7f678019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678019ecd0 con 0x7f6780102d50 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678019ee40 con 0x7f67801011a0 2024-03-12T05:11:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f6785e9d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678019f000 con 0x7f6780102390 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.363+0000 7f677ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 0x7f678019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 0x7f678019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52162/0 (socket says 172.21.15.100:52162) 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 -- 172.21.15.100:0/761685069 learned_addr learned my addr 172.21.15.100:0/761685069 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677effd640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780102390 0x7f6780197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677f7fe640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67801011a0 0x7f67801971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 -- 172.21.15.100:0/761685069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780102390 msgr2=0x7f6780197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780102390 0x7f6780197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 -- 172.21.15.100:0/761685069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67801011a0 msgr2=0x7f67801971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67801011a0 0x7f67801971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 -- 172.21.15.100:0/761685069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6770009560 con 0x7f6780102d50 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677f7fe640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67801011a0 0x7f67801971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677ffff640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 0x7f678019e790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6778009fd0 tx=0x7f677800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677800bea0 con 0x7f6780102d50 2024-03-12T05:11:45.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678019f2e0 con 0x7f6780102d50 2024-03-12T05:11:45.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6778009940 con 0x7f6780102d50 2024-03-12T05:11:45.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67780106c0 con 0x7f6780102d50 2024-03-12T05:11:45.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.364+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678019f7a0 con 0x7f6780102d50 2024-03-12T05:11:45.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.365+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67801084c0 con 0x7f6780102d50 2024-03-12T05:11:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.366+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6778004240 con 0x7f6780102d50 2024-03-12T05:11:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.367+0000 7f677cff9640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f67680a1a30 0x7f67680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.367+0000 7f677f7fe640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f67680a1a30 0x7f67680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.367+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67780cd1f0 con 0x7f6780102d50 2024-03-12T05:11:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.367+0000 7f677f7fe640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f67680a1a30 0x7f67680a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6770036040 tx=0x7f6770000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.371+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6778096400 con 0x7f6780102d50 2024-03-12T05:11:45.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.545+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f678002ce70 con 0x7f6780102d50 2024-03-12T05:11:45.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.547+0000 7f677cff9640 1 -- 172.21.15.100:0/761685069 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7f6778017090 con 0x7f6780102d50 2024-03-12T05:11:45.545 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:45.545 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f67680a1a30 msgr2=0x7f67680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f67680a1a30 0x7f67680a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6770036040 tx=0x7f6770000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 msgr2=0x7f678019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 0x7f678019e790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6778009fd0 tx=0x7f677800b260 comp rx=0 tx=0).stop 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 shutdown_connections 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f67680a1a30 0x7f67680a3ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6780102d50 0x7f678019e790 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780102390 0x7f6780197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 --2- 172.21.15.100:0/761685069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67801011a0 0x7f67801971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 >> 172.21.15.100:0/761685069 conn(0x7f67800fc730 msgr2=0x7f6780105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 shutdown_connections 2024-03-12T05:11:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.549+0000 7f6785e9d640 1 -- 172.21.15.100:0/761685069 wait complete. 2024-03-12T05:11:45.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-12T05:09:54.274771+0000', 'modified': '2024-03-12T05:09:59.961590+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24461, 'mds_1': 24451, 'mds_2': 24439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2592644588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2592644588}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2592644588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/692407308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 692407308}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 692407308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/854337146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 854337146}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 854337146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24439': {'gid': 24439, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6837/2463295988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 2463295988}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 2463295988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.161:6839/1712807460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 1712807460}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 1712807460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/1209044969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1209044969}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1209044969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-12T05:11:45.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-12T05:11:45.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 -- 172.21.15.100:0/3445321706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c100080 msgr2=0x7f6e5c1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 --2- 172.21.15.100:0/3445321706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c100080 0x7f6e5c1004a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009900 tx=0x7f6e5002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 -- 172.21.15.100:0/3445321706 shutdown_connections 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 --2- 172.21.15.100:0/3445321706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c101c30 0x7f6e5c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 --2- 172.21.15.100:0/3445321706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e5c101270 0x7f6e5c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 --2- 172.21.15.100:0/3445321706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c100080 0x7f6e5c1004a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 -- 172.21.15.100:0/3445321706 >> 172.21.15.100:0/3445321706 conn(0x7f6e5c0fb610 msgr2=0x7f6e5c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 -- 172.21.15.100:0/3445321706 shutdown_connections 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.663+0000 7f6e618ff640 1 -- 172.21.15.100:0/3445321706 wait complete. 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 Processor -- start 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 -- start start 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c100080 0x7f6e5c068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e5c101270 0x7f6e5c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 0x7f6e5c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c19ee20 con 0x7f6e5c101c30 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c19ef90 con 0x7f6e5c100080 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e618ff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c19f150 con 0x7f6e5c101270 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 0x7f6e5c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 0x7f6e5c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52180/0 (socket says 172.21.15.100:52180) 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1466364040 learned_addr learned my addr 172.21.15.100:0/1466364040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e5c101270 0x7f6e5c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.664+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c100080 0x7f6e5c068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1466364040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e5c101270 msgr2=0x7f6e5c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e5c101270 0x7f6e5c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1466364040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c100080 msgr2=0x7f6e5c068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c100080 0x7f6e5c068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1466364040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009560 con 0x7f6e5c101c30 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c100080 0x7f6e5c068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 0x7f6e5c19e640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009fd0 tx=0x7f6e5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e608fd640 1 -- 172.21.15.100:0/1466364040 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400bea0 con 0x7f6e5c101c30 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e608fd640 1 -- 172.21.15.100:0/1466364040 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e54009940 con 0x7f6e5c101c30 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c19f3d0 con 0x7f6e5c101c30 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e608fd640 1 -- 172.21.15.100:0/1466364040 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540106c0 con 0x7f6e5c101c30 2024-03-12T05:11:45.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.665+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c19f890 con 0x7f6e5c101c30 2024-03-12T05:11:45.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.667+0000 7f6e608fd640 1 -- 172.21.15.100:0/1466364040 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6e54004240 con 0x7f6e5c101c30 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.667+0000 7f6e645aa740 1 Processor -- start 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.667+0000 7f6e608fd640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e440a1a30 0x7f6e440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.667+0000 7f6e645aa740 1 -- start start 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e608fd640 1 -- 172.21.15.100:0/1466364040 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e540cd120 con 0x7f6e5c101c30 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e440a1a30 0x7f6e440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbf4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e645aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dba8cd0 con 0x55778dbf2ae0 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e645aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778db5bf20 con 0x55778dbecf80 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e645aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778db59450 con 0x55778dbed970 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbf4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbf4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52190/0 (socket says 172.21.15.100:52190) 2024-03-12T05:11:45.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1490305032 learned_addr learned my addr 172.21.15.100:0/1490305032 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e440a1a30 0x7f6e440a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c068bc0 tx=0x7f6e4c00ba60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1490305032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 msgr2=0x55778dbf2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1490305032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 msgr2=0x55778dbed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1490305032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55778db09f20 con 0x55778dbf2ae0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbf4f70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6e540a2300 tx=0x7f6e54005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a66d992e312d2e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e41ffb640 1 -- 172.21.15.100:0/1490305032 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540a2830 con 0x55778dbf2ae0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e41ffb640 1 -- 172.21.15.100:0/1490305032 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e54004000 con 0x55778dbf2ae0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.668+0000 7f6e41ffb640 1 -- 172.21.15.100:0/1490305032 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540994e0 con 0x55778dbf2ae0 2024-03-12T05:11:45.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 -- 172.21.15.100:0/1490305032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 msgr2=0x55778dbf4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbf4f70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6e540a2300 tx=0x7f6e54005b90 comp rx=0 tx=0).stop 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 -- 172.21.15.100:0/1490305032 shutdown_connections 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbf4f70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1490305032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 -- 172.21.15.100:0/1490305032 >> 172.21.15.100:0/1490305032 conn(0x55778dbe84f0 msgr2=0x55778dbe88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 -- 172.21.15.100:0/1490305032 shutdown_connections 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 -- 172.21.15.100:0/1490305032 wait complete. 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.669+0000 7f6e645aa740 1 -- start start 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 Processor -- start 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbf7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbfe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778db54c30 con 0x55778dbf2ae0 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778daf9d30 con 0x55778dbecf80 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d828310 con 0x55778dbed970 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 10 client.0 _ll_register_callbacks cb 0x55778db01d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbfe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbfe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52192/0 (socket says 172.21.15.100:52192) 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/156318364 learned_addr learned my addr 172.21.15.100:0/156318364 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5affd640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbf7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/156318364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 msgr2=0x55778dbf7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 msgr2=0x55778dbf7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbf7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/156318364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c19f890 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbfe6b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e54012040 tx=0x7f6e540969a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80707e103108efea server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e3affd640 1 -- 172.21.15.100:0/156318364 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540a2830 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e3affd640 1 -- 172.21.15.100:0/156318364 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e5407b030 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.670+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55778d8546c0 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e3affd640 1 -- 172.21.15.100:0/156318364 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e540996f0 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 10 client.15195 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55778db532b0 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e3affd640 1 -- 172.21.15.100:0/156318364 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55778d8546c0 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e3affd640 1 -- 172.21.15.100:0/156318364 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e5409f100 con 0x55778dbf2ae0 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 10 client.15195 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 10 client.15195 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T05:11:45.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 20 client.15195 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc02f70 0x55778dc07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 4 client.15195 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55778d8a5270 con 0x55778dc02f70 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.671+0000 7f6e5affd640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc02f70 0x55778dc07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e3affd640 10 client.15195 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e5affd640 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc02f70 0x55778dc07a30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bd2693d16aebc0 server_cookie=3003ea1ee1f03e83 in_seq=0 out_seq=0 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e3affd640 1 -- 172.21.15.100:0/156318364 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55778d8a5270 con 0x55778dc02f70 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e3affd640 10 client.15195 handle_command_reply: tid=0 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 client.15195 shutdown 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 msgr2=0x55778dbfe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbfe6b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e54012040 tx=0x7f6e540969a0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 shutdown_connections 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc02f70 0x55778dc07a30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dbfe6b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbf7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e3affd640 0 client.15195 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:45.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 --2- 172.21.15.100:0/156318364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbf7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 >> 172.21.15.100:0/156318364 conn(0x55778dbe84f0 msgr2=0x55778dbe77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 shutdown_connections 2024-03-12T05:11:45.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.672+0000 7f6e645aa740 1 -- 172.21.15.100:0/156318364 wait complete. 2024-03-12T05:11:45.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.673+0000 7f6e645aa740 20 client.15195 trim_cache size 0 max 16384 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 Processor -- start 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 -- start start 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbe6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 0x55778dbfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbf2ae0 0x55778dbeedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dada490 con 0x55778dbed970 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dc03e40 con 0x55778dbecf80 2024-03-12T05:11:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e645aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dbf5100 con 0x55778dbf2ae0 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 0x55778dbfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 0x55778dbfe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52200/0 (socket says 172.21.15.100:52200) 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 -- 172.21.15.100:0/465458437 learned_addr learned my addr 172.21.15.100:0/465458437 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5affd640 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbe6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 -- 172.21.15.100:0/465458437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbf2ae0 msgr2=0x55778dbeedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbf2ae0 0x55778dbeedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 -- 172.21.15.100:0/465458437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 msgr2=0x55778dbe6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbe6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 -- 172.21.15.100:0/465458437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55778daefa20 con 0x55778dbed970 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 0x55778dbfe670 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c003e20 tx=0x7f6e4c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d198647b94e450 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.688+0000 7f6e417fa640 1 -- 172.21.15.100:0/465458437 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c029640 con 0x55778dbed970 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e417fa640 1 -- 172.21.15.100:0/465458437 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c011920 con 0x55778dbed970 2024-03-12T05:11:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e417fa640 1 -- 172.21.15.100:0/465458437 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c01e9b0 con 0x55778dbed970 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 -- 172.21.15.100:0/465458437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 msgr2=0x55778dbfe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 0x55778dbfe670 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c003e20 tx=0x7f6e4c015000 comp rx=0 tx=0).stop 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 -- 172.21.15.100:0/465458437 shutdown_connections 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbf2ae0 0x55778dbeedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbed970 0x55778dbfe670 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 --2- 172.21.15.100:0/465458437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbe6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 -- 172.21.15.100:0/465458437 >> 172.21.15.100:0/465458437 conn(0x55778dbe84f0 msgr2=0x55778dbe88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 -- 172.21.15.100:0/465458437 shutdown_connections 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 -- 172.21.15.100:0/465458437 wait complete. 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.689+0000 7f6e645aa740 1 -- start start 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 Processor -- start 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbfb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbfb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dc596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dbff330 con 0x55778dbf2ae0 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dbf2690 con 0x55778dbecf80 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778dbffa30 con 0x55778dbed970 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 10 client.0 _ll_register_callbacks cb 0x7f6e540963b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dc596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dc596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52210/0 (socket says 172.21.15.100:52210) 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1555099521 learned_addr learned my addr 172.21.15.100:0/1555099521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5a7fc640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbfb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1555099521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 msgr2=0x55778dbfb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbfb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbfb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 msgr2=0x55778dbfb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbfb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 -- 172.21.15.100:0/1555099521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c003ab0 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e5b7fe640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dc596f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e540964f0 tx=0x7f6e54099cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d963835cc4ca83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e3affd640 1 -- 172.21.15.100:0/1555099521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54099e60 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e3affd640 1 -- 172.21.15.100:0/1555099521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e5407b030 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55778dc045e0 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.690+0000 7f6e3affd640 1 -- 172.21.15.100:0/1555099521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e540a2830 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 10 client.15201 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55778db15780 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e3affd640 1 -- 172.21.15.100:0/1555099521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55778dc045e0 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e3affd640 1 -- 172.21.15.100:0/1555099521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e54098f50 con 0x55778dbf2ae0 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 10 client.15201 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 10 client.15201 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 20 client.15201 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc5e660 0x55778dc5ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 4 client.15201 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55778dc03f80 con 0x55778dc5e660 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.691+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc5e660 0x55778dc5ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e3affd640 10 client.15201 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e5affd640 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc5e660 0x55778dc5ab30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58a78a9a5c5c81b2 server_cookie=55c6cb25056c0647 in_seq=0 out_seq=0 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e3affd640 1 -- 172.21.15.100:0/1555099521 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55778dc03f80 con 0x55778dc5e660 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e3affd640 10 client.15201 handle_command_reply: tid=0 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 client.15201 shutdown 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 msgr2=0x55778dc596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dc596f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e540964f0 tx=0x7f6e54099cd0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 shutdown_connections 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55778dc5e660 0x55778dc5ab30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55778dbf2ae0 0x55778dc596f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e3affd640 0 client.15201 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55778dbed970 0x55778dbfb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 --2- 172.21.15.100:0/1555099521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55778dbecf80 0x55778dbfb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 >> 172.21.15.100:0/1555099521 conn(0x55778dbe84f0 msgr2=0x55778dbe6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 shutdown_connections 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 1 -- 172.21.15.100:0/1555099521 wait complete. 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e645aa740 20 client.15201 trim_cache size 0 max 16384 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-03-12T05:11:45.696 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24439, 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 106.74014036200001, 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 22, 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 88, 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 86, 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 127.252523146 2024-03-12T05:11:45.697 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e440a1a30 msgr2=0x7f6e440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e618ff640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e440a1a30 0x7f6e440a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c068bc0 tx=0x7f6e4c00ba60 comp rx=0 tx=0).stop 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 msgr2=0x7f6e5c19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.692+0000 7f6e618ff640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 0x7f6e5c19e640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009fd0 tx=0x7f6e5400b260 comp rx=0 tx=0).stop 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 shutdown_connections 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e440a1a30 0x7f6e440a3ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e5c101c30 0x7f6e5c19e640 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e5c101270 0x7f6e5c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 --2- 172.21.15.100:0/1466364040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c100080 0x7f6e5c068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 >> 172.21.15.100:0/1466364040 conn(0x7f6e5c0fb610 msgr2=0x7f6e5c104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 shutdown_connections 2024-03-12T05:11:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.693+0000 7f6e618ff640 1 -- 172.21.15.100:0/1466364040 wait complete. 2024-03-12T05:11:45.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 -- 172.21.15.100:0/1549675170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641011b0 msgr2=0x7fbd641015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 --2- 172.21.15.100:0/1549675170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641011b0 0x7fbd641015d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009900 tx=0x7fbd5c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 -- 172.21.15.100:0/1549675170 shutdown_connections 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 --2- 172.21.15.100:0/1549675170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd64102d60 0x7fbd641095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 --2- 172.21.15.100:0/1549675170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641023a0 0x7fbd64102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 --2- 172.21.15.100:0/1549675170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641011b0 0x7fbd641015d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 -- 172.21.15.100:0/1549675170 >> 172.21.15.100:0/1549675170 conn(0x7fbd640fc720 msgr2=0x7fbd640feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 -- 172.21.15.100:0/1549675170 shutdown_connections 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.805+0000 7fbd68aae640 1 -- 172.21.15.100:0/1549675170 wait complete. 2024-03-12T05:11:45.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 Processor -- start 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 -- start start 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641011b0 0x7fbd64199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 0x7fbd64199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd64102d60 0x7fbd641a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641a1030 con 0x7fbd641023a0 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641a11a0 con 0x7fbd641011b0 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd68aae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641a1360 con 0x7fbd64102d60 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 0x7fbd64199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 0x7fbd64199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52218/0 (socket says 172.21.15.100:52218) 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd61d9b640 1 -- 172.21.15.100:0/463051459 learned_addr learned my addr 172.21.15.100:0/463051459 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.806+0000 7fbd6259c640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641011b0 0x7fbd64199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd61d9b640 1 -- 172.21.15.100:0/463051459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd64102d60 msgr2=0x7fbd641a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd64102d60 0x7fbd641a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd61d9b640 1 -- 172.21.15.100:0/463051459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641011b0 msgr2=0x7fbd64199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641011b0 0x7fbd64199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd61d9b640 1 -- 172.21.15.100:0/463051459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009560 con 0x7fbd641023a0 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd6259c640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641011b0 0x7fbd64199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 0x7fbd64199a80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800ba50 tx=0x7fbd5800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd537fe640 1 -- 172.21.15.100:0/463051459 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800cbe0 con 0x7fbd641023a0 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd641a1640 con 0x7fbd641023a0 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd537fe640 1 -- 172.21.15.100:0/463051459 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd5800cd40 con 0x7fbd641023a0 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd537fe640 1 -- 172.21.15.100:0/463051459 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580077d0 con 0x7fbd641023a0 2024-03-12T05:11:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.807+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd641a1b50 con 0x7fbd641023a0 2024-03-12T05:11:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.809+0000 7fbd537fe640 1 -- 172.21.15.100:0/463051459 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbd58014440 con 0x7fbd641023a0 2024-03-12T05:11:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.809+0000 7fbd6b322740 1 Processor -- start 2024-03-12T05:11:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 -- start start 2024-03-12T05:11:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c46dcd0 con 0x55a69c4b2970 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c420f20 con 0x55a69c4b7ae0 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd537fe640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd4c0a1a30 0x7fbd4c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6b322740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c41e450 con 0x55a69c4b1f80 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd62d9d640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd4c0a1a30 0x7fbd4c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52234/0 (socket says 172.21.15.100:52234) 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 -- 172.21.15.100:0/2520012533 learned_addr learned my addr 172.21.15.100:0/2520012533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd537fe640 1 -- 172.21.15.100:0/463051459 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd580ccd80 con 0x7fbd641023a0 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd62d9d640 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 -- 172.21.15.100:0/2520012533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 msgr2=0x55a69c4b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 -- 172.21.15.100:0/2520012533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 msgr2=0x55a69c4b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 -- 172.21.15.100:0/2520012533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a69c3cef20 con 0x55a69c4b2970 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd62d9d640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd4c0a1a30 0x7fbd4c0a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd54005d00 tx=0x7fbd54014840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.810+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4b7470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009900 tx=0x7fbd5c02c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079064771352960 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd50ff9640 1 -- 172.21.15.100:0/2520012533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0046c0 con 0x55a69c4b2970 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd50ff9640 1 -- 172.21.15.100:0/2520012533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd5c004510 con 0x55a69c4b2970 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd50ff9640 1 -- 172.21.15.100:0/2520012533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0054a0 con 0x55a69c4b2970 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 -- 172.21.15.100:0/2520012533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 msgr2=0x55a69c4b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4b7470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009900 tx=0x7fbd5c02c960 comp rx=0 tx=0).stop 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 -- 172.21.15.100:0/2520012533 shutdown_connections 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4b7470 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2520012533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 -- 172.21.15.100:0/2520012533 >> 172.21.15.100:0/2520012533 conn(0x55a69c4ad4f0 msgr2=0x55a69c4ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 -- 172.21.15.100:0/2520012533 shutdown_connections 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.811+0000 7fbd6b322740 1 -- 172.21.15.100:0/2520012533 wait complete. 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 -- start start 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 Processor -- start 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c419c30 con 0x55a69c4b2970 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c3bed30 con 0x55a69c4b7ae0 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c0ed310 con 0x55a69c4b1f80 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6b322740 10 client.0 _ll_register_callbacks cb 0x55a69c3c6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4bc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52250/0 (socket says 172.21.15.100:52250) 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 -- 172.21.15.100:0/1430590986 learned_addr learned my addr 172.21.15.100:0/1430590986 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd62d9d640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6259c640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 -- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 msgr2=0x55a69c4bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4bc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 -- 172.21.15.100:0/1430590986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 msgr2=0x55a69c4c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd61d9b640 1 -- 172.21.15.100:0/1430590986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd64109ef0 con 0x55a69c4b2970 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd62d9d640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.812+0000 7fbd6259c640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4bc5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000f80 tx=0x7fbd5809a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1222890a6a1c50ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd49ffb640 1 -- 172.21.15.100:0/1430590986 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580b1070 con 0x55a69c4b2970 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd49ffb640 1 -- 172.21.15.100:0/1430590986 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd5809de40 con 0x55a69c4b2970 2024-03-12T05:11:45.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a69c1196c0 con 0x55a69c4b2970 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd49ffb640 1 -- 172.21.15.100:0/1430590986 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd5809f0e0 con 0x55a69c4b2970 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 10 client.15213 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a69c4182b0 con 0x55a69c4b2970 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd49ffb640 1 -- 172.21.15.100:0/1430590986 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd5807c080 con 0x55a69c4b2970 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd49ffb640 1 -- 172.21.15.100:0/1430590986 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbd58099f50 con 0x55a69c4b2970 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 10 client.15213 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 20 client.15213 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c4c8a40 0x55a69c4c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 4 client.15213 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.813+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a69c16a270 con 0x55a69c4c8a40 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.814+0000 7fbd6259c640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c4c8a40 0x55a69c4c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.814+0000 7fbd6259c640 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c4c8a40 0x55a69c4c7d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc2190088e6aca1f server_cookie=58c417d180e078b5 in_seq=0 out_seq=0 2024-03-12T05:11:45.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.814+0000 7fbd49ffb640 10 client.15213 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd49ffb640 1 -- 172.21.15.100:0/1430590986 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a69c16a270 con 0x55a69c4c8a40 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd49ffb640 10 client.15213 handle_command_reply: tid=0 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 client.15213 shutdown 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 msgr2=0x55a69c4bc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4bc5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000f80 tx=0x7fbd5809a3f0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 shutdown_connections 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c4c8a40 0x55a69c4c7d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4bc5f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd49ffb640 0 client.15213 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 --2- 172.21.15.100:0/1430590986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b1f80 0x55a69c4bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 >> 172.21.15.100:0/1430590986 conn(0x55a69c4ad4f0 msgr2=0x55a69c4ac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 shutdown_connections 2024-03-12T05:11:45.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 1 -- 172.21.15.100:0/1430590986 wait complete. 2024-03-12T05:11:45.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.815+0000 7fbd6b322740 20 client.15213 trim_cache size 0 max 16384 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 Processor -- start 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 -- start start 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 0x55a69c4d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b2970 0x55a69c4c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c39f490 con 0x55a69c4b1f80 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c4c8610 con 0x55a69c4b2970 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.830+0000 7fbd6b322740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c4caa20 con 0x55a69c4b7ae0 2024-03-12T05:11:45.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 0x55a69c4d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 0x55a69c4d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52258/0 (socket says 172.21.15.100:52258) 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 -- 172.21.15.100:0/2373812038 learned_addr learned my addr 172.21.15.100:0/2373812038 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 -- 172.21.15.100:0/2373812038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 msgr2=0x55a69c4b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 -- 172.21.15.100:0/2373812038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b2970 msgr2=0x55a69c4c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b2970 0x55a69c4c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 -- 172.21.15.100:0/2373812038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a69c3b4a20 con 0x55a69c4b1f80 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 0x55a69c4d0ee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c000c00 tx=0x7fbd5c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765f9965a3230981 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd4bfff640 1 -- 172.21.15.100:0/2373812038 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c02cad0 con 0x55a69c4b1f80 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd4bfff640 1 -- 172.21.15.100:0/2373812038 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd5c004f10 con 0x55a69c4b1f80 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.831+0000 7fbd4bfff640 1 -- 172.21.15.100:0/2373812038 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c04fd90 con 0x55a69c4b1f80 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- 172.21.15.100:0/2373812038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 msgr2=0x55a69c4d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 0x55a69c4d0ee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c000c00 tx=0x7fbd5c040040 comp rx=0 tx=0).stop 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- 172.21.15.100:0/2373812038 shutdown_connections 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 0x55a69c4b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b2970 0x55a69c4c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2373812038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b1f80 0x55a69c4d0ee0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- 172.21.15.100:0/2373812038 >> 172.21.15.100:0/2373812038 conn(0x55a69c4ad4f0 msgr2=0x55a69c4ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- 172.21.15.100:0/2373812038 shutdown_connections 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- 172.21.15.100:0/2373812038 wait complete. 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- start start 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 Processor -- start 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b1f80 0x55a69c4c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c4c2250 con 0x55a69c4b2970 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c4c23c0 con 0x55a69c4b1f80 2024-03-12T05:11:45.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69c4c2580 con 0x55a69c4b7ae0 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd6b322740 10 client.0 _ll_register_callbacks cb 0x7fbd5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4c0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52268/0 (socket says 172.21.15.100:52268) 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 -- 172.21.15.100:0/2775038103 learned_addr learned my addr 172.21.15.100:0/2775038103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 -- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 msgr2=0x55a69c4c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 -- 172.21.15.100:0/2775038103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b1f80 msgr2=0x55a69c4c0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.832+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b1f80 0x55a69c4c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd61d9b640 1 -- 172.21.15.100:0/2775038103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a69c3cef20 con 0x55a69c4b2970 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd61d9b640 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4c0d70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd58096670 tx=0x7fbd58099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e8f52dc2ce71db server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd49ffb640 1 -- 172.21.15.100:0/2775038103 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580b1070 con 0x55a69c4b2970 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd49ffb640 1 -- 172.21.15.100:0/2775038103 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd5809de40 con 0x55a69c4b2970 2024-03-12T05:11:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a69c4c8750 con 0x55a69c4b2970 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd49ffb640 1 -- 172.21.15.100:0/2775038103 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd58095f90 con 0x55a69c4b2970 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 10 client.15219 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a69c3da780 con 0x55a69c4b2970 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd49ffb640 1 -- 172.21.15.100:0/2775038103 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd580a6020 con 0x55a69c4b2970 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd49ffb640 1 -- 172.21.15.100:0/2775038103 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbd580146f0 con 0x55a69c4b2970 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 10 client.15219 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 20 client.15219 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c523f30 0x55a69c520650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 4 client.15219 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.833+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fbd5c004510 con 0x55a69c523f30 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c523f30 0x55a69c520650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd49ffb640 10 client.15219 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6259c640 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c523f30 0x55a69c520650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94e478b873d3b31a server_cookie=d97d3e57f51e7d04 in_seq=0 out_seq=0 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd49ffb640 1 -- 172.21.15.100:0/2775038103 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7fbd5c004510 con 0x55a69c523f30 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd49ffb640 10 client.15219 handle_command_reply: tid=0 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 client.15219 shutdown 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 msgr2=0x55a69c4c0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4c0d70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd58096670 tx=0x7fbd58099d60 comp rx=0 tx=0).stop 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 shutdown_connections 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a69c4b7ae0 0x55a69c4c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69c4b2970 0x55a69c4c0d70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a69c523f30 0x55a69c520650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 --2- 172.21.15.100:0/2775038103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a69c4b1f80 0x55a69c4c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 >> 172.21.15.100:0/2775038103 conn(0x55a69c4ad4f0 msgr2=0x55a69c4d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd49ffb640 0 client.15219 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 shutdown_connections 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 1 -- 172.21.15.100:0/2775038103 wait complete. 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.834+0000 7fbd6b322740 20 client.15219 trim_cache size 0 max 16384 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24451, 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-03-12T05:11:45.839 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 107.892220885, 2024-03-12T05:11:45.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 22, 2024-03-12T05:11:45.840 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 88, 2024-03-12T05:11:45.840 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 86, 2024-03-12T05:11:45.840 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 124.611614664 2024-03-12T05:11:45.840 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd4c0a1a30 msgr2=0x7fbd4c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd4c0a1a30 0x7fbd4c0a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd54005d00 tx=0x7fbd54014840 comp rx=0 tx=0).stop 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 msgr2=0x7fbd64199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 0x7fbd64199a80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800ba50 tx=0x7fbd5800bf10 comp rx=0 tx=0).stop 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 shutdown_connections 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fbd4c0a1a30 0x7fbd4c0a3ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd64102d60 0x7fbd641a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd641023a0 0x7fbd64199a80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 --2- 172.21.15.100:0/463051459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641011b0 0x7fbd64199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 >> 172.21.15.100:0/463051459 conn(0x7fbd640fc720 msgr2=0x7fbd640733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 shutdown_connections 2024-03-12T05:11:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.835+0000 7fbd68aae640 1 -- 172.21.15.100:0/463051459 wait complete. 2024-03-12T05:11:45.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-12T05:11:45.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 -- 172.21.15.100:0/489674138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 msgr2=0x7f7e341004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 --2- 172.21.15.100:0/489674138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e341004a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009900 tx=0x7f7e3002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:45.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 -- 172.21.15.100:0/489674138 shutdown_connections 2024-03-12T05:11:45.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 --2- 172.21.15.100:0/489674138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e34101c30 0x7f7e341084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 --2- 172.21.15.100:0/489674138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e34101270 0x7f7e341016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 --2- 172.21.15.100:0/489674138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e341004a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 -- 172.21.15.100:0/489674138 >> 172.21.15.100:0/489674138 conn(0x7f7e340fb610 msgr2=0x7f7e340fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 -- 172.21.15.100:0/489674138 shutdown_connections 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 -- 172.21.15.100:0/489674138 wait complete. 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 Processor -- start 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.949+0000 7f7e3c283640 1 -- start start 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e34197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e34101270 0x7f7e341977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e34101c30 0x7f7e3419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3419ed70 con 0x7f7e34100080 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3419eee0 con 0x7f7e34101270 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3419f0a0 con 0x7f7e34101c30 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e34197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e34197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52280/0 (socket says 172.21.15.100:52280) 2024-03-12T05:11:45.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 -- 172.21.15.100:0/3883662395 learned_addr learned my addr 172.21.15.100:0/3883662395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3981e640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e34101270 0x7f7e341977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 -- 172.21.15.100:0/3883662395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e34101c30 msgr2=0x7f7e3419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e34101c30 0x7f7e3419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 -- 172.21.15.100:0/3883662395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e34101270 msgr2=0x7f7e341977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e34101270 0x7f7e341977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 -- 172.21.15.100:0/3883662395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30009560 con 0x7f7e34100080 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e34197280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7e30000c00 tx=0x7f7e300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e2b7fe640 1 -- 172.21.15.100:0/3883662395 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e300044e0 con 0x7f7e34100080 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e2b7fe640 1 -- 172.21.15.100:0/3883662395 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e300054e0 con 0x7f7e34100080 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3419f320 con 0x7f7e34100080 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e2b7fe640 1 -- 172.21.15.100:0/3883662395 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3003e450 con 0x7f7e34100080 2024-03-12T05:11:45.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.950+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3419f830 con 0x7f7e34100080 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.952+0000 7f7e2b7fe640 1 -- 172.21.15.100:0/3883662395 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7e300035f0 con 0x7f7e34100080 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 Processor -- start 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 -- start start 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e2b7fe640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e200a1a30 0x7f7e200a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af70ff80 0x5615af7103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 0x5615af715470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af715ae0 0x5615af717f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af6cbcd0 con 0x5615af710970 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af67ef20 con 0x5615af70ff80 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af67c450 con 0x5615af715ae0 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3981e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 0x5615af715470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3a820640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e200a1a30 0x7f7e200a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3981e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 0x5615af715470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52292/0 (socket says 172.21.15.100:52292) 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3981e640 1 -- 172.21.15.100:0/404498778 learned_addr learned my addr 172.21.15.100:0/404498778 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e2b7fe640 1 -- 172.21.15.100:0/3883662395 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7e300efde0 con 0x7f7e34100080 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.953+0000 7f7e3a820640 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af715ae0 0x5615af717f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af70ff80 0x5615af7103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3981e640 1 -- 172.21.15.100:0/404498778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af715ae0 msgr2=0x5615af717f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3981e640 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af715ae0 0x5615af717f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3981e640 1 -- 172.21.15.100:0/404498778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af70ff80 msgr2=0x5615af7103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3981e640 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af70ff80 0x5615af7103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3981e640 1 -- 172.21.15.100:0/404498778 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615af62cf20 con 0x5615af710970 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3a820640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e200a1a30 0x7f7e200a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c005fd0 tx=0x7f7e2c009c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e3981e640 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 0x5615af715470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400cbc0 tx=0x7f7e2400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2be94af1130424e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e28bfe640 1 -- 172.21.15.100:0/404498778 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24010630 con 0x5615af710970 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e28bfe640 1 -- 172.21.15.100:0/404498778 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2400eed0 con 0x5615af710970 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.954+0000 7f7e28bfe640 1 -- 172.21.15.100:0/404498778 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2401ab60 con 0x5615af710970 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/404498778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 msgr2=0x5615af715470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 0x5615af715470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400cbc0 tx=0x7f7e2400e720 comp rx=0 tx=0).stop 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/404498778 shutdown_connections 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af715ae0 0x5615af717f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af710970 0x5615af715470 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/404498778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af70ff80 0x5615af7103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/404498778 >> 172.21.15.100:0/404498778 conn(0x5615af70b4f0 msgr2=0x5615af70b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/404498778 shutdown_connections 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/404498778 wait complete. 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- start start 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 Processor -- start 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af7216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af677c30 con 0x5615af715ae0 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af61cd30 con 0x5615af710970 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.955+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af34b310 con 0x5615af70ff80 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3ee4b740 10 client.0 _ll_register_callbacks cb 0x5615af624d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af7216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af7216f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52308/0 (socket says 172.21.15.100:52308) 2024-03-12T05:11:45.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 -- 172.21.15.100:0/3925218178 learned_addr learned my addr 172.21.15.100:0/3925218178 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3981e640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 -- 172.21.15.100:0/3925218178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 msgr2=0x5615af71a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 -- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 msgr2=0x5615af71a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 -- 172.21.15.100:0/3925218178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2400c7f0 con 0x5615af715ae0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3a820640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af7216f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c003e70 tx=0x7f7e2c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d30f6253e1d1968 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e1dffb640 1 -- 172.21.15.100:0/3925218178 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c01a070 con 0x5615af715ae0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e1dffb640 1 -- 172.21.15.100:0/3925218178 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7e2c009960 con 0x5615af715ae0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615af3776c0 con 0x5615af715ae0 2024-03-12T05:11:45.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e1dffb640 1 -- 172.21.15.100:0/3925218178 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e2c003ea0 con 0x5615af715ae0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3ee4b740 10 client.15231 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5615af6762b0 con 0x5615af715ae0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.956+0000 7f7e1dffb640 1 -- 172.21.15.100:0/3925218178 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5615af3776c0 con 0x5615af715ae0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e1dffb640 1 -- 172.21.15.100:0/3925218178 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e2c026020 con 0x5615af715ae0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3ee4b740 10 client.15231 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3ee4b740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3ee4b740 20 client.15231 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af726bf0 0x5615af725e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3ee4b740 4 client.15231 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615af3c8270 con 0x5615af726bf0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af726bf0 0x5615af725e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e1dffb640 10 client.15231 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.957+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af726bf0 0x5615af725e10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97a58f3d8e56e7 server_cookie=f851b41196c755c in_seq=0 out_seq=0 2024-03-12T05:11:45.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e1dffb640 1 -- 172.21.15.100:0/3925218178 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615af3c8270 con 0x5615af726bf0 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e1dffb640 10 client.15231 handle_command_reply: tid=0 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 client.15231 shutdown 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 msgr2=0x5615af7216f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af7216f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c003e70 tx=0x7f7e2c005bd0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 shutdown_connections 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af726bf0 0x5615af725e10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af7216f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e1dffb640 0 client.15231 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/3925218178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 >> 172.21.15.100:0/3925218178 conn(0x5615af70b4f0 msgr2=0x5615af70a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.958+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 shutdown_connections 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.959+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/3925218178 wait complete. 2024-03-12T05:11:45.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.959+0000 7f7e3ee4b740 20 client.15231 trim_cache size 0 max 16384 2024-03-12T05:11:45.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 Processor -- start 2024-03-12T05:11:45.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 -- start start 2024-03-12T05:11:45.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af70b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af7216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af711e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af5fd490 con 0x5615af715ae0 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af3c8270 con 0x5615af710970 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af3776c0 con 0x5615af70ff80 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3a820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af711e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3a820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af711e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52324/0 (socket says 172.21.15.100:52324) 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.974+0000 7f7e3a820640 1 -- 172.21.15.100:0/2160856932 learned_addr learned my addr 172.21.15.100:0/2160856932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3a820640 1 -- 172.21.15.100:0/2160856932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 msgr2=0x5615af70b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3a820640 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af70b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3a820640 1 -- 172.21.15.100:0/2160856932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 msgr2=0x5615af7216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3a820640 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af7216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3a820640 1 -- 172.21.15.100:0/2160856932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615af612a20 con 0x5615af715ae0 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3a820640 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af711e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c028960 tx=0x7f7e2c003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a13ce644caf38 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e1ffff640 1 -- 172.21.15.100:0/2160856932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c003c80 con 0x5615af715ae0 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e1ffff640 1 -- 172.21.15.100:0/2160856932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2c028bd0 con 0x5615af715ae0 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e1ffff640 1 -- 172.21.15.100:0/2160856932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c03c460 con 0x5615af715ae0 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/2160856932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 msgr2=0x5615af711e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af711e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c028960 tx=0x7f7e2c003f20 comp rx=0 tx=0).stop 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/2160856932 shutdown_connections 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af711e30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af7216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/2160856932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af70b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.975+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/2160856932 >> 172.21.15.100:0/2160856932 conn(0x5615af70b4f0 msgr2=0x5615af718cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/2160856932 shutdown_connections 2024-03-12T05:11:45.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/2160856932 wait complete. 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 -- start start 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 Processor -- start 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af77c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af71ee50 con 0x5615af715ae0 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af71efc0 con 0x5615af710970 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615af77cd30 con 0x5615af70ff80 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3ee4b740 10 client.0 _ll_register_callbacks cb 0x7f7e2400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af77c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af77c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52334/0 (socket says 172.21.15.100:52334) 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 -- 172.21.15.100:0/4097339064 learned_addr learned my addr 172.21.15.100:0/4097339064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3981e640 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 -- 172.21.15.100:0/4097339064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 msgr2=0x5615af71e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 -- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 msgr2=0x5615af71e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 -- 172.21.15.100:0/4097339064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615af6762b0 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.976+0000 7f7e3a820640 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af77c7f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0094c0 tx=0x7f7e2c028ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1eb687824b7c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e1dffb640 1 -- 172.21.15.100:0/4097339064 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c01a070 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e1dffb640 1 -- 172.21.15.100:0/4097339064 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7e2c028400 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615af726900 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e1dffb640 1 -- 172.21.15.100:0/4097339064 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e2c0139b0 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 10 client.15237 fetch_fsmap learned FSMap version 22 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5615af638780 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e1dffb640 1 -- 172.21.15.100:0/4097339064 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5615af726900 con 0x5615af715ae0 2024-03-12T05:11:45.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e1dffb640 1 -- 172.21.15.100:0/4097339064 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e2c026050 con 0x5615af715ae0 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 10 client.15237 fetch_fsmap finished waiting for FSMap version 22 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 20 client.15237 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af781b60 0x5615af77e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 4 client.15237 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f7e2400eed0 con 0x5615af781b60 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.977+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af781b60 0x5615af77e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e1dffb640 10 client.15237 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3a01f640 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af781b60 0x5615af77e020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e00e9c7c3f99ef server_cookie=edb0b631a253c4c1 in_seq=0 out_seq=0 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e1dffb640 1 -- 172.21.15.100:0/4097339064 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f7e2400eed0 con 0x5615af781b60 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e1dffb640 10 client.15237 handle_command_reply: tid=0 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 client.15237 shutdown 2024-03-12T05:11:45.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 msgr2=0x5615af77c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af77c7f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0094c0 tx=0x7f7e2c028ea0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 shutdown_connections 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615af781b60 0x5615af77e020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615af715ae0 0x5615af77c7f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e1dffb640 0 client.15237 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615af710970 0x5615af71e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 --2- 172.21.15.100:0/4097339064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615af70ff80 0x5615af71e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 >> 172.21.15.100:0/4097339064 conn(0x5615af70b4f0 msgr2=0x5615af70b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 shutdown_connections 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 1 -- 172.21.15.100:0/4097339064 wait complete. 2024-03-12T05:11:45.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.978+0000 7f7e3ee4b740 20 client.15237 trim_cache size 0 max 16384 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a1680ae-e02d-11ee-95c8-87774f69a715", 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24461, 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 111.051944718, 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 22, 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 88, 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 86, 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 116.284834426 2024-03-12T05:11:45.983 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:45.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e200a1a30 msgr2=0x7f7e200a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e200a1a30 0x7f7e200a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c005fd0 tx=0x7f7e2c009c30 comp rx=0 tx=0).stop 2024-03-12T05:11:45.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 msgr2=0x7f7e34197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:45.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e34197280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7e30000c00 tx=0x7f7e300042f0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 shutdown_connections 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e200a1a30 0x7f7e200a3ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e34101c30 0x7f7e3419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e34101270 0x7f7e341977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 --2- 172.21.15.100:0/3883662395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e34100080 0x7f7e34197280 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 >> 172.21.15.100:0/3883662395 conn(0x7f7e340fb610 msgr2=0x7f7e340682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 shutdown_connections 2024-03-12T05:11:45.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:45.979+0000 7f7e3c283640 1 -- 172.21.15.100:0/3883662395 wait complete. 2024-03-12T05:11:45.987 INFO:teuthology.run_tasks:Running task kclient... 2024-03-12T05:11:46.012 INFO:tasks.kclient:Mounting kernel clients... 2024-03-12T05:11:46.012 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-12T05:11:46.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:46.013 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-03-12T05:11:46.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:46.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-03-12T05:11:46.093 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/378342690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:46.093 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/244882289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:46.093 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/761685069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:46.094 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:45 smithi100 ceph-mon[32210]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:46.097 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-03-12T05:11:46.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:11:46.099 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-12T05:11:46.099 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T05:11:46.099 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T05:11:46.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-12T05:11:46.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/378342690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:46.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/244882289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:46.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/761685069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:46.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:45 smithi156 ceph-mon[36270]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:46.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/378342690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:46.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/244882289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:46.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/761685069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:46.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:45 smithi161 ceph-mon[36043]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:46.376 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:11:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 -- 172.21.15.100:0/2679661413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500100ba0 msgr2=0x7f8500101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 --2- 172.21.15.100:0/2679661413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500100ba0 0x7f8500101020 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009990 tx=0x7f84f0009ca0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 -- 172.21.15.100:0/2679661413 shutdown_connections 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 --2- 172.21.15.100:0/2679661413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500106bd0 0x7f8500106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 --2- 172.21.15.100:0/2679661413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500101560 0x7f850010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 --2- 172.21.15.100:0/2679661413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500100ba0 0x7f8500101020 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 -- 172.21.15.100:0/2679661413 >> 172.21.15.100:0/2679661413 conn(0x7f8500076bc0 msgr2=0x7f8500076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 -- 172.21.15.100:0/2679661413 shutdown_connections 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 -- 172.21.15.100:0/2679661413 wait complete. 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 Processor -- start 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.191+0000 7f850764f700 1 -- start start 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f850764f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500100ba0 0x7f8500199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f850764f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 0x7f8500199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f850764f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500106bd0 0x7f85001a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f850764f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500199f10 con 0x7f8500101560 2024-03-12T05:11:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f850764f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001a46f0 con 0x7f8500100ba0 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f850764f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001a4830 con 0x7f8500106bd0 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 0x7f8500199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 0x7f8500199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52362/0 (socket says 172.21.15.100:52362) 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8505bec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500106bd0 0x7f85001a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 -- 172.21.15.100:0/4029382545 learned_addr learned my addr 172.21.15.100:0/4029382545 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f85053eb700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500100ba0 0x7f8500199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 -- 172.21.15.100:0/4029382545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500106bd0 msgr2=0x7f85001a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500106bd0 0x7f85001a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 -- 172.21.15.100:0/4029382545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500100ba0 msgr2=0x7f8500199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500100ba0 0x7f8500199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 -- 172.21.15.100:0/4029382545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00095f0 con 0x7f8500101560 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f85053eb700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500100ba0 0x7f8500199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f8504bea700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 0x7f8500199670 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f84f0000c00 tx=0x7f84f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:48.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.192+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f001d070 con 0x7f8500101560 2024-03-12T05:11:48.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.193+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84f0003680 con 0x7f8500101560 2024-03-12T05:11:48.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.193+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00173f0 con 0x7f8500101560 2024-03-12T05:11:48.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.193+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85001a4a80 con 0x7f8500101560 2024-03-12T05:11:48.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.193+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85001a4f70 con 0x7f8500101560 2024-03-12T05:11:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.194+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8500109060 con 0x7f8500101560 2024-03-12T05:11:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.195+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f84f0022720 con 0x7f8500101560 2024-03-12T05:11:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.195+0000 7f84f67fc700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f84ec0a1b40 0x7f84ec0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.195+0000 7f85053eb700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f84ec0a1b40 0x7f84ec0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.195+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84f00ce4c0 con 0x7f8500101560 2024-03-12T05:11:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.196+0000 7f85053eb700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f84ec0a1b40 0x7f84ec0a4000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f84fc005fd0 tx=0x7f84fc005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:48.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.200+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84f0097760 con 0x7f8500101560 2024-03-12T05:11:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.471+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f850006a7a0 con 0x7f8500101560 2024-03-12T05:11:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.471+0000 7f84f67fc700 1 -- 172.21.15.100:0/4029382545 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84f009ae20 con 0x7f8500101560 2024-03-12T05:11:48.469 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-12T05:11:48.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.473+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f84ec0a1b40 msgr2=0x7f84ec0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:48.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.473+0000 7f850764f700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f84ec0a1b40 0x7f84ec0a4000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f84fc005fd0 tx=0x7f84fc005940 comp rx=0 tx=0).stop 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.473+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 msgr2=0x7f8500199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 0x7f8500199670 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f84f0000c00 tx=0x7f84f000f690 comp rx=0 tx=0).stop 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 shutdown_connections 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f84ec0a1b40 0x7f84ec0a4000 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8500100ba0 0x7f8500199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500106bd0 0x7f85001a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 --2- 172.21.15.100:0/4029382545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8500101560 0x7f8500199670 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 >> 172.21.15.100:0/4029382545 conn(0x7f8500076bc0 msgr2=0x7f85000ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 shutdown_connections 2024-03-12T05:11:48.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:48.474+0000 7f850764f700 1 -- 172.21.15.100:0/4029382545 wait complete. 2024-03-12T05:11:49.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:48 smithi100 ceph-mon[32210]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:49.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:48 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4029382545' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T05:11:49.173 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T05:11:49.174 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-12T05:11:49.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.174 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-03-12T05:11:49.189 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T05:11:49.189 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-03-12T05:11:49.190 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T05:11:49.191 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-03-12T05:11:49.191 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-12T05:11:49.191 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-03-12T05:11:49.191 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-03-12T05:11:49.191 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-03-12T05:11:49.191 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-12T05:11:49.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.191 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-03-12T05:11:49.192 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-12T05:11:49.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-03-12T05:11:49.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-03-12T05:11:49.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-03-12T05:11:49.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-12T05:11:49.192 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-12T05:11:49.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:48 smithi156 ceph-mon[36270]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:49.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:48 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4029382545' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-12T05:11:49.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:48 smithi161 ceph-mon[36043]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-12T05:11:49.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:48 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4029382545' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-12T05:11:49.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.368 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T05:11:49.394 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-03-12T05:11:49.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.395 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-03-12T05:11:49.468 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-03-12T05:11:49.468 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T05:11:49.468 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-12T05:11:49.468 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-12T05:11:49.468 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T05:11:49.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.469 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-03-12T05:11:49.469 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-12T05:11:49.469 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-12T05:11:49.469 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-12T05:11:49.469 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-12T05:11:49.469 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-12T05:11:49.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.733 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-03-12T05:11:49.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.748 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-03-12T05:11:49.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.803 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-03-12T05:11:49.803 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-12T05:11:49.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T05:11:49.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-12T05:11:49.804 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-12T05:11:49.886 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-12T05:11:49.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:49.887 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-03-12T05:11:49.887 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-12T05:11:49.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-12T05:11:49.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-12T05:11:49.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-12T05:11:49.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-12T05:11:49.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-12T05:11:49.888 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-12T05:11:50.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.032 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-03-12T05:11:50.032 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-12T05:11:50.033 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-03-12T05:11:50.033 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-03-12T05:11:50.033 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-12T05:11:50.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T05:11:50.074 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-12T05:11:50.129 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-12T05:11:50.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T05:11:50.130 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-12T05:11:50.183 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-12T05:11:50.183 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-12T05:11:50.183 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/b1f3ef68-e29d-48c5-a957-f42b5b34b829 2024-03-12T05:11:50.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.184 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/b1f3ef68-e29d-48c5-a957-f42b5b34b829 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-03-12T05:11:50.268 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-03-12T05:11:50.268 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-03-12T05:11:50.268 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-03-12T05:11:50.385 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-03-12T05:11:50.385 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-03-12T05:11:50.386 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-03-12T05:11:50.386 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.156:6789,172.21.15.161:6789" 2024-03-12T05:11:50.386 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.156:6789,172.21.15.161:6789:/volumes/qa/sv_1/b1f3ef68-e29d-48c5-a957-f42b5b34b829 2024-03-12T05:11:50.386 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=8a1680ae-e02d-11ee-95c8-87774f69a715" will pass to kernel 2024-03-12T05:11:50.387 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-12T05:11:50.387 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-03-12T05:11:50.387 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-12T05:11:50.387 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-03-12T05:11:50.387 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-12T05:11:50.388 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-12T05:11:50.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.389 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-12T05:11:50.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.417 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T05:11:50.474 INFO:teuthology.orchestra.run.smithi100.stdout:client15246 2024-03-12T05:11:50.474 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:11:50.475 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T05:11:50.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.532 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T05:11:50.588 INFO:teuthology.orchestra.run.smithi100.stdout:client15246 2024-03-12T05:11:50.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.589 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T05:11:50.645 INFO:teuthology.orchestra.run.smithi100.stdout:client15246 2024-03-12T05:11:50.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:50.646 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715.client15246/status') 2024-03-12T05:11:50.716 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.15246 (3)192.168.0.1:0/451921804 2024-03-12T05:11:50.716 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-03-12T05:11:50.716 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-03-12T05:11:50.717 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-03-12T05:11:50.717 INFO:teuthology.orchestra.run.smithi100.stderr:69 bytes copied, 2.1989e-05 s, 3.1 MB/s 2024-03-12T05:11:50.718 INFO:teuthology.run_tasks:Running task exec... 2024-03-12T05:11:50.730 INFO:teuthology.task.exec:Executing custom commands... 2024-03-12T05:11:50.731 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:11:50.731 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-12T05:11:50.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3451816347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b280ff490 msgr2=0x7f0b280ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:50.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3451816347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b280ff490 0x7f0b280ff8b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009ca0 tx=0x7f0b2402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:50.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3451816347 shutdown_connections 2024-03-12T05:11:50.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3451816347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b28102e20 0x7f0b281096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:50.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3451816347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102460 0x7f0b281028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:50.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3451816347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b280ff490 0x7f0b280ff8b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3451816347 >> 172.21.15.100:0/3451816347 conn(0x7f0b280fadc0 msgr2=0x7f0b280fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3451816347 shutdown_connections 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3451816347 wait complete. 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 Processor -- start 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- start start 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b280ff490 0x7f0b28101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 0x7f0b28100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102e20 0x7f0b28100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281010b0 con 0x7f0b28102460 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28101220 con 0x7f0b280ff490 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.866+0000 7f0b2f3a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281a1260 con 0x7f0b28102e20 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 0x7f0b28100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 0x7f0b28100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43888/0 (socket says 172.21.15.100:43888) 2024-03-12T05:11:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 -- 172.21.15.100:0/3744739815 learned_addr learned my addr 172.21.15.100:0/3744739815 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2d944640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102e20 0x7f0b28100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 -- 172.21.15.100:0/3744739815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102e20 msgr2=0x7f0b28100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102e20 0x7f0b28100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 -- 172.21.15.100:0/3744739815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b280ff490 msgr2=0x7f0b28101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b280ff490 0x7f0b28101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 -- 172.21.15.100:0/3744739815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b24009900 con 0x7f0b28102460 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2d944640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102e20 0x7f0b28100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2c942640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 0x7f0b28100410 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d980 tx=0x7f0b1800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18009c30 con 0x7f0b28102460 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b1800b660 con 0x7f0b28102460 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b281a1510 con 0x7f0b28102460 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800f6b0 con 0x7f0b28102460 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.867+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b281a1a70 con 0x7f0b28102460 2024-03-12T05:11:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.868+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b28108590 con 0x7f0b28102460 2024-03-12T05:11:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.869+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f0b18010460 con 0x7f0b28102460 2024-03-12T05:11:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.870+0000 7f0b1e7fc640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0b140a1b50 0x7f0b140a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.870+0000 7f0b2d143640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0b140a1b50 0x7f0b140a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.870+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0b180cd0f0 con 0x7f0b28102460 2024-03-12T05:11:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.870+0000 7f0b2d143640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0b140a1b50 0x7f0b140a4010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006010 tx=0x7f0b24009890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:50.874+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b18096300 con 0x7f0b28102460 2024-03-12T05:11:51.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:50 smithi100 ceph-mon[32210]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:51.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.087+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f0b2802ce40 con 0x7f0b28102460 2024-03-12T05:11:51.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:50 smithi156 ceph-mon[36270]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:51.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:50 smithi161 ceph-mon[36043]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-12T05:11:51.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.798+0000 7f0b1e7fc640 1 -- 172.21.15.100:0/3744739815 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0b180999c0 con 0x7f0b28102460 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0b140a1b50 msgr2=0x7f0b140a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0b140a1b50 0x7f0b140a4010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006010 tx=0x7f0b24009890 comp rx=0 tx=0).stop 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 msgr2=0x7f0b28100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 0x7f0b28100410 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d980 tx=0x7f0b1800de40 comp rx=0 tx=0).stop 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 shutdown_connections 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f0b140a1b50 0x7f0b140a4010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b28102e20 0x7f0b28100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b28102460 0x7f0b28100410 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 --2- 172.21.15.100:0/3744739815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b280ff490 0x7f0b28101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.800+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 >> 172.21.15.100:0/3744739815 conn(0x7f0b280fadc0 msgr2=0x7f0b28105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.801+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 shutdown_connections 2024-03-12T05:11:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.801+0000 7f0b2f3a7640 1 -- 172.21.15.100:0/3744739815 wait complete. 2024-03-12T05:11:51.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-12T05:11:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 -- 172.21.15.100:0/179298986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc8102d60 msgr2=0x7f4fc81095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 --2- 172.21.15.100:0/179298986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc8102d60 0x7f4fc81095f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009ca0 tx=0x7f4fc002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 -- 172.21.15.100:0/179298986 shutdown_connections 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 --2- 172.21.15.100:0/179298986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc8102d60 0x7f4fc81095f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 --2- 172.21.15.100:0/179298986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc81023a0 0x7f4fc8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 --2- 172.21.15.100:0/179298986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc81011b0 0x7f4fc81015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 -- 172.21.15.100:0/179298986 >> 172.21.15.100:0/179298986 conn(0x7f4fc80fc740 msgr2=0x7f4fc80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 -- 172.21.15.100:0/179298986 shutdown_connections 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.919+0000 7f4fce457640 1 -- 172.21.15.100:0/179298986 wait complete. 2024-03-12T05:11:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 Processor -- start 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 -- start start 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 0x7f4fc8074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc81023a0 0x7f4fc8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8102d60 0x7f4fc8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8074210 con 0x7f4fc81011b0 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8074380 con 0x7f4fc8102d60 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fce457640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81a10d0 con 0x7f4fc81023a0 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 0x7f4fc8074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 0x7f4fc8074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43902/0 (socket says 172.21.15.100:43902) 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 -- 172.21.15.100:0/1671143906 learned_addr learned my addr 172.21.15.100:0/1671143906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fcc9f4640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8102d60 0x7f4fc8073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 -- 172.21.15.100:0/1671143906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc81023a0 msgr2=0x7f4fc8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc81023a0 0x7f4fc8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 -- 172.21.15.100:0/1671143906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8102d60 msgr2=0x7f4fc8073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8102d60 0x7f4fc8073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.920+0000 7f4fc7fff640 1 -- 172.21.15.100:0/1671143906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0009900 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.921+0000 7f4fc7fff640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 0x7f4fc8074f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c9a0 tx=0x7f4fbc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.921+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc007c10 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.921+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc81a1380 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.921+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fbc007d70 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.921+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00f870 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.921+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc81a18e0 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.922+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8108400 con 0x7f4fc81011b0 2024-03-12T05:11:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.923+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4fbc00fa10 con 0x7f4fc81011b0 2024-03-12T05:11:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.923+0000 7f4fc57fa640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4fb00a1ad0 0x7f4fb00a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.924+0000 7f4fc77fe640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4fb00a1ad0 0x7f4fb00a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.924+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4fbc013070 con 0x7f4fc81011b0 2024-03-12T05:11:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.924+0000 7f4fc77fe640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4fb00a1ad0 0x7f4fb00a3f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8074ca0 tx=0x7f4fb8005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:51.928+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4fbc096a60 con 0x7f4fc81011b0 2024-03-12T05:11:52.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3744739815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-12T05:11:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.129+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fc802cee0 con 0x7f4fb00a1ad0 2024-03-12T05:11:52.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.149+0000 7f4fc57fa640 1 -- 172.21.15.100:0/1671143906 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f4fc802cee0 con 0x7f4fb00a1ad0 2024-03-12T05:11:52.146 INFO:teuthology.orchestra.run.smithi100.stdout:{} 2024-03-12T05:11:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4fb00a1ad0 msgr2=0x7f4fb00a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4fb00a1ad0 0x7f4fb00a3f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8074ca0 tx=0x7f4fb8005bf0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 msgr2=0x7f4fc8074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 0x7f4fc8074f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c9a0 tx=0x7f4fbc00ce60 comp rx=0 tx=0).stop 2024-03-12T05:11:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 shutdown_connections 2024-03-12T05:11:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4fb00a1ad0 0x7f4fb00a3f90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8102d60 0x7f4fc8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc81023a0 0x7f4fc8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 --2- 172.21.15.100:0/1671143906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc81011b0 0x7f4fc8074f30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 >> 172.21.15.100:0/1671143906 conn(0x7f4fc80fc740 msgr2=0x7f4fc81055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:52.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 shutdown_connections 2024-03-12T05:11:52.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.151+0000 7f4fce457640 1 -- 172.21.15.100:0/1671143906 wait complete. 2024-03-12T05:11:52.159 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-12T05:11:52.169 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-12T05:11:52.181 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:52.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3744739815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-12T05:11:52.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3744739815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-12T05:11:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 -- 172.21.15.100:0/1619531435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c1011b0 msgr2=0x7fdd5c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 --2- 172.21.15.100:0/1619531435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c1011b0 0x7fdd5c1015d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009900 tx=0x7fdd4c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 -- 172.21.15.100:0/1619531435 shutdown_connections 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 --2- 172.21.15.100:0/1619531435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5c102d60 0x7fdd5c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 --2- 172.21.15.100:0/1619531435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c1023a0 0x7fdd5c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 --2- 172.21.15.100:0/1619531435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c1011b0 0x7fdd5c1015d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 -- 172.21.15.100:0/1619531435 >> 172.21.15.100:0/1619531435 conn(0x7fdd5c0fc760 msgr2=0x7fdd5c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 -- 172.21.15.100:0/1619531435 shutdown_connections 2024-03-12T05:11:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.287+0000 7fdd617e2640 1 -- 172.21.15.100:0/1619531435 wait complete. 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 Processor -- start 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 -- start start 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5c1011b0 0x7fdd5c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c1023a0 0x7fdd5c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 0x7fdd5c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19ed70 con 0x7fdd5c102d60 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19eee0 con 0x7fdd5c1011b0 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd617e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19f0a0 con 0x7fdd5c1023a0 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 0x7fdd5c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 0x7fdd5c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43936/0 (socket says 172.21.15.100:43936) 2024-03-12T05:11:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 -- 172.21.15.100:0/412545119 learned_addr learned my addr 172.21.15.100:0/412545119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5affd640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5c1011b0 0x7fdd5c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd527fc640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c1023a0 0x7fdd5c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 -- 172.21.15.100:0/412545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c1023a0 msgr2=0x7fdd5c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c1023a0 0x7fdd5c1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 -- 172.21.15.100:0/412545119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5c1011b0 msgr2=0x7fdd5c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5c1011b0 0x7fdd5c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.288+0000 7fdd5b7fe640 1 -- 172.21.15.100:0/412545119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009560 con 0x7fdd5c102d60 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.289+0000 7fdd5b7fe640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 0x7fdd5c19e830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400eaa0 tx=0x7fdd5400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.289+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400cdb0 con 0x7fdd5c102d60 2024-03-12T05:11:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.289+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c19f380 con 0x7fdd5c102d60 2024-03-12T05:11:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.289+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd54005020 con 0x7fdd5c102d60 2024-03-12T05:11:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.289+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540116b0 con 0x7fdd5c102d60 2024-03-12T05:11:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.289+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c19f840 con 0x7fdd5c102d60 2024-03-12T05:11:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.290+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c1085a0 con 0x7fdd5c102d60 2024-03-12T05:11:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.291+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fdd54005520 con 0x7fdd5c102d60 2024-03-12T05:11:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.291+0000 7fdd58ff9640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd480a1a30 0x7fdd480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.292+0000 7fdd5affd640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd480a1a30 0x7fdd480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.292+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdd540cd630 con 0x7fdd5c102d60 2024-03-12T05:11:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.292+0000 7fdd5affd640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd480a1a30 0x7fdd480a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005ab0 tx=0x7fdd4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.296+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd54096840 con 0x7fdd5c102d60 2024-03-12T05:11:52.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.469+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5c02ce40 con 0x7fdd5c102d60 2024-03-12T05:11:52.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.471+0000 7fdd58ff9640 1 -- 172.21.15.100:0/412545119 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7fdd54099f00 con 0x7fdd5c102d60 2024-03-12T05:11:52.468 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:52.468 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:52.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd480a1a30 msgr2=0x7fdd480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd480a1a30 0x7fdd480a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005ab0 tx=0x7fdd4c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 msgr2=0x7fdd5c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 0x7fdd5c19e830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400eaa0 tx=0x7fdd5400ef60 comp rx=0 tx=0).stop 2024-03-12T05:11:52.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 shutdown_connections 2024-03-12T05:11:52.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdd480a1a30 0x7fdd480a3ef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd5c102d60 0x7fdd5c19e830 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5c1023a0 0x7fdd5c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 --2- 172.21.15.100:0/412545119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5c1011b0 0x7fdd5c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 >> 172.21.15.100:0/412545119 conn(0x7fdd5c0fc760 msgr2=0x7fdd5c105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:52.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 shutdown_connections 2024-03-12T05:11:52.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.473+0000 7fdd617e2640 1 -- 172.21.15.100:0/412545119 wait complete. 2024-03-12T05:11:52.480 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:52.133+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:11:52.480 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:52.133+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:11:52.480 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:52.133+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:11:52.480 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:52.133+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:11:52.480 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:52.133+0000 7fbc02a7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:11:52.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:52.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:52.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 -- 172.21.15.100:0/1226476022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58075140 msgr2=0x7f6e58073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/1226476022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58075140 0x7f6e58073590 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009900 tx=0x7f6e4c02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 -- 172.21.15.100:0/1226476022 shutdown_connections 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/1226476022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e58109050 0x7f6e5810b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/1226476022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58073b60 0x7f6e58073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/1226476022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58075140 0x7f6e58073590 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 -- 172.21.15.100:0/1226476022 >> 172.21.15.100:0/1226476022 conn(0x7f6e580fc710 msgr2=0x7f6e580feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 -- 172.21.15.100:0/1226476022 shutdown_connections 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.590+0000 7f6e5eec0640 1 -- 172.21.15.100:0/1226476022 wait complete. 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 Processor -- start 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 -- start start 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e58073b60 0x7f6e581993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58075140 0x7f6e58199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 0x7f6e581a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a0ec0 con 0x7f6e58109050 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a1030 con 0x7f6e58075140 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5eec0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a11f0 con 0x7f6e58073b60 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 0x7f6e581a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 0x7f6e581a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43948/0 (socket says 172.21.15.100:43948) 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 -- 172.21.15.100:0/2698489342 learned_addr learned my addr 172.21.15.100:0/2698489342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5cc5c640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e58073b60 0x7f6e581993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e57fff640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58075140 0x7f6e58199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 -- 172.21.15.100:0/2698489342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e58073b60 msgr2=0x7f6e581993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e58073b60 0x7f6e581993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 -- 172.21.15.100:0/2698489342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58075140 msgr2=0x7f6e58199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58075140 0x7f6e58199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 -- 172.21.15.100:0/2698489342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c009560 con 0x7f6e58109050 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e57fff640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58075140 0x7f6e58199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e5d45d640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 0x7f6e581a0980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000eb10 tx=0x7f6e5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.591+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50004b20 con 0x7f6e58109050 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.592+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e50004c80 con 0x7f6e58109050 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.592+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e581a14d0 con 0x7f6e58109050 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.592+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50010650 con 0x7f6e58109050 2024-03-12T05:11:52.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.592+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581a19e0 con 0x7f6e58109050 2024-03-12T05:11:52.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.593+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e58107e10 con 0x7f6e58109050 2024-03-12T05:11:52.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.593+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6e50005020 con 0x7f6e58109050 2024-03-12T05:11:52.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.594+0000 7f6e55ffb640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e480a1a30 0x7f6e480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.594+0000 7f6e5cc5c640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e480a1a30 0x7f6e480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.594+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e50014070 con 0x7f6e58109050 2024-03-12T05:11:52.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.595+0000 7f6e5cc5c640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e480a1a30 0x7f6e480a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009230 tx=0x7f6e4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:52.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.598+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e50096c10 con 0x7f6e58109050 2024-03-12T05:11:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.769+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e5802ce40 con 0x7f6e58109050 2024-03-12T05:11:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.770+0000 7f6e55ffb640 1 -- 172.21.15.100:0/2698489342 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e5009a2d0 con 0x7f6e58109050 2024-03-12T05:11:52.767 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e480a1a30 msgr2=0x7f6e480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e480a1a30 0x7f6e480a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009230 tx=0x7f6e4c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 msgr2=0x7f6e581a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 0x7f6e581a0980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000eb10 tx=0x7f6e5000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 shutdown_connections 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e480a1a30 0x7f6e480a3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58109050 0x7f6e581a0980 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58075140 0x7f6e58199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 --2- 172.21.15.100:0/2698489342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e58073b60 0x7f6e581993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 >> 172.21.15.100:0/2698489342 conn(0x7f6e580fc710 msgr2=0x7f6e58104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 shutdown_connections 2024-03-12T05:11:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.772+0000 7f6e5eec0640 1 -- 172.21.15.100:0/2698489342 wait complete. 2024-03-12T05:11:52.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:52.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 -- 172.21.15.100:0/2447622771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc101c30 msgr2=0x7f51fc1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 --2- 172.21.15.100:0/2447622771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc101c30 0x7f51fc1084c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009900 tx=0x7f51ec02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 -- 172.21.15.100:0/2447622771 shutdown_connections 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 --2- 172.21.15.100:0/2447622771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc101c30 0x7f51fc1084c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 --2- 172.21.15.100:0/2447622771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51fc101270 0x7f51fc1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 --2- 172.21.15.100:0/2447622771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51fc100080 0x7f51fc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.890+0000 7f5201b13640 1 -- 172.21.15.100:0/2447622771 >> 172.21.15.100:0/2447622771 conn(0x7f51fc0fb610 msgr2=0x7f51fc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 -- 172.21.15.100:0/2447622771 shutdown_connections 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 -- 172.21.15.100:0/2447622771 wait complete. 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 Processor -- start 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 -- start start 2024-03-12T05:11:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 0x7f51fc197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51fc101270 0x7f51fc197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51fc101c30 0x7f51fc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19ed00 con 0x7f51fc100080 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19ee70 con 0x7f51fc101c30 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f5201b13640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19f030 con 0x7f51fc101270 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 0x7f51fc197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.891+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 0x7f51fc197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43970/0 (socket says 172.21.15.100:43970) 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 -- 172.21.15.100:0/2633972588 learned_addr learned my addr 172.21.15.100:0/2633972588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51faffd640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51fc101270 0x7f51fc197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 -- 172.21.15.100:0/2633972588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51fc101270 msgr2=0x7f51fc197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51fc101270 0x7f51fc197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 -- 172.21.15.100:0/2633972588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51fc101c30 msgr2=0x7f51fc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51fc101c30 0x7f51fc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 -- 172.21.15.100:0/2633972588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009560 con 0x7f51fc100080 2024-03-12T05:11:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51fb7fe640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 0x7f51fc197210 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f51f400eb70 tx=0x7f51f400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4003b70 con 0x7f51fc100080 2024-03-12T05:11:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51f400ee70 con 0x7f51fc100080 2024-03-12T05:11:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc19f310 con 0x7f51fc100080 2024-03-12T05:11:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40107f0 con 0x7f51fc100080 2024-03-12T05:11:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.892+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc19f7d0 con 0x7f51fc100080 2024-03-12T05:11:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.893+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc10ba90 con 0x7f51fc100080 2024-03-12T05:11:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.894+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f51f4004020 con 0x7f51fc100080 2024-03-12T05:11:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.895+0000 7f51f8ff9640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f51e80a1b40 0x7f51e80a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.895+0000 7f51faffd640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f51e80a1b40 0x7f51e80a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.895+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51f4014070 con 0x7f51fc100080 2024-03-12T05:11:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.895+0000 7f51faffd640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f51e80a1b40 0x7f51e80a4000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51fc198720 tx=0x7f51e400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:52.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:52.899+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51f4096cf0 con 0x7f51fc100080 2024-03-12T05:11:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-mon[32210]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-12T05:11:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3744739815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-12T05:11:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:11:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/412545119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:53.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2698489342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.073+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51fc19fa50 con 0x7f51fc100080 2024-03-12T05:11:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.073+0000 7f51f8ff9640 1 -- 172.21.15.100:0/2633972588 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51f409a3b0 con 0x7f51fc100080 2024-03-12T05:11:53.070 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f51e80a1b40 msgr2=0x7f51e80a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f51e80a1b40 0x7f51e80a4000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51fc198720 tx=0x7f51e400b390 comp rx=0 tx=0).stop 2024-03-12T05:11:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 msgr2=0x7f51fc197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 0x7f51fc197210 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f51f400eb70 tx=0x7f51f400c6a0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 shutdown_connections 2024-03-12T05:11:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f51e80a1b40 0x7f51e80a4000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51fc101c30 0x7f51fc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51fc101270 0x7f51fc197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 --2- 172.21.15.100:0/2633972588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51fc100080 0x7f51fc197210 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 >> 172.21.15.100:0/2633972588 conn(0x7f51fc0fb610 msgr2=0x7f51fc0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 shutdown_connections 2024-03-12T05:11:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.075+0000 7f5201b13640 1 -- 172.21.15.100:0/2633972588 wait complete. 2024-03-12T05:11:53.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2051491451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4102eb0 msgr2=0x7f3ba4109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2051491451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4102eb0 0x7f3ba4109740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009ca0 tx=0x7f3b9c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2051491451 shutdown_connections 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2051491451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4102eb0 0x7f3ba4109740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2051491451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba41024f0 0x7f3ba4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2051491451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 0x7f3ba40ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2051491451 >> 172.21.15.100:0/2051491451 conn(0x7f3ba40fadb0 msgr2=0x7f3ba40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2051491451 shutdown_connections 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.191+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2051491451 wait complete. 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 Processor -- start 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 -- start start 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 0x7f3ba41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 0x7f3ba4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba4102eb0 0x7f3ba419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419ece0 con 0x7f3ba41024f0 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419ee50 con 0x7f3ba4102eb0 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3baa8b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419f010 con 0x7f3ba40ff480 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 0x7f3ba4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 0x7f3ba4197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43980/0 (socket says 172.21.15.100:43980) 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3ba37fe640 1 -- 172.21.15.100:0/2699726259 learned_addr learned my addr 172.21.15.100:0/2699726259 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.192+0000 7f3ba3fff640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 0x7f3ba41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba8e52640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba4102eb0 0x7f3ba419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba37fe640 1 -- 172.21.15.100:0/2699726259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 msgr2=0x7f3ba41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba37fe640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 0x7f3ba41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba37fe640 1 -- 172.21.15.100:0/2699726259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba4102eb0 msgr2=0x7f3ba419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba37fe640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba4102eb0 0x7f3ba419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba37fe640 1 -- 172.21.15.100:0/2699726259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009900 con 0x7f3ba41024f0 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba3fff640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 0x7f3ba41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba8e52640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba4102eb0 0x7f3ba419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba37fe640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 0x7f3ba4197730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e9f0 tx=0x7f3b9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98009700 con 0x7f3ba41024f0 2024-03-12T05:11:53.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b9800cd80 con 0x7f3ba41024f0 2024-03-12T05:11:53.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba419f2f0 con 0x7f3ba41024f0 2024-03-12T05:11:53.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98018920 con 0x7f3ba41024f0 2024-03-12T05:11:53.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.193+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba419f800 con 0x7f3ba41024f0 2024-03-12T05:11:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.194+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba41085c0 con 0x7f3ba41024f0 2024-03-12T05:11:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.195+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f3b98018a80 con 0x7f3ba41024f0 2024-03-12T05:11:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.196+0000 7f3ba17fa640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3b8c0a1ac0 0x7f3b8c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.196+0000 7f3ba3fff640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3b8c0a1ac0 0x7f3b8c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.196+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b98014070 con 0x7f3ba41024f0 2024-03-12T05:11:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.196+0000 7f3ba3fff640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3b8c0a1ac0 0x7f3b8c0a3f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b94005d00 tx=0x7f3b94005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:53.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.200+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b98052d20 con 0x7f3ba41024f0 2024-03-12T05:11:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:52 smithi156 ceph-mon[36270]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-12T05:11:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3744739815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-12T05:11:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:52 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:11:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/412545119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:53.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2698489342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:52 smithi161 ceph-mon[36043]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-12T05:11:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3744739815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-12T05:11:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:52 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:11:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/412545119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:53.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2698489342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.373+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ba402ce40 con 0x7f3ba41024f0 2024-03-12T05:11:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.374+0000 7f3ba17fa640 1 -- 172.21.15.100:0/2699726259 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b98096a80 con 0x7f3ba41024f0 2024-03-12T05:11:53.371 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:53.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3b8c0a1ac0 msgr2=0x7f3b8c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3b8c0a1ac0 0x7f3b8c0a3f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b94005d00 tx=0x7f3b94005c90 comp rx=0 tx=0).stop 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 msgr2=0x7f3ba4197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 0x7f3ba4197730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e9f0 tx=0x7f3b9800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 shutdown_connections 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f3b8c0a1ac0 0x7f3b8c0a3f80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba4102eb0 0x7f3ba419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba41024f0 0x7f3ba4197730 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 --2- 172.21.15.100:0/2699726259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba40ff480 0x7f3ba41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 >> 172.21.15.100:0/2699726259 conn(0x7f3ba40fadb0 msgr2=0x7f3ba4105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 shutdown_connections 2024-03-12T05:11:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.376+0000 7f3baa8b5640 1 -- 172.21.15.100:0/2699726259 wait complete. 2024-03-12T05:11:53.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/3223899536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0101c40 msgr2=0x7fe1e01084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/3223899536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0101c40 0x7fe1e01084d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009ca0 tx=0x7fe1d802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/3223899536 shutdown_connections 2024-03-12T05:11:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/3223899536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0101c40 0x7fe1e01084d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/3223899536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1e0101280 0x7fe1e0101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/3223899536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0100090 0x7fe1e01004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/3223899536 >> 172.21.15.100:0/3223899536 conn(0x7fe1e00fb620 msgr2=0x7fe1e00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/3223899536 shutdown_connections 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/3223899536 wait complete. 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 Processor -- start 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- start start 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 0x7fe1e006a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0101280 0x7fe1e019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1e0101c40 0x7fe1e019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e019ecf0 con 0x7fe1e0100090 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e019ee60 con 0x7fe1e0101c40 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.489+0000 7fe1e7e8f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e019f020 con 0x7fe1e0101280 2024-03-12T05:11:53.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 0x7fe1e006a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 0x7fe1e006a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44000/0 (socket says 172.21.15.100:44000) 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 -- 172.21.15.100:0/1243082824 learned_addr learned my addr 172.21.15.100:0/1243082824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e542a640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0101280 0x7fe1e019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e642c640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1e0101c40 0x7fe1e019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 -- 172.21.15.100:0/1243082824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0101280 msgr2=0x7fe1e019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0101280 0x7fe1e019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 -- 172.21.15.100:0/1243082824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1e0101c40 msgr2=0x7fe1e019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1e0101c40 0x7fe1e019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 -- 172.21.15.100:0/1243082824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009900 con 0x7fe1e0100090 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e5c2b640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 0x7fe1e006a3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d950 tx=0x7fe1dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc009c30 con 0x7fe1e0100090 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc009d90 con 0x7fe1e0100090 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e019f300 con 0x7fe1e0100090 2024-03-12T05:11:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00f720 con 0x7fe1e0100090 2024-03-12T05:11:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.490+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e019f7c0 con 0x7fe1e0100090 2024-03-12T05:11:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.491+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e01073b0 con 0x7fe1e0100090 2024-03-12T05:11:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.493+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fe1dc010a60 con 0x7fe1e0100090 2024-03-12T05:11:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.493+0000 7fe1d6ffd640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1cc0a1ac0 0x7fe1cc0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.494+0000 7fe1e542a640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1cc0a1ac0 0x7fe1cc0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.494+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe1dc0cd220 con 0x7fe1e0100090 2024-03-12T05:11:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.494+0000 7fe1e542a640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1cc0a1ac0 0x7fe1cc0a3f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1e006a130 tx=0x7fe1d000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.497+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1dc096340 con 0x7fe1e0100090 2024-03-12T05:11:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.669+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e010bc20 con 0x7fe1e0100090 2024-03-12T05:11:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.671+0000 7fe1d6ffd640 1 -- 172.21.15.100:0/1243082824 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7fe1e010bc20 con 0x7fe1e0100090 2024-03-12T05:11:53.668 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1cc0a1ac0 msgr2=0x7fe1cc0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1cc0a1ac0 0x7fe1cc0a3f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1e006a130 tx=0x7fe1d000a2d0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 msgr2=0x7fe1e006a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 0x7fe1e006a3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d950 tx=0x7fe1dc00de10 comp rx=0 tx=0).stop 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 shutdown_connections 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fe1cc0a1ac0 0x7fe1cc0a3f80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1e0101c40 0x7fe1e019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0101280 0x7fe1e019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 --2- 172.21.15.100:0/1243082824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e0100090 0x7fe1e006a3c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 >> 172.21.15.100:0/1243082824 conn(0x7fe1e00fb620 msgr2=0x7fe1e0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 shutdown_connections 2024-03-12T05:11:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.673+0000 7fe1e7e8f640 1 -- 172.21.15.100:0/1243082824 wait complete. 2024-03-12T05:11:53.682 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:53.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:53.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 -- 172.21.15.100:0/3340718016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444106ec0 msgr2=0x7f04441092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 --2- 172.21.15.100:0/3340718016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444106ec0 0x7f04441092b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f043c009ca0 tx=0x7f043c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 -- 172.21.15.100:0/3340718016 shutdown_connections 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 --2- 172.21.15.100:0/3340718016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444106ec0 0x7f04441092b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 --2- 172.21.15.100:0/3340718016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444104590 0x7f0444106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 --2- 172.21.15.100:0/3340718016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0444101c60 0x7f0444104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.789+0000 7f044b8c0640 1 -- 172.21.15.100:0/3340718016 >> 172.21.15.100:0/3340718016 conn(0x7f04440fb610 msgr2=0x7f04440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 -- 172.21.15.100:0/3340718016 shutdown_connections 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 -- 172.21.15.100:0/3340718016 wait complete. 2024-03-12T05:11:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 Processor -- start 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 -- start start 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 0x7f0444197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0444104590 0x7f0444197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444106ec0 0x7f044419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419ed30 con 0x7f0444101c60 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419eea0 con 0x7f0444106ec0 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044b8c0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419f060 con 0x7f0444104590 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044965c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 0x7f0444197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044965c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 0x7f0444197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44008/0 (socket says 172.21.15.100:44008) 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f044965c640 1 -- 172.21.15.100:0/1236126393 learned_addr learned my addr 172.21.15.100:0/1236126393 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.790+0000 7f0448e5b640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0444104590 0x7f0444197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044965c640 1 -- 172.21.15.100:0/1236126393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0444104590 msgr2=0x7f0444197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044965c640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0444104590 0x7f0444197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044965c640 1 -- 172.21.15.100:0/1236126393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444106ec0 msgr2=0x7f044419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f0449e5d640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444106ec0 0x7f044419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044965c640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444106ec0 0x7f044419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044965c640 1 -- 172.21.15.100:0/1236126393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c009900 con 0x7f0444101c60 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f0449e5d640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444106ec0 0x7f044419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044965c640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 0x7f0444197240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f044000c890 tx=0x7f044000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440004220 con 0x7f0444101c60 2024-03-12T05:11:53.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0440007e90 con 0x7f0444101c60 2024-03-12T05:11:53.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044419f340 con 0x7f0444101c60 2024-03-12T05:11:53.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440004a00 con 0x7f0444101c60 2024-03-12T05:11:53.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.791+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044419f850 con 0x7f0444101c60 2024-03-12T05:11:53.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.792+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04440fd100 con 0x7f0444101c60 2024-03-12T05:11:53.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.793+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f04400079b0 con 0x7f0444101c60 2024-03-12T05:11:53.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.793+0000 7f043a7fc640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04300a1ac0 0x7f04300a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:53.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.794+0000 7f0448e5b640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04300a1ac0 0x7f04300a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:53.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.794+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04400cd2d0 con 0x7f0444101c60 2024-03-12T05:11:53.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.794+0000 7f0448e5b640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04300a1ac0 0x7f04300a3f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0444198750 tx=0x7f043400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:53.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.798+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0440096380 con 0x7f0444101c60 2024-03-12T05:11:53.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.975+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f044410cb90 con 0x7f0444101c60 2024-03-12T05:11:53.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.976+0000 7f043a7fc640 1 -- 172.21.15.100:0/1236126393 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0440099a40 con 0x7f0444101c60 2024-03-12T05:11:53.973 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04300a1ac0 msgr2=0x7f04300a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04300a1ac0 0x7f04300a3f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0444198750 tx=0x7f043400a2d0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 msgr2=0x7f0444197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 0x7f0444197240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f044000c890 tx=0x7f044000cd50 comp rx=0 tx=0).stop 2024-03-12T05:11:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 shutdown_connections 2024-03-12T05:11:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f04300a1ac0 0x7f04300a3f80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444106ec0 0x7f044419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0444104590 0x7f0444197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 --2- 172.21.15.100:0/1236126393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0444101c60 0x7f0444197240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 >> 172.21.15.100:0/1236126393 conn(0x7f04440fb610 msgr2=0x7f04441025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 shutdown_connections 2024-03-12T05:11:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:53.978+0000 7f044b8c0640 1 -- 172.21.15.100:0/1236126393 wait complete. 2024-03-12T05:11:53.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:53 smithi100 ceph-mon[32210]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:53.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:53 smithi100 ceph-mon[32210]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-12T05:11:53.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2633972588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:53.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2699726259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:53.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1243082824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:53.986 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.094+0000 7f363b898640 1 -- 172.21.15.100:0/1355401105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101270 msgr2=0x7f36341016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.094+0000 7f363b898640 1 --2- 172.21.15.100:0/1355401105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101270 0x7f36341016f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3624009900 tx=0x7f362402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- 172.21.15.100:0/1355401105 shutdown_connections 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 --2- 172.21.15.100:0/1355401105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101c30 0x7f36341084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 --2- 172.21.15.100:0/1355401105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101270 0x7f36341016f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 --2- 172.21.15.100:0/1355401105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3634100080 0x7f36341004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- 172.21.15.100:0/1355401105 >> 172.21.15.100:0/1355401105 conn(0x7f36340fb610 msgr2=0x7f36340fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- 172.21.15.100:0/1355401105 shutdown_connections 2024-03-12T05:11:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- 172.21.15.100:0/1355401105 wait complete. 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 Processor -- start 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- start start 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3634100080 0x7f36341961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101270 0x7f3634196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 0x7f363419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363419dcd0 con 0x7f3634101c30 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363419de40 con 0x7f3634101270 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f363b898640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363419e000 con 0x7f3634100080 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f3639e35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 0x7f363419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f3639e35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 0x7f363419d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44028/0 (socket says 172.21.15.100:44028) 2024-03-12T05:11:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.095+0000 7f3639e35640 1 -- 172.21.15.100:0/3364850996 learned_addr learned my addr 172.21.15.100:0/3364850996 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639634640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3634100080 0x7f36341961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639e35640 1 -- 172.21.15.100:0/3364850996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3634100080 msgr2=0x7f36341961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3638e33640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101270 0x7f3634196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639e35640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3634100080 0x7f36341961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639e35640 1 -- 172.21.15.100:0/3364850996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101270 msgr2=0x7f3634196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639e35640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101270 0x7f3634196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639e35640 1 -- 172.21.15.100:0/3364850996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624009560 con 0x7f3634101c30 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3638e33640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101270 0x7f3634196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f3639e35640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 0x7f363419d790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b970 tx=0x7f362c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00cb40 con 0x7f3634101c30 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362c00cca0 con 0x7f3634101c30 2024-03-12T05:11:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363419e2e0 con 0x7f3634101c30 2024-03-12T05:11:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0077d0 con 0x7f3634101c30 2024-03-12T05:11:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.096+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363419e870 con 0x7f3634101c30 2024-03-12T05:11:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.097+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3634107420 con 0x7f3634101c30 2024-03-12T05:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.098+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f362c014440 con 0x7f3634101c30 2024-03-12T05:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.099+0000 7f362a7fc640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f36200a1b40 0x7f36200a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.099+0000 7f3639634640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f36200a1b40 0x7f36200a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.099+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f362c0cd130 con 0x7f3634101c30 2024-03-12T05:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.099+0000 7f3639634640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f36200a1b40 0x7f36200a4000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f363000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:54.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.103+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f362c0963a0 con 0x7f3634101c30 2024-03-12T05:11:54.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:53 smithi156 ceph-mon[36270]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:54.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:53 smithi156 ceph-mon[36270]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-12T05:11:54.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2633972588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2699726259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1243082824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:53 smithi161 ceph-mon[36043]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:11:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:53 smithi161 ceph-mon[36043]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-12T05:11:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2633972588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2699726259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1243082824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:54.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.274+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f363410baf0 con 0x7f3634101c30 2024-03-12T05:11:54.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.275+0000 7f362a7fc640 1 -- 172.21.15.100:0/3364850996 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f362c019020 con 0x7f3634101c30 2024-03-12T05:11:54.272 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f36200a1b40 msgr2=0x7f36200a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f36200a1b40 0x7f36200a4000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f363000b390 comp rx=0 tx=0).stop 2024-03-12T05:11:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 msgr2=0x7f363419d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 0x7f363419d790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b970 tx=0x7f362c00be30 comp rx=0 tx=0).stop 2024-03-12T05:11:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 shutdown_connections 2024-03-12T05:11:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f36200a1b40 0x7f36200a4000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634101c30 0x7f363419d790 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3634101270 0x7f3634196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 --2- 172.21.15.100:0/3364850996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3634100080 0x7f36341961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.276+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 >> 172.21.15.100:0/3364850996 conn(0x7f36340fb610 msgr2=0x7f36341045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.277+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 shutdown_connections 2024-03-12T05:11:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.277+0000 7f363b898640 1 -- 172.21.15.100:0/3364850996 wait complete. 2024-03-12T05:11:54.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 -- 172.21.15.100:0/2877667046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18741045c0 msgr2=0x7f18741069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 --2- 172.21.15.100:0/2877667046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18741045c0 0x7f18741069b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1868009900 tx=0x7f186802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 -- 172.21.15.100:0/2877667046 shutdown_connections 2024-03-12T05:11:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 --2- 172.21.15.100:0/2877667046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1874106ef0 0x7f18741092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 --2- 172.21.15.100:0/2877667046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18741045c0 0x7f18741069b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 --2- 172.21.15.100:0/2877667046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874101c90 0x7f1874104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 -- 172.21.15.100:0/2877667046 >> 172.21.15.100:0/2877667046 conn(0x7f18740fb620 msgr2=0x7f18740fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 -- 172.21.15.100:0/2877667046 shutdown_connections 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 -- 172.21.15.100:0/2877667046 wait complete. 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 Processor -- start 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.393+0000 7f187a999640 1 -- start start 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 0x7f18741971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18741045c0 0x7f1874197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1874106ef0 0x7f187419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187419ecd0 con 0x7f1874101c90 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187419ee40 con 0x7f1874106ef0 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187419f000 con 0x7f18741045c0 2024-03-12T05:11:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 0x7f18741971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 0x7f18741971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44040/0 (socket says 172.21.15.100:44040) 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 -- 172.21.15.100:0/58668799 learned_addr learned my addr 172.21.15.100:0/58668799 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f18737fe640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18741045c0 0x7f1874197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 -- 172.21.15.100:0/58668799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18741045c0 msgr2=0x7f1874197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18741045c0 0x7f1874197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 -- 172.21.15.100:0/58668799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1874106ef0 msgr2=0x7f187419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1874106ef0 0x7f187419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 -- 172.21.15.100:0/58668799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1868009560 con 0x7f1874101c90 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f1873fff640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 0x7f18741971e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f186400c920 tx=0x7f186400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864007c70 con 0x7f1874101c90 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1864007dd0 con 0x7f1874101c90 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187419f2e0 con 0x7f1874101c90 2024-03-12T05:11:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864018860 con 0x7f1874101c90 2024-03-12T05:11:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.394+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187419f840 con 0x7f1874101c90 2024-03-12T05:11:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.395+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18740fd110 con 0x7f1874101c90 2024-03-12T05:11:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.396+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f186400fac0 con 0x7f1874101c90 2024-03-12T05:11:54.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.397+0000 7f18717fa640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f185c0a1b40 0x7f185c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.397+0000 7f18737fe640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f185c0a1b40 0x7f185c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.397+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f18640cd730 con 0x7f1874101c90 2024-03-12T05:11:54.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.397+0000 7f18737fe640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f185c0a1b40 0x7f185c0a4000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1868036040 tx=0x7f1868005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:54.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.401+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18640967e0 con 0x7f1874101c90 2024-03-12T05:11:54.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.572+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f187402cee0 con 0x7f1874101c90 2024-03-12T05:11:54.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.573+0000 7f18717fa640 1 -- 172.21.15.100:0/58668799 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1864099ea0 con 0x7f1874101c90 2024-03-12T05:11:54.570 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:54.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f185c0a1b40 msgr2=0x7f185c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f185c0a1b40 0x7f185c0a4000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1868036040 tx=0x7f1868005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 msgr2=0x7f18741971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 0x7f18741971e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f186400c920 tx=0x7f186400cde0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 shutdown_connections 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f185c0a1b40 0x7f185c0a4000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1874106ef0 0x7f187419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18741045c0 0x7f1874197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 --2- 172.21.15.100:0/58668799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874101c90 0x7f18741971e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 >> 172.21.15.100:0/58668799 conn(0x7f18740fb620 msgr2=0x7f18741026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 shutdown_connections 2024-03-12T05:11:54.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.575+0000 7f187a999640 1 -- 172.21.15.100:0/58668799 wait complete. 2024-03-12T05:11:54.583 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-12T05:11:54.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:54.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 -- 172.21.15.100:0/2131639874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930101280 msgr2=0x7f8930101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 --2- 172.21.15.100:0/2131639874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930101280 0x7f8930101700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8920009900 tx=0x7f892002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:54.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 -- 172.21.15.100:0/2131639874 shutdown_connections 2024-03-12T05:11:54.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 --2- 172.21.15.100:0/2131639874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8930101c40 0x7f89301084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 --2- 172.21.15.100:0/2131639874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930101280 0x7f8930101700 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 --2- 172.21.15.100:0/2131639874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100090 0x7f89301004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 -- 172.21.15.100:0/2131639874 >> 172.21.15.100:0/2131639874 conn(0x7f89300fb620 msgr2=0x7f89300fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 -- 172.21.15.100:0/2131639874 shutdown_connections 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 -- 172.21.15.100:0/2131639874 wait complete. 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 Processor -- start 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.688+0000 7f89363d0640 1 -- start start 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 0x7f8930196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8930101280 0x7f8930196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930101c40 0x7f893019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019db00 con 0x7f8930100090 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019dc70 con 0x7f8930101c40 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019de30 con 0x7f8930101280 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 0x7f8930196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 0x7f8930196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44062/0 (socket says 172.21.15.100:44062) 2024-03-12T05:11:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 -- 172.21.15.100:0/1091593381 learned_addr learned my addr 172.21.15.100:0/1091593381 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f893496d640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930101c40 0x7f893019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 -- 172.21.15.100:0/1091593381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8930101280 msgr2=0x7f8930196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8930101280 0x7f8930196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 -- 172.21.15.100:0/1091593381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930101c40 msgr2=0x7f893019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930101c40 0x7f893019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 -- 172.21.15.100:0/1091593381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920009560 con 0x7f8930100090 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892ffff640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 0x7f8930196010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f891800c970 tx=0x7f891800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918007c10 con 0x7f8930100090 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893019e110 con 0x7f8930100090 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8918007d70 con 0x7f8930100090 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800f870 con 0x7f8930100090 2024-03-12T05:11:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.689+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893019e670 con 0x7f8930100090 2024-03-12T05:11:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.690+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89301072e0 con 0x7f8930100090 2024-03-12T05:11:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.691+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f8918018a60 con 0x7f8930100090 2024-03-12T05:11:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.692+0000 7f892d7fa640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f891c0a1ac0 0x7f891c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.692+0000 7f892f7fe640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f891c0a1ac0 0x7f891c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.692+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8918013070 con 0x7f8930100090 2024-03-12T05:11:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.692+0000 7f892f7fe640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f891c0a1ac0 0x7f891c0a3f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f892002c6a0 tx=0x7f8920000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:54.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.696+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8918096ad0 con 0x7f8930100090 2024-03-12T05:11:54.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.908+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f893010baa0 con 0x7f8930100090 2024-03-12T05:11:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.909+0000 7f892d7fa640 1 -- 172.21.15.100:0/1091593381 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f891809a190 con 0x7f8930100090 2024-03-12T05:11:54.906 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:54.906 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:11:51.088786+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:54.907 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 23 2024-03-12T05:11:54.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f891c0a1ac0 msgr2=0x7f891c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f891c0a1ac0 0x7f891c0a3f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f892002c6a0 tx=0x7f8920000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 msgr2=0x7f8930196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:54.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 0x7f8930196010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f891800c970 tx=0x7f891800ce30 comp rx=0 tx=0).stop 2024-03-12T05:11:54.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 shutdown_connections 2024-03-12T05:11:54.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f891c0a1ac0 0x7f891c0a3f80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930101c40 0x7f893019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8930101280 0x7f8930196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 --2- 172.21.15.100:0/1091593381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8930100090 0x7f8930196010 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 >> 172.21.15.100:0/1091593381 conn(0x7f89300fb620 msgr2=0x7f89301044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 shutdown_connections 2024-03-12T05:11:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:54.911+0000 7f89363d0640 1 -- 172.21.15.100:0/1091593381 wait complete. 2024-03-12T05:11:54.918 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:54 smithi100 ceph-mon[32210]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-12T05:11:54.918 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1236126393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.918 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3364850996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.918 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/58668799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:54.919 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-12T05:11:54.919 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-12T05:11:54.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 -- 172.21.15.100:0/3085526247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e78100bb0 msgr2=0x7f7e78105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3085526247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e78100bb0 0x7f7e78105450 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009ca0 tx=0x7f7e7002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 -- 172.21.15.100:0/3085526247 shutdown_connections 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3085526247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e78100bb0 0x7f7e78105450 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3085526247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e781001f0 0x7f7e78100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3085526247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e780ff490 0x7f7e780ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 -- 172.21.15.100:0/3085526247 >> 172.21.15.100:0/3085526247 conn(0x7f7e780fadc0 msgr2=0x7f7e780fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.024+0000 7f7e7f839640 1 -- 172.21.15.100:0/3085526247 shutdown_connections 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 -- 172.21.15.100:0/3085526247 wait complete. 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 Processor -- start 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 -- start start 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 0x7f7e7819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e781001f0 0x7f7e7819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e78100bb0 0x7f7e781a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a5320 con 0x7f7e780ff490 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a5490 con 0x7f7e781001f0 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7f839640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a5650 con 0x7f7e78100bb0 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7d5d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 0x7f7e7819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7d5d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 0x7f7e7819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44068/0 (socket says 172.21.15.100:44068) 2024-03-12T05:11:55.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7d5d5640 1 -- 172.21.15.100:0/3637854771 learned_addr learned my addr 172.21.15.100:0/3637854771 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.025+0000 7f7e7cdd4640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e781001f0 0x7f7e7819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7ddd6640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e78100bb0 0x7f7e781a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7d5d5640 1 -- 172.21.15.100:0/3637854771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e78100bb0 msgr2=0x7f7e781a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7d5d5640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e78100bb0 0x7f7e781a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7d5d5640 1 -- 172.21.15.100:0/3637854771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e781001f0 msgr2=0x7f7e7819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7d5d5640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e781001f0 0x7f7e7819dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7d5d5640 1 -- 172.21.15.100:0/3637854771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009900 con 0x7f7e780ff490 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7d5d5640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 0x7f7e7819d830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400d950 tx=0x7f7e7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74009c30 con 0x7f7e780ff490 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e74009d90 con 0x7f7e780ff490 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e781a5930 con 0x7f7e780ff490 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400f7e0 con 0x7f7e780ff490 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.026+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e781a5e40 con 0x7f7e780ff490 2024-03-12T05:11:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.027+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e78104260 con 0x7f7e780ff490 2024-03-12T05:11:55.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.028+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f7e74010ad0 con 0x7f7e780ff490 2024-03-12T05:11:55.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.029+0000 7f7e6e7fc640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e640a1ac0 0x7f7e640a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.029+0000 7f7e7cdd4640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e640a1ac0 0x7f7e640a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.029+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7e740cd220 con 0x7f7e780ff490 2024-03-12T05:11:55.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.029+0000 7f7e7cdd4640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e640a1ac0 0x7f7e640a3f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e7819ed40 tx=0x7f7e6800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.033+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e740962d0 con 0x7f7e780ff490 2024-03-12T05:11:55.106 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:54 smithi156 ceph-mon[36270]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-12T05:11:55.106 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1236126393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:55.106 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3364850996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:55.106 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/58668799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:55.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.203+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e7802cee0 con 0x7f7e640a1ac0 2024-03-12T05:11:55.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.210+0000 7f7e6e7fc640 1 -- 172.21.15.100:0/3637854771 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258840 (secure 0 0 0) 0x7f7e7802cee0 con 0x7f7e640a1ac0 2024-03-12T05:11:55.210 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:55.211 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:11:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e640a1ac0 msgr2=0x7f7e640a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e640a1ac0 0x7f7e640a3f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e7819ed40 tx=0x7f7e6800b390 comp rx=0 tx=0).stop 2024-03-12T05:11:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 msgr2=0x7f7e7819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 0x7f7e7819d830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400d950 tx=0x7f7e7400de10 comp rx=0 tx=0).stop 2024-03-12T05:11:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 shutdown_connections 2024-03-12T05:11:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f7e640a1ac0 0x7f7e640a3f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e78100bb0 0x7f7e781a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e781001f0 0x7f7e7819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 --2- 172.21.15.100:0/3637854771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e780ff490 0x7f7e7819d830 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 >> 172.21.15.100:0/3637854771 conn(0x7f7e780fadc0 msgr2=0x7f7e78101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:55.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 shutdown_connections 2024-03-12T05:11:55.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.213+0000 7f7e7f839640 1 -- 172.21.15.100:0/3637854771 wait complete. 2024-03-12T05:11:55.221 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-12T05:11:53.739187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656879,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":831,"num_read_kb":537,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374988,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566836,"statfs":{"total":1151940427776,"available":1151556440064,"internally_reserved":0,"allocated":13676544,"data_stored":7483095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002812"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744383+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:11:03.744383+0000","last_peered":"2024-03-12T05:11:03.744383+0000","last_clean":"2024-03-12T05:11:03.744383+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:11:03.744383+0000","last_undegraded":"2024-03-12T05:11:03.744383+0000","last_fullsized":"2024-03-12T05:11:03.744383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.739101+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:11:03.739101+0000","last_peered":"2024-03-12T05:11:03.739101+0000","last_clean":"2024-03-12T05:11:03.739101+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:11:03.739101+0000","last_undegraded":"2024-03-12T05:11:03.739101+0000","last_fullsized":"2024-03-12T05:11:03.739101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757927+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:11:03.757927+0000","last_peered":"2024-03-12T05:11:03.757927+0000","last_clean":"2024-03-12T05:11:03.757927+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:11:03.757927+0000","last_undegraded":"2024-03-12T05:11:03.757927+0000","last_fullsized":"2024-03-12T05:11:03.757927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733109+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:11:03.733109+0000","last_peered":"2024-03-12T05:11:03.733109+0000","last_clean":"2024-03-12T05:11:03.733109+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:11:03.733109+0000","last_undegraded":"2024-03-12T05:11:03.733109+0000","last_fullsized":"2024-03-12T05:11:03.733109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743525+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:11:03.743525+0000","last_peered":"2024-03-12T05:11:03.743525+0000","last_clean":"2024-03-12T05:11:03.743525+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:11:03.743525+0000","last_undegraded":"2024-03-12T05:11:03.743525+0000","last_fullsized":"2024-03-12T05:11:03.743525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745830+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:11:03.745830+0000","last_peered":"2024-03-12T05:11:03.745830+0000","last_clean":"2024-03-12T05:11:03.745830+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:11:03.745830+0000","last_undegraded":"2024-03-12T05:11:03.745830+0000","last_fullsized":"2024-03-12T05:11:03.745830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750437+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:11:03.750437+0000","last_peered":"2024-03-12T05:11:03.750437+0000","last_clean":"2024-03-12T05:11:03.750437+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:11:03.750437+0000","last_undegraded":"2024-03-12T05:11:03.750437+0000","last_fullsized":"2024-03-12T05:11:03.750437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747959+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:11:03.747959+0000","last_peered":"2024-03-12T05:11:03.747959+0000","last_clean":"2024-03-12T05:11:03.747959+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:11:03.747959+0000","last_undegraded":"2024-03-12T05:11:03.747959+0000","last_fullsized":"2024-03-12T05:11:03.747959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.752826+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:11:03.752826+0000","last_peered":"2024-03-12T05:11:03.752826+0000","last_clean":"2024-03-12T05:11:03.752826+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:11:03.752826+0000","last_undegraded":"2024-03-12T05:11:03.752826+0000","last_fullsized":"2024-03-12T05:11:03.752826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744896+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:11:03.744896+0000","last_peered":"2024-03-12T05:11:03.744896+0000","last_clean":"2024-03-12T05:11:03.744896+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:11:03.744896+0000","last_undegraded":"2024-03-12T05:11:03.744896+0000","last_fullsized":"2024-03-12T05:11:03.744896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758668+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:11:03.758668+0000","last_peered":"2024-03-12T05:11:03.758668+0000","last_clean":"2024-03-12T05:11:03.758668+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:11:03.758668+0000","last_undegraded":"2024-03-12T05:11:03.758668+0000","last_fullsized":"2024-03-12T05:11:03.758668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748754+0000","last_change":"2024-03-12T05:10:04.986947+0000","last_active":"2024-03-12T05:11:03.748754+0000","last_peered":"2024-03-12T05:11:03.748754+0000","last_clean":"2024-03-12T05:11:03.748754+0000","last_became_active":"2024-03-12T05:10:04.986526+0000","last_became_peered":"2024-03-12T05:10:04.986526+0000","last_unstale":"2024-03-12T05:11:03.748754+0000","last_undegraded":"2024-03-12T05:11:03.748754+0000","last_fullsized":"2024-03-12T05:11:03.748754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:30:47.611206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743287+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:11:03.743287+0000","last_peered":"2024-03-12T05:11:03.743287+0000","last_clean":"2024-03-12T05:11:03.743287+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:11:03.743287+0000","last_undegraded":"2024-03-12T05:11:03.743287+0000","last_fullsized":"2024-03-12T05:11:03.743287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749425+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:11:03.749425+0000","last_peered":"2024-03-12T05:11:03.749425+0000","last_clean":"2024-03-12T05:11:03.749425+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:11:03.749425+0000","last_undegraded":"2024-03-12T05:11:03.749425+0000","last_fullsized":"2024-03-12T05:11:03.749425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.740596+0000","last_change":"2024-03-12T05:10:04.989251+0000","last_active":"2024-03-12T05:11:03.740596+0000","last_peered":"2024-03-12T05:11:03.740596+0000","last_clean":"2024-03-12T05:11:03.740596+0000","last_became_active":"2024-03-12T05:10:04.988845+0000","last_became_peered":"2024-03-12T05:10:04.988845+0000","last_unstale":"2024-03-12T05:11:03.740596+0000","last_undegraded":"2024-03-12T05:11:03.740596+0000","last_fullsized":"2024-03-12T05:11:03.740596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:21:42.513931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760056+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:11:03.760056+0000","last_peered":"2024-03-12T05:11:03.760056+0000","last_clean":"2024-03-12T05:11:03.760056+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:11:03.760056+0000","last_undegraded":"2024-03-12T05:11:03.760056+0000","last_fullsized":"2024-03-12T05:11:03.760056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744673+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:11:03.744673+0000","last_peered":"2024-03-12T05:11:03.744673+0000","last_clean":"2024-03-12T05:11:03.744673+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:11:03.744673+0000","last_undegraded":"2024-03-12T05:11:03.744673+0000","last_fullsized":"2024-03-12T05:11:03.744673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749801+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:11:03.749801+0000","last_peered":"2024-03-12T05:11:03.749801+0000","last_clean":"2024-03-12T05:11:03.749801+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:11:03.749801+0000","last_undegraded":"2024-03-12T05:11:03.749801+0000","last_fullsized":"2024-03-12T05:11:03.749801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748000+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:11:03.748000+0000","last_peered":"2024-03-12T05:11:03.748000+0000","last_clean":"2024-03-12T05:11:03.748000+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:11:03.748000+0000","last_undegraded":"2024-03-12T05:11:03.748000+0000","last_fullsized":"2024-03-12T05:11:03.748000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983601+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:10:04.983601+0000","last_peered":"2024-03-12T05:10:04.983601+0000","last_clean":"2024-03-12T05:10:04.983601+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:10:04.983601+0000","last_undegraded":"2024-03-12T05:10:04.983601+0000","last_fullsized":"2024-03-12T05:10:04.983601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745669+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:11:03.745669+0000","last_peered":"2024-03-12T05:11:03.745669+0000","last_clean":"2024-03-12T05:11:03.745669+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:11:03.745669+0000","last_undegraded":"2024-03-12T05:11:03.745669+0000","last_fullsized":"2024-03-12T05:11:03.745669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985285+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:10:04.985285+0000","last_peered":"2024-03-12T05:10:04.985285+0000","last_clean":"2024-03-12T05:10:04.985285+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:10:04.985285+0000","last_undegraded":"2024-03-12T05:10:04.985285+0000","last_fullsized":"2024-03-12T05:10:04.985285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757822+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:11:03.757822+0000","last_peered":"2024-03-12T05:11:03.757822+0000","last_clean":"2024-03-12T05:11:03.757822+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:11:03.757822+0000","last_undegraded":"2024-03-12T05:11:03.757822+0000","last_fullsized":"2024-03-12T05:11:03.757822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985028+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:10:04.985028+0000","last_peered":"2024-03-12T05:10:04.985028+0000","last_clean":"2024-03-12T05:10:04.985028+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:10:04.985028+0000","last_undegraded":"2024-03-12T05:10:04.985028+0000","last_fullsized":"2024-03-12T05:10:04.985028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758666+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:11:03.758666+0000","last_peered":"2024-03-12T05:11:03.758666+0000","last_clean":"2024-03-12T05:11:03.758666+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:11:03.758666+0000","last_undegraded":"2024-03-12T05:11:03.758666+0000","last_fullsized":"2024-03-12T05:11:03.758666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985370+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:10:04.985370+0000","last_peered":"2024-03-12T05:10:04.985370+0000","last_clean":"2024-03-12T05:10:04.985370+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:10:04.985370+0000","last_undegraded":"2024-03-12T05:10:04.985370+0000","last_fullsized":"2024-03-12T05:10:04.985370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748647+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:11:03.748647+0000","last_peered":"2024-03-12T05:11:03.748647+0000","last_clean":"2024-03-12T05:11:03.748647+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:11:03.748647+0000","last_undegraded":"2024-03-12T05:11:03.748647+0000","last_fullsized":"2024-03-12T05:11:03.748647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":140,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:48.156731+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:11:48.156731+0000","last_peered":"2024-03-12T05:11:48.156731+0000","last_clean":"2024-03-12T05:11:48.156731+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:11:48.156731+0000","last_undegraded":"2024-03-12T05:11:48.156731+0000","last_fullsized":"2024-03-12T05:11:48.156731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748799+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:11:03.748799+0000","last_peered":"2024-03-12T05:11:03.748799+0000","last_clean":"2024-03-12T05:11:03.748799+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:11:03.748799+0000","last_undegraded":"2024-03-12T05:11:03.748799+0000","last_fullsized":"2024-03-12T05:11:03.748799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983832+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:10:04.983832+0000","last_peered":"2024-03-12T05:10:04.983832+0000","last_clean":"2024-03-12T05:10:04.983832+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:10:04.983832+0000","last_undegraded":"2024-03-12T05:10:04.983832+0000","last_fullsized":"2024-03-12T05:10:04.983832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748879+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:11:03.748879+0000","last_peered":"2024-03-12T05:11:03.748879+0000","last_clean":"2024-03-12T05:11:03.748879+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:11:03.748879+0000","last_undegraded":"2024-03-12T05:11:03.748879+0000","last_fullsized":"2024-03-12T05:11:03.748879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.993890+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:10:04.993890+0000","last_peered":"2024-03-12T05:10:04.993890+0000","last_clean":"2024-03-12T05:10:04.993890+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:10:04.993890+0000","last_undegraded":"2024-03-12T05:10:04.993890+0000","last_fullsized":"2024-03-12T05:10:04.993890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745430+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:11:03.745430+0000","last_peered":"2024-03-12T05:11:03.745430+0000","last_clean":"2024-03-12T05:11:03.745430+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:11:03.745430+0000","last_undegraded":"2024-03-12T05:11:03.745430+0000","last_fullsized":"2024-03-12T05:11:03.745430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990178+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:10:04.990178+0000","last_peered":"2024-03-12T05:10:04.990178+0000","last_clean":"2024-03-12T05:10:04.990178+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:10:04.990178+0000","last_undegraded":"2024-03-12T05:10:04.990178+0000","last_fullsized":"2024-03-12T05:10:04.990178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748530+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:11:03.748530+0000","last_peered":"2024-03-12T05:11:03.748530+0000","last_clean":"2024-03-12T05:11:03.748530+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:11:03.748530+0000","last_undegraded":"2024-03-12T05:11:03.748530+0000","last_fullsized":"2024-03-12T05:11:03.748530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:49.158080+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:11:49.158080+0000","last_peered":"2024-03-12T05:11:49.158080+0000","last_clean":"2024-03-12T05:11:49.158080+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:11:49.158080+0000","last_undegraded":"2024-03-12T05:11:49.158080+0000","last_fullsized":"2024-03-12T05:11:49.158080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.756261+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:11:03.756261+0000","last_peered":"2024-03-12T05:11:03.756261+0000","last_clean":"2024-03-12T05:11:03.756261+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:11:03.756261+0000","last_undegraded":"2024-03-12T05:11:03.756261+0000","last_fullsized":"2024-03-12T05:11:03.756261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981550+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:10:04.981550+0000","last_peered":"2024-03-12T05:10:04.981550+0000","last_clean":"2024-03-12T05:10:04.981550+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:10:04.981550+0000","last_undegraded":"2024-03-12T05:10:04.981550+0000","last_fullsized":"2024-03-12T05:10:04.981550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748739+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:11:03.748739+0000","last_peered":"2024-03-12T05:11:03.748739+0000","last_clean":"2024-03-12T05:11:03.748739+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:11:03.748739+0000","last_undegraded":"2024-03-12T05:11:03.748739+0000","last_fullsized":"2024-03-12T05:11:03.748739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990717+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:10:04.990717+0000","last_peered":"2024-03-12T05:10:04.990717+0000","last_clean":"2024-03-12T05:10:04.990717+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:10:04.990717+0000","last_undegraded":"2024-03-12T05:10:04.990717+0000","last_fullsized":"2024-03-12T05:10:04.990717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.759675+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:11:03.759675+0000","last_peered":"2024-03-12T05:11:03.759675+0000","last_clean":"2024-03-12T05:11:03.759675+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:11:03.759675+0000","last_undegraded":"2024-03-12T05:11:03.759675+0000","last_fullsized":"2024-03-12T05:11:03.759675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":283,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:50.387679+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:11:50.387679+0000","last_peered":"2024-03-12T05:11:50.387679+0000","last_clean":"2024-03-12T05:11:50.387679+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:11:50.387679+0000","last_undegraded":"2024-03-12T05:11:50.387679+0000","last_fullsized":"2024-03-12T05:11:50.387679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61603,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":231,"num_read_kb":175,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751355+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:11:03.751355+0000","last_peered":"2024-03-12T05:11:03.751355+0000","last_clean":"2024-03-12T05:11:03.751355+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:11:03.751355+0000","last_undegraded":"2024-03-12T05:11:03.751355+0000","last_fullsized":"2024-03-12T05:11:03.751355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.998175+0000","last_change":"2024-03-12T05:10:04.997655+0000","last_active":"2024-03-12T05:10:04.998175+0000","last_peered":"2024-03-12T05:10:04.998175+0000","last_clean":"2024-03-12T05:10:04.998175+0000","last_became_active":"2024-03-12T05:10:04.985974+0000","last_became_peered":"2024-03-12T05:10:04.985974+0000","last_unstale":"2024-03-12T05:10:04.998175+0000","last_undegraded":"2024-03-12T05:10:04.998175+0000","last_fullsized":"2024-03-12T05:10:04.998175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:50:52.646109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751063+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:11:03.751063+0000","last_peered":"2024-03-12T05:11:03.751063+0000","last_clean":"2024-03-12T05:11:03.751063+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:11:03.751063+0000","last_undegraded":"2024-03-12T05:11:03.751063+0000","last_fullsized":"2024-03-12T05:11:03.751063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.989829+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:10:04.989829+0000","last_peered":"2024-03-12T05:10:04.989829+0000","last_clean":"2024-03-12T05:10:04.989829+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:10:04.989829+0000","last_undegraded":"2024-03-12T05:10:04.989829+0000","last_fullsized":"2024-03-12T05:10:04.989829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750601+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:11:03.750601+0000","last_peered":"2024-03-12T05:11:03.750601+0000","last_clean":"2024-03-12T05:11:03.750601+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:11:03.750601+0000","last_undegraded":"2024-03-12T05:11:03.750601+0000","last_fullsized":"2024-03-12T05:11:03.750601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":139,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:49.159185+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:11:49.159185+0000","last_peered":"2024-03-12T05:11:49.159185+0000","last_clean":"2024-03-12T05:11:49.159185+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:11:49.159185+0000","last_undegraded":"2024-03-12T05:11:49.159185+0000","last_fullsized":"2024-03-12T05:11:49.159185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751248+0000","last_change":"2024-03-12T05:10:04.985396+0000","last_active":"2024-03-12T05:11:03.751248+0000","last_peered":"2024-03-12T05:11:03.751248+0000","last_clean":"2024-03-12T05:11:03.751248+0000","last_became_active":"2024-03-12T05:10:04.984994+0000","last_became_peered":"2024-03-12T05:10:04.984994+0000","last_unstale":"2024-03-12T05:11:03.751248+0000","last_undegraded":"2024-03-12T05:11:03.751248+0000","last_fullsized":"2024-03-12T05:11:03.751248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:26:42.993458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984101+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:10:04.984101+0000","last_peered":"2024-03-12T05:10:04.984101+0000","last_clean":"2024-03-12T05:10:04.984101+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:10:04.984101+0000","last_undegraded":"2024-03-12T05:10:04.984101+0000","last_fullsized":"2024-03-12T05:10:04.984101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.738306+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:11:03.738306+0000","last_peered":"2024-03-12T05:11:03.738306+0000","last_clean":"2024-03-12T05:11:03.738306+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:11:03.738306+0000","last_undegraded":"2024-03-12T05:11:03.738306+0000","last_fullsized":"2024-03-12T05:11:03.738306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984702+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:10:04.984702+0000","last_peered":"2024-03-12T05:10:04.984702+0000","last_clean":"2024-03-12T05:10:04.984702+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:10:04.984702+0000","last_undegraded":"2024-03-12T05:10:04.984702+0000","last_fullsized":"2024-03-12T05:10:04.984702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.742201+0000","last_change":"2024-03-12T05:10:04.995448+0000","last_active":"2024-03-12T05:11:03.742201+0000","last_peered":"2024-03-12T05:11:03.742201+0000","last_clean":"2024-03-12T05:11:03.742201+0000","last_became_active":"2024-03-12T05:10:04.995056+0000","last_became_peered":"2024-03-12T05:10:04.995056+0000","last_unstale":"2024-03-12T05:11:03.742201+0000","last_undegraded":"2024-03-12T05:11:03.742201+0000","last_fullsized":"2024-03-12T05:11:03.742201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:23:27.643478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983622+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:10:04.983622+0000","last_peered":"2024-03-12T05:10:04.983622+0000","last_clean":"2024-03-12T05:10:04.983622+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:10:04.983622+0000","last_undegraded":"2024-03-12T05:10:04.983622+0000","last_fullsized":"2024-03-12T05:10:04.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747337+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:11:03.747337+0000","last_peered":"2024-03-12T05:11:03.747337+0000","last_clean":"2024-03-12T05:11:03.747337+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:11:03.747337+0000","last_undegraded":"2024-03-12T05:11:03.747337+0000","last_fullsized":"2024-03-12T05:11:03.747337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981281+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:10:04.981281+0000","last_peered":"2024-03-12T05:10:04.981281+0000","last_clean":"2024-03-12T05:10:04.981281+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:10:04.981281+0000","last_undegraded":"2024-03-12T05:10:04.981281+0000","last_fullsized":"2024-03-12T05:10:04.981281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737885+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:11:03.737885+0000","last_peered":"2024-03-12T05:11:03.737885+0000","last_clean":"2024-03-12T05:11:03.737885+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:11:03.737885+0000","last_undegraded":"2024-03-12T05:11:03.737885+0000","last_fullsized":"2024-03-12T05:11:03.737885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745048+0000","last_change":"2024-03-12T05:10:04.997291+0000","last_active":"2024-03-12T05:11:03.745048+0000","last_peered":"2024-03-12T05:11:03.745048+0000","last_clean":"2024-03-12T05:11:03.745048+0000","last_became_active":"2024-03-12T05:10:04.996814+0000","last_became_peered":"2024-03-12T05:10:04.996814+0000","last_unstale":"2024-03-12T05:11:03.745048+0000","last_undegraded":"2024-03-12T05:11:03.745048+0000","last_fullsized":"2024-03-12T05:11:03.745048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:42.585416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:48.157450+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:11:48.157450+0000","last_peered":"2024-03-12T05:11:48.157450+0000","last_clean":"2024-03-12T05:11:48.157450+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:11:48.157450+0000","last_undegraded":"2024-03-12T05:11:48.157450+0000","last_fullsized":"2024-03-12T05:11:48.157450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746635+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:11:03.746635+0000","last_peered":"2024-03-12T05:11:03.746635+0000","last_clean":"2024-03-12T05:11:03.746635+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:11:03.746635+0000","last_undegraded":"2024-03-12T05:11:03.746635+0000","last_fullsized":"2024-03-12T05:11:03.746635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996155+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:10:04.996155+0000","last_peered":"2024-03-12T05:10:04.996155+0000","last_clean":"2024-03-12T05:10:04.996155+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:10:04.996155+0000","last_undegraded":"2024-03-12T05:10:04.996155+0000","last_fullsized":"2024-03-12T05:10:04.996155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981209+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:10:04.981209+0000","last_peered":"2024-03-12T05:10:04.981209+0000","last_clean":"2024-03-12T05:10:04.981209+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:10:04.981209+0000","last_undegraded":"2024-03-12T05:10:04.981209+0000","last_fullsized":"2024-03-12T05:10:04.981209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748237+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:11:03.748237+0000","last_peered":"2024-03-12T05:11:03.748237+0000","last_clean":"2024-03-12T05:11:03.748237+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:11:03.748237+0000","last_undegraded":"2024-03-12T05:11:03.748237+0000","last_fullsized":"2024-03-12T05:11:03.748237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981470+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:10:04.981470+0000","last_peered":"2024-03-12T05:10:04.981470+0000","last_clean":"2024-03-12T05:10:04.981470+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:10:04.981470+0000","last_undegraded":"2024-03-12T05:10:04.981470+0000","last_fullsized":"2024-03-12T05:10:04.981470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984811+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:10:04.984811+0000","last_peered":"2024-03-12T05:10:04.984811+0000","last_clean":"2024-03-12T05:10:04.984811+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:10:04.984811+0000","last_undegraded":"2024-03-12T05:10:04.984811+0000","last_fullsized":"2024-03-12T05:10:04.984811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744189+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:11:03.744189+0000","last_peered":"2024-03-12T05:11:03.744189+0000","last_clean":"2024-03-12T05:11:03.744189+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:11:03.744189+0000","last_undegraded":"2024-03-12T05:11:03.744189+0000","last_fullsized":"2024-03-12T05:11:03.744189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.941869+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:11:03.941869+0000","last_peered":"2024-03-12T05:11:03.941869+0000","last_clean":"2024-03-12T05:11:03.941869+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:11:03.941869+0000","last_undegraded":"2024-03-12T05:11:03.941869+0000","last_fullsized":"2024-03-12T05:11:03.941869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750882+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:11:03.750882+0000","last_peered":"2024-03-12T05:11:03.750882+0000","last_clean":"2024-03-12T05:11:03.750882+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:11:03.750882+0000","last_undegraded":"2024-03-12T05:11:03.750882+0000","last_fullsized":"2024-03-12T05:11:03.750882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996107+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:10:04.996107+0000","last_peered":"2024-03-12T05:10:04.996107+0000","last_clean":"2024-03-12T05:10:04.996107+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:10:04.996107+0000","last_undegraded":"2024-03-12T05:10:04.996107+0000","last_fullsized":"2024-03-12T05:10:04.996107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748414+0000","last_change":"2024-03-12T05:10:04.997304+0000","last_active":"2024-03-12T05:11:03.748414+0000","last_peered":"2024-03-12T05:11:03.748414+0000","last_clean":"2024-03-12T05:11:03.748414+0000","last_became_active":"2024-03-12T05:10:04.996794+0000","last_became_peered":"2024-03-12T05:10:04.996794+0000","last_unstale":"2024-03-12T05:11:03.748414+0000","last_undegraded":"2024-03-12T05:11:03.748414+0000","last_fullsized":"2024-03-12T05:11:03.748414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:33.408355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.139977+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:10:05.139977+0000","last_peered":"2024-03-12T05:10:05.139977+0000","last_clean":"2024-03-12T05:10:05.139977+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:10:05.139977+0000","last_undegraded":"2024-03-12T05:10:05.139977+0000","last_fullsized":"2024-03-12T05:10:05.139977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733684+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:11:03.733684+0000","last_peered":"2024-03-12T05:11:03.733684+0000","last_clean":"2024-03-12T05:11:03.733684+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:11:03.733684+0000","last_undegraded":"2024-03-12T05:11:03.733684+0000","last_fullsized":"2024-03-12T05:11:03.733684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984298+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:10:04.984298+0000","last_peered":"2024-03-12T05:10:04.984298+0000","last_clean":"2024-03-12T05:10:04.984298+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:10:04.984298+0000","last_undegraded":"2024-03-12T05:10:04.984298+0000","last_fullsized":"2024-03-12T05:10:04.984298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749472+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:11:03.749472+0000","last_peered":"2024-03-12T05:11:03.749472+0000","last_clean":"2024-03-12T05:11:03.749472+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:11:03.749472+0000","last_undegraded":"2024-03-12T05:11:03.749472+0000","last_fullsized":"2024-03-12T05:11:03.749472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.000638+0000","last_change":"2024-03-12T05:10:05.000161+0000","last_active":"2024-03-12T05:10:05.000638+0000","last_peered":"2024-03-12T05:10:05.000638+0000","last_clean":"2024-03-12T05:10:05.000638+0000","last_became_active":"2024-03-12T05:10:04.985941+0000","last_became_peered":"2024-03-12T05:10:04.985941+0000","last_unstale":"2024-03-12T05:10:05.000638+0000","last_undegraded":"2024-03-12T05:10:05.000638+0000","last_fullsized":"2024-03-12T05:10:05.000638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:34.170134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745090+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:11:03.745090+0000","last_peered":"2024-03-12T05:11:03.745090+0000","last_clean":"2024-03-12T05:11:03.745090+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:11:03.745090+0000","last_undegraded":"2024-03-12T05:11:03.745090+0000","last_fullsized":"2024-03-12T05:11:03.745090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984347+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:10:04.984347+0000","last_peered":"2024-03-12T05:10:04.984347+0000","last_clean":"2024-03-12T05:10:04.984347+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:10:04.984347+0000","last_undegraded":"2024-03-12T05:10:04.984347+0000","last_fullsized":"2024-03-12T05:10:04.984347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758283+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:11:03.758283+0000","last_peered":"2024-03-12T05:11:03.758283+0000","last_clean":"2024-03-12T05:11:03.758283+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:11:03.758283+0000","last_undegraded":"2024-03-12T05:11:03.758283+0000","last_fullsized":"2024-03-12T05:11:03.758283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985425+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:10:04.985425+0000","last_peered":"2024-03-12T05:10:04.985425+0000","last_clean":"2024-03-12T05:10:04.985425+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:10:04.985425+0000","last_undegraded":"2024-03-12T05:10:04.985425+0000","last_fullsized":"2024-03-12T05:10:04.985425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758796+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:11:03.758796+0000","last_peered":"2024-03-12T05:11:03.758796+0000","last_clean":"2024-03-12T05:11:03.758796+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:11:03.758796+0000","last_undegraded":"2024-03-12T05:11:03.758796+0000","last_fullsized":"2024-03-12T05:11:03.758796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990547+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:10:04.990547+0000","last_peered":"2024-03-12T05:10:04.990547+0000","last_clean":"2024-03-12T05:10:04.990547+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:10:04.990547+0000","last_undegraded":"2024-03-12T05:10:04.990547+0000","last_fullsized":"2024-03-12T05:10:04.990547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749176+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:11:03.749176+0000","last_peered":"2024-03-12T05:11:03.749176+0000","last_clean":"2024-03-12T05:11:03.749176+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:11:03.749176+0000","last_undegraded":"2024-03-12T05:11:03.749176+0000","last_fullsized":"2024-03-12T05:11:03.749176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.994225+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:10:04.994225+0000","last_peered":"2024-03-12T05:10:04.994225+0000","last_clean":"2024-03-12T05:10:04.994225+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:10:04.994225+0000","last_undegraded":"2024-03-12T05:10:04.994225+0000","last_fullsized":"2024-03-12T05:10:04.994225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751718+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:11:03.751718+0000","last_peered":"2024-03-12T05:11:03.751718+0000","last_clean":"2024-03-12T05:11:03.751718+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:11:03.751718+0000","last_undegraded":"2024-03-12T05:11:03.751718+0000","last_fullsized":"2024-03-12T05:11:03.751718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749780+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:11:03.749780+0000","last_peered":"2024-03-12T05:11:03.749780+0000","last_clean":"2024-03-12T05:11:03.749780+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:11:03.749780+0000","last_undegraded":"2024-03-12T05:11:03.749780+0000","last_fullsized":"2024-03-12T05:11:03.749780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743933+0000","last_change":"2024-03-12T05:10:04.992158+0000","last_active":"2024-03-12T05:11:03.743933+0000","last_peered":"2024-03-12T05:11:03.743933+0000","last_clean":"2024-03-12T05:11:03.743933+0000","last_became_active":"2024-03-12T05:10:04.991721+0000","last_became_peered":"2024-03-12T05:10:04.991721+0000","last_unstale":"2024-03-12T05:11:03.743933+0000","last_undegraded":"2024-03-12T05:11:03.743933+0000","last_fullsized":"2024-03-12T05:11:03.743933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:24.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748140+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:11:03.748140+0000","last_peered":"2024-03-12T05:11:03.748140+0000","last_clean":"2024-03-12T05:11:03.748140+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:11:03.748140+0000","last_undegraded":"2024-03-12T05:11:03.748140+0000","last_fullsized":"2024-03-12T05:11:03.748140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748105+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:11:03.748105+0000","last_peered":"2024-03-12T05:11:03.748105+0000","last_clean":"2024-03-12T05:11:03.748105+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:11:03.748105+0000","last_undegraded":"2024-03-12T05:11:03.748105+0000","last_fullsized":"2024-03-12T05:11:03.748105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748285+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:11:03.748285+0000","last_peered":"2024-03-12T05:11:03.748285+0000","last_clean":"2024-03-12T05:11:03.748285+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:11:03.748285+0000","last_undegraded":"2024-03-12T05:11:03.748285+0000","last_fullsized":"2024-03-12T05:11:03.748285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737606+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:11:03.737606+0000","last_peered":"2024-03-12T05:11:03.737606+0000","last_clean":"2024-03-12T05:11:03.737606+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:11:03.737606+0000","last_undegraded":"2024-03-12T05:11:03.737606+0000","last_fullsized":"2024-03-12T05:11:03.737606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743523+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:11:03.743523+0000","last_peered":"2024-03-12T05:11:03.743523+0000","last_clean":"2024-03-12T05:11:03.743523+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:11:03.743523+0000","last_undegraded":"2024-03-12T05:11:03.743523+0000","last_fullsized":"2024-03-12T05:11:03.743523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745476+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:11:03.745476+0000","last_peered":"2024-03-12T05:11:03.745476+0000","last_clean":"2024-03-12T05:11:03.745476+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:11:03.745476+0000","last_undegraded":"2024-03-12T05:11:03.745476+0000","last_fullsized":"2024-03-12T05:11:03.745476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744920+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:11:03.744920+0000","last_peered":"2024-03-12T05:11:03.744920+0000","last_clean":"2024-03-12T05:11:03.744920+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:11:03.744920+0000","last_undegraded":"2024-03-12T05:11:03.744920+0000","last_fullsized":"2024-03-12T05:11:03.744920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746188+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:11:03.746188+0000","last_peered":"2024-03-12T05:11:03.746188+0000","last_clean":"2024-03-12T05:11:03.746188+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:11:03.746188+0000","last_undegraded":"2024-03-12T05:11:03.746188+0000","last_fullsized":"2024-03-12T05:11:03.746188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760230+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:11:03.760230+0000","last_peered":"2024-03-12T05:11:03.760230+0000","last_clean":"2024-03-12T05:11:03.760230+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:11:03.760230+0000","last_undegraded":"2024-03-12T05:11:03.760230+0000","last_fullsized":"2024-03-12T05:11:03.760230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751293+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:11:03.751293+0000","last_peered":"2024-03-12T05:11:03.751293+0000","last_clean":"2024-03-12T05:11:03.751293+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:11:03.751293+0000","last_undegraded":"2024-03-12T05:11:03.751293+0000","last_fullsized":"2024-03-12T05:11:03.751293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747355+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:11:03.747355+0000","last_peered":"2024-03-12T05:11:03.747355+0000","last_clean":"2024-03-12T05:11:03.747355+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:11:03.747355+0000","last_undegraded":"2024-03-12T05:11:03.747355+0000","last_fullsized":"2024-03-12T05:11:03.747355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66277,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":695,"num_read_kb":418,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":205460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1113530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":460150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1053455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1024000,"data_stored":520627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":460810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":459850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":460508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1051825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:11:55.225 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T05:11:55.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:54 smithi161 ceph-mon[36043]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-12T05:11:55.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1236126393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:55.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3364850996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:55.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/58668799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.365+0000 7f4453681640 1 -- 172.21.15.100:0/2640853728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 msgr2=0x7f444c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.365+0000 7f4453681640 1 --2- 172.21.15.100:0/2640853728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c1092d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4444009ca0 tx=0x7f444402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.365+0000 7f4453681640 1 -- 172.21.15.100:0/2640853728 shutdown_connections 2024-03-12T05:11:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.365+0000 7f4453681640 1 --2- 172.21.15.100:0/2640853728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c1092d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.365+0000 7f4453681640 1 --2- 172.21.15.100:0/2640853728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 0x7f444c1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.365+0000 7f4453681640 1 --2- 172.21.15.100:0/2640853728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c101c80 0x7f444c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- 172.21.15.100:0/2640853728 >> 172.21.15.100:0/2640853728 conn(0x7f444c0fb610 msgr2=0x7f444c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- 172.21.15.100:0/2640853728 shutdown_connections 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- 172.21.15.100:0/2640853728 wait complete. 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 Processor -- start 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- start start 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c101c80 0x7f444c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 0x7f444c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c19ed90 con 0x7f444c106ee0 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c19ef00 con 0x7f444c1045b0 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4453681640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c19f0c0 con 0x7f444c101c80 2024-03-12T05:11:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4451c1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4451c1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44082/0 (socket says 172.21.15.100:44082) 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.366+0000 7f4451c1e640 1 -- 172.21.15.100:0/1559411113 learned_addr learned my addr 172.21.15.100:0/1559411113 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f445141d640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c101c80 0x7f444c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4451c1e640 1 -- 172.21.15.100:0/1559411113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c101c80 msgr2=0x7f444c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4451c1e640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c101c80 0x7f444c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4450c1c640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 0x7f444c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4451c1e640 1 -- 172.21.15.100:0/1559411113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 msgr2=0x7f444c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4451c1e640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 0x7f444c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4451c1e640 1 -- 172.21.15.100:0/1559411113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009900 con 0x7f444c106ee0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4450c1c640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 0x7f444c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4451c1e640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c19e850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4444037040 tx=0x7f4444034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440329b0 con 0x7f444c106ee0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4444003e50 con 0x7f444c106ee0 2024-03-12T05:11:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c19f340 con 0x7f444c106ee0 2024-03-12T05:11:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444040450 con 0x7f444c106ee0 2024-03-12T05:11:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.367+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c19f840 con 0x7f444c106ee0 2024-03-12T05:11:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.368+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c0fd100 con 0x7f444c106ee0 2024-03-12T05:11:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.369+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f444400a040 con 0x7f444c106ee0 2024-03-12T05:11:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.370+0000 7f44427fc640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f44380a1b40 0x7f44380a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.370+0000 7f445141d640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f44380a1b40 0x7f44380a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.370+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44440f0410 con 0x7f444c106ee0 2024-03-12T05:11:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.370+0000 7f445141d640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f44380a1b40 0x7f44380a4000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4448009aa0 tx=0x7f4448005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:55.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.374+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44440b95d0 con 0x7f444c106ee0 2024-03-12T05:11:55.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.543+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f444c0615c0 con 0x7f44380a1b40 2024-03-12T05:11:55.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.550+0000 7f44427fc640 1 -- 172.21.15.100:0/1559411113 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258840 (secure 0 0 0) 0x7f444c0615c0 con 0x7f44380a1b40 2024-03-12T05:11:55.550 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:55.551 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f44380a1b40 msgr2=0x7f44380a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f44380a1b40 0x7f44380a4000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4448009aa0 tx=0x7f4448005f90 comp rx=0 tx=0).stop 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 msgr2=0x7f444c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c19e850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4444037040 tx=0x7f4444034a90 comp rx=0 tx=0).stop 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 shutdown_connections 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f44380a1b40 0x7f44380a4000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f444c106ee0 0x7f444c19e850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f444c1045b0 0x7f444c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 --2- 172.21.15.100:0/1559411113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c101c80 0x7f444c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 >> 172.21.15.100:0/1559411113 conn(0x7f444c0fb610 msgr2=0x7f444c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 shutdown_connections 2024-03-12T05:11:55.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.553+0000 7f4453681640 1 -- 172.21.15.100:0/1559411113 wait complete. 2024-03-12T05:11:55.561 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-12T05:11:53.739187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656879,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":831,"num_read_kb":537,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374988,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566836,"statfs":{"total":1151940427776,"available":1151556440064,"internally_reserved":0,"allocated":13676544,"data_stored":7483095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002812"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744383+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:11:03.744383+0000","last_peered":"2024-03-12T05:11:03.744383+0000","last_clean":"2024-03-12T05:11:03.744383+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:11:03.744383+0000","last_undegraded":"2024-03-12T05:11:03.744383+0000","last_fullsized":"2024-03-12T05:11:03.744383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.739101+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:11:03.739101+0000","last_peered":"2024-03-12T05:11:03.739101+0000","last_clean":"2024-03-12T05:11:03.739101+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:11:03.739101+0000","last_undegraded":"2024-03-12T05:11:03.739101+0000","last_fullsized":"2024-03-12T05:11:03.739101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757927+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:11:03.757927+0000","last_peered":"2024-03-12T05:11:03.757927+0000","last_clean":"2024-03-12T05:11:03.757927+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:11:03.757927+0000","last_undegraded":"2024-03-12T05:11:03.757927+0000","last_fullsized":"2024-03-12T05:11:03.757927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733109+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:11:03.733109+0000","last_peered":"2024-03-12T05:11:03.733109+0000","last_clean":"2024-03-12T05:11:03.733109+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:11:03.733109+0000","last_undegraded":"2024-03-12T05:11:03.733109+0000","last_fullsized":"2024-03-12T05:11:03.733109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743525+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:11:03.743525+0000","last_peered":"2024-03-12T05:11:03.743525+0000","last_clean":"2024-03-12T05:11:03.743525+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:11:03.743525+0000","last_undegraded":"2024-03-12T05:11:03.743525+0000","last_fullsized":"2024-03-12T05:11:03.743525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745830+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:11:03.745830+0000","last_peered":"2024-03-12T05:11:03.745830+0000","last_clean":"2024-03-12T05:11:03.745830+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:11:03.745830+0000","last_undegraded":"2024-03-12T05:11:03.745830+0000","last_fullsized":"2024-03-12T05:11:03.745830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750437+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:11:03.750437+0000","last_peered":"2024-03-12T05:11:03.750437+0000","last_clean":"2024-03-12T05:11:03.750437+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:11:03.750437+0000","last_undegraded":"2024-03-12T05:11:03.750437+0000","last_fullsized":"2024-03-12T05:11:03.750437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747959+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:11:03.747959+0000","last_peered":"2024-03-12T05:11:03.747959+0000","last_clean":"2024-03-12T05:11:03.747959+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:11:03.747959+0000","last_undegraded":"2024-03-12T05:11:03.747959+0000","last_fullsized":"2024-03-12T05:11:03.747959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.752826+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:11:03.752826+0000","last_peered":"2024-03-12T05:11:03.752826+0000","last_clean":"2024-03-12T05:11:03.752826+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:11:03.752826+0000","last_undegraded":"2024-03-12T05:11:03.752826+0000","last_fullsized":"2024-03-12T05:11:03.752826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744896+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:11:03.744896+0000","last_peered":"2024-03-12T05:11:03.744896+0000","last_clean":"2024-03-12T05:11:03.744896+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:11:03.744896+0000","last_undegraded":"2024-03-12T05:11:03.744896+0000","last_fullsized":"2024-03-12T05:11:03.744896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758668+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:11:03.758668+0000","last_peered":"2024-03-12T05:11:03.758668+0000","last_clean":"2024-03-12T05:11:03.758668+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:11:03.758668+0000","last_undegraded":"2024-03-12T05:11:03.758668+0000","last_fullsized":"2024-03-12T05:11:03.758668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748754+0000","last_change":"2024-03-12T05:10:04.986947+0000","last_active":"2024-03-12T05:11:03.748754+0000","last_peered":"2024-03-12T05:11:03.748754+0000","last_clean":"2024-03-12T05:11:03.748754+0000","last_became_active":"2024-03-12T05:10:04.986526+0000","last_became_peered":"2024-03-12T05:10:04.986526+0000","last_unstale":"2024-03-12T05:11:03.748754+0000","last_undegraded":"2024-03-12T05:11:03.748754+0000","last_fullsized":"2024-03-12T05:11:03.748754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:30:47.611206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743287+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:11:03.743287+0000","last_peered":"2024-03-12T05:11:03.743287+0000","last_clean":"2024-03-12T05:11:03.743287+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:11:03.743287+0000","last_undegraded":"2024-03-12T05:11:03.743287+0000","last_fullsized":"2024-03-12T05:11:03.743287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749425+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:11:03.749425+0000","last_peered":"2024-03-12T05:11:03.749425+0000","last_clean":"2024-03-12T05:11:03.749425+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:11:03.749425+0000","last_undegraded":"2024-03-12T05:11:03.749425+0000","last_fullsized":"2024-03-12T05:11:03.749425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.740596+0000","last_change":"2024-03-12T05:10:04.989251+0000","last_active":"2024-03-12T05:11:03.740596+0000","last_peered":"2024-03-12T05:11:03.740596+0000","last_clean":"2024-03-12T05:11:03.740596+0000","last_became_active":"2024-03-12T05:10:04.988845+0000","last_became_peered":"2024-03-12T05:10:04.988845+0000","last_unstale":"2024-03-12T05:11:03.740596+0000","last_undegraded":"2024-03-12T05:11:03.740596+0000","last_fullsized":"2024-03-12T05:11:03.740596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:21:42.513931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760056+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:11:03.760056+0000","last_peered":"2024-03-12T05:11:03.760056+0000","last_clean":"2024-03-12T05:11:03.760056+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:11:03.760056+0000","last_undegraded":"2024-03-12T05:11:03.760056+0000","last_fullsized":"2024-03-12T05:11:03.760056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744673+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:11:03.744673+0000","last_peered":"2024-03-12T05:11:03.744673+0000","last_clean":"2024-03-12T05:11:03.744673+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:11:03.744673+0000","last_undegraded":"2024-03-12T05:11:03.744673+0000","last_fullsized":"2024-03-12T05:11:03.744673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749801+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:11:03.749801+0000","last_peered":"2024-03-12T05:11:03.749801+0000","last_clean":"2024-03-12T05:11:03.749801+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:11:03.749801+0000","last_undegraded":"2024-03-12T05:11:03.749801+0000","last_fullsized":"2024-03-12T05:11:03.749801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748000+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:11:03.748000+0000","last_peered":"2024-03-12T05:11:03.748000+0000","last_clean":"2024-03-12T05:11:03.748000+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:11:03.748000+0000","last_undegraded":"2024-03-12T05:11:03.748000+0000","last_fullsized":"2024-03-12T05:11:03.748000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983601+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:10:04.983601+0000","last_peered":"2024-03-12T05:10:04.983601+0000","last_clean":"2024-03-12T05:10:04.983601+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:10:04.983601+0000","last_undegraded":"2024-03-12T05:10:04.983601+0000","last_fullsized":"2024-03-12T05:10:04.983601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745669+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:11:03.745669+0000","last_peered":"2024-03-12T05:11:03.745669+0000","last_clean":"2024-03-12T05:11:03.745669+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:11:03.745669+0000","last_undegraded":"2024-03-12T05:11:03.745669+0000","last_fullsized":"2024-03-12T05:11:03.745669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985285+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:10:04.985285+0000","last_peered":"2024-03-12T05:10:04.985285+0000","last_clean":"2024-03-12T05:10:04.985285+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:10:04.985285+0000","last_undegraded":"2024-03-12T05:10:04.985285+0000","last_fullsized":"2024-03-12T05:10:04.985285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757822+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:11:03.757822+0000","last_peered":"2024-03-12T05:11:03.757822+0000","last_clean":"2024-03-12T05:11:03.757822+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:11:03.757822+0000","last_undegraded":"2024-03-12T05:11:03.757822+0000","last_fullsized":"2024-03-12T05:11:03.757822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985028+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:10:04.985028+0000","last_peered":"2024-03-12T05:10:04.985028+0000","last_clean":"2024-03-12T05:10:04.985028+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:10:04.985028+0000","last_undegraded":"2024-03-12T05:10:04.985028+0000","last_fullsized":"2024-03-12T05:10:04.985028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758666+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:11:03.758666+0000","last_peered":"2024-03-12T05:11:03.758666+0000","last_clean":"2024-03-12T05:11:03.758666+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:11:03.758666+0000","last_undegraded":"2024-03-12T05:11:03.758666+0000","last_fullsized":"2024-03-12T05:11:03.758666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985370+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:10:04.985370+0000","last_peered":"2024-03-12T05:10:04.985370+0000","last_clean":"2024-03-12T05:10:04.985370+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:10:04.985370+0000","last_undegraded":"2024-03-12T05:10:04.985370+0000","last_fullsized":"2024-03-12T05:10:04.985370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748647+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:11:03.748647+0000","last_peered":"2024-03-12T05:11:03.748647+0000","last_clean":"2024-03-12T05:11:03.748647+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:11:03.748647+0000","last_undegraded":"2024-03-12T05:11:03.748647+0000","last_fullsized":"2024-03-12T05:11:03.748647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":140,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:48.156731+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:11:48.156731+0000","last_peered":"2024-03-12T05:11:48.156731+0000","last_clean":"2024-03-12T05:11:48.156731+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:11:48.156731+0000","last_undegraded":"2024-03-12T05:11:48.156731+0000","last_fullsized":"2024-03-12T05:11:48.156731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748799+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:11:03.748799+0000","last_peered":"2024-03-12T05:11:03.748799+0000","last_clean":"2024-03-12T05:11:03.748799+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:11:03.748799+0000","last_undegraded":"2024-03-12T05:11:03.748799+0000","last_fullsized":"2024-03-12T05:11:03.748799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983832+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:10:04.983832+0000","last_peered":"2024-03-12T05:10:04.983832+0000","last_clean":"2024-03-12T05:10:04.983832+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:10:04.983832+0000","last_undegraded":"2024-03-12T05:10:04.983832+0000","last_fullsized":"2024-03-12T05:10:04.983832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748879+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:11:03.748879+0000","last_peered":"2024-03-12T05:11:03.748879+0000","last_clean":"2024-03-12T05:11:03.748879+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:11:03.748879+0000","last_undegraded":"2024-03-12T05:11:03.748879+0000","last_fullsized":"2024-03-12T05:11:03.748879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.993890+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:10:04.993890+0000","last_peered":"2024-03-12T05:10:04.993890+0000","last_clean":"2024-03-12T05:10:04.993890+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:10:04.993890+0000","last_undegraded":"2024-03-12T05:10:04.993890+0000","last_fullsized":"2024-03-12T05:10:04.993890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745430+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:11:03.745430+0000","last_peered":"2024-03-12T05:11:03.745430+0000","last_clean":"2024-03-12T05:11:03.745430+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:11:03.745430+0000","last_undegraded":"2024-03-12T05:11:03.745430+0000","last_fullsized":"2024-03-12T05:11:03.745430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990178+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:10:04.990178+0000","last_peered":"2024-03-12T05:10:04.990178+0000","last_clean":"2024-03-12T05:10:04.990178+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:10:04.990178+0000","last_undegraded":"2024-03-12T05:10:04.990178+0000","last_fullsized":"2024-03-12T05:10:04.990178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748530+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:11:03.748530+0000","last_peered":"2024-03-12T05:11:03.748530+0000","last_clean":"2024-03-12T05:11:03.748530+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:11:03.748530+0000","last_undegraded":"2024-03-12T05:11:03.748530+0000","last_fullsized":"2024-03-12T05:11:03.748530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:49.158080+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:11:49.158080+0000","last_peered":"2024-03-12T05:11:49.158080+0000","last_clean":"2024-03-12T05:11:49.158080+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:11:49.158080+0000","last_undegraded":"2024-03-12T05:11:49.158080+0000","last_fullsized":"2024-03-12T05:11:49.158080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.756261+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:11:03.756261+0000","last_peered":"2024-03-12T05:11:03.756261+0000","last_clean":"2024-03-12T05:11:03.756261+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:11:03.756261+0000","last_undegraded":"2024-03-12T05:11:03.756261+0000","last_fullsized":"2024-03-12T05:11:03.756261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981550+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:10:04.981550+0000","last_peered":"2024-03-12T05:10:04.981550+0000","last_clean":"2024-03-12T05:10:04.981550+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:10:04.981550+0000","last_undegraded":"2024-03-12T05:10:04.981550+0000","last_fullsized":"2024-03-12T05:10:04.981550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748739+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:11:03.748739+0000","last_peered":"2024-03-12T05:11:03.748739+0000","last_clean":"2024-03-12T05:11:03.748739+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:11:03.748739+0000","last_undegraded":"2024-03-12T05:11:03.748739+0000","last_fullsized":"2024-03-12T05:11:03.748739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990717+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:10:04.990717+0000","last_peered":"2024-03-12T05:10:04.990717+0000","last_clean":"2024-03-12T05:10:04.990717+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:10:04.990717+0000","last_undegraded":"2024-03-12T05:10:04.990717+0000","last_fullsized":"2024-03-12T05:10:04.990717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.759675+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:11:03.759675+0000","last_peered":"2024-03-12T05:11:03.759675+0000","last_clean":"2024-03-12T05:11:03.759675+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:11:03.759675+0000","last_undegraded":"2024-03-12T05:11:03.759675+0000","last_fullsized":"2024-03-12T05:11:03.759675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":283,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:50.387679+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:11:50.387679+0000","last_peered":"2024-03-12T05:11:50.387679+0000","last_clean":"2024-03-12T05:11:50.387679+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:11:50.387679+0000","last_undegraded":"2024-03-12T05:11:50.387679+0000","last_fullsized":"2024-03-12T05:11:50.387679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61603,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":231,"num_read_kb":175,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751355+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:11:03.751355+0000","last_peered":"2024-03-12T05:11:03.751355+0000","last_clean":"2024-03-12T05:11:03.751355+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:11:03.751355+0000","last_undegraded":"2024-03-12T05:11:03.751355+0000","last_fullsized":"2024-03-12T05:11:03.751355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.998175+0000","last_change":"2024-03-12T05:10:04.997655+0000","last_active":"2024-03-12T05:10:04.998175+0000","last_peered":"2024-03-12T05:10:04.998175+0000","last_clean":"2024-03-12T05:10:04.998175+0000","last_became_active":"2024-03-12T05:10:04.985974+0000","last_became_peered":"2024-03-12T05:10:04.985974+0000","last_unstale":"2024-03-12T05:10:04.998175+0000","last_undegraded":"2024-03-12T05:10:04.998175+0000","last_fullsized":"2024-03-12T05:10:04.998175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:50:52.646109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751063+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:11:03.751063+0000","last_peered":"2024-03-12T05:11:03.751063+0000","last_clean":"2024-03-12T05:11:03.751063+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:11:03.751063+0000","last_undegraded":"2024-03-12T05:11:03.751063+0000","last_fullsized":"2024-03-12T05:11:03.751063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.989829+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:10:04.989829+0000","last_peered":"2024-03-12T05:10:04.989829+0000","last_clean":"2024-03-12T05:10:04.989829+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:10:04.989829+0000","last_undegraded":"2024-03-12T05:10:04.989829+0000","last_fullsized":"2024-03-12T05:10:04.989829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750601+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:11:03.750601+0000","last_peered":"2024-03-12T05:11:03.750601+0000","last_clean":"2024-03-12T05:11:03.750601+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:11:03.750601+0000","last_undegraded":"2024-03-12T05:11:03.750601+0000","last_fullsized":"2024-03-12T05:11:03.750601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":139,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:49.159185+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:11:49.159185+0000","last_peered":"2024-03-12T05:11:49.159185+0000","last_clean":"2024-03-12T05:11:49.159185+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:11:49.159185+0000","last_undegraded":"2024-03-12T05:11:49.159185+0000","last_fullsized":"2024-03-12T05:11:49.159185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751248+0000","last_change":"2024-03-12T05:10:04.985396+0000","last_active":"2024-03-12T05:11:03.751248+0000","last_peered":"2024-03-12T05:11:03.751248+0000","last_clean":"2024-03-12T05:11:03.751248+0000","last_became_active":"2024-03-12T05:10:04.984994+0000","last_became_peered":"2024-03-12T05:10:04.984994+0000","last_unstale":"2024-03-12T05:11:03.751248+0000","last_undegraded":"2024-03-12T05:11:03.751248+0000","last_fullsized":"2024-03-12T05:11:03.751248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:26:42.993458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984101+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:10:04.984101+0000","last_peered":"2024-03-12T05:10:04.984101+0000","last_clean":"2024-03-12T05:10:04.984101+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:10:04.984101+0000","last_undegraded":"2024-03-12T05:10:04.984101+0000","last_fullsized":"2024-03-12T05:10:04.984101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.738306+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:11:03.738306+0000","last_peered":"2024-03-12T05:11:03.738306+0000","last_clean":"2024-03-12T05:11:03.738306+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:11:03.738306+0000","last_undegraded":"2024-03-12T05:11:03.738306+0000","last_fullsized":"2024-03-12T05:11:03.738306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984702+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:10:04.984702+0000","last_peered":"2024-03-12T05:10:04.984702+0000","last_clean":"2024-03-12T05:10:04.984702+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:10:04.984702+0000","last_undegraded":"2024-03-12T05:10:04.984702+0000","last_fullsized":"2024-03-12T05:10:04.984702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.742201+0000","last_change":"2024-03-12T05:10:04.995448+0000","last_active":"2024-03-12T05:11:03.742201+0000","last_peered":"2024-03-12T05:11:03.742201+0000","last_clean":"2024-03-12T05:11:03.742201+0000","last_became_active":"2024-03-12T05:10:04.995056+0000","last_became_peered":"2024-03-12T05:10:04.995056+0000","last_unstale":"2024-03-12T05:11:03.742201+0000","last_undegraded":"2024-03-12T05:11:03.742201+0000","last_fullsized":"2024-03-12T05:11:03.742201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:23:27.643478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983622+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:10:04.983622+0000","last_peered":"2024-03-12T05:10:04.983622+0000","last_clean":"2024-03-12T05:10:04.983622+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:10:04.983622+0000","last_undegraded":"2024-03-12T05:10:04.983622+0000","last_fullsized":"2024-03-12T05:10:04.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747337+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:11:03.747337+0000","last_peered":"2024-03-12T05:11:03.747337+0000","last_clean":"2024-03-12T05:11:03.747337+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:11:03.747337+0000","last_undegraded":"2024-03-12T05:11:03.747337+0000","last_fullsized":"2024-03-12T05:11:03.747337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981281+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:10:04.981281+0000","last_peered":"2024-03-12T05:10:04.981281+0000","last_clean":"2024-03-12T05:10:04.981281+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:10:04.981281+0000","last_undegraded":"2024-03-12T05:10:04.981281+0000","last_fullsized":"2024-03-12T05:10:04.981281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737885+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:11:03.737885+0000","last_peered":"2024-03-12T05:11:03.737885+0000","last_clean":"2024-03-12T05:11:03.737885+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:11:03.737885+0000","last_undegraded":"2024-03-12T05:11:03.737885+0000","last_fullsized":"2024-03-12T05:11:03.737885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745048+0000","last_change":"2024-03-12T05:10:04.997291+0000","last_active":"2024-03-12T05:11:03.745048+0000","last_peered":"2024-03-12T05:11:03.745048+0000","last_clean":"2024-03-12T05:11:03.745048+0000","last_became_active":"2024-03-12T05:10:04.996814+0000","last_became_peered":"2024-03-12T05:10:04.996814+0000","last_unstale":"2024-03-12T05:11:03.745048+0000","last_undegraded":"2024-03-12T05:11:03.745048+0000","last_fullsized":"2024-03-12T05:11:03.745048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:42.585416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:48.157450+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:11:48.157450+0000","last_peered":"2024-03-12T05:11:48.157450+0000","last_clean":"2024-03-12T05:11:48.157450+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:11:48.157450+0000","last_undegraded":"2024-03-12T05:11:48.157450+0000","last_fullsized":"2024-03-12T05:11:48.157450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746635+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:11:03.746635+0000","last_peered":"2024-03-12T05:11:03.746635+0000","last_clean":"2024-03-12T05:11:03.746635+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:11:03.746635+0000","last_undegraded":"2024-03-12T05:11:03.746635+0000","last_fullsized":"2024-03-12T05:11:03.746635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996155+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:10:04.996155+0000","last_peered":"2024-03-12T05:10:04.996155+0000","last_clean":"2024-03-12T05:10:04.996155+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:10:04.996155+0000","last_undegraded":"2024-03-12T05:10:04.996155+0000","last_fullsized":"2024-03-12T05:10:04.996155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981209+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:10:04.981209+0000","last_peered":"2024-03-12T05:10:04.981209+0000","last_clean":"2024-03-12T05:10:04.981209+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:10:04.981209+0000","last_undegraded":"2024-03-12T05:10:04.981209+0000","last_fullsized":"2024-03-12T05:10:04.981209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748237+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:11:03.748237+0000","last_peered":"2024-03-12T05:11:03.748237+0000","last_clean":"2024-03-12T05:11:03.748237+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:11:03.748237+0000","last_undegraded":"2024-03-12T05:11:03.748237+0000","last_fullsized":"2024-03-12T05:11:03.748237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981470+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:10:04.981470+0000","last_peered":"2024-03-12T05:10:04.981470+0000","last_clean":"2024-03-12T05:10:04.981470+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:10:04.981470+0000","last_undegraded":"2024-03-12T05:10:04.981470+0000","last_fullsized":"2024-03-12T05:10:04.981470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984811+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:10:04.984811+0000","last_peered":"2024-03-12T05:10:04.984811+0000","last_clean":"2024-03-12T05:10:04.984811+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:10:04.984811+0000","last_undegraded":"2024-03-12T05:10:04.984811+0000","last_fullsized":"2024-03-12T05:10:04.984811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744189+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:11:03.744189+0000","last_peered":"2024-03-12T05:11:03.744189+0000","last_clean":"2024-03-12T05:11:03.744189+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:11:03.744189+0000","last_undegraded":"2024-03-12T05:11:03.744189+0000","last_fullsized":"2024-03-12T05:11:03.744189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.941869+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:11:03.941869+0000","last_peered":"2024-03-12T05:11:03.941869+0000","last_clean":"2024-03-12T05:11:03.941869+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:11:03.941869+0000","last_undegraded":"2024-03-12T05:11:03.941869+0000","last_fullsized":"2024-03-12T05:11:03.941869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750882+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:11:03.750882+0000","last_peered":"2024-03-12T05:11:03.750882+0000","last_clean":"2024-03-12T05:11:03.750882+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:11:03.750882+0000","last_undegraded":"2024-03-12T05:11:03.750882+0000","last_fullsized":"2024-03-12T05:11:03.750882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996107+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:10:04.996107+0000","last_peered":"2024-03-12T05:10:04.996107+0000","last_clean":"2024-03-12T05:10:04.996107+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:10:04.996107+0000","last_undegraded":"2024-03-12T05:10:04.996107+0000","last_fullsized":"2024-03-12T05:10:04.996107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748414+0000","last_change":"2024-03-12T05:10:04.997304+0000","last_active":"2024-03-12T05:11:03.748414+0000","last_peered":"2024-03-12T05:11:03.748414+0000","last_clean":"2024-03-12T05:11:03.748414+0000","last_became_active":"2024-03-12T05:10:04.996794+0000","last_became_peered":"2024-03-12T05:10:04.996794+0000","last_unstale":"2024-03-12T05:11:03.748414+0000","last_undegraded":"2024-03-12T05:11:03.748414+0000","last_fullsized":"2024-03-12T05:11:03.748414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:33.408355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.139977+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:10:05.139977+0000","last_peered":"2024-03-12T05:10:05.139977+0000","last_clean":"2024-03-12T05:10:05.139977+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:10:05.139977+0000","last_undegraded":"2024-03-12T05:10:05.139977+0000","last_fullsized":"2024-03-12T05:10:05.139977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733684+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:11:03.733684+0000","last_peered":"2024-03-12T05:11:03.733684+0000","last_clean":"2024-03-12T05:11:03.733684+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:11:03.733684+0000","last_undegraded":"2024-03-12T05:11:03.733684+0000","last_fullsized":"2024-03-12T05:11:03.733684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984298+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:10:04.984298+0000","last_peered":"2024-03-12T05:10:04.984298+0000","last_clean":"2024-03-12T05:10:04.984298+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:10:04.984298+0000","last_undegraded":"2024-03-12T05:10:04.984298+0000","last_fullsized":"2024-03-12T05:10:04.984298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749472+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:11:03.749472+0000","last_peered":"2024-03-12T05:11:03.749472+0000","last_clean":"2024-03-12T05:11:03.749472+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:11:03.749472+0000","last_undegraded":"2024-03-12T05:11:03.749472+0000","last_fullsized":"2024-03-12T05:11:03.749472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.000638+0000","last_change":"2024-03-12T05:10:05.000161+0000","last_active":"2024-03-12T05:10:05.000638+0000","last_peered":"2024-03-12T05:10:05.000638+0000","last_clean":"2024-03-12T05:10:05.000638+0000","last_became_active":"2024-03-12T05:10:04.985941+0000","last_became_peered":"2024-03-12T05:10:04.985941+0000","last_unstale":"2024-03-12T05:10:05.000638+0000","last_undegraded":"2024-03-12T05:10:05.000638+0000","last_fullsized":"2024-03-12T05:10:05.000638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:34.170134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745090+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:11:03.745090+0000","last_peered":"2024-03-12T05:11:03.745090+0000","last_clean":"2024-03-12T05:11:03.745090+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:11:03.745090+0000","last_undegraded":"2024-03-12T05:11:03.745090+0000","last_fullsized":"2024-03-12T05:11:03.745090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984347+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:10:04.984347+0000","last_peered":"2024-03-12T05:10:04.984347+0000","last_clean":"2024-03-12T05:10:04.984347+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:10:04.984347+0000","last_undegraded":"2024-03-12T05:10:04.984347+0000","last_fullsized":"2024-03-12T05:10:04.984347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758283+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:11:03.758283+0000","last_peered":"2024-03-12T05:11:03.758283+0000","last_clean":"2024-03-12T05:11:03.758283+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:11:03.758283+0000","last_undegraded":"2024-03-12T05:11:03.758283+0000","last_fullsized":"2024-03-12T05:11:03.758283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985425+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:10:04.985425+0000","last_peered":"2024-03-12T05:10:04.985425+0000","last_clean":"2024-03-12T05:10:04.985425+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:10:04.985425+0000","last_undegraded":"2024-03-12T05:10:04.985425+0000","last_fullsized":"2024-03-12T05:10:04.985425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758796+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:11:03.758796+0000","last_peered":"2024-03-12T05:11:03.758796+0000","last_clean":"2024-03-12T05:11:03.758796+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:11:03.758796+0000","last_undegraded":"2024-03-12T05:11:03.758796+0000","last_fullsized":"2024-03-12T05:11:03.758796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990547+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:10:04.990547+0000","last_peered":"2024-03-12T05:10:04.990547+0000","last_clean":"2024-03-12T05:10:04.990547+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:10:04.990547+0000","last_undegraded":"2024-03-12T05:10:04.990547+0000","last_fullsized":"2024-03-12T05:10:04.990547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749176+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:11:03.749176+0000","last_peered":"2024-03-12T05:11:03.749176+0000","last_clean":"2024-03-12T05:11:03.749176+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:11:03.749176+0000","last_undegraded":"2024-03-12T05:11:03.749176+0000","last_fullsized":"2024-03-12T05:11:03.749176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.994225+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:10:04.994225+0000","last_peered":"2024-03-12T05:10:04.994225+0000","last_clean":"2024-03-12T05:10:04.994225+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:10:04.994225+0000","last_undegraded":"2024-03-12T05:10:04.994225+0000","last_fullsized":"2024-03-12T05:10:04.994225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751718+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:11:03.751718+0000","last_peered":"2024-03-12T05:11:03.751718+0000","last_clean":"2024-03-12T05:11:03.751718+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:11:03.751718+0000","last_undegraded":"2024-03-12T05:11:03.751718+0000","last_fullsized":"2024-03-12T05:11:03.751718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749780+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:11:03.749780+0000","last_peered":"2024-03-12T05:11:03.749780+0000","last_clean":"2024-03-12T05:11:03.749780+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:11:03.749780+0000","last_undegraded":"2024-03-12T05:11:03.749780+0000","last_fullsized":"2024-03-12T05:11:03.749780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743933+0000","last_change":"2024-03-12T05:10:04.992158+0000","last_active":"2024-03-12T05:11:03.743933+0000","last_peered":"2024-03-12T05:11:03.743933+0000","last_clean":"2024-03-12T05:11:03.743933+0000","last_became_active":"2024-03-12T05:10:04.991721+0000","last_became_peered":"2024-03-12T05:10:04.991721+0000","last_unstale":"2024-03-12T05:11:03.743933+0000","last_undegraded":"2024-03-12T05:11:03.743933+0000","last_fullsized":"2024-03-12T05:11:03.743933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:24.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748140+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:11:03.748140+0000","last_peered":"2024-03-12T05:11:03.748140+0000","last_clean":"2024-03-12T05:11:03.748140+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:11:03.748140+0000","last_undegraded":"2024-03-12T05:11:03.748140+0000","last_fullsized":"2024-03-12T05:11:03.748140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748105+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:11:03.748105+0000","last_peered":"2024-03-12T05:11:03.748105+0000","last_clean":"2024-03-12T05:11:03.748105+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:11:03.748105+0000","last_undegraded":"2024-03-12T05:11:03.748105+0000","last_fullsized":"2024-03-12T05:11:03.748105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748285+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:11:03.748285+0000","last_peered":"2024-03-12T05:11:03.748285+0000","last_clean":"2024-03-12T05:11:03.748285+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:11:03.748285+0000","last_undegraded":"2024-03-12T05:11:03.748285+0000","last_fullsized":"2024-03-12T05:11:03.748285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737606+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:11:03.737606+0000","last_peered":"2024-03-12T05:11:03.737606+0000","last_clean":"2024-03-12T05:11:03.737606+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:11:03.737606+0000","last_undegraded":"2024-03-12T05:11:03.737606+0000","last_fullsized":"2024-03-12T05:11:03.737606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743523+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:11:03.743523+0000","last_peered":"2024-03-12T05:11:03.743523+0000","last_clean":"2024-03-12T05:11:03.743523+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:11:03.743523+0000","last_undegraded":"2024-03-12T05:11:03.743523+0000","last_fullsized":"2024-03-12T05:11:03.743523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745476+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:11:03.745476+0000","last_peered":"2024-03-12T05:11:03.745476+0000","last_clean":"2024-03-12T05:11:03.745476+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:11:03.745476+0000","last_undegraded":"2024-03-12T05:11:03.745476+0000","last_fullsized":"2024-03-12T05:11:03.745476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744920+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:11:03.744920+0000","last_peered":"2024-03-12T05:11:03.744920+0000","last_clean":"2024-03-12T05:11:03.744920+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:11:03.744920+0000","last_undegraded":"2024-03-12T05:11:03.744920+0000","last_fullsized":"2024-03-12T05:11:03.744920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746188+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:11:03.746188+0000","last_peered":"2024-03-12T05:11:03.746188+0000","last_clean":"2024-03-12T05:11:03.746188+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:11:03.746188+0000","last_undegraded":"2024-03-12T05:11:03.746188+0000","last_fullsized":"2024-03-12T05:11:03.746188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760230+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:11:03.760230+0000","last_peered":"2024-03-12T05:11:03.760230+0000","last_clean":"2024-03-12T05:11:03.760230+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:11:03.760230+0000","last_undegraded":"2024-03-12T05:11:03.760230+0000","last_fullsized":"2024-03-12T05:11:03.760230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751293+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:11:03.751293+0000","last_peered":"2024-03-12T05:11:03.751293+0000","last_clean":"2024-03-12T05:11:03.751293+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:11:03.751293+0000","last_undegraded":"2024-03-12T05:11:03.751293+0000","last_fullsized":"2024-03-12T05:11:03.751293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747355+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:11:03.747355+0000","last_peered":"2024-03-12T05:11:03.747355+0000","last_clean":"2024-03-12T05:11:03.747355+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:11:03.747355+0000","last_undegraded":"2024-03-12T05:11:03.747355+0000","last_fullsized":"2024-03-12T05:11:03.747355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66277,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":695,"num_read_kb":418,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":205460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1113530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":460150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1053455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704656,"statfs":{"total":95995035648,"available":95953567744,"internally_reserved":0,"allocated":1024000,"data_stored":520627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":460810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":459850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":460508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1051825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:11:55.565 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-12T05:11:55.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T05:11:55.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- 172.21.15.100:0/2917339200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41011a0 msgr2=0x7fcec41015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/2917339200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41011a0 0x7fcec41015c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- 172.21.15.100:0/2917339200 shutdown_connections 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/2917339200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec4102d50 0x7fcec41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/2917339200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102390 0x7fcec4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/2917339200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41011a0 0x7fcec41015c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- 172.21.15.100:0/2917339200 >> 172.21.15.100:0/2917339200 conn(0x7fcec40fc730 msgr2=0x7fcec40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- 172.21.15.100:0/2917339200 shutdown_connections 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- 172.21.15.100:0/2917339200 wait complete. 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 Processor -- start 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- start start 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec41011a0 0x7fcec41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 0x7fcec4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102d50 0x7fcec419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419ece0 con 0x7fcec4102390 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419ee50 con 0x7fcec4102d50 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.706+0000 7fcecbd6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419f010 con 0x7fcec41011a0 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 0x7fcec4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 0x7fcec4197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44112/0 (socket says 172.21.15.100:44112) 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 -- 172.21.15.100:0/1839409805 learned_addr learned my addr 172.21.15.100:0/1839409805 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9b08640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec41011a0 0x7fcec41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fceca309640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102d50 0x7fcec419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 -- 172.21.15.100:0/1839409805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec41011a0 msgr2=0x7fcec41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec41011a0 0x7fcec41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 -- 172.21.15.100:0/1839409805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102d50 msgr2=0x7fcec419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102d50 0x7fcec419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 -- 172.21.15.100:0/1839409805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc009560 con 0x7fcec4102390 2024-03-12T05:11:55.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fceca309640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102d50 0x7fcec419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcec9307640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 0x7fcec4197730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d950 tx=0x7fceb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4009c30 con 0x7fcec4102390 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fceb4009d90 con 0x7fcec4102390 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400f7e0 con 0x7fcec4102390 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419f2f0 con 0x7fcec4102390 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.707+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec419f850 con 0x7fcec4102390 2024-03-12T05:11:55.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.708+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec41084c0 con 0x7fcec4102390 2024-03-12T05:11:55.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.710+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fceb4010ad0 con 0x7fcec4102390 2024-03-12T05:11:55.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.710+0000 7fcebaffd640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fceb00a1b40 0x7fceb00a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:55.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.711+0000 7fcec9b08640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fceb00a1b40 0x7fceb00a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:55.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.711+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fceb40cd140 con 0x7fcec4102390 2024-03-12T05:11:55.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.711+0000 7fcec9b08640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fceb00a1b40 0x7fceb00a4000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005ab0 tx=0x7fcebc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:55.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.714+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fceb4096270 con 0x7fcec4102390 2024-03-12T05:11:55.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.887+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcec402ceb0 con 0x7fceb00a1b40 2024-03-12T05:11:55.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.894+0000 7fcebaffd640 1 -- 172.21.15.100:0/1839409805 <== mgr.14152 v2:172.21.15.100:6800/327651532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258840 (secure 0 0 0) 0x7fcec402ceb0 con 0x7fceb00a1b40 2024-03-12T05:11:55.895 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:55.896 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fceb00a1b40 msgr2=0x7fceb00a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fceb00a1b40 0x7fceb00a4000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005ab0 tx=0x7fcebc005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 msgr2=0x7fcec4197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 0x7fcec4197730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d950 tx=0x7fceb400de10 comp rx=0 tx=0).stop 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 shutdown_connections 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fceb00a1b40 0x7fceb00a4000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec4102d50 0x7fcec419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4102390 0x7fcec4197730 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 --2- 172.21.15.100:0/1839409805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec41011a0 0x7fcec41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 >> 172.21.15.100:0/1839409805 conn(0x7fcec40fc730 msgr2=0x7fcec4105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:55.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 shutdown_connections 2024-03-12T05:11:55.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:55.897+0000 7fcecbd6c640 1 -- 172.21.15.100:0/1839409805 wait complete. 2024-03-12T05:11:55.905 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-03-12T05:11:55.739621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657270,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":842,"num_read_kb":543,"num_write":355,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374996,"kb_used_data":13364,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566828,"statfs":{"total":1151940427776,"available":1151556431872,"internally_reserved":0,"allocated":13684736,"data_stored":7489558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":36,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002801"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744383+0000","last_change":"2024-03-12T05:09:54.714065+0000","last_active":"2024-03-12T05:11:03.744383+0000","last_peered":"2024-03-12T05:11:03.744383+0000","last_clean":"2024-03-12T05:11:03.744383+0000","last_became_active":"2024-03-12T05:09:54.713229+0000","last_became_peered":"2024-03-12T05:09:54.713229+0000","last_unstale":"2024-03-12T05:11:03.744383+0000","last_undegraded":"2024-03-12T05:11:03.744383+0000","last_fullsized":"2024-03-12T05:11:03.744383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:21:06.573465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.739101+0000","last_change":"2024-03-12T05:09:54.702706+0000","last_active":"2024-03-12T05:11:03.739101+0000","last_peered":"2024-03-12T05:11:03.739101+0000","last_clean":"2024-03-12T05:11:03.739101+0000","last_became_active":"2024-03-12T05:09:54.702376+0000","last_became_peered":"2024-03-12T05:09:54.702376+0000","last_unstale":"2024-03-12T05:11:03.739101+0000","last_undegraded":"2024-03-12T05:11:03.739101+0000","last_fullsized":"2024-03-12T05:11:03.739101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:36:15.307331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757927+0000","last_change":"2024-03-12T05:09:54.711076+0000","last_active":"2024-03-12T05:11:03.757927+0000","last_peered":"2024-03-12T05:11:03.757927+0000","last_clean":"2024-03-12T05:11:03.757927+0000","last_became_active":"2024-03-12T05:09:54.710746+0000","last_became_peered":"2024-03-12T05:09:54.710746+0000","last_unstale":"2024-03-12T05:11:03.757927+0000","last_undegraded":"2024-03-12T05:11:03.757927+0000","last_fullsized":"2024-03-12T05:11:03.757927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:51.966695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733109+0000","last_change":"2024-03-12T05:09:54.704220+0000","last_active":"2024-03-12T05:11:03.733109+0000","last_peered":"2024-03-12T05:11:03.733109+0000","last_clean":"2024-03-12T05:11:03.733109+0000","last_became_active":"2024-03-12T05:09:54.703596+0000","last_became_peered":"2024-03-12T05:09:54.703596+0000","last_unstale":"2024-03-12T05:11:03.733109+0000","last_undegraded":"2024-03-12T05:11:03.733109+0000","last_fullsized":"2024-03-12T05:11:03.733109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:31.224452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743525+0000","last_change":"2024-03-12T05:09:54.706179+0000","last_active":"2024-03-12T05:11:03.743525+0000","last_peered":"2024-03-12T05:11:03.743525+0000","last_clean":"2024-03-12T05:11:03.743525+0000","last_became_active":"2024-03-12T05:09:54.705820+0000","last_became_peered":"2024-03-12T05:09:54.705820+0000","last_unstale":"2024-03-12T05:11:03.743525+0000","last_undegraded":"2024-03-12T05:11:03.743525+0000","last_fullsized":"2024-03-12T05:11:03.743525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.405722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745830+0000","last_change":"2024-03-12T05:09:54.699916+0000","last_active":"2024-03-12T05:11:03.745830+0000","last_peered":"2024-03-12T05:11:03.745830+0000","last_clean":"2024-03-12T05:11:03.745830+0000","last_became_active":"2024-03-12T05:09:54.699590+0000","last_became_peered":"2024-03-12T05:09:54.699590+0000","last_unstale":"2024-03-12T05:11:03.745830+0000","last_undegraded":"2024-03-12T05:11:03.745830+0000","last_fullsized":"2024-03-12T05:11:03.745830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:02:36.543426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750437+0000","last_change":"2024-03-12T05:09:54.714558+0000","last_active":"2024-03-12T05:11:03.750437+0000","last_peered":"2024-03-12T05:11:03.750437+0000","last_clean":"2024-03-12T05:11:03.750437+0000","last_became_active":"2024-03-12T05:09:54.713464+0000","last_became_peered":"2024-03-12T05:09:54.713464+0000","last_unstale":"2024-03-12T05:11:03.750437+0000","last_undegraded":"2024-03-12T05:11:03.750437+0000","last_fullsized":"2024-03-12T05:11:03.750437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:21:58.286429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747959+0000","last_change":"2024-03-12T05:09:54.704139+0000","last_active":"2024-03-12T05:11:03.747959+0000","last_peered":"2024-03-12T05:11:03.747959+0000","last_clean":"2024-03-12T05:11:03.747959+0000","last_became_active":"2024-03-12T05:09:54.703838+0000","last_became_peered":"2024-03-12T05:09:54.703838+0000","last_unstale":"2024-03-12T05:11:03.747959+0000","last_undegraded":"2024-03-12T05:11:03.747959+0000","last_fullsized":"2024-03-12T05:11:03.747959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:17:58.471367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.752826+0000","last_change":"2024-03-12T05:09:54.708961+0000","last_active":"2024-03-12T05:11:03.752826+0000","last_peered":"2024-03-12T05:11:03.752826+0000","last_clean":"2024-03-12T05:11:03.752826+0000","last_became_active":"2024-03-12T05:09:54.708645+0000","last_became_peered":"2024-03-12T05:09:54.708645+0000","last_unstale":"2024-03-12T05:11:03.752826+0000","last_undegraded":"2024-03-12T05:11:03.752826+0000","last_fullsized":"2024-03-12T05:11:03.752826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:39:00.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744896+0000","last_change":"2024-03-12T05:09:54.697979+0000","last_active":"2024-03-12T05:11:03.744896+0000","last_peered":"2024-03-12T05:11:03.744896+0000","last_clean":"2024-03-12T05:11:03.744896+0000","last_became_active":"2024-03-12T05:09:54.697386+0000","last_became_peered":"2024-03-12T05:09:54.697386+0000","last_unstale":"2024-03-12T05:11:03.744896+0000","last_undegraded":"2024-03-12T05:11:03.744896+0000","last_fullsized":"2024-03-12T05:11:03.744896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:33.449168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758668+0000","last_change":"2024-03-12T05:09:54.714844+0000","last_active":"2024-03-12T05:11:03.758668+0000","last_peered":"2024-03-12T05:11:03.758668+0000","last_clean":"2024-03-12T05:11:03.758668+0000","last_became_active":"2024-03-12T05:09:54.713420+0000","last_became_peered":"2024-03-12T05:09:54.713420+0000","last_unstale":"2024-03-12T05:11:03.758668+0000","last_undegraded":"2024-03-12T05:11:03.758668+0000","last_fullsized":"2024-03-12T05:11:03.758668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:27:59.363890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748754+0000","last_change":"2024-03-12T05:10:04.986947+0000","last_active":"2024-03-12T05:11:03.748754+0000","last_peered":"2024-03-12T05:11:03.748754+0000","last_clean":"2024-03-12T05:11:03.748754+0000","last_became_active":"2024-03-12T05:10:04.986526+0000","last_became_peered":"2024-03-12T05:10:04.986526+0000","last_unstale":"2024-03-12T05:11:03.748754+0000","last_undegraded":"2024-03-12T05:11:03.748754+0000","last_fullsized":"2024-03-12T05:11:03.748754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:30:47.611206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743287+0000","last_change":"2024-03-12T05:09:54.699450+0000","last_active":"2024-03-12T05:11:03.743287+0000","last_peered":"2024-03-12T05:11:03.743287+0000","last_clean":"2024-03-12T05:11:03.743287+0000","last_became_active":"2024-03-12T05:09:54.699108+0000","last_became_peered":"2024-03-12T05:09:54.699108+0000","last_unstale":"2024-03-12T05:11:03.743287+0000","last_undegraded":"2024-03-12T05:11:03.743287+0000","last_fullsized":"2024-03-12T05:11:03.743287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:31:36.364961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749425+0000","last_change":"2024-03-12T05:09:54.703665+0000","last_active":"2024-03-12T05:11:03.749425+0000","last_peered":"2024-03-12T05:11:03.749425+0000","last_clean":"2024-03-12T05:11:03.749425+0000","last_became_active":"2024-03-12T05:09:54.703349+0000","last_became_peered":"2024-03-12T05:09:54.703349+0000","last_unstale":"2024-03-12T05:11:03.749425+0000","last_undegraded":"2024-03-12T05:11:03.749425+0000","last_fullsized":"2024-03-12T05:11:03.749425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:18:57.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.740596+0000","last_change":"2024-03-12T05:10:04.989251+0000","last_active":"2024-03-12T05:11:03.740596+0000","last_peered":"2024-03-12T05:11:03.740596+0000","last_clean":"2024-03-12T05:11:03.740596+0000","last_became_active":"2024-03-12T05:10:04.988845+0000","last_became_peered":"2024-03-12T05:10:04.988845+0000","last_unstale":"2024-03-12T05:11:03.740596+0000","last_undegraded":"2024-03-12T05:11:03.740596+0000","last_fullsized":"2024-03-12T05:11:03.740596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:21:42.513931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760056+0000","last_change":"2024-03-12T05:09:54.709659+0000","last_active":"2024-03-12T05:11:03.760056+0000","last_peered":"2024-03-12T05:11:03.760056+0000","last_clean":"2024-03-12T05:11:03.760056+0000","last_became_active":"2024-03-12T05:09:54.709349+0000","last_became_peered":"2024-03-12T05:09:54.709349+0000","last_unstale":"2024-03-12T05:11:03.760056+0000","last_undegraded":"2024-03-12T05:11:03.760056+0000","last_fullsized":"2024-03-12T05:11:03.760056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:23:36.166357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744673+0000","last_change":"2024-03-12T05:09:54.713733+0000","last_active":"2024-03-12T05:11:03.744673+0000","last_peered":"2024-03-12T05:11:03.744673+0000","last_clean":"2024-03-12T05:11:03.744673+0000","last_became_active":"2024-03-12T05:09:54.713356+0000","last_became_peered":"2024-03-12T05:09:54.713356+0000","last_unstale":"2024-03-12T05:11:03.744673+0000","last_undegraded":"2024-03-12T05:11:03.744673+0000","last_fullsized":"2024-03-12T05:11:03.744673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:10:11.096436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749801+0000","last_change":"2024-03-12T05:09:54.700530+0000","last_active":"2024-03-12T05:11:03.749801+0000","last_peered":"2024-03-12T05:11:03.749801+0000","last_clean":"2024-03-12T05:11:03.749801+0000","last_became_active":"2024-03-12T05:09:54.700249+0000","last_became_peered":"2024-03-12T05:09:54.700249+0000","last_unstale":"2024-03-12T05:11:03.749801+0000","last_undegraded":"2024-03-12T05:11:03.749801+0000","last_fullsized":"2024-03-12T05:11:03.749801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:05:41.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748000+0000","last_change":"2024-03-12T05:09:54.706685+0000","last_active":"2024-03-12T05:11:03.748000+0000","last_peered":"2024-03-12T05:11:03.748000+0000","last_clean":"2024-03-12T05:11:03.748000+0000","last_became_active":"2024-03-12T05:09:54.706362+0000","last_became_peered":"2024-03-12T05:09:54.706362+0000","last_unstale":"2024-03-12T05:11:03.748000+0000","last_undegraded":"2024-03-12T05:11:03.748000+0000","last_fullsized":"2024-03-12T05:11:03.748000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:59:16.140104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983601+0000","last_change":"2024-03-12T05:09:53.939683+0000","last_active":"2024-03-12T05:10:04.983601+0000","last_peered":"2024-03-12T05:10:04.983601+0000","last_clean":"2024-03-12T05:10:04.983601+0000","last_became_active":"2024-03-12T05:09:53.939300+0000","last_became_peered":"2024-03-12T05:09:53.939300+0000","last_unstale":"2024-03-12T05:10:04.983601+0000","last_undegraded":"2024-03-12T05:10:04.983601+0000","last_fullsized":"2024-03-12T05:10:04.983601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:43.596025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745669+0000","last_change":"2024-03-12T05:09:54.712120+0000","last_active":"2024-03-12T05:11:03.745669+0000","last_peered":"2024-03-12T05:11:03.745669+0000","last_clean":"2024-03-12T05:11:03.745669+0000","last_became_active":"2024-03-12T05:09:54.711808+0000","last_became_peered":"2024-03-12T05:09:54.711808+0000","last_unstale":"2024-03-12T05:11:03.745669+0000","last_undegraded":"2024-03-12T05:11:03.745669+0000","last_fullsized":"2024-03-12T05:11:03.745669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:40.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985285+0000","last_change":"2024-03-12T05:09:53.942434+0000","last_active":"2024-03-12T05:10:04.985285+0000","last_peered":"2024-03-12T05:10:04.985285+0000","last_clean":"2024-03-12T05:10:04.985285+0000","last_became_active":"2024-03-12T05:09:53.942114+0000","last_became_peered":"2024-03-12T05:09:53.942114+0000","last_unstale":"2024-03-12T05:10:04.985285+0000","last_undegraded":"2024-03-12T05:10:04.985285+0000","last_fullsized":"2024-03-12T05:10:04.985285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:39.229507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.757822+0000","last_change":"2024-03-12T05:09:54.710239+0000","last_active":"2024-03-12T05:11:03.757822+0000","last_peered":"2024-03-12T05:11:03.757822+0000","last_clean":"2024-03-12T05:11:03.757822+0000","last_became_active":"2024-03-12T05:09:54.709572+0000","last_became_peered":"2024-03-12T05:09:54.709572+0000","last_unstale":"2024-03-12T05:11:03.757822+0000","last_undegraded":"2024-03-12T05:11:03.757822+0000","last_fullsized":"2024-03-12T05:11:03.757822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:30.293852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985028+0000","last_change":"2024-03-12T05:09:53.943080+0000","last_active":"2024-03-12T05:10:04.985028+0000","last_peered":"2024-03-12T05:10:04.985028+0000","last_clean":"2024-03-12T05:10:04.985028+0000","last_became_active":"2024-03-12T05:09:53.942758+0000","last_became_peered":"2024-03-12T05:09:53.942758+0000","last_unstale":"2024-03-12T05:10:04.985028+0000","last_undegraded":"2024-03-12T05:10:04.985028+0000","last_fullsized":"2024-03-12T05:10:04.985028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:01:19.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758666+0000","last_change":"2024-03-12T05:09:54.711227+0000","last_active":"2024-03-12T05:11:03.758666+0000","last_peered":"2024-03-12T05:11:03.758666+0000","last_clean":"2024-03-12T05:11:03.758666+0000","last_became_active":"2024-03-12T05:09:54.710906+0000","last_became_peered":"2024-03-12T05:09:54.710906+0000","last_unstale":"2024-03-12T05:11:03.758666+0000","last_undegraded":"2024-03-12T05:11:03.758666+0000","last_fullsized":"2024-03-12T05:11:03.758666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:11:11.031073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985370+0000","last_change":"2024-03-12T05:09:53.938308+0000","last_active":"2024-03-12T05:10:04.985370+0000","last_peered":"2024-03-12T05:10:04.985370+0000","last_clean":"2024-03-12T05:10:04.985370+0000","last_became_active":"2024-03-12T05:09:53.937993+0000","last_became_peered":"2024-03-12T05:09:53.937993+0000","last_unstale":"2024-03-12T05:10:04.985370+0000","last_undegraded":"2024-03-12T05:10:04.985370+0000","last_fullsized":"2024-03-12T05:10:04.985370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:03:23.750600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748647+0000","last_change":"2024-03-12T05:09:54.705780+0000","last_active":"2024-03-12T05:11:03.748647+0000","last_peered":"2024-03-12T05:11:03.748647+0000","last_clean":"2024-03-12T05:11:03.748647+0000","last_became_active":"2024-03-12T05:09:54.705432+0000","last_became_peered":"2024-03-12T05:09:54.705432+0000","last_unstale":"2024-03-12T05:11:03.748647+0000","last_undegraded":"2024-03-12T05:11:03.748647+0000","last_fullsized":"2024-03-12T05:11:03.748647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:34:27.907895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":147,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:53.165558+0000","last_change":"2024-03-12T05:09:53.939694+0000","last_active":"2024-03-12T05:11:53.165558+0000","last_peered":"2024-03-12T05:11:53.165558+0000","last_clean":"2024-03-12T05:11:53.165558+0000","last_became_active":"2024-03-12T05:09:53.939106+0000","last_became_peered":"2024-03-12T05:09:53.939106+0000","last_unstale":"2024-03-12T05:11:53.165558+0000","last_undegraded":"2024-03-12T05:11:53.165558+0000","last_fullsized":"2024-03-12T05:11:53.165558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:29.782195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748799+0000","last_change":"2024-03-12T05:09:54.705843+0000","last_active":"2024-03-12T05:11:03.748799+0000","last_peered":"2024-03-12T05:11:03.748799+0000","last_clean":"2024-03-12T05:11:03.748799+0000","last_became_active":"2024-03-12T05:09:54.705271+0000","last_became_peered":"2024-03-12T05:09:54.705271+0000","last_unstale":"2024-03-12T05:11:03.748799+0000","last_undegraded":"2024-03-12T05:11:03.748799+0000","last_fullsized":"2024-03-12T05:11:03.748799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:16:54.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983832+0000","last_change":"2024-03-12T05:09:53.941120+0000","last_active":"2024-03-12T05:10:04.983832+0000","last_peered":"2024-03-12T05:10:04.983832+0000","last_clean":"2024-03-12T05:10:04.983832+0000","last_became_active":"2024-03-12T05:09:53.940553+0000","last_became_peered":"2024-03-12T05:09:53.940553+0000","last_unstale":"2024-03-12T05:10:04.983832+0000","last_undegraded":"2024-03-12T05:10:04.983832+0000","last_fullsized":"2024-03-12T05:10:04.983832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:09:40.724989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748879+0000","last_change":"2024-03-12T05:09:54.707006+0000","last_active":"2024-03-12T05:11:03.748879+0000","last_peered":"2024-03-12T05:11:03.748879+0000","last_clean":"2024-03-12T05:11:03.748879+0000","last_became_active":"2024-03-12T05:09:54.706535+0000","last_became_peered":"2024-03-12T05:09:54.706535+0000","last_unstale":"2024-03-12T05:11:03.748879+0000","last_undegraded":"2024-03-12T05:11:03.748879+0000","last_fullsized":"2024-03-12T05:11:03.748879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:12:50.252774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.993890+0000","last_change":"2024-03-12T05:09:53.936299+0000","last_active":"2024-03-12T05:10:04.993890+0000","last_peered":"2024-03-12T05:10:04.993890+0000","last_clean":"2024-03-12T05:10:04.993890+0000","last_became_active":"2024-03-12T05:09:53.935825+0000","last_became_peered":"2024-03-12T05:09:53.935825+0000","last_unstale":"2024-03-12T05:10:04.993890+0000","last_undegraded":"2024-03-12T05:10:04.993890+0000","last_fullsized":"2024-03-12T05:10:04.993890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:07:09.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745430+0000","last_change":"2024-03-12T05:09:54.705419+0000","last_active":"2024-03-12T05:11:03.745430+0000","last_peered":"2024-03-12T05:11:03.745430+0000","last_clean":"2024-03-12T05:11:03.745430+0000","last_became_active":"2024-03-12T05:09:54.704223+0000","last_became_peered":"2024-03-12T05:09:54.704223+0000","last_unstale":"2024-03-12T05:11:03.745430+0000","last_undegraded":"2024-03-12T05:11:03.745430+0000","last_fullsized":"2024-03-12T05:11:03.745430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:54:25.728410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990178+0000","last_change":"2024-03-12T05:09:53.940251+0000","last_active":"2024-03-12T05:10:04.990178+0000","last_peered":"2024-03-12T05:10:04.990178+0000","last_clean":"2024-03-12T05:10:04.990178+0000","last_became_active":"2024-03-12T05:09:53.939961+0000","last_became_peered":"2024-03-12T05:09:53.939961+0000","last_unstale":"2024-03-12T05:10:04.990178+0000","last_undegraded":"2024-03-12T05:10:04.990178+0000","last_fullsized":"2024-03-12T05:10:04.990178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:03.843167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748530+0000","last_change":"2024-03-12T05:09:54.703986+0000","last_active":"2024-03-12T05:11:03.748530+0000","last_peered":"2024-03-12T05:11:03.748530+0000","last_clean":"2024-03-12T05:11:03.748530+0000","last_became_active":"2024-03-12T05:09:54.703663+0000","last_became_peered":"2024-03-12T05:09:54.703663+0000","last_unstale":"2024-03-12T05:11:03.748530+0000","last_undegraded":"2024-03-12T05:11:03.748530+0000","last_fullsized":"2024-03-12T05:11:03.748530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:48:38.942809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:49.158080+0000","last_change":"2024-03-12T05:09:53.943111+0000","last_active":"2024-03-12T05:11:49.158080+0000","last_peered":"2024-03-12T05:11:49.158080+0000","last_clean":"2024-03-12T05:11:49.158080+0000","last_became_active":"2024-03-12T05:09:53.942785+0000","last_became_peered":"2024-03-12T05:09:53.942785+0000","last_unstale":"2024-03-12T05:11:49.158080+0000","last_undegraded":"2024-03-12T05:11:49.158080+0000","last_fullsized":"2024-03-12T05:11:49.158080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:41:47.692706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.756261+0000","last_change":"2024-03-12T05:09:54.702015+0000","last_active":"2024-03-12T05:11:03.756261+0000","last_peered":"2024-03-12T05:11:03.756261+0000","last_clean":"2024-03-12T05:11:03.756261+0000","last_became_active":"2024-03-12T05:09:54.701633+0000","last_became_peered":"2024-03-12T05:09:54.701633+0000","last_unstale":"2024-03-12T05:11:03.756261+0000","last_undegraded":"2024-03-12T05:11:03.756261+0000","last_fullsized":"2024-03-12T05:11:03.756261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:07:30.135086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981550+0000","last_change":"2024-03-12T05:09:53.941178+0000","last_active":"2024-03-12T05:10:04.981550+0000","last_peered":"2024-03-12T05:10:04.981550+0000","last_clean":"2024-03-12T05:10:04.981550+0000","last_became_active":"2024-03-12T05:09:53.940832+0000","last_became_peered":"2024-03-12T05:09:53.940832+0000","last_unstale":"2024-03-12T05:10:04.981550+0000","last_undegraded":"2024-03-12T05:10:04.981550+0000","last_fullsized":"2024-03-12T05:10:04.981550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T12:35:35.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748739+0000","last_change":"2024-03-12T05:09:54.707824+0000","last_active":"2024-03-12T05:11:03.748739+0000","last_peered":"2024-03-12T05:11:03.748739+0000","last_clean":"2024-03-12T05:11:03.748739+0000","last_became_active":"2024-03-12T05:09:54.707506+0000","last_became_peered":"2024-03-12T05:09:54.707506+0000","last_unstale":"2024-03-12T05:11:03.748739+0000","last_undegraded":"2024-03-12T05:11:03.748739+0000","last_fullsized":"2024-03-12T05:11:03.748739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:14.004871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990717+0000","last_change":"2024-03-12T05:09:53.935937+0000","last_active":"2024-03-12T05:10:04.990717+0000","last_peered":"2024-03-12T05:10:04.990717+0000","last_clean":"2024-03-12T05:10:04.990717+0000","last_became_active":"2024-03-12T05:09:53.935600+0000","last_became_peered":"2024-03-12T05:09:53.935600+0000","last_unstale":"2024-03-12T05:10:04.990717+0000","last_undegraded":"2024-03-12T05:10:04.990717+0000","last_fullsized":"2024-03-12T05:10:04.990717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:44:12.405968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.759675+0000","last_change":"2024-03-12T05:09:54.711280+0000","last_active":"2024-03-12T05:11:03.759675+0000","last_peered":"2024-03-12T05:11:03.759675+0000","last_clean":"2024-03-12T05:11:03.759675+0000","last_became_active":"2024-03-12T05:09:54.710996+0000","last_became_peered":"2024-03-12T05:09:54.710996+0000","last_unstale":"2024-03-12T05:11:03.759675+0000","last_undegraded":"2024-03-12T05:11:03.759675+0000","last_fullsized":"2024-03-12T05:11:03.759675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:20:10.758541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":283,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:50.387679+0000","last_change":"2024-03-12T05:09:53.941765+0000","last_active":"2024-03-12T05:11:50.387679+0000","last_peered":"2024-03-12T05:11:50.387679+0000","last_clean":"2024-03-12T05:11:50.387679+0000","last_became_active":"2024-03-12T05:09:53.941431+0000","last_became_peered":"2024-03-12T05:09:53.941431+0000","last_unstale":"2024-03-12T05:11:50.387679+0000","last_undegraded":"2024-03-12T05:11:50.387679+0000","last_fullsized":"2024-03-12T05:11:50.387679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:01:19.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61603,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":231,"num_read_kb":175,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751355+0000","last_change":"2024-03-12T05:09:54.705047+0000","last_active":"2024-03-12T05:11:03.751355+0000","last_peered":"2024-03-12T05:11:03.751355+0000","last_clean":"2024-03-12T05:11:03.751355+0000","last_became_active":"2024-03-12T05:09:54.704750+0000","last_became_peered":"2024-03-12T05:09:54.704750+0000","last_unstale":"2024-03-12T05:11:03.751355+0000","last_undegraded":"2024-03-12T05:11:03.751355+0000","last_fullsized":"2024-03-12T05:11:03.751355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:45:47.056125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.998175+0000","last_change":"2024-03-12T05:10:04.997655+0000","last_active":"2024-03-12T05:10:04.998175+0000","last_peered":"2024-03-12T05:10:04.998175+0000","last_clean":"2024-03-12T05:10:04.998175+0000","last_became_active":"2024-03-12T05:10:04.985974+0000","last_became_peered":"2024-03-12T05:10:04.985974+0000","last_unstale":"2024-03-12T05:10:04.998175+0000","last_undegraded":"2024-03-12T05:10:04.998175+0000","last_fullsized":"2024-03-12T05:10:04.998175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:50:52.646109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751063+0000","last_change":"2024-03-12T05:09:54.706564+0000","last_active":"2024-03-12T05:11:03.751063+0000","last_peered":"2024-03-12T05:11:03.751063+0000","last_clean":"2024-03-12T05:11:03.751063+0000","last_became_active":"2024-03-12T05:09:54.706263+0000","last_became_peered":"2024-03-12T05:09:54.706263+0000","last_unstale":"2024-03-12T05:11:03.751063+0000","last_undegraded":"2024-03-12T05:11:03.751063+0000","last_fullsized":"2024-03-12T05:11:03.751063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:37:12.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.989829+0000","last_change":"2024-03-12T05:09:53.938188+0000","last_active":"2024-03-12T05:10:04.989829+0000","last_peered":"2024-03-12T05:10:04.989829+0000","last_clean":"2024-03-12T05:10:04.989829+0000","last_became_active":"2024-03-12T05:09:53.937919+0000","last_became_peered":"2024-03-12T05:09:53.937919+0000","last_unstale":"2024-03-12T05:10:04.989829+0000","last_undegraded":"2024-03-12T05:10:04.989829+0000","last_fullsized":"2024-03-12T05:10:04.989829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:55:44.077721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750601+0000","last_change":"2024-03-12T05:09:54.706993+0000","last_active":"2024-03-12T05:11:03.750601+0000","last_peered":"2024-03-12T05:11:03.750601+0000","last_clean":"2024-03-12T05:11:03.750601+0000","last_became_active":"2024-03-12T05:09:54.706333+0000","last_became_peered":"2024-03-12T05:09:54.706333+0000","last_unstale":"2024-03-12T05:11:03.750601+0000","last_undegraded":"2024-03-12T05:11:03.750601+0000","last_fullsized":"2024-03-12T05:11:03.750601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:15.791409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":139,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:49.159185+0000","last_change":"2024-03-12T05:09:53.939355+0000","last_active":"2024-03-12T05:11:49.159185+0000","last_peered":"2024-03-12T05:11:49.159185+0000","last_clean":"2024-03-12T05:11:49.159185+0000","last_became_active":"2024-03-12T05:09:53.938959+0000","last_became_peered":"2024-03-12T05:09:53.938959+0000","last_unstale":"2024-03-12T05:11:49.159185+0000","last_undegraded":"2024-03-12T05:11:49.159185+0000","last_fullsized":"2024-03-12T05:11:49.159185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:24:06.526045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751248+0000","last_change":"2024-03-12T05:10:04.985396+0000","last_active":"2024-03-12T05:11:03.751248+0000","last_peered":"2024-03-12T05:11:03.751248+0000","last_clean":"2024-03-12T05:11:03.751248+0000","last_became_active":"2024-03-12T05:10:04.984994+0000","last_became_peered":"2024-03-12T05:10:04.984994+0000","last_unstale":"2024-03-12T05:11:03.751248+0000","last_undegraded":"2024-03-12T05:11:03.751248+0000","last_fullsized":"2024-03-12T05:11:03.751248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:26:42.993458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984101+0000","last_change":"2024-03-12T05:09:53.942301+0000","last_active":"2024-03-12T05:10:04.984101+0000","last_peered":"2024-03-12T05:10:04.984101+0000","last_clean":"2024-03-12T05:10:04.984101+0000","last_became_active":"2024-03-12T05:09:53.941299+0000","last_became_peered":"2024-03-12T05:09:53.941299+0000","last_unstale":"2024-03-12T05:10:04.984101+0000","last_undegraded":"2024-03-12T05:10:04.984101+0000","last_fullsized":"2024-03-12T05:10:04.984101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:24:27.315269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.738306+0000","last_change":"2024-03-12T05:09:54.708217+0000","last_active":"2024-03-12T05:11:03.738306+0000","last_peered":"2024-03-12T05:11:03.738306+0000","last_clean":"2024-03-12T05:11:03.738306+0000","last_became_active":"2024-03-12T05:09:54.707884+0000","last_became_peered":"2024-03-12T05:09:54.707884+0000","last_unstale":"2024-03-12T05:11:03.738306+0000","last_undegraded":"2024-03-12T05:11:03.738306+0000","last_fullsized":"2024-03-12T05:11:03.738306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:55:49.048300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984702+0000","last_change":"2024-03-12T05:09:53.937270+0000","last_active":"2024-03-12T05:10:04.984702+0000","last_peered":"2024-03-12T05:10:04.984702+0000","last_clean":"2024-03-12T05:10:04.984702+0000","last_became_active":"2024-03-12T05:09:53.936927+0000","last_became_peered":"2024-03-12T05:09:53.936927+0000","last_unstale":"2024-03-12T05:10:04.984702+0000","last_undegraded":"2024-03-12T05:10:04.984702+0000","last_fullsized":"2024-03-12T05:10:04.984702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:41:50.335674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.742201+0000","last_change":"2024-03-12T05:10:04.995448+0000","last_active":"2024-03-12T05:11:03.742201+0000","last_peered":"2024-03-12T05:11:03.742201+0000","last_clean":"2024-03-12T05:11:03.742201+0000","last_became_active":"2024-03-12T05:10:04.995056+0000","last_became_peered":"2024-03-12T05:10:04.995056+0000","last_unstale":"2024-03-12T05:11:03.742201+0000","last_undegraded":"2024-03-12T05:11:03.742201+0000","last_fullsized":"2024-03-12T05:11:03.742201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:23:27.643478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.983622+0000","last_change":"2024-03-12T05:09:53.940516+0000","last_active":"2024-03-12T05:10:04.983622+0000","last_peered":"2024-03-12T05:10:04.983622+0000","last_clean":"2024-03-12T05:10:04.983622+0000","last_became_active":"2024-03-12T05:09:53.939912+0000","last_became_peered":"2024-03-12T05:09:53.939912+0000","last_unstale":"2024-03-12T05:10:04.983622+0000","last_undegraded":"2024-03-12T05:10:04.983622+0000","last_fullsized":"2024-03-12T05:10:04.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:20.794723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747337+0000","last_change":"2024-03-12T05:09:54.706692+0000","last_active":"2024-03-12T05:11:03.747337+0000","last_peered":"2024-03-12T05:11:03.747337+0000","last_clean":"2024-03-12T05:11:03.747337+0000","last_became_active":"2024-03-12T05:09:54.706247+0000","last_became_peered":"2024-03-12T05:09:54.706247+0000","last_unstale":"2024-03-12T05:11:03.747337+0000","last_undegraded":"2024-03-12T05:11:03.747337+0000","last_fullsized":"2024-03-12T05:11:03.747337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:41:29.534023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981281+0000","last_change":"2024-03-12T05:09:53.940811+0000","last_active":"2024-03-12T05:10:04.981281+0000","last_peered":"2024-03-12T05:10:04.981281+0000","last_clean":"2024-03-12T05:10:04.981281+0000","last_became_active":"2024-03-12T05:09:53.940509+0000","last_became_peered":"2024-03-12T05:09:53.940509+0000","last_unstale":"2024-03-12T05:10:04.981281+0000","last_undegraded":"2024-03-12T05:10:04.981281+0000","last_fullsized":"2024-03-12T05:10:04.981281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:44.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737885+0000","last_change":"2024-03-12T05:09:54.698923+0000","last_active":"2024-03-12T05:11:03.737885+0000","last_peered":"2024-03-12T05:11:03.737885+0000","last_clean":"2024-03-12T05:11:03.737885+0000","last_became_active":"2024-03-12T05:09:54.698620+0000","last_became_peered":"2024-03-12T05:09:54.698620+0000","last_unstale":"2024-03-12T05:11:03.737885+0000","last_undegraded":"2024-03-12T05:11:03.737885+0000","last_fullsized":"2024-03-12T05:11:03.737885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:39:49.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745048+0000","last_change":"2024-03-12T05:10:04.997291+0000","last_active":"2024-03-12T05:11:03.745048+0000","last_peered":"2024-03-12T05:11:03.745048+0000","last_clean":"2024-03-12T05:11:03.745048+0000","last_became_active":"2024-03-12T05:10:04.996814+0000","last_became_peered":"2024-03-12T05:10:04.996814+0000","last_unstale":"2024-03-12T05:11:03.745048+0000","last_undegraded":"2024-03-12T05:11:03.745048+0000","last_fullsized":"2024-03-12T05:11:03.745048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:39:42.585416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":140,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:53.164675+0000","last_change":"2024-03-12T05:09:53.935367+0000","last_active":"2024-03-12T05:11:53.164675+0000","last_peered":"2024-03-12T05:11:53.164675+0000","last_clean":"2024-03-12T05:11:53.164675+0000","last_became_active":"2024-03-12T05:09:53.934855+0000","last_became_peered":"2024-03-12T05:09:53.934855+0000","last_unstale":"2024-03-12T05:11:53.164675+0000","last_undegraded":"2024-03-12T05:11:53.164675+0000","last_fullsized":"2024-03-12T05:11:53.164675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:37:09.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746635+0000","last_change":"2024-03-12T05:09:54.708774+0000","last_active":"2024-03-12T05:11:03.746635+0000","last_peered":"2024-03-12T05:11:03.746635+0000","last_clean":"2024-03-12T05:11:03.746635+0000","last_became_active":"2024-03-12T05:09:54.708523+0000","last_became_peered":"2024-03-12T05:09:54.708523+0000","last_unstale":"2024-03-12T05:11:03.746635+0000","last_undegraded":"2024-03-12T05:11:03.746635+0000","last_fullsized":"2024-03-12T05:11:03.746635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:35.346595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996155+0000","last_change":"2024-03-12T05:09:53.936325+0000","last_active":"2024-03-12T05:10:04.996155+0000","last_peered":"2024-03-12T05:10:04.996155+0000","last_clean":"2024-03-12T05:10:04.996155+0000","last_became_active":"2024-03-12T05:09:53.935909+0000","last_became_peered":"2024-03-12T05:09:53.935909+0000","last_unstale":"2024-03-12T05:10:04.996155+0000","last_undegraded":"2024-03-12T05:10:04.996155+0000","last_fullsized":"2024-03-12T05:10:04.996155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:24:51.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981209+0000","last_change":"2024-03-12T05:08:43.695655+0000","last_active":"2024-03-12T05:10:04.981209+0000","last_peered":"2024-03-12T05:10:04.981209+0000","last_clean":"2024-03-12T05:10:04.981209+0000","last_became_active":"2024-03-12T05:08:43.695213+0000","last_became_peered":"2024-03-12T05:08:43.695213+0000","last_unstale":"2024-03-12T05:10:04.981209+0000","last_undegraded":"2024-03-12T05:10:04.981209+0000","last_fullsized":"2024-03-12T05:10:04.981209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:05:27.979663+0000","last_clean_scrub_stamp":"2024-03-12T05:05:27.979663+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:53:25.917143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748237+0000","last_change":"2024-03-12T05:09:54.700894+0000","last_active":"2024-03-12T05:11:03.748237+0000","last_peered":"2024-03-12T05:11:03.748237+0000","last_clean":"2024-03-12T05:11:03.748237+0000","last_became_active":"2024-03-12T05:09:54.700478+0000","last_became_peered":"2024-03-12T05:09:54.700478+0000","last_unstale":"2024-03-12T05:11:03.748237+0000","last_undegraded":"2024-03-12T05:11:03.748237+0000","last_fullsized":"2024-03-12T05:11:03.748237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:23:00.794501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.981470+0000","last_change":"2024-03-12T05:09:53.937603+0000","last_active":"2024-03-12T05:10:04.981470+0000","last_peered":"2024-03-12T05:10:04.981470+0000","last_clean":"2024-03-12T05:10:04.981470+0000","last_became_active":"2024-03-12T05:09:53.937264+0000","last_became_peered":"2024-03-12T05:09:53.937264+0000","last_unstale":"2024-03-12T05:10:04.981470+0000","last_undegraded":"2024-03-12T05:10:04.981470+0000","last_fullsized":"2024-03-12T05:10:04.981470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:43:05.950744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984811+0000","last_change":"2024-03-12T05:09:53.940522+0000","last_active":"2024-03-12T05:10:04.984811+0000","last_peered":"2024-03-12T05:10:04.984811+0000","last_clean":"2024-03-12T05:10:04.984811+0000","last_became_active":"2024-03-12T05:09:53.940150+0000","last_became_peered":"2024-03-12T05:09:53.940150+0000","last_unstale":"2024-03-12T05:10:04.984811+0000","last_undegraded":"2024-03-12T05:10:04.984811+0000","last_fullsized":"2024-03-12T05:10:04.984811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:16:44.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744189+0000","last_change":"2024-03-12T05:09:54.696222+0000","last_active":"2024-03-12T05:11:03.744189+0000","last_peered":"2024-03-12T05:11:03.744189+0000","last_clean":"2024-03-12T05:11:03.744189+0000","last_became_active":"2024-03-12T05:09:54.695860+0000","last_became_peered":"2024-03-12T05:09:54.695860+0000","last_unstale":"2024-03-12T05:11:03.744189+0000","last_undegraded":"2024-03-12T05:11:03.744189+0000","last_fullsized":"2024-03-12T05:11:03.744189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:51.651193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.941869+0000","last_change":"2024-03-12T05:09:53.937184+0000","last_active":"2024-03-12T05:11:03.941869+0000","last_peered":"2024-03-12T05:11:03.941869+0000","last_clean":"2024-03-12T05:11:03.941869+0000","last_became_active":"2024-03-12T05:09:53.936386+0000","last_became_peered":"2024-03-12T05:09:53.936386+0000","last_unstale":"2024-03-12T05:11:03.941869+0000","last_undegraded":"2024-03-12T05:11:03.941869+0000","last_fullsized":"2024-03-12T05:11:03.941869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:57:06.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.750882+0000","last_change":"2024-03-12T05:09:54.712804+0000","last_active":"2024-03-12T05:11:03.750882+0000","last_peered":"2024-03-12T05:11:03.750882+0000","last_clean":"2024-03-12T05:11:03.750882+0000","last_became_active":"2024-03-12T05:09:54.712449+0000","last_became_peered":"2024-03-12T05:09:54.712449+0000","last_unstale":"2024-03-12T05:11:03.750882+0000","last_undegraded":"2024-03-12T05:11:03.750882+0000","last_fullsized":"2024-03-12T05:11:03.750882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:13:49.290488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.996107+0000","last_change":"2024-03-12T05:09:53.939149+0000","last_active":"2024-03-12T05:10:04.996107+0000","last_peered":"2024-03-12T05:10:04.996107+0000","last_clean":"2024-03-12T05:10:04.996107+0000","last_became_active":"2024-03-12T05:09:53.938746+0000","last_became_peered":"2024-03-12T05:09:53.938746+0000","last_unstale":"2024-03-12T05:10:04.996107+0000","last_undegraded":"2024-03-12T05:10:04.996107+0000","last_fullsized":"2024-03-12T05:10:04.996107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:50:13.174562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748414+0000","last_change":"2024-03-12T05:10:04.997304+0000","last_active":"2024-03-12T05:11:03.748414+0000","last_peered":"2024-03-12T05:11:03.748414+0000","last_clean":"2024-03-12T05:11:03.748414+0000","last_became_active":"2024-03-12T05:10:04.996794+0000","last_became_peered":"2024-03-12T05:10:04.996794+0000","last_unstale":"2024-03-12T05:11:03.748414+0000","last_undegraded":"2024-03-12T05:11:03.748414+0000","last_fullsized":"2024-03-12T05:11:03.748414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T15:43:33.408355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.139977+0000","last_change":"2024-03-12T05:09:53.938488+0000","last_active":"2024-03-12T05:10:05.139977+0000","last_peered":"2024-03-12T05:10:05.139977+0000","last_clean":"2024-03-12T05:10:05.139977+0000","last_became_active":"2024-03-12T05:09:53.938180+0000","last_became_peered":"2024-03-12T05:09:53.938180+0000","last_unstale":"2024-03-12T05:10:05.139977+0000","last_undegraded":"2024-03-12T05:10:05.139977+0000","last_fullsized":"2024-03-12T05:10:05.139977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:19:29.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.733684+0000","last_change":"2024-03-12T05:09:54.700610+0000","last_active":"2024-03-12T05:11:03.733684+0000","last_peered":"2024-03-12T05:11:03.733684+0000","last_clean":"2024-03-12T05:11:03.733684+0000","last_became_active":"2024-03-12T05:09:54.700288+0000","last_became_peered":"2024-03-12T05:09:54.700288+0000","last_unstale":"2024-03-12T05:11:03.733684+0000","last_undegraded":"2024-03-12T05:11:03.733684+0000","last_fullsized":"2024-03-12T05:11:03.733684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:32:50.865660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984298+0000","last_change":"2024-03-12T05:09:53.947337+0000","last_active":"2024-03-12T05:10:04.984298+0000","last_peered":"2024-03-12T05:10:04.984298+0000","last_clean":"2024-03-12T05:10:04.984298+0000","last_became_active":"2024-03-12T05:09:53.946709+0000","last_became_peered":"2024-03-12T05:09:53.946709+0000","last_unstale":"2024-03-12T05:10:04.984298+0000","last_undegraded":"2024-03-12T05:10:04.984298+0000","last_fullsized":"2024-03-12T05:10:04.984298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:57:42.495199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749472+0000","last_change":"2024-03-12T05:09:54.711244+0000","last_active":"2024-03-12T05:11:03.749472+0000","last_peered":"2024-03-12T05:11:03.749472+0000","last_clean":"2024-03-12T05:11:03.749472+0000","last_became_active":"2024-03-12T05:09:54.710346+0000","last_became_peered":"2024-03-12T05:09:54.710346+0000","last_unstale":"2024-03-12T05:11:03.749472+0000","last_undegraded":"2024-03-12T05:11:03.749472+0000","last_fullsized":"2024-03-12T05:11:03.749472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:44:55.483920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:05.000638+0000","last_change":"2024-03-12T05:10:05.000161+0000","last_active":"2024-03-12T05:10:05.000638+0000","last_peered":"2024-03-12T05:10:05.000638+0000","last_clean":"2024-03-12T05:10:05.000638+0000","last_became_active":"2024-03-12T05:10:04.985941+0000","last_became_peered":"2024-03-12T05:10:04.985941+0000","last_unstale":"2024-03-12T05:10:05.000638+0000","last_undegraded":"2024-03-12T05:10:05.000638+0000","last_fullsized":"2024-03-12T05:10:05.000638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:34.170134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745090+0000","last_change":"2024-03-12T05:09:54.705911+0000","last_active":"2024-03-12T05:11:03.745090+0000","last_peered":"2024-03-12T05:11:03.745090+0000","last_clean":"2024-03-12T05:11:03.745090+0000","last_became_active":"2024-03-12T05:09:54.705306+0000","last_became_peered":"2024-03-12T05:09:54.705306+0000","last_unstale":"2024-03-12T05:11:03.745090+0000","last_undegraded":"2024-03-12T05:11:03.745090+0000","last_fullsized":"2024-03-12T05:11:03.745090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:43:07.373188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.984347+0000","last_change":"2024-03-12T05:09:53.943150+0000","last_active":"2024-03-12T05:10:04.984347+0000","last_peered":"2024-03-12T05:10:04.984347+0000","last_clean":"2024-03-12T05:10:04.984347+0000","last_became_active":"2024-03-12T05:09:53.942804+0000","last_became_peered":"2024-03-12T05:09:53.942804+0000","last_unstale":"2024-03-12T05:10:04.984347+0000","last_undegraded":"2024-03-12T05:10:04.984347+0000","last_fullsized":"2024-03-12T05:10:04.984347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:29:41.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758283+0000","last_change":"2024-03-12T05:09:54.714662+0000","last_active":"2024-03-12T05:11:03.758283+0000","last_peered":"2024-03-12T05:11:03.758283+0000","last_clean":"2024-03-12T05:11:03.758283+0000","last_became_active":"2024-03-12T05:09:54.713363+0000","last_became_peered":"2024-03-12T05:09:54.713363+0000","last_unstale":"2024-03-12T05:11:03.758283+0000","last_undegraded":"2024-03-12T05:11:03.758283+0000","last_fullsized":"2024-03-12T05:11:03.758283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:18:38.388588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.985425+0000","last_change":"2024-03-12T05:09:53.943889+0000","last_active":"2024-03-12T05:10:04.985425+0000","last_peered":"2024-03-12T05:10:04.985425+0000","last_clean":"2024-03-12T05:10:04.985425+0000","last_became_active":"2024-03-12T05:09:53.943573+0000","last_became_peered":"2024-03-12T05:09:53.943573+0000","last_unstale":"2024-03-12T05:10:04.985425+0000","last_undegraded":"2024-03-12T05:10:04.985425+0000","last_fullsized":"2024-03-12T05:10:04.985425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:09:08.165647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.758796+0000","last_change":"2024-03-12T05:09:54.713760+0000","last_active":"2024-03-12T05:11:03.758796+0000","last_peered":"2024-03-12T05:11:03.758796+0000","last_clean":"2024-03-12T05:11:03.758796+0000","last_became_active":"2024-03-12T05:09:54.713113+0000","last_became_peered":"2024-03-12T05:09:54.713113+0000","last_unstale":"2024-03-12T05:11:03.758796+0000","last_undegraded":"2024-03-12T05:11:03.758796+0000","last_fullsized":"2024-03-12T05:11:03.758796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:51:33.311616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.990547+0000","last_change":"2024-03-12T05:09:53.936942+0000","last_active":"2024-03-12T05:10:04.990547+0000","last_peered":"2024-03-12T05:10:04.990547+0000","last_clean":"2024-03-12T05:10:04.990547+0000","last_became_active":"2024-03-12T05:09:53.936608+0000","last_became_peered":"2024-03-12T05:09:53.936608+0000","last_unstale":"2024-03-12T05:10:04.990547+0000","last_undegraded":"2024-03-12T05:10:04.990547+0000","last_fullsized":"2024-03-12T05:10:04.990547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T13:30:22.384155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749176+0000","last_change":"2024-03-12T05:09:54.713212+0000","last_active":"2024-03-12T05:11:03.749176+0000","last_peered":"2024-03-12T05:11:03.749176+0000","last_clean":"2024-03-12T05:11:03.749176+0000","last_became_active":"2024-03-12T05:09:54.712824+0000","last_became_peered":"2024-03-12T05:09:54.712824+0000","last_unstale":"2024-03-12T05:11:03.749176+0000","last_undegraded":"2024-03-12T05:11:03.749176+0000","last_fullsized":"2024-03-12T05:11:03.749176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:22:16.643379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:10:04.994225+0000","last_change":"2024-03-12T05:09:53.938332+0000","last_active":"2024-03-12T05:10:04.994225+0000","last_peered":"2024-03-12T05:10:04.994225+0000","last_clean":"2024-03-12T05:10:04.994225+0000","last_became_active":"2024-03-12T05:09:53.938032+0000","last_became_peered":"2024-03-12T05:09:53.938032+0000","last_unstale":"2024-03-12T05:10:04.994225+0000","last_undegraded":"2024-03-12T05:10:04.994225+0000","last_fullsized":"2024-03-12T05:10:04.994225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:52.917391+0000","last_clean_scrub_stamp":"2024-03-12T05:09:52.917391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:34:22.208827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751718+0000","last_change":"2024-03-12T05:09:54.712608+0000","last_active":"2024-03-12T05:11:03.751718+0000","last_peered":"2024-03-12T05:11:03.751718+0000","last_clean":"2024-03-12T05:11:03.751718+0000","last_became_active":"2024-03-12T05:09:54.712269+0000","last_became_peered":"2024-03-12T05:09:54.712269+0000","last_unstale":"2024-03-12T05:11:03.751718+0000","last_undegraded":"2024-03-12T05:11:03.751718+0000","last_fullsized":"2024-03-12T05:11:03.751718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:26:19.560602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.749780+0000","last_change":"2024-03-12T05:09:54.711750+0000","last_active":"2024-03-12T05:11:03.749780+0000","last_peered":"2024-03-12T05:11:03.749780+0000","last_clean":"2024-03-12T05:11:03.749780+0000","last_became_active":"2024-03-12T05:09:54.711422+0000","last_became_peered":"2024-03-12T05:09:54.711422+0000","last_unstale":"2024-03-12T05:11:03.749780+0000","last_undegraded":"2024-03-12T05:11:03.749780+0000","last_fullsized":"2024-03-12T05:11:03.749780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:22.856049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743933+0000","last_change":"2024-03-12T05:10:04.992158+0000","last_active":"2024-03-12T05:11:03.743933+0000","last_peered":"2024-03-12T05:11:03.743933+0000","last_clean":"2024-03-12T05:11:03.743933+0000","last_became_active":"2024-03-12T05:10:04.991721+0000","last_became_peered":"2024-03-12T05:10:04.991721+0000","last_unstale":"2024-03-12T05:11:03.743933+0000","last_undegraded":"2024-03-12T05:11:03.743933+0000","last_fullsized":"2024-03-12T05:11:03.743933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:19:24.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748140+0000","last_change":"2024-03-12T05:09:54.709582+0000","last_active":"2024-03-12T05:11:03.748140+0000","last_peered":"2024-03-12T05:11:03.748140+0000","last_clean":"2024-03-12T05:11:03.748140+0000","last_became_active":"2024-03-12T05:09:54.709257+0000","last_became_peered":"2024-03-12T05:09:54.709257+0000","last_unstale":"2024-03-12T05:11:03.748140+0000","last_undegraded":"2024-03-12T05:11:03.748140+0000","last_fullsized":"2024-03-12T05:11:03.748140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:12.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748105+0000","last_change":"2024-03-12T05:09:54.704231+0000","last_active":"2024-03-12T05:11:03.748105+0000","last_peered":"2024-03-12T05:11:03.748105+0000","last_clean":"2024-03-12T05:11:03.748105+0000","last_became_active":"2024-03-12T05:09:54.703913+0000","last_became_peered":"2024-03-12T05:09:54.703913+0000","last_unstale":"2024-03-12T05:11:03.748105+0000","last_undegraded":"2024-03-12T05:11:03.748105+0000","last_fullsized":"2024-03-12T05:11:03.748105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:47:18.592339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.748285+0000","last_change":"2024-03-12T05:09:54.700975+0000","last_active":"2024-03-12T05:11:03.748285+0000","last_peered":"2024-03-12T05:11:03.748285+0000","last_clean":"2024-03-12T05:11:03.748285+0000","last_became_active":"2024-03-12T05:09:54.700619+0000","last_became_peered":"2024-03-12T05:09:54.700619+0000","last_unstale":"2024-03-12T05:11:03.748285+0000","last_undegraded":"2024-03-12T05:11:03.748285+0000","last_fullsized":"2024-03-12T05:11:03.748285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:06:22.578541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.737606+0000","last_change":"2024-03-12T05:09:54.706525+0000","last_active":"2024-03-12T05:11:03.737606+0000","last_peered":"2024-03-12T05:11:03.737606+0000","last_clean":"2024-03-12T05:11:03.737606+0000","last_became_active":"2024-03-12T05:09:54.706100+0000","last_became_peered":"2024-03-12T05:09:54.706100+0000","last_unstale":"2024-03-12T05:11:03.737606+0000","last_undegraded":"2024-03-12T05:11:03.737606+0000","last_fullsized":"2024-03-12T05:11:03.737606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T11:06:50.908478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.743523+0000","last_change":"2024-03-12T05:09:54.703549+0000","last_active":"2024-03-12T05:11:03.743523+0000","last_peered":"2024-03-12T05:11:03.743523+0000","last_clean":"2024-03-12T05:11:03.743523+0000","last_became_active":"2024-03-12T05:09:54.702795+0000","last_became_peered":"2024-03-12T05:09:54.702795+0000","last_unstale":"2024-03-12T05:11:03.743523+0000","last_undegraded":"2024-03-12T05:11:03.743523+0000","last_fullsized":"2024-03-12T05:11:03.743523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:24:46.954817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.745476+0000","last_change":"2024-03-12T05:09:54.707615+0000","last_active":"2024-03-12T05:11:03.745476+0000","last_peered":"2024-03-12T05:11:03.745476+0000","last_clean":"2024-03-12T05:11:03.745476+0000","last_became_active":"2024-03-12T05:09:54.706500+0000","last_became_peered":"2024-03-12T05:09:54.706500+0000","last_unstale":"2024-03-12T05:11:03.745476+0000","last_undegraded":"2024-03-12T05:11:03.745476+0000","last_fullsized":"2024-03-12T05:11:03.745476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:45:31.084794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.744920+0000","last_change":"2024-03-12T05:09:54.705070+0000","last_active":"2024-03-12T05:11:03.744920+0000","last_peered":"2024-03-12T05:11:03.744920+0000","last_clean":"2024-03-12T05:11:03.744920+0000","last_became_active":"2024-03-12T05:09:54.704719+0000","last_became_peered":"2024-03-12T05:09:54.704719+0000","last_unstale":"2024-03-12T05:11:03.744920+0000","last_undegraded":"2024-03-12T05:11:03.744920+0000","last_fullsized":"2024-03-12T05:11:03.744920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:53.105963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.746188+0000","last_change":"2024-03-12T05:09:54.712862+0000","last_active":"2024-03-12T05:11:03.746188+0000","last_peered":"2024-03-12T05:11:03.746188+0000","last_clean":"2024-03-12T05:11:03.746188+0000","last_became_active":"2024-03-12T05:09:54.712495+0000","last_became_peered":"2024-03-12T05:09:54.712495+0000","last_unstale":"2024-03-12T05:11:03.746188+0000","last_undegraded":"2024-03-12T05:11:03.746188+0000","last_fullsized":"2024-03-12T05:11:03.746188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T16:09:49.492323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.760230+0000","last_change":"2024-03-12T05:09:54.709705+0000","last_active":"2024-03-12T05:11:03.760230+0000","last_peered":"2024-03-12T05:11:03.760230+0000","last_clean":"2024-03-12T05:11:03.760230+0000","last_became_active":"2024-03-12T05:09:54.709329+0000","last_became_peered":"2024-03-12T05:09:54.709329+0000","last_unstale":"2024-03-12T05:11:03.760230+0000","last_undegraded":"2024-03-12T05:11:03.760230+0000","last_fullsized":"2024-03-12T05:11:03.760230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T14:27:20.062670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.751293+0000","last_change":"2024-03-12T05:09:54.705627+0000","last_active":"2024-03-12T05:11:03.751293+0000","last_peered":"2024-03-12T05:11:03.751293+0000","last_clean":"2024-03-12T05:11:03.751293+0000","last_became_active":"2024-03-12T05:09:54.705280+0000","last_became_peered":"2024-03-12T05:09:54.705280+0000","last_unstale":"2024-03-12T05:11:03.751293+0000","last_undegraded":"2024-03-12T05:11:03.751293+0000","last_fullsized":"2024-03-12T05:11:03.751293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T10:52:26.120511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-12T05:11:03.747355+0000","last_change":"2024-03-12T05:09:54.701447+0000","last_active":"2024-03-12T05:11:03.747355+0000","last_peered":"2024-03-12T05:11:03.747355+0000","last_clean":"2024-03-12T05:11:03.747355+0000","last_became_active":"2024-03-12T05:09:54.700983+0000","last_became_peered":"2024-03-12T05:09:54.700983+0000","last_unstale":"2024-03-12T05:11:03.747355+0000","last_undegraded":"2024-03-12T05:11:03.747355+0000","last_fullsized":"2024-03-12T05:11:03.747355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T05:09:53.920753+0000","last_clean_scrub_stamp":"2024-03-12T05:09:53.920753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:43.485435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66668,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":706,"num_read_kb":424,"num_write":172,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":211923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1113530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":460150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1053455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":459539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1032192,"data_stored":526699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":460810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":459850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":460508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1052216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T05:11:55.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:56.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 -- 172.21.15.100:0/1053141937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658101290 msgr2=0x7f1658101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 --2- 172.21.15.100:0/1053141937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658101290 0x7f1658101710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1654009900 tx=0x7f165402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 -- 172.21.15.100:0/1053141937 shutdown_connections 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 --2- 172.21.15.100:0/1053141937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 0x7f16581084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 --2- 172.21.15.100:0/1053141937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658101290 0x7f1658101710 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 --2- 172.21.15.100:0/1053141937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16581000a0 0x7f16581004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 -- 172.21.15.100:0/1053141937 >> 172.21.15.100:0/1053141937 conn(0x7f16580fb610 msgr2=0x7f16580fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 -- 172.21.15.100:0/1053141937 shutdown_connections 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 -- 172.21.15.100:0/1053141937 wait complete. 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 Processor -- start 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.051+0000 7f165f34f640 1 -- start start 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 0x7f1658197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658101290 0x7f1658197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 0x7f165819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165819ed10 con 0x7f16581000a0 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165819ee80 con 0x7f1658101c50 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165819f040 con 0x7f1658101290 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 0x7f1658197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 0x7f1658197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44128/0 (socket says 172.21.15.100:44128) 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 -- 172.21.15.100:0/2245219654 learned_addr learned my addr 172.21.15.100:0/2245219654 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d8ec640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 0x7f165819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165c8ea640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658101290 0x7f1658197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 -- 172.21.15.100:0/2245219654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658101290 msgr2=0x7f1658197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658101290 0x7f1658197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 -- 172.21.15.100:0/2245219654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 msgr2=0x7f165819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 0x7f165819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 -- 172.21.15.100:0/2245219654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654009560 con 0x7f16581000a0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d8ec640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 0x7f165819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165d0eb640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 0x7f1658197220 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f164800e9c0 tx=0x7f164800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648009700 con 0x7f16581000a0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f164800cd80 con 0x7f16581000a0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648010430 con 0x7f16581000a0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165819f320 con 0x7f16581000a0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.052+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165819f830 con 0x7f16581000a0 2024-03-12T05:11:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.053+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16581073c0 con 0x7f16581000a0 2024-03-12T05:11:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.054+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f1648003680 con 0x7f16581000a0 2024-03-12T05:11:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.055+0000 7f164e7fc640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f16440a1ac0 0x7f16440a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.055+0000 7f165c8ea640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f16440a1ac0 0x7f16440a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.055+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1648014070 con 0x7f16581000a0 2024-03-12T05:11:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.055+0000 7f165c8ea640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f16440a1ac0 0x7f16440a3f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1658198730 tx=0x7f1654000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.059+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1648096b90 con 0x7f16581000a0 2024-03-12T05:11:56.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1091593381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:56.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1091593381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:56.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1091593381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.234+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f165810cbb0 con 0x7f16581000a0 2024-03-12T05:11:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.236+0000 7f164e7fc640 1 -- 172.21.15.100:0/2245219654 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7f164809a250 con 0x7f16581000a0 2024-03-12T05:11:56.233 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:56.234 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:56.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f16440a1ac0 msgr2=0x7f16440a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f16440a1ac0 0x7f16440a3f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1658198730 tx=0x7f1654000bc0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 msgr2=0x7f1658197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 0x7f1658197220 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f164800e9c0 tx=0x7f164800ee80 comp rx=0 tx=0).stop 2024-03-12T05:11:56.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 shutdown_connections 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f16440a1ac0 0x7f16440a3f80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101c50 0x7f165819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658101290 0x7f1658197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 --2- 172.21.15.100:0/2245219654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16581000a0 0x7f1658197220 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 >> 172.21.15.100:0/2245219654 conn(0x7f16580fb610 msgr2=0x7f1658104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 shutdown_connections 2024-03-12T05:11:56.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.238+0000 7f165f34f640 1 -- 172.21.15.100:0/2245219654 wait complete. 2024-03-12T05:11:56.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:11:56.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:11:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/3607523476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8102380 msgr2=0x7fd8c81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/3607523476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8102380 0x7fd8c81047f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009900 tx=0x7fd8b802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/3607523476 shutdown_connections 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/3607523476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8c8104d30 0x7fd8c81051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/3607523476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8102380 0x7fd8c81047f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/3607523476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8c80ffa10 0x7fd8c8101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/3607523476 >> 172.21.15.100:0/3607523476 conn(0x7fd8c80fb630 msgr2=0x7fd8c80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/3607523476 shutdown_connections 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.355+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/3607523476 wait complete. 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 Processor -- start 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 -- start start 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8c80ffa10 0x7fd8c819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8c8102380 0x7fd8c819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 0x7fd8c81a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81a5300 con 0x7fd8c8104d30 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81a5470 con 0x7fd8c80ffa10 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cfbd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81a5630 con 0x7fd8c8102380 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 0x7fd8c81a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 0x7fd8c81a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44142/0 (socket says 172.21.15.100:44142) 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 -- 172.21.15.100:0/305608938 learned_addr learned my addr 172.21.15.100:0/305608938 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cd96c640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8c80ffa10 0x7fd8c819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8cd16b640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8c8102380 0x7fd8c819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 -- 172.21.15.100:0/305608938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8c8102380 msgr2=0x7fd8c819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8c8102380 0x7fd8c819dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 -- 172.21.15.100:0/305608938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8c80ffa10 msgr2=0x7fd8c819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.356+0000 7fd8ce16d640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8c80ffa10 0x7fd8c819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8ce16d640 1 -- 172.21.15.100:0/305608938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8009560 con 0x7fd8c8104d30 2024-03-12T05:11:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8ce16d640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 0x7fd8c81a4dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000eaa0 tx=0x7fd8c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000cdb0 con 0x7fd8c8104d30 2024-03-12T05:11:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8c0005020 con 0x7fd8c8104d30 2024-03-12T05:11:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c81a5910 con 0x7fd8c8104d30 2024-03-12T05:11:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00116b0 con 0x7fd8c8104d30 2024-03-12T05:11:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.357+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c81a5e20 con 0x7fd8c8104d30 2024-03-12T05:11:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.358+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c80fd120 con 0x7fd8c8104d30 2024-03-12T05:11:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.359+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fd8c0005520 con 0x7fd8c8104d30 2024-03-12T05:11:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.359+0000 7fd8beffd640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd8b40a1ac0 0x7fd8b40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.359+0000 7fd8cd96c640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd8b40a1ac0 0x7fd8b40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.359+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8c00cd740 con 0x7fd8c8104d30 2024-03-12T05:11:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.360+0000 7fd8cd96c640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd8b40a1ac0 0x7fd8b40a3f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4005d00 tx=0x7fd8c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.364+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8c00967f0 con 0x7fd8c8104d30 2024-03-12T05:11:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.536+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8c802ce40 con 0x7fd8c8104d30 2024-03-12T05:11:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.536+0000 7fd8beffd640 1 -- 172.21.15.100:0/305608938 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8c0099eb0 con 0x7fd8c8104d30 2024-03-12T05:11:56.534 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd8b40a1ac0 msgr2=0x7fd8b40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd8b40a1ac0 0x7fd8b40a3f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4005d00 tx=0x7fd8c400b410 comp rx=0 tx=0).stop 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 msgr2=0x7fd8c81a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 0x7fd8c81a4dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000eaa0 tx=0x7fd8c000ef60 comp rx=0 tx=0).stop 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 shutdown_connections 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fd8b40a1ac0 0x7fd8b40a3f80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8104d30 0x7fd8c81a4dc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8c8102380 0x7fd8c819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 --2- 172.21.15.100:0/305608938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8c80ffa10 0x7fd8c819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 >> 172.21.15.100:0/305608938 conn(0x7fd8c80fb630 msgr2=0x7fd8c8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 shutdown_connections 2024-03-12T05:11:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.538+0000 7fd8cfbd0640 1 -- 172.21.15.100:0/305608938 wait complete. 2024-03-12T05:11:56.547 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:11:56.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 -- 172.21.15.100:0/2370074188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 msgr2=0x7f1d081015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 --2- 172.21.15.100:0/2370074188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d081015b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009900 tx=0x7f1cf802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 -- 172.21.15.100:0/2370074188 shutdown_connections 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 --2- 172.21.15.100:0/2370074188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102d40 0x7f1d081095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 --2- 172.21.15.100:0/2370074188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d08102380 0x7f1d08102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 --2- 172.21.15.100:0/2370074188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d081015b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 -- 172.21.15.100:0/2370074188 >> 172.21.15.100:0/2370074188 conn(0x7f1d080fc720 msgr2=0x7f1d080feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 -- 172.21.15.100:0/2370074188 shutdown_connections 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 -- 172.21.15.100:0/2370074188 wait complete. 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 Processor -- start 2024-03-12T05:11:56.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.653+0000 7f1d0d527640 1 -- start start 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d08197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102380 0x7f1d081977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d08102d40 0x7f1d0819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0819ed60 con 0x7f1d08101190 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0819eed0 con 0x7f1d08102d40 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0819f090 con 0x7f1d08102380 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d08197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d08197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44166/0 (socket says 172.21.15.100:44166) 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 -- 172.21.15.100:0/1871311343 learned_addr learned my addr 172.21.15.100:0/1871311343 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d077fe640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d08102d40 0x7f1d0819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1cfffff640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102380 0x7f1d081977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 -- 172.21.15.100:0/1871311343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102380 msgr2=0x7f1d081977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102380 0x7f1d081977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 -- 172.21.15.100:0/1871311343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d08102d40 msgr2=0x7f1d0819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d08102d40 0x7f1d0819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 -- 172.21.15.100:0/1871311343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8009560 con 0x7f1d08101190 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1cfffff640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102380 0x7f1d081977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d06ffd640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d08197270 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8000c00 tx=0x7f1cf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8004df0 con 0x7f1d08101190 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0819f310 con 0x7f1d08101190 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cf8005430 con 0x7f1d08101190 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf803e450 con 0x7f1d08101190 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.654+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0819f870 con 0x7f1d08101190 2024-03-12T05:11:56.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.655+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08108530 con 0x7f1d08101190 2024-03-12T05:11:56.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.656+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f1cf80055a0 con 0x7f1d08101190 2024-03-12T05:11:56.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.657+0000 7f1d04ff9640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1cf40a1ac0 0x7f1cf40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.657+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1cf80efff0 con 0x7f1d08101190 2024-03-12T05:11:56.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.657+0000 7f1cfffff640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1cf40a1ac0 0x7f1cf40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.657+0000 7f1cfffff640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1cf40a1ac0 0x7f1cf40a3f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d08198780 tx=0x7f1cf0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.660+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1cf80b91b0 con 0x7f1d08101190 2024-03-12T05:11:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.834+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d0819f4a0 con 0x7f1d08101190 2024-03-12T05:11:56.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.834+0000 7f1d04ff9640 1 -- 172.21.15.100:0/1871311343 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d0819f4a0 con 0x7f1d08101190 2024-03-12T05:11:56.832 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1cf40a1ac0 msgr2=0x7f1cf40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1cf40a1ac0 0x7f1cf40a3f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d08198780 tx=0x7f1cf0005e50 comp rx=0 tx=0).stop 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 msgr2=0x7f1d08197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d08197270 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8000c00 tx=0x7f1cf8004270 comp rx=0 tx=0).stop 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 shutdown_connections 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f1cf40a1ac0 0x7f1cf40a3f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d08102d40 0x7f1d0819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d08102380 0x7f1d081977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 --2- 172.21.15.100:0/1871311343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d08101190 0x7f1d08197270 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.836+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 >> 172.21.15.100:0/1871311343 conn(0x7f1d080fc720 msgr2=0x7f1d08105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.837+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 shutdown_connections 2024-03-12T05:11:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.837+0000 7f1d0d527640 1 -- 172.21.15.100:0/1871311343 wait complete. 2024-03-12T05:11:56.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:11:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 -- 172.21.15.100:0/1037940390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102390 msgr2=0x7f2a94102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 --2- 172.21.15.100:0/1037940390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102390 0x7f2a94102810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009900 tx=0x7f2a8802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 -- 172.21.15.100:0/1037940390 shutdown_connections 2024-03-12T05:11:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 --2- 172.21.15.100:0/1037940390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a94102d50 0x7f2a941095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 --2- 172.21.15.100:0/1037940390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102390 0x7f2a94102810 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 --2- 172.21.15.100:0/1037940390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a941011a0 0x7f2a941015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 -- 172.21.15.100:0/1037940390 >> 172.21.15.100:0/1037940390 conn(0x7f2a940fc710 msgr2=0x7f2a940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 -- 172.21.15.100:0/1037940390 shutdown_connections 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.952+0000 7f2a99840640 1 -- 172.21.15.100:0/1037940390 wait complete. 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 Processor -- start 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 -- start start 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a941011a0 0x7f2a94197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94102390 0x7f2a94197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 0x7f2a9419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9419ed30 con 0x7f2a94102d50 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9419eea0 con 0x7f2a941011a0 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a99840640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9419f060 con 0x7f2a94102390 2024-03-12T05:11:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 0x7f2a9419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 0x7f2a9419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44192/0 (socket says 172.21.15.100:44192) 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a937fe640 1 -- 172.21.15.100:0/2778796174 learned_addr learned my addr 172.21.15.100:0/2778796174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a92ffd640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a941011a0 0x7f2a94197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.953+0000 7f2a927fc640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94102390 0x7f2a94197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a937fe640 1 -- 172.21.15.100:0/2778796174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94102390 msgr2=0x7f2a94197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a937fe640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94102390 0x7f2a94197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a937fe640 1 -- 172.21.15.100:0/2778796174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a941011a0 msgr2=0x7f2a94197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a937fe640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a941011a0 0x7f2a94197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a937fe640 1 -- 172.21.15.100:0/2778796174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a88009560 con 0x7f2a94102d50 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a937fe640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 0x7f2a9419e7f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00cad0 tx=0x7f2a8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00ae20 con 0x7f2a94102d50 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c005020 con 0x7f2a94102d50 2024-03-12T05:11:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9419f340 con 0x7f2a94102d50 2024-03-12T05:11:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00f6b0 con 0x7f2a94102d50 2024-03-12T05:11:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.954+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9419f850 con 0x7f2a94102d50 2024-03-12T05:11:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.955+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a941084c0 con 0x7f2a94102d50 2024-03-12T05:11:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.956+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f2a8c005520 con 0x7f2a94102d50 2024-03-12T05:11:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.957+0000 7f2a9883e640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b40 0x7f2a7c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.957+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a8c0cd570 con 0x7f2a94102d50 2024-03-12T05:11:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.957+0000 7f2a92ffd640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b40 0x7f2a7c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.957+0000 7f2a92ffd640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b40 0x7f2a7c0a4000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2a941021f0 tx=0x7f2a8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:56.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:56.961+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a8c096620 con 0x7f2a94102d50 2024-03-12T05:11:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:56 smithi100 ceph-mon[32210]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:56 smithi100 ceph-mon[32210]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:56 smithi100 ceph-mon[32210]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-12T05:11:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:56 smithi100 ceph-mon[32210]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2245219654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:57.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/305608938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:57.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.134+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a9402ce40 con 0x7f2a94102d50 2024-03-12T05:11:57.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.134+0000 7f2a9883e640 1 -- 172.21.15.100:0/2778796174 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a8c099ce0 con 0x7f2a94102d50 2024-03-12T05:11:57.133 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:11:57.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b40 msgr2=0x7f2a7c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b40 0x7f2a7c0a4000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2a941021f0 tx=0x7f2a8400b390 comp rx=0 tx=0).stop 2024-03-12T05:11:57.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 msgr2=0x7f2a9419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 0x7f2a9419e7f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00cad0 tx=0x7f2a8c00cf90 comp rx=0 tx=0).stop 2024-03-12T05:11:57.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 shutdown_connections 2024-03-12T05:11:57.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f2a7c0a1b40 0x7f2a7c0a4000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a94102d50 0x7f2a9419e7f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94102390 0x7f2a94197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 --2- 172.21.15.100:0/2778796174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a941011a0 0x7f2a94197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 >> 172.21.15.100:0/2778796174 conn(0x7f2a940fc710 msgr2=0x7f2a94105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 shutdown_connections 2024-03-12T05:11:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.136+0000 7f2a99840640 1 -- 172.21.15.100:0/2778796174 wait complete. 2024-03-12T05:11:57.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:11:57.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:56 smithi156 ceph-mon[36270]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:56 smithi156 ceph-mon[36270]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:56 smithi156 ceph-mon[36270]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-12T05:11:57.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:56 smithi156 ceph-mon[36270]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2245219654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:57.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/305608938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:56 smithi161 ceph-mon[36043]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:56 smithi161 ceph-mon[36043]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:56 smithi161 ceph-mon[36043]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-12T05:11:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:56 smithi161 ceph-mon[36043]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T05:11:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2245219654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:57.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/305608938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:11:57.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3859723601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac1011b0 msgr2=0x7f99ac1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3859723601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac1011b0 0x7f99ac1015d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009ca0 tx=0x7f99a402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3859723601 shutdown_connections 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3859723601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac102d60 0x7f99ac1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3859723601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99ac1023a0 0x7f99ac102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3859723601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac1011b0 0x7f99ac1015d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3859723601 >> 172.21.15.100:0/3859723601 conn(0x7f99ac0fc720 msgr2=0x7f99ac0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3859723601 shutdown_connections 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3859723601 wait complete. 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.249+0000 7f99b3d2c640 1 Processor -- start 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 -- start start 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99ac1011b0 0x7f99ac197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac1023a0 0x7f99ac197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 0x7f99ac19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19ed40 con 0x7f99ac102d60 2024-03-12T05:11:57.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19eeb0 con 0x7f99ac1023a0 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19f070 con 0x7f99ac1011b0 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 0x7f99ac19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 0x7f99ac19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44216/0 (socket says 172.21.15.100:44216) 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 -- 172.21.15.100:0/3925904237 learned_addr learned my addr 172.21.15.100:0/3925904237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b12c7640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac1023a0 0x7f99ac197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b1ac8640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99ac1011b0 0x7f99ac197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 -- 172.21.15.100:0/3925904237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99ac1011b0 msgr2=0x7f99ac197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99ac1011b0 0x7f99ac197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 -- 172.21.15.100:0/3925904237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac1023a0 msgr2=0x7f99ac197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac1023a0 0x7f99ac197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 -- 172.21.15.100:0/3925904237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4009900 con 0x7f99ac102d60 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b12c7640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac1023a0 0x7f99ac197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:57.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b22c9640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 0x7f99ac19e800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f99a800d750 tx=0x7f99a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8009bd0 con 0x7f99ac102d60 2024-03-12T05:11:57.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99a800de60 con 0x7f99ac102d60 2024-03-12T05:11:57.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac19f350 con 0x7f99ac102d60 2024-03-12T05:11:57.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8004e70 con 0x7f99ac102d60 2024-03-12T05:11:57.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.250+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac19f810 con 0x7f99ac102d60 2024-03-12T05:11:57.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.251+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac108550 con 0x7f99ac102d60 2024-03-12T05:11:57.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.252+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f99a8005020 con 0x7f99ac102d60 2024-03-12T05:11:57.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.253+0000 7f99a2ffd640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f99980a1ac0 0x7f99980a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.253+0000 7f99b1ac8640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f99980a1ac0 0x7f99980a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.253+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99a80ccf90 con 0x7f99ac102d60 2024-03-12T05:11:57.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.253+0000 7f99b1ac8640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f99980a1ac0 0x7f99980a3f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f99a402c700 tx=0x7f99a4009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.257+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99a8099c30 con 0x7f99ac102d60 2024-03-12T05:11:57.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.464+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99ac02ce40 con 0x7f99ac102d60 2024-03-12T05:11:57.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.464+0000 7f99a2ffd640 1 -- 172.21.15.100:0/3925904237 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f99a8020090 con 0x7f99ac102d60 2024-03-12T05:11:57.463 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:57.464 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:11:51.088786+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:11:57.464 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 23 2024-03-12T05:11:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f99980a1ac0 msgr2=0x7f99980a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f99980a1ac0 0x7f99980a3f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f99a402c700 tx=0x7f99a4009800 comp rx=0 tx=0).stop 2024-03-12T05:11:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 msgr2=0x7f99ac19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 0x7f99ac19e800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f99a800d750 tx=0x7f99a800dc10 comp rx=0 tx=0).stop 2024-03-12T05:11:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 shutdown_connections 2024-03-12T05:11:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f99980a1ac0 0x7f99980a3f80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ac102d60 0x7f99ac19e800 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99ac1023a0 0x7f99ac197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 --2- 172.21.15.100:0/3925904237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99ac1011b0 0x7f99ac197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 >> 172.21.15.100:0/3925904237 conn(0x7f99ac0fc720 msgr2=0x7f99ac105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 shutdown_connections 2024-03-12T05:11:57.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.466+0000 7f99b3d2c640 1 -- 172.21.15.100:0/3925904237 wait complete. 2024-03-12T05:11:57.477 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:11:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 -- 172.21.15.100:0/11554075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864100120 msgr2=0x7fc864100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 --2- 172.21.15.100:0/11554075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864100120 0x7fc864100540 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc860009900 tx=0x7fc86002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 -- 172.21.15.100:0/11554075 shutdown_connections 2024-03-12T05:11:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 --2- 172.21.15.100:0/11554075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 0x7fc864108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 --2- 172.21.15.100:0/11554075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864101310 0x7fc864101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 --2- 172.21.15.100:0/11554075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864100120 0x7fc864100540 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 -- 172.21.15.100:0/11554075 >> 172.21.15.100:0/11554075 conn(0x7fc8640fb6b0 msgr2=0x7fc8640fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 -- 172.21.15.100:0/11554075 shutdown_connections 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.580+0000 7fc86c20e640 1 -- 172.21.15.100:0/11554075 wait complete. 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 Processor -- start 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 -- start start 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864100120 0x7fc864196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 0x7fc8641966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 0x7fc86419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86419dc80 con 0x7fc864101310 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86419ddf0 con 0x7fc864101cd0 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86419dfb0 con 0x7fc864100120 2024-03-12T05:11:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 0x7fc8641966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 0x7fc8641966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44244/0 (socket says 172.21.15.100:44244) 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 -- 172.21.15.100:0/1637655322 learned_addr learned my addr 172.21.15.100:0/1637655322 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc869faa640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864100120 0x7fc864196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86a7ab640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 0x7fc86419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 -- 172.21.15.100:0/1637655322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864100120 msgr2=0x7fc864196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864100120 0x7fc864196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 -- 172.21.15.100:0/1637655322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 msgr2=0x7fc86419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 0x7fc86419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 -- 172.21.15.100:0/1637655322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860009560 con 0x7fc864101310 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86a7ab640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 0x7fc86419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc8697a9640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 0x7fc8641966d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc85400d770 tx=0x7fc85400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854009c30 con 0x7fc864101310 2024-03-12T05:11:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc85400de80 con 0x7fc864101310 2024-03-12T05:11:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86419e290 con 0x7fc864101310 2024-03-12T05:11:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854004e80 con 0x7fc864101310 2024-03-12T05:11:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.581+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86419e750 con 0x7fc864101310 2024-03-12T05:11:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.582+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8641074c0 con 0x7fc864101310 2024-03-12T05:11:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.584+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fc854005000 con 0x7fc864101310 2024-03-12T05:11:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.584+0000 7fc85affd640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8500a1ac0 0x7fc8500a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.584+0000 7fc869faa640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8500a1ac0 0x7fc8500a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.584+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8540cd3f0 con 0x7fc864101310 2024-03-12T05:11:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.585+0000 7fc869faa640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8500a1ac0 0x7fc8500a3f80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc860005ab0 tx=0x7fc860005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.588+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8540964a0 con 0x7fc864101310 2024-03-12T05:11:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.762+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc86410bd10 con 0x7fc864101310 2024-03-12T05:11:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.763+0000 7fc85affd640 1 -- 172.21.15.100:0/1637655322 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22672 (secure 0 0 0) 0x7fc854017020 con 0x7fc864101310 2024-03-12T05:11:57.762 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:11:57.762 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":88,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:10:04.977359+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8500a1ac0 msgr2=0x7fc8500a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8500a1ac0 0x7fc8500a3f80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc860005ab0 tx=0x7fc860005fd0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 msgr2=0x7fc8641966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 0x7fc8641966d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc85400d770 tx=0x7fc85400dc30 comp rx=0 tx=0).stop 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 shutdown_connections 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fc8500a1ac0 0x7fc8500a3f80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc864101cd0 0x7fc86419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc864101310 0x7fc8641966d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 --2- 172.21.15.100:0/1637655322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864100120 0x7fc864196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.765+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 >> 172.21.15.100:0/1637655322 conn(0x7fc8640fb6b0 msgr2=0x7fc864104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.766+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 shutdown_connections 2024-03-12T05:11:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.766+0000 7fc86c20e640 1 -- 172.21.15.100:0/1637655322 wait complete. 2024-03-12T05:11:57.775 DEBUG:tasks.fwd_scrub:Yielding 2024-03-12T05:11:57.775 INFO:teuthology.run_tasks:Running task exec... 2024-03-12T05:11:57.785 INFO:teuthology.task.exec:Executing custom commands... 2024-03-12T05:11:57.785 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:11:57.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-12T05:11:57.785 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-12T05:11:57.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:11:57.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:11:57.786 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 -- 172.21.15.100:0/3657869963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b50102d90 msgr2=0x7f4b50109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/3657869963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b50102d90 0x7f4b50109620 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009ca0 tx=0x7f4b4802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 -- 172.21.15.100:0/3657869963 shutdown_connections 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/3657869963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b50102d90 0x7f4b50109620 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/3657869963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b501023d0 0x7f4b50102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/3657869963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b501011e0 0x7f4b50101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 -- 172.21.15.100:0/3657869963 >> 172.21.15.100:0/3657869963 conn(0x7f4b500fc750 msgr2=0x7f4b500feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 -- 172.21.15.100:0/3657869963 shutdown_connections 2024-03-12T05:11:57.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 -- 172.21.15.100:0/3657869963 wait complete. 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 Processor -- start 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.892+0000 7f4b56eb5640 1 -- start start 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 0x7f4b50197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b501023d0 0x7f4b50197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b50102d90 0x7f4b5019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5019ec10 con 0x7f4b501011e0 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5019ed80 con 0x7f4b501023d0 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5019ef40 con 0x7f4b50102d90 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 0x7f4b50197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 0x7f4b50197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44266/0 (socket says 172.21.15.100:44266) 2024-03-12T05:11:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 -- 172.21.15.100:0/4021990792 learned_addr learned my addr 172.21.15.100:0/4021990792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b55452640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b50102d90 0x7f4b5019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 -- 172.21.15.100:0/4021990792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b50102d90 msgr2=0x7f4b5019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b50102d90 0x7f4b5019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 -- 172.21.15.100:0/4021990792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b501023d0 msgr2=0x7f4b50197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b501023d0 0x7f4b50197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 -- 172.21.15.100:0/4021990792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48009900 con 0x7f4b501011e0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b55452640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b50102d90 0x7f4b5019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b54c51640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 0x7f4b50197120 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d770 tx=0x7f4b4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44009c30 con 0x7f4b501011e0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b4400de80 con 0x7f4b501011e0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44010460 con 0x7f4b501011e0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.893+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5019f220 con 0x7f4b501011e0 2024-03-12T05:11:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.894+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5019f780 con 0x7f4b501011e0 2024-03-12T05:11:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.894+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b501084a0 con 0x7f4b501011e0 2024-03-12T05:11:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.895+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f4b44003f60 con 0x7f4b501011e0 2024-03-12T05:11:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.896+0000 7f4b4dffb640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4b400a1bf0 0x7f4b400a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.896+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b440cd2d0 con 0x7f4b501011e0 2024-03-12T05:11:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.896+0000 7f4b4ffff640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4b400a1bf0 0x7f4b400a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.896+0000 7f4b4ffff640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4b400a1bf0 0x7f4b400a40b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c005fd0 tx=0x7f4b3c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.900+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b44096380 con 0x7f4b501011e0 2024-03-12T05:11:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2872798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 msgr2=0x7f25b4106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2872798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4106960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009900 tx=0x7f25a402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2872798752 shutdown_connections 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2872798752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b4106ea0 0x7f25b41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2872798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4106960 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2872798752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 0x7f25b4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2872798752 >> 172.21.15.100:0/2872798752 conn(0x7f25b40fb630 msgr2=0x7f25b40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2872798752 shutdown_connections 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.901+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2872798752 wait complete. 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 Processor -- start 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 -- start start 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 0x7f25b41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b4106ea0 0x7f25b419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b419ed90 con 0x7f25b4104570 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b419ef00 con 0x7f25b4106ea0 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bc7c2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b419f0c0 con 0x7f25b4101c40 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44276/0 (socket says 172.21.15.100:44276) 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 -- 172.21.15.100:0/2255708535 learned_addr learned my addr 172.21.15.100:0/2255708535 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25bad5f640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b4106ea0 0x7f25b419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25ba55e640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 0x7f25b41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 -- 172.21.15.100:0/2255708535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 msgr2=0x7f25b41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 0x7f25b41972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 -- 172.21.15.100:0/2255708535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b4106ea0 msgr2=0x7f25b419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b4106ea0 0x7f25b419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25b9d5d640 1 -- 172.21.15.100:0/2255708535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4009560 con 0x7f25b4104570 2024-03-12T05:11:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.902+0000 7f25ba55e640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 0x7f25b41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.903+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4197830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25a4000c00 tx=0x7f25a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.903+0000 7f25ab7fe640 1 -- 172.21.15.100:0/2255708535 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a402c5d0 con 0x7f25b4104570 2024-03-12T05:11:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.903+0000 7f25ab7fe640 1 -- 172.21.15.100:0/2255708535 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25a40044e0 con 0x7f25b4104570 2024-03-12T05:11:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.903+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b419f340 con 0x7f25b4104570 2024-03-12T05:11:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.903+0000 7f25ab7fe640 1 -- 172.21.15.100:0/2255708535 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4003e20 con 0x7f25b4104570 2024-03-12T05:11:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.903+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b419f850 con 0x7f25b4104570 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25ab7fe640 1 -- 172.21.15.100:0/2255708535 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f25a403e450 con 0x7f25b4104570 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 Processor -- start 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 -- start start 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739ce08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739ce1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739c4ff90 con 0x556739ce2050 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739c4d4c0 con 0x556739ce04c0 2024-03-12T05:11:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bf38b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739c48ca0 con 0x556739ce1690 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25b9d5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25b9d5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44288/0 (socket says 172.21.15.100:44288) 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25b9d5d640 1 -- 172.21.15.100:0/139754805 learned_addr learned my addr 172.21.15.100:0/139754805 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25bad5f640 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739ce08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25ab7fe640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f25a00a1ac0 0x7f25a00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.906+0000 7f25ba55e640 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739ce1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25bad5f640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f25a00a1ac0 0x7f25a00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25ab7fe640 1 -- 172.21.15.100:0/2255708535 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25a40efdd0 con 0x7f25b4104570 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25b9d5d640 1 -- 172.21.15.100:0/139754805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 msgr2=0x556739ce1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739ce1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25b9d5d640 1 -- 172.21.15.100:0/139754805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 msgr2=0x556739ce08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739ce08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25b9d5d640 1 -- 172.21.15.100:0/139754805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556739c0b0b0 con 0x556739ce2050 2024-03-12T05:11:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25bad5f640 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739ce08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce88e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ed7d0 tx=0x7f25a402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada663cd572dc137 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25a8ff9640 1 -- 172.21.15.100:0/139754805 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40bc830 con 0x556739ce2050 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25a8ff9640 1 -- 172.21.15.100:0/139754805 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25a40bc650 con 0x556739ce2050 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.907+0000 7f25a8ff9640 1 -- 172.21.15.100:0/139754805 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40edaa0 con 0x556739ce2050 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- 172.21.15.100:0/139754805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 msgr2=0x556739ce88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bad5f640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f25a00a1ac0 0x7f25a00a3f80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x556739ce1590 tx=0x7f25ac004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce88e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ed7d0 tx=0x7f25a402bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- 172.21.15.100:0/139754805 shutdown_connections 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce88e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739ce1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- 172.21.15.100:0/139754805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739ce08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- 172.21.15.100:0/139754805 >> 172.21.15.100:0/139754805 conn(0x556739cdb6a0 msgr2=0x556739cdbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- 172.21.15.100:0/139754805 shutdown_connections 2024-03-12T05:11:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- 172.21.15.100:0/139754805 wait complete. 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- start start 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 Processor -- start 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 0x556739cea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce2050 0x556739cf1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739bedf40 con 0x556739ce04c0 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673991c4b0 con 0x556739ce2050 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739bce670 con 0x556739ce1690 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.908+0000 7f25bf38b740 10 client.0 _ll_register_callbacks cb 0x556739c3bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 0x556739cea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 0x556739cea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44300/0 (socket says 172.21.15.100:44300) 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 -- 172.21.15.100:0/4137550542 learned_addr learned my addr 172.21.15.100:0/4137550542 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25bad5f640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce2050 0x556739cf1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 -- 172.21.15.100:0/4137550542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 msgr2=0x556739cea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 -- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce2050 msgr2=0x556739cf1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce2050 0x556739cf1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 -- 172.21.15.100:0/4137550542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b419f850 con 0x556739ce04c0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25ba55e640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 0x556739cea3e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009f90 tx=0x7f25b0011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49833424639d120e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f259dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f259dffb640 1 -- 172.21.15.100:0/4137550542 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0012920 con 0x556739ce04c0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f259dffb640 1 -- 172.21.15.100:0/4137550542 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25b0012a80 con 0x556739ce04c0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556739c22f40 con 0x556739ce04c0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f259dffb640 1 -- 172.21.15.100:0/4137550542 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b0012c40 con 0x556739ce04c0 2024-03-12T05:11:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25bf38b740 10 client.15390 fetch_fsmap learned FSMap version 23 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556739c47320 con 0x556739ce04c0 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.909+0000 7f259dffb640 1 -- 172.21.15.100:0/4137550542 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556739c22f40 con 0x556739ce04c0 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f259dffb640 1 -- 172.21.15.100:0/4137550542 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25b000c070 con 0x556739ce04c0 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25bf38b740 10 client.15390 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25bf38b740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25bf38b740 20 client.15390 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25bf38b740 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739cf6730 0x556739cf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25bf38b740 4 client.15390 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556739c3f310 con 0x556739cf6730 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739cf6730 0x556739cf8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739cf6730 0x556739cf8b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b13d7d31b2e0b7a server_cookie=ca2204e056576893 in_seq=0 out_seq=0 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.910+0000 7f259dffb640 10 client.15390 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f259dffb640 1 -- 172.21.15.100:0/4137550542 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556739c3f310 con 0x556739cf6730 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f259dffb640 10 client.15390 handle_command_reply: tid=0 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 client.15390 shutdown 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 msgr2=0x556739cea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 0x556739cea3e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009f90 tx=0x7f25b0011870 comp rx=0 tx=0).stop 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 shutdown_connections 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739cf6730 0x556739cf8b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce2050 0x556739cf1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f259dffb640 0 client.15390 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 --2- 172.21.15.100:0/4137550542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce04c0 0x556739cea3e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 >> 172.21.15.100:0/4137550542 conn(0x556739cdb6a0 msgr2=0x556739cda8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 shutdown_connections 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 1 -- 172.21.15.100:0/4137550542 wait complete. 2024-03-12T05:11:57.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.911+0000 7f25bf38b740 20 client.15390 trim_cache size 0 max 16384 2024-03-12T05:11:57.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 Processor -- start 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 -- start start 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce04c0 0x556739cdb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce1690 0x556739cf1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739ce9530 con 0x556739ce2050 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739ce93f0 con 0x556739ce1690 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bf38b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739cd6be0 con 0x556739ce04c0 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bad5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bad5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44316/0 (socket says 172.21.15.100:44316) 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25bad5f640 1 -- 172.21.15.100:0/159957078 learned_addr learned my addr 172.21.15.100:0/159957078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25ba55e640 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce04c0 0x556739cdb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.926+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce1690 0x556739cf1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25bad5f640 1 -- 172.21.15.100:0/159957078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce04c0 msgr2=0x556739cdb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25bad5f640 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce04c0 0x556739cdb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25bad5f640 1 -- 172.21.15.100:0/159957078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce1690 msgr2=0x556739cf1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25bad5f640 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce1690 0x556739cf1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25bad5f640 1 -- 172.21.15.100:0/159957078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556739be3c00 con 0x556739ce2050 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce1690 0x556739cf1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f25bad5f640 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce96b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007e40 tx=0x7f25ac014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283962027dbe48c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f259ffff640 1 -- 172.21.15.100:0/159957078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0119f0 con 0x556739ce2050 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f259ffff640 1 -- 172.21.15.100:0/159957078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25ac01f400 con 0x556739ce2050 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.927+0000 7f259ffff640 1 -- 172.21.15.100:0/159957078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac010630 con 0x556739ce2050 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- 172.21.15.100:0/159957078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 msgr2=0x556739ce96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce96b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007e40 tx=0x7f25ac014070 comp rx=0 tx=0).stop 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- 172.21.15.100:0/159957078 shutdown_connections 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739ce96b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce1690 0x556739cf1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- 172.21.15.100:0/159957078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce04c0 0x556739cdb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- 172.21.15.100:0/159957078 >> 172.21.15.100:0/159957078 conn(0x556739cdb6a0 msgr2=0x556739cdf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- 172.21.15.100:0/159957078 shutdown_connections 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- 172.21.15.100:0/159957078 wait complete. 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- start start 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 Processor -- start 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739cf0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cf0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739d4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739d4d1c0 con 0x556739ce2050 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739d4d300 con 0x556739ce04c0 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739d4d440 con 0x556739ce1690 2024-03-12T05:11:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bf38b740 10 client.0 _ll_register_callbacks cb 0x7f25a40f3790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bad5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739d4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bad5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739d4cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44318/0 (socket says 172.21.15.100:44318) 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25bad5f640 1 -- 172.21.15.100:0/1415445479 learned_addr learned my addr 172.21.15.100:0/1415445479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.928+0000 7f25b9d5d640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cf0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bad5f640 1 -- 172.21.15.100:0/1415445479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 msgr2=0x556739cf0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bad5f640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cf0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25ba55e640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739cf0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bad5f640 1 -- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 msgr2=0x556739cf0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bad5f640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739cf0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bad5f640 1 -- 172.21.15.100:0/1415445479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac012510 con 0x556739ce2050 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bad5f640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739d4cc80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fb0 tx=0x7f25ac011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d1669fb80bef9e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f259dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f259dffb640 1 -- 172.21.15.100:0/1415445479 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac029840 con 0x556739ce2050 2024-03-12T05:11:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f259dffb640 1 -- 172.21.15.100:0/1415445479 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25ac011e60 con 0x556739ce2050 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556739c6e900 con 0x556739ce2050 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f259dffb640 1 -- 172.21.15.100:0/1415445479 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac01fbc0 con 0x556739ce2050 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 10 client.15396 fetch_fsmap learned FSMap version 23 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556739c09910 con 0x556739ce2050 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f259dffb640 1 -- 172.21.15.100:0/1415445479 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556739c6e900 con 0x556739ce2050 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f259dffb640 1 -- 172.21.15.100:0/1415445479 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25ac0338c0 con 0x556739ce2050 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 10 client.15396 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 20 client.15396 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739d4ef00 0x556739d51320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 4 client.15396 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.929+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556739948860 con 0x556739d4ef00 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.930+0000 7f25ba55e640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739d4ef00 0x556739d51320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.930+0000 7f259dffb640 10 client.15396 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.930+0000 7f25ba55e640 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739d4ef00 0x556739d51320 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9e576ba0824cc0 server_cookie=2a8e936042a168b3 in_seq=0 out_seq=0 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.931+0000 7f259dffb640 1 -- 172.21.15.100:0/1415445479 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556739948860 con 0x556739d4ef00 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.931+0000 7f259dffb640 10 client.15396 handle_command_reply: tid=0 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.931+0000 7f25bf38b740 1 client.15396 shutdown 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 msgr2=0x556739d4cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739d4cc80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fb0 tx=0x7f25ac011c70 comp rx=0 tx=0).stop 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 shutdown_connections 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556739d4ef00 0x556739d51320 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556739ce2050 0x556739d4cc80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f259dffb640 0 client.15396 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556739ce1690 0x556739cf0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 --2- 172.21.15.100:0/1415445479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556739ce04c0 0x556739cf0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 >> 172.21.15.100:0/1415445479 conn(0x556739cdb6a0 msgr2=0x556739cda050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 shutdown_connections 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 1 -- 172.21.15.100:0/1415445479 wait complete. 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bf38b740 20 client.15396 trim_cache size 0 max 16384 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "0ef25f63-046a-4f21-99e5-846dd3a4f010", 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:11:57.936 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f25a00a1ac0 msgr2=0x7f25a00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f25a00a1ac0 0x7f25a00a3f80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x556739ce1590 tx=0x7f25ac004ab0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 msgr2=0x7f25b4197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4197830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25a4000c00 tx=0x7f25a40042f0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 shutdown_connections 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f25a00a1ac0 0x7f25a00a3f80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b4106ea0 0x7f25b419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b4104570 0x7f25b4197830 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 --2- 172.21.15.100:0/2255708535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25b4101c40 0x7f25b41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 >> 172.21.15.100:0/2255708535 conn(0x7f25b40fb630 msgr2=0x7f25b4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 shutdown_connections 2024-03-12T05:11:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:57.932+0000 7f25bc7c2640 1 -- 172.21.15.100:0/2255708535 wait complete. 2024-03-12T05:11:57.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:11:57.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.048+0000 7f60534d4640 1 -- 172.21.15.100:0/1105469052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c101270 msgr2=0x7f604c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.048+0000 7f60534d4640 1 --2- 172.21.15.100:0/1105469052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c101270 0x7f604c1016f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f603c009900 tx=0x7f603c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- 172.21.15.100:0/1105469052 shutdown_connections 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 --2- 172.21.15.100:0/1105469052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 0x7f604c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 --2- 172.21.15.100:0/1105469052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c101270 0x7f604c1016f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 --2- 172.21.15.100:0/1105469052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c100080 0x7f604c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- 172.21.15.100:0/1105469052 >> 172.21.15.100:0/1105469052 conn(0x7f604c0fb5f0 msgr2=0x7f604c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- 172.21.15.100:0/1105469052 shutdown_connections 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- 172.21.15.100:0/1105469052 wait complete. 2024-03-12T05:11:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 Processor -- start 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- start start 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 0x7f604c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c101270 0x7f604c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 0x7f604c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c19eca0 con 0x7f604c100080 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c19ee10 con 0x7f604c101c30 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f60534d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c19efd0 con 0x7f604c101270 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f6051270640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 0x7f604c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f6051270640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 0x7f604c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44336/0 (socket says 172.21.15.100:44336) 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.049+0000 7f6051270640 1 -- 172.21.15.100:0/1459142147 learned_addr learned my addr 172.21.15.100:0/1459142147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6050a6f640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c101270 0x7f604c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051270640 1 -- 172.21.15.100:0/1459142147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c101270 msgr2=0x7f604c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051a71640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 0x7f604c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051270640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c101270 0x7f604c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051270640 1 -- 172.21.15.100:0/1459142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 msgr2=0x7f604c19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051270640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 0x7f604c19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051270640 1 -- 172.21.15.100:0/1459142147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c009560 con 0x7f604c100080 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051a71640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 0x7f604c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f6051270640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 0x7f604c1971b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f604400c970 tx=0x7f604400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f60427fc640 1 -- 172.21.15.100:0/1459142147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044007c10 con 0x7f604c100080 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c19f2b0 con 0x7f604c100080 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f60427fc640 1 -- 172.21.15.100:0/1459142147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6044007d70 con 0x7f604c100080 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f60427fc640 1 -- 172.21.15.100:0/1459142147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400f870 con 0x7f604c100080 2024-03-12T05:11:58.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.050+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c19f7c0 con 0x7f604c100080 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f60427fc640 1 -- 172.21.15.100:0/1459142147 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f6044018a60 con 0x7f604c100080 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 Processor -- start 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 -- start start 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6c2d0 0x557a15c6c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6ccc0 0x557a15c717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15bdc280 con 0x557a15c71e30 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15bd97b0 con 0x557a15c6ccc0 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6056186740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15bd4f90 con 0x557a15c6c2d0 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6050a6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6050a6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44350/0 (socket says 172.21.15.100:44350) 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.053+0000 7f6050a6f640 1 -- 172.21.15.100:0/1782639399 learned_addr learned my addr 172.21.15.100:0/1782639399 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f60427fc640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f60380a1ac0 0x7f60380a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6051270640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6ccc0 0x557a15c717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6051a71640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6c2d0 0x557a15c6c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6051a71640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f60380a1ac0 0x7f60380a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6050a6f640 1 -- 172.21.15.100:0/1782639399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6c2d0 msgr2=0x557a15c6c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6050a6f640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6c2d0 0x557a15c6c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6050a6f640 1 -- 172.21.15.100:0/1782639399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6ccc0 msgr2=0x557a15c717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6050a6f640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6ccc0 0x557a15c717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6050a6f640 1 -- 172.21.15.100:0/1782639399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a15b93570 con 0x557a15c71e30 2024-03-12T05:11:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6051a71640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6c2d0 0x557a15c6c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6051270640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6ccc0 0x557a15c717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f60427fc640 1 -- 172.21.15.100:0/1459142147 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6044013070 con 0x7f604c100080 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6050a6f640 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c742c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f603c02c170 tx=0x7f603c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2748581b63a879 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f6051a71640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f60380a1ac0 0x7f60380a3f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f604800afd0 tx=0x7f6048005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f60377fe640 1 -- 172.21.15.100:0/1782639399 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0048c0 con 0x557a15c71e30 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f60377fe640 1 -- 172.21.15.100:0/1782639399 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f603c004510 con 0x557a15c71e30 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.054+0000 7f60377fe640 1 -- 172.21.15.100:0/1782639399 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0055a0 con 0x557a15c71e30 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 -- 172.21.15.100:0/1782639399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 msgr2=0x557a15c742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c742c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f603c02c170 tx=0x7f603c02ca10 comp rx=0 tx=0).stop 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 -- 172.21.15.100:0/1782639399 shutdown_connections 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c742c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6ccc0 0x557a15c717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 --2- 172.21.15.100:0/1782639399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6c2d0 0x557a15c6c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 -- 172.21.15.100:0/1782639399 >> 172.21.15.100:0/1782639399 conn(0x557a15c677f0 msgr2=0x557a15c67bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 -- 172.21.15.100:0/1782639399 shutdown_connections 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 -- 172.21.15.100:0/1782639399 wait complete. 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 -- start start 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.055+0000 7f6056186740 1 Processor -- start 2024-03-12T05:11:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c76700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c7d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a158a8430 con 0x557a15c71e30 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15864e00 con 0x557a15c6c2d0 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15b59fd0 con 0x557a15c6ccc0 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6056186740 10 client.0 _ll_register_callbacks cb 0x557a15baffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c7d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c7d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44364/0 (socket says 172.21.15.100:44364) 2024-03-12T05:11:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 -- 172.21.15.100:0/321517257 learned_addr learned my addr 172.21.15.100:0/321517257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051270640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6050a6f640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c76700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 -- 172.21.15.100:0/321517257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 msgr2=0x557a15c76700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c76700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 -- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 msgr2=0x557a15c761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 -- 172.21.15.100:0/321517257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c108dc0 con 0x557a15c71e30 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6051a71640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c7d740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f604800db20 tx=0x7f6048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2395c5e66d908e48 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.056+0000 7f6034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6034ff9640 1 -- 172.21.15.100:0/321517257 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604801d440 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a158d47e0 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6034ff9640 1 -- 172.21.15.100:0/321517257 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f604801d5a0 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6034ff9640 1 -- 172.21.15.100:0/321517257 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604801d760 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 10 client.15408 fetch_fsmap learned FSMap version 23 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557a15bd3610 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6034ff9640 1 -- 172.21.15.100:0/321517257 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a158d47e0 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6034ff9640 1 -- 172.21.15.100:0/321517257 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f604801a9d0 con 0x557a15c71e30 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 10 client.15408 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:11:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 20 client.15408 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15c824d0 0x557a15c84910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 4 client.15408 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a15911720 con 0x557a15c824d0 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.057+0000 7f6051270640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15c824d0 0x557a15c84910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.058+0000 7f6034ff9640 10 client.15408 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.058+0000 7f6051270640 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15c824d0 0x557a15c84910 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc482323d6d96952 server_cookie=b85f190af550a87d in_seq=0 out_seq=0 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.058+0000 7f6034ff9640 1 -- 172.21.15.100:0/321517257 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a15911720 con 0x557a15c824d0 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.058+0000 7f6034ff9640 10 client.15408 handle_command_reply: tid=0 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.058+0000 7f6056186740 1 client.15408 shutdown 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 msgr2=0x557a15c7d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c7d740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f604800db20 tx=0x7f6048004a40 comp rx=0 tx=0).stop 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 shutdown_connections 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15c824d0 0x557a15c84910 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c7d740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c76700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 --2- 172.21.15.100:0/321517257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6034ff9640 0 client.15408 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 >> 172.21.15.100:0/321517257 conn(0x557a15c677f0 msgr2=0x557a15c66a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 shutdown_connections 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 1 -- 172.21.15.100:0/321517257 wait complete. 2024-03-12T05:11:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.059+0000 7f6056186740 20 client.15408 trim_cache size 0 max 16384 2024-03-12T05:11:58.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1871311343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:58.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2778796174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:58.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3925904237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:58.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:57 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1637655322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 Processor -- start 2024-03-12T05:11:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 -- start start 2024-03-12T05:11:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c71550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c6df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15c80960 con 0x557a15c71e30 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15c752d0 con 0x557a15c6c2d0 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.074+0000 7f6056186740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15c6da60 con 0x557a15c6ccc0 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c6df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c6df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44372/0 (socket says 172.21.15.100:44372) 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 -- 172.21.15.100:0/1693766936 learned_addr learned my addr 172.21.15.100:0/1693766936 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6050a6f640 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c71550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051270640 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 -- 172.21.15.100:0/1693766936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 msgr2=0x557a15c71550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c71550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 -- 172.21.15.100:0/1693766936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 msgr2=0x557a15c6d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c6d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 -- 172.21.15.100:0/1693766936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a15b6fac0 con 0x557a15c71e30 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051270640 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6051a71640 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c6df00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6048012e60 tx=0x7f6048013f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bdfd942bb16521 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6036ffd640 1 -- 172.21.15.100:0/1693766936 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604801cee0 con 0x557a15c71e30 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6036ffd640 1 -- 172.21.15.100:0/1693766936 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f604801c7a0 con 0x557a15c71e30 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.075+0000 7f6036ffd640 1 -- 172.21.15.100:0/1693766936 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604801d6a0 con 0x557a15c71e30 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 -- 172.21.15.100:0/1693766936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 msgr2=0x557a15c6df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c6df00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6048012e60 tx=0x7f6048013f20 comp rx=0 tx=0).stop 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 -- 172.21.15.100:0/1693766936 shutdown_connections 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15c6df00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15c71550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 --2- 172.21.15.100:0/1693766936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15c6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 -- 172.21.15.100:0/1693766936 >> 172.21.15.100:0/1693766936 conn(0x557a15c677f0 msgr2=0x557a15c67bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 -- 172.21.15.100:0/1693766936 shutdown_connections 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 -- 172.21.15.100:0/1693766936 wait complete. 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 -- start start 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.076+0000 7f6056186740 1 Processor -- start 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15cd5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15cd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15cdc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15cdcb60 con 0x557a15c71e30 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15cdcca0 con 0x557a15c6c2d0 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a15cdce10 con 0x557a15c6ccc0 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 10 client.0 _ll_register_callbacks cb 0x7f603c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:11:58.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15cdc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15cdc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44376/0 (socket says 172.21.15.100:44376) 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 -- 172.21.15.100:0/1907746261 learned_addr learned my addr 172.21.15.100:0/1907746261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6050a6f640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15cd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051270640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15cd5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 -- 172.21.15.100:0/1907746261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 msgr2=0x557a15cd5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15cd5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 -- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 msgr2=0x557a15cd5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15cd5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 -- 172.21.15.100:0/1907746261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a15bd3610 con 0x557a15c71e30 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6051a71640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15cdc620 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6048015040 tx=0x7f6048004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677c9d98aeac49fa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6034ff9640 1 -- 172.21.15.100:0/1907746261 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604801d440 con 0x557a15c71e30 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f603c004510 con 0x557a15c71e30 2024-03-12T05:11:58.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.077+0000 7f6034ff9640 1 -- 172.21.15.100:0/1907746261 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f604801d5a0 con 0x557a15c71e30 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6034ff9640 1 -- 172.21.15.100:0/1907746261 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604801d760 con 0x557a15c71e30 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 10 client.15414 fetch_fsmap learned FSMap version 23 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557a15b956a0 con 0x557a15c71e30 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6034ff9640 1 -- 172.21.15.100:0/1907746261 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f603c004510 con 0x557a15c71e30 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6034ff9640 1 -- 172.21.15.100:0/1907746261 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6048030900 con 0x557a15c71e30 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 10 client.15414 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 20 client.15414 populate_metadata read hostname 'smithi100' 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15cde510 0x557a15ce0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 4 client.15414 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a15960890 con 0x557a15cde510 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.078+0000 7f6051270640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15cde510 0x557a15ce0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6034ff9640 10 client.15414 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6051270640 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15cde510 0x557a15ce0930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f238f13727f1a33 server_cookie=427b356f43f689e0 in_seq=0 out_seq=0 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6034ff9640 1 -- 172.21.15.100:0/1907746261 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557a15960890 con 0x557a15cde510 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6034ff9640 10 client.15414 handle_command_reply: tid=0 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 client.15414 shutdown 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 msgr2=0x557a15cdc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15cdc620 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6048015040 tx=0x7f6048004ab0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 shutdown_connections 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557a15cde510 0x557a15ce0930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a15c71e30 0x557a15cdc620 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a15c6ccc0 0x557a15cd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 --2- 172.21.15.100:0/1907746261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a15c6c2d0 0x557a15cd5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 >> 172.21.15.100:0/1907746261 conn(0x557a15c677f0 msgr2=0x557a15c76160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6034ff9640 0 client.15414 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:11:58.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 shutdown_connections 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 1 -- 172.21.15.100:0/1907746261 wait complete. 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.079+0000 7f6056186740 20 client.15414 trim_cache size 0 max 16384 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout: "0ef25f63-046a-4f21-99e5-846dd3a4f010": { 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "0ef25f63-046a-4f21-99e5-846dd3a4f010", 2024-03-12T05:11:58.086 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:11:58.087 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:11:58.087 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:11:58.087 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f60380a1ac0 msgr2=0x7f60380a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f60380a1ac0 0x7f60380a3f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f604800afd0 tx=0x7f6048005e80 comp rx=0 tx=0).stop 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 msgr2=0x7f604c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 0x7f604c1971b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f604400c970 tx=0x7f604400ce30 comp rx=0 tx=0).stop 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 shutdown_connections 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f60380a1ac0 0x7f60380a3f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f604c101c30 0x7f604c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c101270 0x7f604c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 --2- 172.21.15.100:0/1459142147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f604c100080 0x7f604c1971b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 >> 172.21.15.100:0/1459142147 conn(0x7f604c0fb5f0 msgr2=0x7f604c0682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 shutdown_connections 2024-03-12T05:11:58.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.080+0000 7f60534d4640 1 -- 172.21.15.100:0/1459142147 wait complete. 2024-03-12T05:11:58.092 INFO:tasks.cephfs.filesystem:scrub status for tag:0ef25f63-046a-4f21-99e5-846dd3a4f010 - {'path': '/', 'tag': '0ef25f63-046a-4f21-99e5-846dd3a4f010', 'options': 'recursive,force'} 2024-03-12T05:11:58.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.144+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f4b5002ce40 con 0x7f4b501011e0 2024-03-12T05:11:58.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:57 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1871311343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:58.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:57 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2778796174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:58.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:57 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3925904237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:58.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:57 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1637655322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:58.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:57 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1871311343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:11:58.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:57 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2778796174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:11:58.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:57 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3925904237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:11:58.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:57 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1637655322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:11:58.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.828+0000 7f4b4dffb640 1 -- 172.21.15.100:0/4021990792 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7f4b44099a40 con 0x7f4b501011e0 2024-03-12T05:11:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4b400a1bf0 msgr2=0x7f4b400a40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4b400a1bf0 0x7f4b400a40b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c005fd0 tx=0x7f4b3c005850 comp rx=0 tx=0).stop 2024-03-12T05:11:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 msgr2=0x7f4b50197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 0x7f4b50197120 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d770 tx=0x7f4b4400dc30 comp rx=0 tx=0).stop 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 shutdown_connections 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f4b400a1bf0 0x7f4b400a40b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b50102d90 0x7f4b5019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b501023d0 0x7f4b50197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 --2- 172.21.15.100:0/4021990792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b501011e0 0x7f4b50197120 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 >> 172.21.15.100:0/4021990792 conn(0x7f4b500fc750 msgr2=0x7f4b50105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 shutdown_connections 2024-03-12T05:11:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.831+0000 7f4b56eb5640 1 -- 172.21.15.100:0/4021990792 wait complete. 2024-03-12T05:11:58.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-12T05:11:58.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 -- 172.21.15.100:0/237156125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 msgr2=0x7fdc8c108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/237156125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c108530 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009ca0 tx=0x7fdc8402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 -- 172.21.15.100:0/237156125 shutdown_connections 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/237156125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c108530 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/237156125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc8c1012e0 0x7fdc8c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/237156125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc8c1000f0 0x7fdc8c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 -- 172.21.15.100:0/237156125 >> 172.21.15.100:0/237156125 conn(0x7fdc8c0fb660 msgr2=0x7fdc8c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 -- 172.21.15.100:0/237156125 shutdown_connections 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.959+0000 7fdc92ae1640 1 -- 172.21.15.100:0/237156125 wait complete. 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 Processor -- start 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 -- start start 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc8c1000f0 0x7fdc8c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc8c1012e0 0x7fdc8c1026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c103360 con 0x7fdc8c101ca0 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c1034d0 con 0x7fdc8c1012e0 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc92ae1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c1a11d0 con 0x7fdc8c1000f0 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44408/0 (socket says 172.21.15.100:44408) 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 -- 172.21.15.100:0/1906512136 learned_addr learned my addr 172.21.15.100:0/1906512136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9087d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc8c1000f0 0x7fdc8c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc8bfff640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc8c1012e0 0x7fdc8c1026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 -- 172.21.15.100:0/1906512136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc8c1000f0 msgr2=0x7fdc8c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc8c1000f0 0x7fdc8c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 -- 172.21.15.100:0/1906512136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc8c1012e0 msgr2=0x7fdc8c1026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc8c1012e0 0x7fdc8c1026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.960+0000 7fdc9107e640 1 -- 172.21.15.100:0/1906512136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc84009900 con 0x7fdc8c101ca0 2024-03-12T05:11:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.961+0000 7fdc9107e640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c102c00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc84033000 tx=0x7fdc84009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.961+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840329b0 con 0x7fdc8c101ca0 2024-03-12T05:11:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.961+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c1a1420 con 0x7fdc8c101ca0 2024-03-12T05:11:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.961+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc84003e50 con 0x7fdc8c101ca0 2024-03-12T05:11:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.961+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84040450 con 0x7fdc8c101ca0 2024-03-12T05:11:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.961+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c1a1920 con 0x7fdc8c101ca0 2024-03-12T05:11:58.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.962+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c10bb00 con 0x7fdc8c101ca0 2024-03-12T05:11:58.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.963+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136348+0+0 (secure 0 0 0) 0x7fdc8400a040 con 0x7fdc8c101ca0 2024-03-12T05:11:58.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.964+0000 7fdc89ffb640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 0x7fdc7c0a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:58.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.964+0000 7fdc9087d640 1 -- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 msgr2=0x7fdc7c0a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:11:58.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.964+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdc840f0260 con 0x7fdc8c101ca0 2024-03-12T05:11:58.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.964+0000 7fdc9087d640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 0x7fdc7c0a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:11:58.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:58.968+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc840b9420 con 0x7fdc8c101ca0 2024-03-12T05:11:59.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.162+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fdc8c10bc90 con 0x7fdc8c101ca0 2024-03-12T05:11:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.165+0000 7fdc9087d640 1 -- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 msgr2=0x7fdc7c0a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:11:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.165+0000 7fdc9087d640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 0x7fdc7c0a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T05:11:59.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.166+0000 7fdc89ffb640 1 -- 172.21.15.100:0/1906512136 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdc840bcae0 con 0x7fdc8c101ca0 2024-03-12T05:11:59.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 msgr2=0x7fdc7c0a4080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:11:59.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 0x7fdc7c0a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 msgr2=0x7fdc8c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c102c00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc84033000 tx=0x7fdc84009360 comp rx=0 tx=0).stop 2024-03-12T05:11:59.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 shutdown_connections 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7fdc7c0a1bc0 0x7fdc7c0a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c101ca0 0x7fdc8c102c00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc8c1012e0 0x7fdc8c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 --2- 172.21.15.100:0/1906512136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc8c1000f0 0x7fdc8c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 >> 172.21.15.100:0/1906512136 conn(0x7fdc8c0fb660 msgr2=0x7fdc8c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 shutdown_connections 2024-03-12T05:11:59.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.168+0000 7fdc92ae1640 1 -- 172.21.15.100:0/1906512136 wait complete. 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: ignoring --setuser ceph since I am not root 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: ignoring --setgroup ceph since I am not root 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-mon[32210]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:11:59.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:58 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4021990792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-12T05:11:59.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-12T05:11:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mon[36270]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-03-12T05:11:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:11:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:11:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:11:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:11:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4021990792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-12T05:11:59.206 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: ignoring --setuser ceph since I am not root 2024-03-12T05:11:59.206 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: ignoring --setgroup ceph since I am not root 2024-03-12T05:11:59.206 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:11:58 smithi156 ceph-mgr[37652]: -- 172.21.15.156:0/4255065727 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5637dcf0a1a0 con 0x5637dcee0800 2024-03-12T05:11:59.223 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: ignoring --setuser ceph since I am not root 2024-03-12T05:11:59.223 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: ignoring --setgroup ceph since I am not root 2024-03-12T05:11:59.223 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mgr[38071]: -- 172.21.15.161:0/1137740881 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55827a9f21a0 con 0x55827a9c8800 2024-03-12T05:11:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mon[36043]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-03-12T05:11:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:11:59.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:11:59.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:11:59.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:11:59.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:58 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4021990792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/842599840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea0101290 msgr2=0x7f6ea0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/842599840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea0101290 0x7f6ea0101710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009900 tx=0x7f6e8802bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/842599840 shutdown_connections 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/842599840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101c50 0x7f6ea01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/842599840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea0101290 0x7f6ea0101710 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/842599840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01000a0 0x7f6ea01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/842599840 >> 172.21.15.100:0/842599840 conn(0x7f6ea00fb610 msgr2=0x7f6ea00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.284+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/842599840 shutdown_connections 2024-03-12T05:11:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/842599840 wait complete. 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 Processor -- start 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 -- start start 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 0x7f6ea0197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101290 0x7f6ea01976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea0101c50 0x7f6ea019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea019ec80 con 0x7f6ea01000a0 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea019edf0 con 0x7f6ea0101290 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6ea5cbb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea019efb0 con 0x7f6ea0101c50 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6e9f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 0x7f6ea0197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6e9f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 0x7f6ea0197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44430/0 (socket says 172.21.15.100:44430) 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.285+0000 7f6e9f7fe640 1 -- 172.21.15.100:0/2830098863 learned_addr learned my addr 172.21.15.100:0/2830098863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e96ffd640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101290 0x7f6ea01976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:59.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9ffff640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea0101c50 0x7f6ea019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9f7fe640 1 -- 172.21.15.100:0/2830098863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea0101c50 msgr2=0x7f6ea019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9f7fe640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea0101c50 0x7f6ea019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9f7fe640 1 -- 172.21.15.100:0/2830098863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101290 msgr2=0x7f6ea01976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9f7fe640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101290 0x7f6ea01976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9f7fe640 1 -- 172.21.15.100:0/2830098863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e88009560 con 0x7f6ea01000a0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e96ffd640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101290 0x7f6ea01976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9f7fe640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 0x7f6ea0197190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000d950 tx=0x7f6e9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90009c30 con 0x7f6ea01000a0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e90009d90 con 0x7f6ea01000a0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000f850 con 0x7f6ea01000a0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea019f290 con 0x7f6ea01000a0 2024-03-12T05:11:59.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.286+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea019f770 con 0x7f6ea01000a0 2024-03-12T05:11:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.288+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea0107360 con 0x7f6ea01000a0 2024-03-12T05:11:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.288+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136348+0+0 (secure 0 0 0) 0x7f6e9000f9b0 con 0x7f6ea01000a0 2024-03-12T05:11:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.289+0000 7f6e9d7fa640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e8c0a1bc0 0x7f6e8c0a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.289+0000 7f6e96ffd640 1 -- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e8c0a1bc0 msgr2=0x7f6e8c0a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:11:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.289+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e900cd220 con 0x7f6ea01000a0 2024-03-12T05:11:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.289+0000 7f6e96ffd640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e8c0a1bc0 0x7f6e8c0a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:11:59.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.293+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e900962d0 con 0x7f6ea01000a0 2024-03-12T05:11:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.476+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6ea010cbb0 con 0x7f6ea01000a0 2024-03-12T05:11:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.479+0000 7f6e9d7fa640 1 -- 172.21.15.100:0/2830098863 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6e90099990 con 0x7f6ea01000a0 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.481+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e8c0a1bc0 msgr2=0x7f6e8c0a4080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.481+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e8c0a1bc0 0x7f6e8c0a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.481+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 msgr2=0x7f6ea0197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.481+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 0x7f6ea0197190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000d950 tx=0x7f6e9000de10 comp rx=0 tx=0).stop 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 shutdown_connections 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f6e8c0a1bc0 0x7f6e8c0a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea0101c50 0x7f6ea019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea0101290 0x7f6ea01976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 --2- 172.21.15.100:0/2830098863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea01000a0 0x7f6ea0197190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 >> 172.21.15.100:0/2830098863 conn(0x7f6ea00fb610 msgr2=0x7f6ea0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 shutdown_connections 2024-03-12T05:11:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.482+0000 7f6ea5cbb640 1 -- 172.21.15.100:0/2830098863 wait complete. 2024-03-12T05:11:59.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-12T05:11:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 -- 172.21.15.100:0/2572707566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280104550 msgr2=0x7f3280106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 --2- 172.21.15.100:0/2572707566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280104550 0x7f3280106940 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3270009900 tx=0x7f327002bc30 comp rx=0 tx=0).stop 2024-03-12T05:11:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 -- 172.21.15.100:0/2572707566 shutdown_connections 2024-03-12T05:11:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 --2- 172.21.15.100:0/2572707566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3280106e80 0x7f32801092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 --2- 172.21.15.100:0/2572707566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280104550 0x7f3280106940 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 --2- 172.21.15.100:0/2572707566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3280101c20 0x7f3280104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.598+0000 7f3285bd0640 1 -- 172.21.15.100:0/2572707566 >> 172.21.15.100:0/2572707566 conn(0x7f32800fb610 msgr2=0x7f32800fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:11:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 -- 172.21.15.100:0/2572707566 shutdown_connections 2024-03-12T05:11:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 -- 172.21.15.100:0/2572707566 wait complete. 2024-03-12T05:11:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 Processor -- start 2024-03-12T05:11:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 -- start start 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3280101c20 0x7f3280197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3280104550 0x7f3280197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280106e80 0x7f328019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328019ed10 con 0x7f3280106e80 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328019ee80 con 0x7f3280104550 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.599+0000 7f3285bd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328019f040 con 0x7f3280101c20 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280106e80 0x7f328019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3280101c20 0x7f3280197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3280104550 0x7f3280197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280106e80 0x7f328019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44442/0 (socket says 172.21.15.100:44442) 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3280101c20 0x7f3280197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52572/0 (socket says 172.21.15.100:52572) 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 -- 172.21.15.100:0/4266942386 learned_addr learned my addr 172.21.15.100:0/4266942386 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3280101c20 msgr2=0x7f3280197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3280101c20 0x7f3280197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3280104550 msgr2=0x7f3280197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3280104550 0x7f3280197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 -- 172.21.15.100:0/4266942386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270009560 con 0x7f3280106e80 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327effd640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3280104550 0x7f3280197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327ffff640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280106e80 0x7f328019e7d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3278009fd0 tx=0x7f327800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800bea0 con 0x7f3280106e80 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328019f320 con 0x7f3280106e80 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3278009940 con 0x7f3280106e80 2024-03-12T05:11:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780106c0 con 0x7f3280106e80 2024-03-12T05:11:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.600+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328019f7b0 con 0x7f3280106e80 2024-03-12T05:11:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.601+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32800fd100 con 0x7f3280106e80 2024-03-12T05:11:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.603+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136348+0+0 (secure 0 0 0) 0x7f3278004240 con 0x7f3280106e80 2024-03-12T05:11:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.604+0000 7f327cff9640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:11:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.604+0000 7f327f7fe640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 msgr2=0x7f32680a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:11:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.604+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32780cd210 con 0x7f3280106e80 2024-03-12T05:11:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.604+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T05:11:59.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.607+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3278096370 con 0x7f3280106e80 2024-03-12T05:11:59.705 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:11:59.400+0000 7fa16173a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:11:59.705 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:11:59.494+0000 7fa16173a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:11:59.723 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:11:59.405+0000 7f4f47d3f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:11:59.723 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:11:59.500+0000 7f4f47d3f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:11:59.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.805+0000 7f327f7fe640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 msgr2=0x7f32680a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:11:59.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.805+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-12T05:11:59.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:11:59.806+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 --> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f328002ceb0 con 0x7f32680a1bc0 2024-03-12T05:11:59.818 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:59.435+0000 7fc563a6d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T05:11:59.818 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:11:59.534+0000 7fc563a6d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T05:12:00.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4021990792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-12T05:12:00.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-mon[36270]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-03-12T05:12:00.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1906512136' entity='client.admin' 2024-03-12T05:12:00.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2830098863' entity='client.admin' 2024-03-12T05:12:00.130 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:11:59 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:12:00.136 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4021990792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-12T05:12:00.136 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-mon[36043]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-03-12T05:12:00.136 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1906512136' entity='client.admin' 2024-03-12T05:12:00.136 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2830098863' entity='client.admin' 2024-03-12T05:12:00.136 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:11:59 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:12:00.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4021990792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-12T05:12:00.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-mon[32210]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-03-12T05:12:00.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1906512136' entity='client.admin' 2024-03-12T05:12:00.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2830098863' entity='client.admin' 2024-03-12T05:12:00.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:11:59 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:12:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:00.206+0000 7f327f7fe640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 msgr2=0x7f32680a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:12:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:00.206+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-12T05:12:00.454 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.133+0000 7fa16173a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:12:00.455 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.226+0000 7fa16173a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:12:00.455 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.368+0000 7fa16173a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:12:00.467 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.139+0000 7f4f47d3f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:12:00.467 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.232+0000 7f4f47d3f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:12:00.467 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.374+0000 7f4f47d3f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:12:00.507 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.182+0000 7fc563a6d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T05:12:00.507 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.275+0000 7fc563a6d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T05:12:00.507 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.415+0000 7fc563a6d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T05:12:00.507 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.509+0000 7fc563a6d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:12:00.722 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.470+0000 7f4f47d3f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:12:00.723 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.553+0000 7f4f47d3f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:12:00.814 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.464+0000 7fa16173a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T05:12:00.814 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.546+0000 7fa16173a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:12:00.817 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.592+0000 7fc563a6d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T05:12:01.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:01.007+0000 7f327f7fe640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 msgr2=0x7f32680a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:12:01.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:01.007+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-12T05:12:01.106 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.817+0000 7fa16173a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:12:01.106 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:00.928+0000 7fa16173a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:12:01.106 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:01.015+0000 7fa16173a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:12:01.106 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:01.109+0000 7fa16173a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:12:01.107 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.819+0000 7f4f47d3f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:12:01.109 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:00.929+0000 7f4f47d3f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:12:01.109 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:01.014+0000 7f4f47d3f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:12:01.110 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:01.108+0000 7f4f47d3f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:12:01.143 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.855+0000 7fc563a6d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T05:12:01.143 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:00.964+0000 7fc563a6d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T05:12:01.143 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:01.051+0000 7fc563a6d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T05:12:01.143 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:01.145+0000 7fc563a6d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T05:12:01.454 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:01.286+0000 7fa16173a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:12:01.472 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:01.286+0000 7f4f47d3f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:12:01.567 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:01.323+0000 7fc563a6d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T05:12:02.567 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:02.278+0000 7fc563a6d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:12:02.568 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:02.408+0000 7fc563a6d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:12:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:02.609+0000 7f327f7fe640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 msgr2=0x7f32680a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/327651532 2024-03-12T05:12:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:02.609+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-12T05:12:02.704 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:02.250+0000 7fa16173a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:12:02.705 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:02.378+0000 7fa16173a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:12:02.718 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:02.254+0000 7f4f47d3f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T05:12:02.718 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:02.382+0000 7f4f47d3f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T05:12:02.972 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:02.720+0000 7f4f47d3f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:12:02.973 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:02.811+0000 7f4f47d3f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:12:03.000 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:02.716+0000 7fa16173a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:12:03.000 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:02.808+0000 7fa16173a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:12:03.000 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:03.003+0000 7fa16173a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:12:03.030 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:02.744+0000 7fc563a6d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T05:12:03.030 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:02.835+0000 7fc563a6d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T05:12:03.317 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:03.033+0000 7fc563a6d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:12:03.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:03.219+0000 7fc563a6d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:12:03.454 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:03.188+0000 7fa16173a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:12:03.472 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:03.006+0000 7f4f47d3f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:12:03.473 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:03.193+0000 7f4f47d3f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T05:12:03.732 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:03.472+0000 7fa16173a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:12:03.732 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:03.735+0000 7fa16173a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:12:03.739 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:03.478+0000 7f4f47d3f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:12:03.761 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:03.505+0000 7fc563a6d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T05:12:04.067 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:03.764+0000 7fc563a6d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:12:04.068 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:04.002+0000 7fc563a6d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:12:04.204 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:03.971+0000 7fa16173a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:12:04.222 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:03.741+0000 7f4f47d3f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T05:12:04.222 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:03.976+0000 7f4f47d3f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T05:12:05.068 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:04.812+0000 7fc563a6d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:12:05.124 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:04.776+0000 7f4f47d3f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:12:05.204 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:04.769+0000 7fa16173a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T05:12:05.450 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:05.354+0000 7f4f47d3f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:12:05.654 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:05.344+0000 7fa16173a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:12:05.654 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:05.442+0000 7fa16173a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:12:05.654 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y[37648]: 2024-03-12T05:12:05.657+0000 7fa16173a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:12:05.704 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.383+0000 7fc563a6d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T05:12:05.704 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.484+0000 7fc563a6d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:12:05.723 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:05.452+0000 7f4f47d3f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T05:12:05.723 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z[38067]: 2024-03-12T05:12:05.669+0000 7f4f47d3f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:12:05.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:05.734+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 21) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f3278096740 con 0x7f3280106e80 2024-03-12T05:12:05.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:05.734+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 msgr2=0x7f32680a4080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:12:05.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:05.734+0000 7f327cff9640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/327651532,v1:172.21.15.100:6801/327651532] conn(0x7f32680a1bc0 0x7f32680a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: Standby manager daemon y restarted 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: Standby manager daemon y started 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: Standby manager daemon z restarted 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: Standby manager daemon z started 2024-03-12T05:12:05.955 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T05:12:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:12:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T05:12:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:12:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: Active manager daemon x restarted 2024-03-12T05:12:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:05 smithi156 ceph-mon[36270]: Activating manager daemon x 2024-03-12T05:12:05.968 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.707+0000 7fc563a6d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T05:12:05.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: Standby manager daemon y restarted 2024-03-12T05:12:05.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: Standby manager daemon y started 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: Standby manager daemon z restarted 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: Standby manager daemon z started 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: Active manager daemon x restarted 2024-03-12T05:12:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-mon[32210]: Activating manager daemon x 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: Standby manager daemon y restarted 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: Standby manager daemon y started 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.156:0/3363036101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:12:05.986 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: Standby manager daemon z restarted 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: Standby manager daemon z started 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: from='mgr.? 172.21.15.161:0/3722845861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: Active manager daemon x restarted 2024-03-12T05:12:05.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:05 smithi161 ceph-mon[36043]: Activating manager daemon x 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.970+0000 7fc4c545d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.970+0000 7fc4c545d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.970+0000 7fc4c545d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.970+0000 7fc4c545d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.970+0000 7fc4c545d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.973+0000 7fc4c2c58700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.973+0000 7fc4c2c58700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.318 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.973+0000 7fc4c2c58700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.319 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.973+0000 7fc4c2c58700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.319 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:12:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:12:05.973+0000 7fc4c2c58700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:12:06.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.758+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f327809b350 con 0x7f3280106e80 2024-03-12T05:12:06.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.758+0000 7f327cff9640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3268050fd0 0x7f32680513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:06.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.758+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 --> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f328002ceb0 con 0x7f3268050fd0 2024-03-12T05:12:06.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.762+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3268050fd0 0x7f32680513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:06.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.764+0000 7f327f7fe640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3268050fd0 0x7f32680513b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3274005d00 tx=0x7f327400b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:06.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.823+0000 7f327cff9640 1 -- 172.21.15.100:0/4266942386 <== mgr.15432 v2:172.21.15.100:6800/2025021937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f328002ceb0 con 0x7f3268050fd0 2024-03-12T05:12:06.822 INFO:teuthology.orchestra.run.smithi100.stdout:Schedule set for path / 2024-03-12T05:12:06.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3268050fd0 msgr2=0x7f32680513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:06.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f3285bd0640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3268050fd0 0x7f32680513b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3274005d00 tx=0x7f327400b410 comp rx=0 tx=0).stop 2024-03-12T05:12:06.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280106e80 msgr2=0x7f328019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:06.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f3285bd0640 1 --2- 172.21.15.100:0/4266942386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3280106e80 0x7f328019e7d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3278009fd0 tx=0x7f327800b260 comp rx=0 tx=0).stop 2024-03-12T05:12:06.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f327ffff640 1 -- 172.21.15.100:0/4266942386 reap_dead start 2024-03-12T05:12:06.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 shutdown_connections 2024-03-12T05:12:06.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.825+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 >> 172.21.15.100:0/4266942386 conn(0x7f32800fb610 msgr2=0x7f3280102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:06.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.826+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 shutdown_connections 2024-03-12T05:12:06.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.826+0000 7f3285bd0640 1 -- 172.21.15.100:0/4266942386 wait complete. 2024-03-12T05:12:06.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-12T05:12:06.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 -- 172.21.15.100:0/3584676346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 msgr2=0x7f4e641092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 --2- 172.21.15.100:0/3584676346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e641092d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009ca0 tx=0x7f4e5c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 -- 172.21.15.100:0/3584676346 shutdown_connections 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 --2- 172.21.15.100:0/3584676346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e641092d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 --2- 172.21.15.100:0/3584676346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e64104550 0x7f4e64106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 --2- 172.21.15.100:0/3584676346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e64101c20 0x7f4e64104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 -- 172.21.15.100:0/3584676346 >> 172.21.15.100:0/3584676346 conn(0x7f4e640fb610 msgr2=0x7f4e640fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 -- 172.21.15.100:0/3584676346 shutdown_connections 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.943+0000 7f4e692f1640 1 -- 172.21.15.100:0/3584676346 wait complete. 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 Processor -- start 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 -- start start 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e64101c20 0x7f4e641971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:06.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e64104550 0x7f4e641976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e6419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6419ec90 con 0x7f4e64106e80 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6419ee00 con 0x7f4e64104550 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e692f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6419efc0 con 0x7f4e64101c20 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e6419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e6419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51142/0 (socket says 172.21.15.100:51142) 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e637fe640 1 -- 172.21.15.100:0/2828194185 learned_addr learned my addr 172.21.15.100:0/2828194185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e62ffd640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e64101c20 0x7f4e641971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.944+0000 7f4e5a7fc640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e64104550 0x7f4e641976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e637fe640 1 -- 172.21.15.100:0/2828194185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e64101c20 msgr2=0x7f4e641971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e637fe640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e64101c20 0x7f4e641971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e637fe640 1 -- 172.21.15.100:0/2828194185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e64104550 msgr2=0x7f4e641976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:06.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e637fe640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e64104550 0x7f4e641976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e637fe640 1 -- 172.21.15.100:0/2828194185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c009900 con 0x7f4e64106e80 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e637fe640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e6419e750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c037040 tx=0x7f4e5c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0329b0 con 0x7f4e64106e80 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c003e50 con 0x7f4e64106e80 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6419f240 con 0x7f4e64106e80 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0048a0 con 0x7f4e64106e80 2024-03-12T05:12:06.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.945+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6419f6f0 con 0x7f4e64106e80 2024-03-12T05:12:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.946+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e640fd100 con 0x7f4e64106e80 2024-03-12T05:12:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.947+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f4e5c00a040 con 0x7f4e64106e80 2024-03-12T05:12:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.948+0000 7f4e60ff9640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4e500a18e0 0x7f4e500a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.948+0000 7f4e62ffd640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4e500a18e0 0x7f4e500a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.948+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4e5c0eff50 con 0x7f4e64106e80 2024-03-12T05:12:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.948+0000 7f4e62ffd640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4e500a18e0 0x7f4e500a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009bd0 tx=0x7f4e54005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:06.953+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e5c0b93b0 con 0x7f4e64106e80 2024-03-12T05:12:07.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: osdmap e89: 12 total, 12 up, 12 in 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: mgrmap e21: x(active, starting, since 0.0152817s), standbys: y, z 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T05:12:07.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:12:07.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:12:07.045 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:12:07.046 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: Manager daemon x is now available 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:07.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T05:12:07.048 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:06 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T05:12:07.053 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: osdmap e89: 12 total, 12 up, 12 in 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: mgrmap e21: x(active, starting, since 0.0152817s), standbys: y, z 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T05:12:07.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:12:07.055 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: Manager daemon x is now available 2024-03-12T05:12:07.056 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:07.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:07.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T05:12:07.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:06 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T05:12:07.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: osdmap e89: 12 total, 12 up, 12 in 2024-03-12T05:12:07.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: mgrmap e21: x(active, starting, since 0.0152817s), standbys: y, z 2024-03-12T05:12:07.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-12T05:12:07.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-12T05:12:07.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-12T05:12:07.084 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-12T05:12:07.085 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T05:12:07.086 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T05:12:07.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T05:12:07.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: Manager daemon x is now available 2024-03-12T05:12:07.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:07.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:07.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-12T05:12:07.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:06 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-12T05:12:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.159+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 --> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e6402cf50 con 0x7f4e500a18e0 2024-03-12T05:12:07.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.215+0000 7f4e60ff9640 1 -- 172.21.15.100:0/2828194185 <== mgr.15432 v2:172.21.15.100:6800/2025021937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f4e6402cf50 con 0x7f4e500a18e0 2024-03-12T05:12:07.212 INFO:teuthology.orchestra.run.smithi100.stdout:Retention added to path / 2024-03-12T05:12:07.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4e500a18e0 msgr2=0x7f4e500a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4e500a18e0 0x7f4e500a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009bd0 tx=0x7f4e54005f90 comp rx=0 tx=0).stop 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 msgr2=0x7f4e6419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e6419e750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c037040 tx=0x7f4e5c009360 comp rx=0 tx=0).stop 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 shutdown_connections 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4e500a18e0 0x7f4e500a3da0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e64106e80 0x7f4e6419e750 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e64104550 0x7f4e641976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 --2- 172.21.15.100:0/2828194185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e64101c20 0x7f4e641971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 >> 172.21.15.100:0/2828194185 conn(0x7f4e640fb610 msgr2=0x7f4e64102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 shutdown_connections 2024-03-12T05:12:07.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.217+0000 7f4e692f1640 1 -- 172.21.15.100:0/2828194185 wait complete. 2024-03-12T05:12:07.225 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-12T05:12:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 -- 172.21.15.100:0/591082452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101290 msgr2=0x7f16ec101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 --2- 172.21.15.100:0/591082452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101290 0x7f16ec101710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009900 tx=0x7f16dc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:12:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 -- 172.21.15.100:0/591082452 shutdown_connections 2024-03-12T05:12:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 --2- 172.21.15.100:0/591082452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec101c50 0x7f16ec1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 --2- 172.21.15.100:0/591082452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101290 0x7f16ec101710 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 --2- 172.21.15.100:0/591082452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec1000a0 0x7f16ec1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 -- 172.21.15.100:0/591082452 >> 172.21.15.100:0/591082452 conn(0x7f16ec0fb610 msgr2=0x7f16ec0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 -- 172.21.15.100:0/591082452 shutdown_connections 2024-03-12T05:12:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 -- 172.21.15.100:0/591082452 wait complete. 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.333+0000 7f16f116c640 1 Processor -- start 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 -- start start 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec1000a0 0x7f16ec1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec101290 0x7f16ec196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 0x7f16ec19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19db90 con 0x7f16ec101c50 2024-03-12T05:12:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19dd00 con 0x7f16ec1000a0 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16f116c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19dec0 con 0x7f16ec101290 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 0x7f16ec19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 0x7f16ec19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51158/0 (socket says 172.21.15.100:51158) 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 -- 172.21.15.100:0/3225700571 learned_addr learned my addr 172.21.15.100:0/3225700571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16ea59c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec101290 0x7f16ec196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16ead9d640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec1000a0 0x7f16ec1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 -- 172.21.15.100:0/3225700571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec101290 msgr2=0x7f16ec196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec101290 0x7f16ec196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 -- 172.21.15.100:0/3225700571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec1000a0 msgr2=0x7f16ec1960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec1000a0 0x7f16ec1960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.334+0000 7f16eb59e640 1 -- 172.21.15.100:0/3225700571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc009560 con 0x7f16ec101c50 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.335+0000 7f16eb59e640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 0x7f16ec19d650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f16e400eaa0 tx=0x7f16e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.335+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400cdb0 con 0x7f16ec101c50 2024-03-12T05:12:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.335+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16e4005020 con 0x7f16ec101c50 2024-03-12T05:12:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.335+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e40116b0 con 0x7f16ec101c50 2024-03-12T05:12:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.335+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec19e1a0 con 0x7f16ec101c50 2024-03-12T05:12:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.335+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec19e680 con 0x7f16ec101c50 2024-03-12T05:12:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.336+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec1073c0 con 0x7f16ec101c50 2024-03-12T05:12:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.337+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f16e4005520 con 0x7f16ec101c50 2024-03-12T05:12:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.338+0000 7f16d3fff640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f16d40a18e0 0x7f16d40a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.338+0000 7f16ead9d640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f16d40a18e0 0x7f16d40a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.338+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f16e40cd670 con 0x7f16ec101c50 2024-03-12T05:12:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.338+0000 7f16ead9d640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f16d40a18e0 0x7f16d40a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16ec1010f0 tx=0x7f16e000b390 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.344+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16e4096af0 con 0x7f16ec101c50 2024-03-12T05:12:07.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.575+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 --> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f16ec02ceb0 con 0x7f16d40a18e0 2024-03-12T05:12:07.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.583+0000 7f16d3fff640 1 -- 172.21.15.100:0/3225700571 <== mgr.15432 v2:172.21.15.100:6800/2025021937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f16ec02ceb0 con 0x7f16d40a18e0 2024-03-12T05:12:07.581 INFO:teuthology.orchestra.run.smithi100.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-12T00:00:00", "created": "2024-03-12T05:12:06", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.585+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f16d40a18e0 msgr2=0x7f16d40a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.585+0000 7f16f116c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f16d40a18e0 0x7f16d40a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16ec1010f0 tx=0x7f16e000b390 comp rx=0 tx=0).stop 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.585+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 msgr2=0x7f16ec19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.585+0000 7f16f116c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 0x7f16ec19d650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f16e400eaa0 tx=0x7f16e400ef60 comp rx=0 tx=0).stop 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.585+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 shutdown_connections 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f16d40a18e0 0x7f16d40a3da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16ec101c50 0x7f16ec19d650 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16ec101290 0x7f16ec196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 --2- 172.21.15.100:0/3225700571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec1000a0 0x7f16ec1960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 >> 172.21.15.100:0/3225700571 conn(0x7f16ec0fb610 msgr2=0x7f16ec104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:07.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 shutdown_connections 2024-03-12T05:12:07.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.586+0000 7f16f116c640 1 -- 172.21.15.100:0/3225700571 wait complete. 2024-03-12T05:12:07.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-12T05:12:07.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 -- 172.21.15.100:0/3919376845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01011d0 msgr2=0x7f91c01015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 --2- 172.21.15.100:0/3919376845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01011d0 0x7f91c01015f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009900 tx=0x7f91bc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:12:07.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 -- 172.21.15.100:0/3919376845 shutdown_connections 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 --2- 172.21.15.100:0/3919376845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c0102d80 0x7f91c0109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 --2- 172.21.15.100:0/3919376845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c01023c0 0x7f91c0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 --2- 172.21.15.100:0/3919376845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01011d0 0x7f91c01015f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.719+0000 7f91c8202640 1 -- 172.21.15.100:0/3919376845 >> 172.21.15.100:0/3919376845 conn(0x7f91c00fc740 msgr2=0x7f91c00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 -- 172.21.15.100:0/3919376845 shutdown_connections 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 -- 172.21.15.100:0/3919376845 wait complete. 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 Processor -- start 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 -- start start 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c01011d0 0x7f91c0074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 0x7f91c0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c0102d80 0x7f91c0073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c0074240 con 0x7f91c01023c0 2024-03-12T05:12:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00743b0 con 0x7f91c01011d0 2024-03-12T05:12:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c8202640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c01a12a0 con 0x7f91c0102d80 2024-03-12T05:12:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c579d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 0x7f91c0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.720+0000 7f91c579d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 0x7f91c0073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51180/0 (socket says 172.21.15.100:51180) 2024-03-12T05:12:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 -- 172.21.15.100:0/936083780 learned_addr learned my addr 172.21.15.100:0/936083780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c679f640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c0102d80 0x7f91c0073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c5f9e640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c01011d0 0x7f91c0074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 -- 172.21.15.100:0/936083780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c0102d80 msgr2=0x7f91c0073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c0102d80 0x7f91c0073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 -- 172.21.15.100:0/936083780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c01011d0 msgr2=0x7f91c0074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c01011d0 0x7f91c0074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 -- 172.21.15.100:0/936083780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc009560 con 0x7f91c01023c0 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c679f640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c0102d80 0x7f91c0073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c5f9e640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c01011d0 0x7f91c0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c579d640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 0x7f91c0073570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f91b000ea20 tx=0x7f91b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:07.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0009720 con 0x7f91c01023c0 2024-03-12T05:12:07.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91b000cd80 con 0x7f91c01023c0 2024-03-12T05:12:07.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c01a1500 con 0x7f91c01023c0 2024-03-12T05:12:07.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0010490 con 0x7f91c01023c0 2024-03-12T05:12:07.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.721+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c01a19c0 con 0x7f91c01023c0 2024-03-12T05:12:07.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.722+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c01084f0 con 0x7f91c01023c0 2024-03-12T05:12:07.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.724+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f91b0005000 con 0x7f91c01023c0 2024-03-12T05:12:07.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.724+0000 7f91b6ffd640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91ac0a18e0 0x7f91ac0a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:07.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.724+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91b0014070 con 0x7f91c01023c0 2024-03-12T05:12:07.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.724+0000 7f91c5f9e640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91ac0a18e0 0x7f91ac0a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:07.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.725+0000 7f91c5f9e640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91ac0a18e0 0x7f91ac0a3da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91bc005ab0 tx=0x7f91bc005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:07.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.728+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91b0096b10 con 0x7f91c01023c0 2024-03-12T05:12:07.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.763+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 23) v1 ==== 136400+0+0 (secure 0 0 0) 0x7f91b0096ee0 con 0x7f91c01023c0 2024-03-12T05:12:07.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.967+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 --> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f91c002ce40 con 0x7f91ac0a18e0 2024-03-12T05:12:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.974+0000 7f91b6ffd640 1 -- 172.21.15.100:0/936083780 <== mgr.15432 v2:172.21.15.100:6800/2025021937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f91c002ce40 con 0x7f91ac0a18e0 2024-03-12T05:12:07.971 INFO:teuthology.orchestra.run.smithi100.stdout:/ 1m 6m3h 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.975+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91ac0a18e0 msgr2=0x7f91ac0a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.975+0000 7f91c8202640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91ac0a18e0 0x7f91ac0a3da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91bc005ab0 tx=0x7f91bc005fd0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 msgr2=0x7f91c0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 0x7f91c0073570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f91b000ea20 tx=0x7f91b000ee20 comp rx=0 tx=0).stop 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 shutdown_connections 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91ac0a18e0 0x7f91ac0a3da0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c0102d80 0x7f91c0073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c01023c0 0x7f91c0073570 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 --2- 172.21.15.100:0/936083780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c01011d0 0x7f91c0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 >> 172.21.15.100:0/936083780 conn(0x7f91c00fc740 msgr2=0x7f91c01056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 shutdown_connections 2024-03-12T05:12:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:07.976+0000 7f91c8202640 1 -- 172.21.15.100:0/936083780 wait complete. 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: [12/Mar/2024:05:12:06] ENGINE Bus STARTING 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: [12/Mar/2024:05:12:06] ENGINE Serving on http://172.21.15.100:8765 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: mgrmap e22: x(active, since 1.03913s), standbys: y, z 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: [12/Mar/2024:05:12:06] ENGINE Serving on https://172.21.15.100:7150 2024-03-12T05:12:07.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:07 smithi100 ceph-mon[32210]: [12/Mar/2024:05:12:06] ENGINE Bus STARTED 2024-03-12T05:12:07.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-12T05:12:08.014 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-12T05:12:08.053 INFO:teuthology.run_tasks:Running task workunit... 2024-03-12T05:12:08.190 INFO:tasks.workunit:Pulling workunits from ref b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-12T05:12:08.191 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-12T05:12:08.191 INFO:tasks.workunit:timeout=3h 2024-03-12T05:12:08.191 INFO:tasks.workunit:cleanup=True 2024-03-12T05:12:08.191 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-12T05:12:08.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: [12/Mar/2024:05:12:06] ENGINE Bus STARTING 2024-03-12T05:12:08.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: [12/Mar/2024:05:12:06] ENGINE Serving on http://172.21.15.100:8765 2024-03-12T05:12:08.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: mgrmap e22: x(active, since 1.03913s), standbys: y, z 2024-03-12T05:12:08.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:08.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:08.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: [12/Mar/2024:05:12:06] ENGINE Serving on https://172.21.15.100:7150 2024-03-12T05:12:08.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:07 smithi156 ceph-mon[36270]: [12/Mar/2024:05:12:06] ENGINE Bus STARTED 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-12 05:10:01.338502902 +0000 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-12 05:10:01.338502902 +0000 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-12 05:11:50.417183593 +0000 2024-03-12T05:12:08.213 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-12 05:10:01.338502902 +0000 2024-03-12T05:12:08.213 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-12T05:12:08.214 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-12T05:12:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: [12/Mar/2024:05:12:06] ENGINE Bus STARTING 2024-03-12T05:12:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: [12/Mar/2024:05:12:06] ENGINE Serving on http://172.21.15.100:8765 2024-03-12T05:12:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: mgrmap e22: x(active, since 1.03913s), standbys: y, z 2024-03-12T05:12:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: [12/Mar/2024:05:12:06] ENGINE Serving on https://172.21.15.100:7150 2024-03-12T05:12:08.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:07 smithi161 ceph-mon[36043]: [12/Mar/2024:05:12:06] ENGINE Bus STARTED 2024-03-12T05:12:08.294 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-12T05:12:08.351 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-12T05:12:09.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:08 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:08 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:09.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:09.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:08 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:12:10.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:09 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:12:10.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:09 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:12:10.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:09 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:12:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:12:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:12:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:12:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:12:11.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.069 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:12:11.070 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:12:11.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:12:11.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.207 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:12:11.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:12:11.209 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-12T05:12:11.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: Updating smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T05:12:11.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:12:11.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:12:13.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:12 smithi100 ceph-mon[32210]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:13.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:12 smithi156 ceph-mon[36270]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:13.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:12 smithi161 ceph-mon[36043]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-12T05:12:15.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:14 smithi100 ceph-mon[32210]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-03-12T05:12:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:14 smithi156 ceph-mon[36270]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-03-12T05:12:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:14 smithi161 ceph-mon[36043]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-03-12T05:12:17.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:16 smithi100 ceph-mon[32210]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-03-12T05:12:17.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:16 smithi156 ceph-mon[36270]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-03-12T05:12:17.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:16 smithi161 ceph-mon[36043]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-03-12T05:12:19.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:18 smithi100 ceph-mon[32210]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-03-12T05:12:19.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:18 smithi156 ceph-mon[36270]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-03-12T05:12:19.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:18 smithi161 ceph-mon[36043]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-03-12T05:12:21.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:20 smithi100 ceph-mon[32210]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-12T05:12:21.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:20 smithi156 ceph-mon[36270]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-12T05:12:21.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:20 smithi161 ceph-mon[36043]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-12T05:12:23.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:22 smithi100 ceph-mon[32210]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-12T05:12:23.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:22 smithi156 ceph-mon[36270]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-12T05:12:23.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:22 smithi161 ceph-mon[36043]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-12T05:12:25.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:24 smithi100 ceph-mon[32210]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-12T05:12:25.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:24 smithi156 ceph-mon[36270]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-12T05:12:25.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:24 smithi161 ceph-mon[36043]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-12T05:12:27.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:26 smithi100 ceph-mon[32210]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:27.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:26 smithi156 ceph-mon[36270]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:27.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:26 smithi161 ceph-mon[36043]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:28.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:12:28.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:12:28.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.214+0000 7fe2c28df640 1 -- 172.21.15.100:0/1563056873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc100270 msgr2=0x7fe2bc100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.214+0000 7fe2c28df640 1 --2- 172.21.15.100:0/1563056873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc100270 0x7fe2bc100650 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009930 tx=0x7fe2ac02bc30 comp rx=0 tx=0).stop 2024-03-12T05:12:28.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.214+0000 7fe2c28df640 1 -- 172.21.15.100:0/1563056873 shutdown_connections 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.214+0000 7fe2c28df640 1 --2- 172.21.15.100:0/1563056873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2bc105340 0x7fe2bc10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.214+0000 7fe2c28df640 1 --2- 172.21.15.100:0/1563056873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc100c20 0x7fe2bc104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.215+0000 7fe2c28df640 1 --2- 172.21.15.100:0/1563056873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc100270 0x7fe2bc100650 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.215+0000 7fe2c28df640 1 -- 172.21.15.100:0/1563056873 >> 172.21.15.100:0/1563056873 conn(0x7fe2bc077230 msgr2=0x7fe2bc077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.215+0000 7fe2c28df640 1 -- 172.21.15.100:0/1563056873 shutdown_connections 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.215+0000 7fe2c28df640 1 -- 172.21.15.100:0/1563056873 wait complete. 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.215+0000 7fe2c28df640 1 Processor -- start 2024-03-12T05:12:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.215+0000 7fe2c28df640 1 -- start start 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c28df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2bc100270 0x7fe2bc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c28df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 0x7fe2bc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c28df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc197610 0x7fe2bc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c28df640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc198000 con 0x7fe2bc105340 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c28df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc198170 con 0x7fe2bc197610 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c28df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc198330 con 0x7fe2bc100270 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 0x7fe2bc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 0x7fe2bc19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40954/0 (socket says 172.21.15.100:40954) 2024-03-12T05:12:28.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/760852442 learned_addr learned my addr 172.21.15.100:0/760852442 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc197610 0x7fe2bc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2bc100270 0x7fe2bc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/760852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2bc100270 msgr2=0x7fe2bc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2bc100270 0x7fe2bc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/760852442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc197610 msgr2=0x7fe2bc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc197610 0x7fe2bc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/760852442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac009560 con 0x7fe2bc105340 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 0x7fe2bc19d950 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000d770 tx=0x7fe2b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.216+0000 7fe2b97fa640 1 -- 172.21.15.100:0/760852442 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0009c30 con 0x7fe2bc105340 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.217+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc1a42d0 con 0x7fe2bc105340 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.217+0000 7fe2b97fa640 1 -- 172.21.15.100:0/760852442 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2b000de80 con 0x7fe2bc105340 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.217+0000 7fe2b97fa640 1 -- 172.21.15.100:0/760852442 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000f6b0 con 0x7fe2bc105340 2024-03-12T05:12:28.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.217+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1a47e0 con 0x7fe2bc105340 2024-03-12T05:12:28.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2b97fa640 1 -- 172.21.15.100:0/760852442 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7fe2b0010460 con 0x7fe2bc105340 2024-03-12T05:12:28.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 Processor -- start 2024-03-12T05:12:28.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 -- start start 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b7eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1ae2280 con 0x5622f1b7c7b0 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1adf7b0 con 0x5622f1b73bd0 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2c5593740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1adaf90 con 0x5622f1b732b0 2024-03-12T05:12:28.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.219+0000 7fe2b97fa640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2a40a1c30 0x7fe2a40a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2a40a1c30 0x7fe2a40a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b7eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41854/0 (socket says 172.21.15.100:41854) 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2c0e7c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 -- 172.21.15.100:0/1362245497 learned_addr learned my addr 172.21.15.100:0/1362245497 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b7eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40960/0 (socket says 172.21.15.100:40960) 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2b97fa640 1 -- 172.21.15.100:0/760852442 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe2b00cd720 con 0x7fe2bc105340 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 -- 172.21.15.100:0/1362245497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 msgr2=0x5622f1b73690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b73690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 -- 172.21.15.100:0/1362245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 msgr2=0x5622f1b7eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b7eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 -- 172.21.15.100:0/1362245497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622f1a99570 con 0x5622f1b73bd0 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b7eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2a40a1c30 0x7fe2a40a40f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fd0 tx=0x7fe2b4009d60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac036040 tx=0x7fe2ac005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0aed934ebb45f47 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2a27fc640 1 -- 172.21.15.100:0/1362245497 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac043070 con 0x5622f1b73bd0 2024-03-12T05:12:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2a27fc640 1 -- 172.21.15.100:0/1362245497 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac004880 con 0x5622f1b73bd0 2024-03-12T05:12:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.220+0000 7fe2a27fc640 1 -- 172.21.15.100:0/1362245497 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac03e650 con 0x5622f1b73bd0 2024-03-12T05:12:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 -- 172.21.15.100:0/1362245497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 msgr2=0x5622f1b7c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac036040 tx=0x7fe2ac005c00 comp rx=0 tx=0).stop 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 -- 172.21.15.100:0/1362245497 shutdown_connections 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b7eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 --2- 172.21.15.100:0/1362245497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b73690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 -- 172.21.15.100:0/1362245497 >> 172.21.15.100:0/1362245497 conn(0x5622f1b6d990 msgr2=0x5622f1b6dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 -- 172.21.15.100:0/1362245497 shutdown_connections 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 -- 172.21.15.100:0/1362245497 wait complete. 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.221+0000 7fe2c5593740 1 -- start start 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 Processor -- start 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b79590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f17ae430 con 0x5622f1b7c7b0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f176ae00 con 0x5622f1b73bd0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1a5ffd0 con 0x5622f1b732b0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c5593740 10 client.0 _ll_register_callbacks cb 0x5622f1ab5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b79cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40970/0 (socket says 172.21.15.100:40970) 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/2487231735 learned_addr learned my addr 172.21.15.100:0/2487231735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b79590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/2487231735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 msgr2=0x5622f1b7c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b7c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 msgr2=0x5622f1b79590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b79590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/2487231735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc10d720 con 0x5622f1b7c7b0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b79590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.222+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b7c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b79cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4012330 tx=0x7fe2b40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f258e9858d369bd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe29b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe29b7fe640 1 -- 172.21.15.100:0/2487231735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4003c40 con 0x5622f1b7c7b0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe29b7fe640 1 -- 172.21.15.100:0/2487231735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe2b4003da0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622f17da7e0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe29b7fe640 1 -- 172.21.15.100:0/2487231735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b40118e0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 10 client.15522 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5622f1ad9610 con 0x5622f1b7c7b0 2024-03-12T05:12:28.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe29b7fe640 1 -- 172.21.15.100:0/2487231735 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5622f17da7e0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe29b7fe640 1 -- 172.21.15.100:0/2487231735 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2b4026020 con 0x5622f1b7c7b0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 10 client.15522 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 20 client.15522 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1b8edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 4 client.15522 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.223+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622f1817720 con 0x5622f1b8c9a0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.224+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1b8edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.224+0000 7fe29b7fe640 10 client.15522 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.224+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1b8edc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e0ede274c30a8a server_cookie=bc56bc9ed39ab157 in_seq=0 out_seq=0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe29b7fe640 1 -- 172.21.15.100:0/2487231735 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622f1817720 con 0x5622f1b8c9a0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe29b7fe640 10 client.15522 handle_command_reply: tid=0 2024-03-12T05:12:28.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 client.15522 shutdown 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 msgr2=0x5622f1b79cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b79cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4012330 tx=0x7fe2b40096a0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 shutdown_connections 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1b8edc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b79cd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b73bd0 0x5622f1b79590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe29b7fe640 0 client.15522 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 --2- 172.21.15.100:0/2487231735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b732b0 0x5622f1b7c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 >> 172.21.15.100:0/2487231735 conn(0x5622f1b6d990 msgr2=0x5622f1b6f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 shutdown_connections 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 1 -- 172.21.15.100:0/2487231735 wait complete. 2024-03-12T05:12:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.225+0000 7fe2c5593740 20 client.15522 trim_cache size 0 max 16384 2024-03-12T05:12:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 Processor -- start 2024-03-12T05:12:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 -- start start 2024-03-12T05:12:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b7bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b7c7b0 0x5622f1b77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1b6e800 con 0x5622f1b73bd0 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1b69440 con 0x5622f1b732b0 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2c5593740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1b6ea60 con 0x5622f1b7c7b0 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40976/0 (socket says 172.21.15.100:40976) 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.240+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/352845899 learned_addr learned my addr 172.21.15.100:0/352845899 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b7bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b7c7b0 0x5622f1b77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/352845899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b7c7b0 msgr2=0x5622f1b77e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b7c7b0 0x5622f1b77e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/352845899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 msgr2=0x5622f1b7bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b7bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bb7fe640 1 -- 172.21.15.100:0/352845899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622f1a75ac0 con 0x5622f1b73bd0 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b7c7b0 0x5622f1b77e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b7bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0096bf0 tx=0x7fe2b006d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e71e6a335f3cda6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2a1ffb640 1 -- 172.21.15.100:0/352845899 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00a2910 con 0x5622f1b73bd0 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2a1ffb640 1 -- 172.21.15.100:0/352845899 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0098520 con 0x5622f1b73bd0 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.241+0000 7fe2a1ffb640 1 -- 172.21.15.100:0/352845899 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00cb250 con 0x5622f1b73bd0 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- 172.21.15.100:0/352845899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 msgr2=0x5622f1b7c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0096bf0 tx=0x7fe2b006d4d0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- 172.21.15.100:0/352845899 shutdown_connections 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b7c7b0 0x5622f1b77e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b73bd0 0x5622f1b7c200 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- 172.21.15.100:0/352845899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b7bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- 172.21.15.100:0/352845899 >> 172.21.15.100:0/352845899 conn(0x5622f1b6d990 msgr2=0x5622f1b74850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- 172.21.15.100:0/352845899 shutdown_connections 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- 172.21.15.100:0/352845899 wait complete. 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- start start 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 Processor -- start 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b83710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b73bd0 0x5622f1b7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b80110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1b80650 con 0x5622f1b7c7b0 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1b807c0 con 0x5622f1b732b0 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f1b80980 con 0x5622f1b73bd0 2024-03-12T05:12:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.242+0000 7fe2c5593740 10 client.0 _ll_register_callbacks cb 0x7fe2ac005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b80110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40992/0 (socket says 172.21.15.100:40992) 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/3480247744 learned_addr learned my addr 172.21.15.100:0/3480247744 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2bb7fe640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b73bd0 0x5622f1b7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b83710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/3480247744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b73bd0 msgr2=0x5622f1b7fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b73bd0 0x5622f1b7fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 msgr2=0x5622f1b83710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b83710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 -- 172.21.15.100:0/3480247744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1a42d0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c0e7c640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b80110 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fa0 tx=0x7fe2b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea9401298b6b002 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe29b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe29b7fe640 1 -- 172.21.15.100:0/3480247744 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40119c0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe29b7fe640 1 -- 172.21.15.100:0/3480247744 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe2b4011b20 con 0x5622f1b7c7b0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2ac004880 con 0x5622f1b7c7b0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe29b7fe640 1 -- 172.21.15.100:0/3480247744 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b4011ce0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c5593740 10 client.15528 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5622f1a9b6a0 con 0x5622f1b7c7b0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.243+0000 7fe29b7fe640 1 -- 172.21.15.100:0/3480247744 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac004880 con 0x5622f1b7c7b0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe29b7fe640 1 -- 172.21.15.100:0/3480247744 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2b4026070 con 0x5622f1b7c7b0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2c5593740 10 client.15528 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2c5593740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2c5593740 20 client.15528 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2c5593740 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1be62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2c5593740 4 client.15528 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622f1866890 con 0x5622f1b8c9a0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1be62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe29b7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.244+0000 7fe2bbfff640 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1be62e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ddef8c7fbc30493 server_cookie=f317b9f46907a9aa in_seq=0 out_seq=0 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe29b7fe640 1 -- 172.21.15.100:0/3480247744 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5622f1866890 con 0x5622f1b8c9a0 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe29b7fe640 10 client.15528 handle_command_reply: tid=0 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 client.15528 shutdown 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 msgr2=0x5622f1b80110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b80110 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fa0 tx=0x7fe2b4004a40 comp rx=0 tx=0).stop 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 shutdown_connections 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5622f1b8c9a0 0x5622f1be62e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622f1b7c7b0 0x5622f1b80110 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622f1b73bd0 0x5622f1b7fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe29b7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 --2- 172.21.15.100:0/3480247744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f1b732b0 0x5622f1b83710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 >> 172.21.15.100:0/3480247744 conn(0x5622f1b6d990 msgr2=0x5622f1b78df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 shutdown_connections 2024-03-12T05:12:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 1 -- 172.21.15.100:0/3480247744 wait complete. 2024-03-12T05:12:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c5593740 20 client.15528 trim_cache size 0 max 16384 2024-03-12T05:12:28.250 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:12:28.250 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:12:28.250 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:12:28.251 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2a40a1c30 msgr2=0x7fe2a40a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c28df640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2a40a1c30 0x7fe2a40a40f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fd0 tx=0x7fe2b4009d60 comp rx=0 tx=0).stop 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 msgr2=0x7fe2bc19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.245+0000 7fe2c28df640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 0x7fe2bc19d950 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000d770 tx=0x7fe2b000dc30 comp rx=0 tx=0).stop 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 shutdown_connections 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2a40a1c30 0x7fe2a40a40f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc197610 0x7fe2bc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2bc105340 0x7fe2bc19d950 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 --2- 172.21.15.100:0/760852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2bc100270 0x7fe2bc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 >> 172.21.15.100:0/760852442 conn(0x7fe2bc077230 msgr2=0x7fe2bc104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 shutdown_connections 2024-03-12T05:12:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.246+0000 7fe2c28df640 1 -- 172.21.15.100:0/760852442 wait complete. 2024-03-12T05:12:28.253 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:12:28.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 -- 172.21.15.100:0/1720792614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 msgr2=0x7f9a7c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 --2- 172.21.15.100:0/1720792614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c101770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009930 tx=0x7f9a6c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 -- 172.21.15.100:0/1720792614 shutdown_connections 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 --2- 172.21.15.100:0/1720792614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 0x7f9a7c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 --2- 172.21.15.100:0/1720792614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a7c101d40 0x7f9a7c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 --2- 172.21.15.100:0/1720792614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c101770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 -- 172.21.15.100:0/1720792614 >> 172.21.15.100:0/1720792614 conn(0x7f9a7c0fcc10 msgr2=0x7f9a7c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.358+0000 7f9a820c0640 1 -- 172.21.15.100:0/1720792614 shutdown_connections 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 -- 172.21.15.100:0/1720792614 wait complete. 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 Processor -- start 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 -- start start 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 0x7f9a7c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a7c1976b0 0x7f9a7c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c198180 con 0x7f9a7c101390 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c1982f0 con 0x7f9a7c1976b0 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a820c0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c1984b0 con 0x7f9a7c106460 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.359+0000 7f9a7b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41010/0 (socket says 172.21.15.100:41010) 2024-03-12T05:12:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 -- 172.21.15.100:0/2144630743 learned_addr learned my addr 172.21.15.100:0/2144630743 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a72ffd640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 0x7f9a7c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7bfff640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a7c1976b0 0x7f9a7c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 -- 172.21.15.100:0/2144630743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 msgr2=0x7f9a7c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 0x7f9a7c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 -- 172.21.15.100:0/2144630743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a7c1976b0 msgr2=0x7f9a7c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a7c1976b0 0x7f9a7c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 -- 172.21.15.100:0/2144630743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c009560 con 0x7f9a7c101390 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a72ffd640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 0x7f9a7c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a7b7fe640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c19d5a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009230 tx=0x7f9a6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c004440 con 0x7f9a7c101390 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c1a43a0 con 0x7f9a7c101390 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a6c0052d0 con 0x7f9a7c101390 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c003d70 con 0x7f9a7c101390 2024-03-12T05:12:28.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.360+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c1a48d0 con 0x7f9a7c101390 2024-03-12T05:12:28.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.361+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c102860 con 0x7f9a7c101390 2024-03-12T05:12:28.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.363+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f9a6c005440 con 0x7f9a7c101390 2024-03-12T05:12:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.363+0000 7f9a797fa640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a680a1c30 0x7f9a680a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.363+0000 7f9a72ffd640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a680a1c30 0x7f9a680a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.364+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9a6c0f02c0 con 0x7f9a7c101390 2024-03-12T05:12:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.364+0000 7f9a72ffd640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a680a1c30 0x7f9a680a40f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c198eb0 tx=0x7f9a64005ec0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.367+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a6c0b9120 con 0x7f9a7c101390 2024-03-12T05:12:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.574+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a7c109690 con 0x7f9a7c101390 2024-03-12T05:12:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.575+0000 7f9a797fa640 1 -- 172.21.15.100:0/2144630743 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f9a6c047e90 con 0x7f9a7c101390 2024-03-12T05:12:28.572 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:12:28.572 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:11:51.088786+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:12:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 23 2024-03-12T05:12:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a680a1c30 msgr2=0x7f9a680a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a680a1c30 0x7f9a680a40f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c198eb0 tx=0x7f9a64005ec0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 msgr2=0x7f9a7c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c19d5a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009230 tx=0x7f9a6c004040 comp rx=0 tx=0).stop 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 shutdown_connections 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a680a1c30 0x7f9a680a40f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a7c1976b0 0x7f9a7c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a7c106460 0x7f9a7c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 --2- 172.21.15.100:0/2144630743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a7c101390 0x7f9a7c19d5a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 >> 172.21.15.100:0/2144630743 conn(0x7f9a7c0fcc10 msgr2=0x7f9a7c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 shutdown_connections 2024-03-12T05:12:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.577+0000 7f9a820c0640 1 -- 172.21.15.100:0/2144630743 wait complete. 2024-03-12T05:12:28.586 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:12:28.586 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:12:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 -- 172.21.15.100:0/1718957854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074100310 msgr2=0x7fa074105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 --2- 172.21.15.100:0/1718957854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074100310 0x7fa074105e40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa068009930 tx=0x7fa06802bc30 comp rx=0 tx=0).stop 2024-03-12T05:12:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 -- 172.21.15.100:0/1718957854 shutdown_connections 2024-03-12T05:12:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 --2- 172.21.15.100:0/1718957854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa074106570 0x7fa07410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 --2- 172.21.15.100:0/1718957854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074100310 0x7fa074105e40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 --2- 172.21.15.100:0/1718957854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 0x7fa0740ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 -- 172.21.15.100:0/1718957854 >> 172.21.15.100:0/1718957854 conn(0x7fa0740766b0 msgr2=0x7fa0740fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 -- 172.21.15.100:0/1718957854 shutdown_connections 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.692+0000 7fa07a948640 1 -- 172.21.15.100:0/1718957854 wait complete. 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 Processor -- start 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 -- start start 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 0x7fa07419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 0x7fa07419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa074197680 0x7fa074197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074198150 con 0x7fa074106570 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741982c0 con 0x7fa0740ff960 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa07a948640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074198480 con 0x7fa074197680 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.693+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 0x7fa07419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 0x7fa07419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41020/0 (socket says 172.21.15.100:41020) 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 -- 172.21.15.100:0/3436279308 learned_addr learned my addr 172.21.15.100:0/3436279308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa078ee5640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa074197680 0x7fa074197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa073fff640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 0x7fa07419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 -- 172.21.15.100:0/3436279308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa074197680 msgr2=0x7fa074197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa074197680 0x7fa074197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 -- 172.21.15.100:0/3436279308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 msgr2=0x7fa07419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 0x7fa07419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 -- 172.21.15.100:0/3436279308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009560 con 0x7fa074106570 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa073fff640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 0x7fa07419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0737fe640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 0x7fa07419db00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa068000c00 tx=0x7fa068004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0717fa640 1 -- 172.21.15.100:0/3436279308 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06802cbc0 con 0x7fa074106570 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0717fa640 1 -- 172.21.15.100:0/3436279308 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0680052d0 con 0x7fa074106570 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0741a43c0 con 0x7fa074106570 2024-03-12T05:12:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa0717fa640 1 -- 172.21.15.100:0/3436279308 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068003e20 con 0x7fa074106570 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.694+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0741a48a0 con 0x7fa074106570 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.696+0000 7fa0717fa640 1 -- 172.21.15.100:0/3436279308 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7fa0680035f0 con 0x7fa074106570 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 Processor -- start 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 -- start start 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75250 0x558f0cd75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75b70 0x558f0cd7e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd80b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cd30960 con 0x558f0cd7e750 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cce4310 con 0x558f0cd75b70 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cce1840 con 0x558f0cd75250 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa0717fa640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa05c0a1c30 0x7fa05c0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd80b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa078ee5640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa05c0a1c30 0x7fa05c0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd80b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41030/0 (socket says 172.21.15.100:41030) 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa0737fe640 1 -- 172.21.15.100:0/2587899659 learned_addr learned my addr 172.21.15.100:0/2587899659 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa073fff640 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75b70 0x558f0cd7e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa078ee5640 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75250 0x558f0cd75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.697+0000 7fa0717fa640 1 -- 172.21.15.100:0/3436279308 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa0680efe80 con 0x7fa074106570 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa0737fe640 1 -- 172.21.15.100:0/2587899659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75250 msgr2=0x558f0cd75630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa0737fe640 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75250 0x558f0cd75630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa0737fe640 1 -- 172.21.15.100:0/2587899659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75b70 msgr2=0x558f0cd7e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa0737fe640 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75b70 0x558f0cd7e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa0737fe640 1 -- 172.21.15.100:0/2587899659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f0cc9f060 con 0x558f0cd7e750 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa078ee5640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa05c0a1c30 0x7fa05c0a40f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007fd0 tx=0x7fa06c007c60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa0737fe640 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd80b40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa0680c4c50 tx=0x7fa06808fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77f8e9010d65329 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa05a7fc640 1 -- 172.21.15.100:0/2587899659 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680bd0e0 con 0x558f0cd7e750 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa05a7fc640 1 -- 172.21.15.100:0/2587899659 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0680bc9e0 con 0x558f0cd7e750 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.698+0000 7fa05a7fc640 1 -- 172.21.15.100:0/2587899659 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068005440 con 0x558f0cd7e750 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- 172.21.15.100:0/2587899659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 msgr2=0x558f0cd80b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd80b40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa0680c4c50 tx=0x7fa06808fc90 comp rx=0 tx=0).stop 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- 172.21.15.100:0/2587899659 shutdown_connections 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd80b40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75b70 0x558f0cd7e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/2587899659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75250 0x558f0cd75630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- 172.21.15.100:0/2587899659 >> 172.21.15.100:0/2587899659 conn(0x558f0cd6f980 msgr2=0x558f0cd6fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- 172.21.15.100:0/2587899659 shutdown_connections 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- 172.21.15.100:0/2587899659 wait complete. 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- start start 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 Processor -- start 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75250 0x558f0cd7b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd7bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd7c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.699+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0ccdd020 con 0x558f0cd7e750 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0c9b0430 con 0x558f0cd75250 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0c96ce00 con 0x558f0cd75b70 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa07d5f8740 10 client.0 _ll_register_callbacks cb 0x558f0cc84cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd7c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd7c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41040/0 (socket says 172.21.15.100:41040) 2024-03-12T05:12:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 -- 172.21.15.100:0/1341512278 learned_addr learned my addr 172.21.15.100:0/1341512278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd7bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa073fff640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75250 0x558f0cd7b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 -- 172.21.15.100:0/1341512278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 msgr2=0x558f0cd7bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd7bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 -- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75250 msgr2=0x558f0cd7b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75250 0x558f0cd7b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 -- 172.21.15.100:0/1341512278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0741a48a0 con 0x558f0cd7e750 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa073fff640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75250 0x558f0cd7b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa078ee5640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd7c210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa06c010cc0 tx=0x7fa06c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424382a7336083d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa0537fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa0537fe640 1 -- 172.21.15.100:0/1341512278 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c010cf0 con 0x558f0cd7e750 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa0537fe640 1 -- 172.21.15.100:0/1341512278 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa06c010e50 con 0x558f0cd7e750 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f0c9dc7e0 con 0x558f0cd7e750 2024-03-12T05:12:28.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa0537fe640 1 -- 172.21.15.100:0/1341512278 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa06c020460 con 0x558f0cd7e750 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa07d5f8740 10 client.15546 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.700+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558f0ccdb6a0 con 0x558f0cd7e750 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa0537fe640 1 -- 172.21.15.100:0/1341512278 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558f0c9dc7e0 con 0x558f0cd7e750 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa0537fe640 1 -- 172.21.15.100:0/1341512278 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa06c01f630 con 0x558f0cd7e750 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa07d5f8740 10 client.15546 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa07d5f8740 10 client.15546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa07d5f8740 20 client.15546 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cd8e890 0x558f0cd90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa07d5f8740 4 client.15546 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f0ca19720 con 0x558f0cd8e890 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.701+0000 7fa073fff640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cd8e890 0x558f0cd90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa0537fe640 10 client.15546 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:12:28.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa073fff640 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cd8e890 0x558f0cd90cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee450f9036f0f62a server_cookie=dd5983b5ca49a826 in_seq=0 out_seq=0 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa0537fe640 1 -- 172.21.15.100:0/1341512278 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f0ca19720 con 0x558f0cd8e890 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa0537fe640 10 client.15546 handle_command_reply: tid=0 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 client.15546 shutdown 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 msgr2=0x558f0cd7c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd7c210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa06c010cc0 tx=0x7fa06c01a000 comp rx=0 tx=0).stop 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 shutdown_connections 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cd8e890 0x558f0cd90cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd7e750 0x558f0cd7c210 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd7bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/1341512278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd75250 0x558f0cd7b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa0537fe640 0 client.15546 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 >> 172.21.15.100:0/1341512278 conn(0x558f0cd6f980 msgr2=0x558f0cd716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.702+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 shutdown_connections 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.703+0000 7fa07d5f8740 1 -- 172.21.15.100:0/1341512278 wait complete. 2024-03-12T05:12:28.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.703+0000 7fa07d5f8740 20 client.15546 trim_cache size 0 max 16384 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 Processor -- start 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 -- start start 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cd7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd6ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cc62130 con 0x558f0cd75250 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0ca19720 con 0x558f0cd7e750 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cd8e660 con 0x558f0cd75b70 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa073fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cd7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa073fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cd7c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41048/0 (socket says 172.21.15.100:41048) 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa073fff640 1 -- 172.21.15.100:0/3774300721 learned_addr learned my addr 172.21.15.100:0/3774300721 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.714+0000 7fa073fff640 1 -- 172.21.15.100:0/3774300721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 msgr2=0x558f0cd6ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa073fff640 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd6ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa073fff640 1 -- 172.21.15.100:0/3774300721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 msgr2=0x558f0cd853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa073fff640 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa073fff640 1 -- 172.21.15.100:0/3774300721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f0cc77c20 con 0x558f0cd75250 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa073fff640 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cd7c030 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x558f0cd915e0 tx=0x7fa06400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e659b0e27cea28 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa059ffb640 1 -- 172.21.15.100:0/3774300721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064016040 con 0x558f0cd75250 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa059ffb640 1 -- 172.21.15.100:0/3774300721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa06400e3b0 con 0x558f0cd75250 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa059ffb640 1 -- 172.21.15.100:0/3774300721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640225c0 con 0x558f0cd75250 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 -- 172.21.15.100:0/3774300721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 msgr2=0x558f0cd7c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cd7c030 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x558f0cd915e0 tx=0x7fa06400f7f0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 -- 172.21.15.100:0/3774300721 shutdown_connections 2024-03-12T05:12:28.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cd6ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/3774300721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cd7c030 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 -- 172.21.15.100:0/3774300721 >> 172.21.15.100:0/3774300721 conn(0x558f0cd6f980 msgr2=0x558f0cd6e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 -- 172.21.15.100:0/3774300721 shutdown_connections 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.715+0000 7fa07d5f8740 1 -- 172.21.15.100:0/3774300721 wait complete. 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 -- start start 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 Processor -- start 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cddf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cddfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cd84f70 con 0x558f0cd75250 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cd850e0 con 0x558f0cd7e750 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0cd852a0 con 0x558f0cd75b70 2024-03-12T05:12:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa07d5f8740 10 client.0 _ll_register_callbacks cb 0x7fa0680ed440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cddf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cddf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41054/0 (socket says 172.21.15.100:41054) 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 -- 172.21.15.100:0/659940272 learned_addr learned my addr 172.21.15.100:0/659940272 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa0737fe640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cddfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa078ee5640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 -- 172.21.15.100:0/659940272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 msgr2=0x558f0cddfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cddfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 -- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 msgr2=0x558f0cd849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd849b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa073fff640 1 -- 172.21.15.100:0/659940272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06400bd30 con 0x558f0cd75250 2024-03-12T05:12:28.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.716+0000 7fa0737fe640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cddfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa073fff640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cddf4f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa06401b5d0 tx=0x7fa06401b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc215e27f4d9d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0537fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0537fe640 1 -- 172.21.15.100:0/659940272 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064016070 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0537fe640 1 -- 172.21.15.100:0/659940272 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa06400e720 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0680bc9e0 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0537fe640 1 -- 172.21.15.100:0/659940272 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa06401b930 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 10 client.15552 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558f0cc9d8c0 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0537fe640 1 -- 172.21.15.100:0/659940272 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0680bc9e0 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0537fe640 1 -- 172.21.15.100:0/659940272 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa064004c10 con 0x558f0cd75250 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 10 client.15552 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 10 client.15552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:12:28.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 20 client.15552 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cde72e0 0x558f0cde9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 4 client.15552 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558f0ca68890 con 0x558f0cde72e0 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.717+0000 7fa0737fe640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cde72e0 0x558f0cde9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa0737fe640 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cde72e0 0x558f0cde9700 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdbfb7e31487b427 server_cookie=7750112ef5f1c737 in_seq=0 out_seq=0 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa0537fe640 10 client.15552 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa0537fe640 1 -- 172.21.15.100:0/659940272 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558f0ca68890 con 0x558f0cde72e0 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa0537fe640 10 client.15552 handle_command_reply: tid=0 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 client.15552 shutdown 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 msgr2=0x558f0cddf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cddf4f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa06401b5d0 tx=0x7fa06401b600 comp rx=0 tx=0).stop 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 shutdown_connections 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x558f0cde72e0 0x558f0cde9700 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f0cd7e750 0x558f0cd849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa0537fe640 0 client.15552 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f0cd75b70 0x558f0cddfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 --2- 172.21.15.100:0/659940272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f0cd75250 0x558f0cddf4f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 >> 172.21.15.100:0/659940272 conn(0x558f0cd6f980 msgr2=0x558f0cd6eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 shutdown_connections 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 1 -- 172.21.15.100:0/659940272 wait complete. 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.718+0000 7fa07d5f8740 20 client.15552 trim_cache size 0 max 16384 2024-03-12T05:12:28.724 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:12:28.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa05c0a1c30 msgr2=0x7fa05c0a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa05c0a1c30 0x7fa05c0a40f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007fd0 tx=0x7fa06c007c60 comp rx=0 tx=0).stop 2024-03-12T05:12:28.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 msgr2=0x7fa07419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 0x7fa07419db00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa068000c00 tx=0x7fa068004270 comp rx=0 tx=0).stop 2024-03-12T05:12:28.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 shutdown_connections 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa05c0a1c30 0x7fa05c0a40f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa074197680 0x7fa074197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa074106570 0x7fa07419db00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 --2- 172.21.15.100:0/3436279308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0740ff960 0x7fa07419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 >> 172.21.15.100:0/3436279308 conn(0x7fa0740766b0 msgr2=0x7fa0740fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 shutdown_connections 2024-03-12T05:12:28.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.719+0000 7fa07a948640 1 -- 172.21.15.100:0/3436279308 wait complete. 2024-03-12T05:12:28.727 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:12:28.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:12:28.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/2157279948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28100cc0 msgr2=0x7f1a28104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/2157279948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28100cc0 0x7f1a28104d00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009930 tx=0x7f1a1802bc30 comp rx=0 tx=0).stop 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/2157279948 shutdown_connections 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/2157279948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 0x7f1a2810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/2157279948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28100cc0 0x7f1a28104d00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/2157279948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28100310 0x7f1a281006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/2157279948 >> 172.21.15.100:0/2157279948 conn(0x7f1a280771e0 msgr2=0x7f1a280775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/2157279948 shutdown_connections 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.835+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/2157279948 wait complete. 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 Processor -- start 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 -- start start 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28100310 0x7f1a28198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 0x7f1a28198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 0x7f1a281a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a3d80 con 0x7f1a28199130 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a3ef0 con 0x7f1a281053e0 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a40b0 con 0x7f1a28100310 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 0x7f1a281a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 0x7f1a281a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41062/0 (socket says 172.21.15.100:41062) 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 -- 172.21.15.100:0/3949962665 learned_addr learned my addr 172.21.15.100:0/3949962665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 0x7f1a28198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2c908640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28100310 0x7f1a28198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 -- 172.21.15.100:0/3949962665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28100310 msgr2=0x7f1a28198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28100310 0x7f1a28198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 -- 172.21.15.100:0/3949962665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 msgr2=0x7f1a28198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 0x7f1a28198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 -- 172.21.15.100:0/3949962665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18009560 con 0x7f1a28199130 2024-03-12T05:12:28.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 0x7f1a28198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2d109640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 0x7f1a281a36e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000cce0 tx=0x7f1a2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a1dffb640 1 -- 172.21.15.100:0/3949962665 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000f450 con 0x7f1a28199130 2024-03-12T05:12:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a1dffb640 1 -- 172.21.15.100:0/3949962665 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a2000fa90 con 0x7f1a28199130 2024-03-12T05:12:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a281a4390 con 0x7f1a28199130 2024-03-12T05:12:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.836+0000 7f1a1dffb640 1 -- 172.21.15.100:0/3949962665 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20018b90 con 0x7f1a28199130 2024-03-12T05:12:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.837+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a281a4850 con 0x7f1a28199130 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a1dffb640 1 -- 172.21.15.100:0/3949962665 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f1a2000fc00 con 0x7f1a28199130 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 Processor -- start 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 -- start start 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a1dffb640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a100a1c30 0x7f1a100a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed417a960 con 0x564ed41bfb70 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed412e310 con 0x564ed41bf250 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.839+0000 7f1a3181e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed412b840 con 0x564ed41c8750 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a100a1c30 0x7f1a100a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1dffb640 1 -- 172.21.15.100:0/3949962665 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1a200ce1d0 con 0x7f1a28199130 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41076/0 (socket says 172.21.15.100:41076) 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 -- 172.21.15.100:0/2370974791 learned_addr learned my addr 172.21.15.100:0/2370974791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a2d109640 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a2c908640 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 -- 172.21.15.100:0/2370974791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 msgr2=0x564ed41cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 -- 172.21.15.100:0/2370974791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 msgr2=0x564ed41bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 -- 172.21.15.100:0/2370974791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed40e9060 con 0x564ed41bfb70 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a2d109640 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41c8210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1a18035000 tx=0x7f1a1802cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5744b91c649877da server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a100a1c30 0x7f1a100a40f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1a1802ccf0 tx=0x7f1a18039000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a0effd640 1 -- 172.21.15.100:0/2370974791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a180046c0 con 0x564ed41bfb70 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a0effd640 1 -- 172.21.15.100:0/2370974791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a18004320 con 0x564ed41bfb70 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.840+0000 7f1a0effd640 1 -- 172.21.15.100:0/2370974791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18003b30 con 0x564ed41bfb70 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 -- 172.21.15.100:0/2370974791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 msgr2=0x564ed41c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41c8210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1a18035000 tx=0x7f1a1802cc80 comp rx=0 tx=0).stop 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 -- 172.21.15.100:0/2370974791 shutdown_connections 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41c8210 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 --2- 172.21.15.100:0/2370974791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 -- 172.21.15.100:0/2370974791 >> 172.21.15.100:0/2370974791 conn(0x564ed41b9980 msgr2=0x564ed41b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 -- 172.21.15.100:0/2370974791 shutdown_connections 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 -- 172.21.15.100:0/2370974791 wait complete. 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 -- start start 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.841+0000 7f1a3181e740 1 Processor -- start 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 0x564ed41c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bfb70 0x564ed41c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41c8750 0x564ed41c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed4127020 con 0x564ed41bf250 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed3dfa430 con 0x564ed41c8750 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed3db6e00 con 0x564ed41bfb70 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a3181e740 10 client.0 _ll_register_callbacks cb 0x564ed40cecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 0x564ed41c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 0x564ed41c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41090/0 (socket says 172.21.15.100:41090) 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 -- 172.21.15.100:0/673887559 learned_addr learned my addr 172.21.15.100:0/673887559 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2d109640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41c8750 0x564ed41c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bfb70 0x564ed41c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 -- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bfb70 msgr2=0x564ed41c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bfb70 0x564ed41c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 -- 172.21.15.100:0/673887559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41c8750 msgr2=0x564ed41c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41c8750 0x564ed41c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 -- 172.21.15.100:0/673887559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2810d7c0 con 0x564ed41bf250 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2d109640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41c8750 0x564ed41c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a2c908640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 0x564ed41c5820 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1a24005a60 tx=0x7f1a2400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9a910fa8ea301a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a07fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.842+0000 7f1a07fff640 1 -- 172.21.15.100:0/673887559 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400bb00 con 0x564ed41bf250 2024-03-12T05:12:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a07fff640 1 -- 172.21.15.100:0/673887559 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1a2400bc60 con 0x564ed41bf250 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ed3e267e0 con 0x564ed41bf250 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a07fff640 1 -- 172.21.15.100:0/673887559 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a2400be20 con 0x564ed41bf250 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 10 client.15564 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564ed41256a0 con 0x564ed41bf250 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a07fff640 1 -- 172.21.15.100:0/673887559 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a24026080 con 0x564ed41bf250 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a07fff640 1 -- 172.21.15.100:0/673887559 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1a24018030 con 0x564ed41bf250 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 10 client.15564 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 10 client.15564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 20 client.15564 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed41d8cd0 0x564ed41db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 4 client.15564 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.843+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ed3e63720 con 0x564ed41d8cd0 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.844+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed41d8cd0 0x564ed41db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.844+0000 7f1a07fff640 10 client.15564 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.844+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed41d8cd0 0x564ed41db0f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec5fadc3c997099b server_cookie=962cfd256e8dc2ed in_seq=0 out_seq=0 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a07fff640 1 -- 172.21.15.100:0/673887559 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ed3e63720 con 0x564ed41d8cd0 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a07fff640 10 client.15564 handle_command_reply: tid=0 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 client.15564 shutdown 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 msgr2=0x564ed41c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 0x564ed41c5820 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1a24005a60 tx=0x7f1a2400fb30 comp rx=0 tx=0).stop 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 shutdown_connections 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed41d8cd0 0x564ed41db0f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41c8750 0x564ed41c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bfb70 0x564ed41c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a07fff640 0 client.15564 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 --2- 172.21.15.100:0/673887559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bf250 0x564ed41c5820 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 >> 172.21.15.100:0/673887559 conn(0x564ed41b9980 msgr2=0x564ed41bb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 shutdown_connections 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 1 -- 172.21.15.100:0/673887559 wait complete. 2024-03-12T05:12:28.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.845+0000 7f1a3181e740 20 client.15564 trim_cache size 0 max 16384 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.856+0000 7f1a3181e740 1 Processor -- start 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.856+0000 7f1a3181e740 1 -- start start 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a3181e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed40ac130 con 0x564ed41bfb70 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a3181e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed41bfa30 con 0x564ed41bf250 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a3181e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed41c1480 con 0x564ed41c8750 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41102/0 (socket says 172.21.15.100:41102) 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 -- 172.21.15.100:0/3977531011 learned_addr learned my addr 172.21.15.100:0/3977531011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 -- 172.21.15.100:0/3977531011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 msgr2=0x564ed41c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a2c908640 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 -- 172.21.15.100:0/3977531011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 msgr2=0x564ed41c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 -- 172.21.15.100:0/3977531011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed40c1c20 con 0x564ed41bfb70 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a2c908640 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41cf980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009900 tx=0x7f1a1802c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67348589b585a85a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a0e7fc640 1 -- 172.21.15.100:0/3977531011 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1805e070 con 0x564ed41bfb70 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a0e7fc640 1 -- 172.21.15.100:0/3977531011 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a180046c0 con 0x564ed41bfb70 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.857+0000 7f1a0e7fc640 1 -- 172.21.15.100:0/3977531011 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1805a440 con 0x564ed41bfb70 2024-03-12T05:12:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 -- 172.21.15.100:0/3977531011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 msgr2=0x564ed41cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41cf980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009900 tx=0x7f1a1802c2b0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 -- 172.21.15.100:0/3977531011 shutdown_connections 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41c8750 0x564ed41c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41bfb70 0x564ed41cf980 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 --2- 172.21.15.100:0/3977531011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bf250 0x564ed41c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 -- 172.21.15.100:0/3977531011 >> 172.21.15.100:0/3977531011 conn(0x564ed41b9980 msgr2=0x564ed41dfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 -- 172.21.15.100:0/3977531011 shutdown_connections 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 -- 172.21.15.100:0/3977531011 wait complete. 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 -- start start 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.858+0000 7f1a3181e740 1 Processor -- start 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bf250 0x564ed4227a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bfb70 0x564ed4227f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 0x564ed422c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed422c590 con 0x564ed41c8750 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed422c700 con 0x564ed41bfb70 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed422c8c0 con 0x564ed41bf250 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 10 client.0 _ll_register_callbacks cb 0x7f1a18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 0x564ed422c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 0x564ed422c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41106/0 (socket says 172.21.15.100:41106) 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 -- 172.21.15.100:0/134322268 learned_addr learned my addr 172.21.15.100:0/134322268 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2c908640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bf250 0x564ed4227a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a1ffff640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bfb70 0x564ed4227f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 -- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bf250 msgr2=0x564ed4227a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bf250 0x564ed4227a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 -- 172.21.15.100:0/134322268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bfb70 msgr2=0x564ed4227f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bfb70 0x564ed4227f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 -- 172.21.15.100:0/134322268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ed40e9060 con 0x564ed41c8750 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2c908640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bf250 0x564ed4227a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a2d109640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 0x564ed422c050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000ab20 tx=0x7f1a2009a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10146f6d1d50eb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a07fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a07fff640 1 -- 172.21.15.100:0/134322268 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2009edd0 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a07fff640 1 -- 172.21.15.100:0/134322268 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1a200a3420 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a18004320 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a07fff640 1 -- 172.21.15.100:0/134322268 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a2004d700 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 10 client.15570 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.859+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564ed40e78c0 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a07fff640 1 -- 172.21.15.100:0/134322268 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a20081030 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a07fff640 1 -- 172.21.15.100:0/134322268 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1a20016020 con 0x564ed41c8750 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a3181e740 10 client.15570 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a3181e740 10 client.15570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a3181e740 20 client.15570 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a3181e740 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed4235240 0x564ed4237660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a3181e740 4 client.15570 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:12:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564ed3eb2890 con 0x564ed4235240 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a2c908640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed4235240 0x564ed4237660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a2c908640 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed4235240 0x564ed4237660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d25e345604068b22 server_cookie=500260bad830ffb9 in_seq=0 out_seq=0 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.860+0000 7f1a07fff640 10 client.15570 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a07fff640 1 -- 172.21.15.100:0/134322268 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564ed3eb2890 con 0x564ed4235240 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a07fff640 10 client.15570 handle_command_reply: tid=0 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 client.15570 shutdown 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 msgr2=0x564ed422c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 0x564ed422c050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000ab20 tx=0x7f1a2009a6a0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 shutdown_connections 2024-03-12T05:12:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed41c8750 0x564ed422c050 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x564ed4235240 0x564ed4237660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ed41bfb70 0x564ed4227f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a07fff640 0 client.15570 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 --2- 172.21.15.100:0/134322268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ed41bf250 0x564ed4227a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 >> 172.21.15.100:0/134322268 conn(0x564ed41b9980 msgr2=0x564ed41b82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 shutdown_connections 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 1 -- 172.21.15.100:0/134322268 wait complete. 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.861+0000 7f1a3181e740 20 client.15570 trim_cache size 0 max 16384 2024-03-12T05:12:28.866 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:12:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a100a1c30 msgr2=0x7f1a100a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a100a1c30 0x7f1a100a40f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1a1802ccf0 tx=0x7f1a18039000 comp rx=0 tx=0).stop 2024-03-12T05:12:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 msgr2=0x7f1a281a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 0x7f1a281a36e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000cce0 tx=0x7f1a2000a620 comp rx=0 tx=0).stop 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 shutdown_connections 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a100a1c30 0x7f1a100a40f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a28199130 0x7f1a281a36e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a281053e0 0x7f1a28198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 --2- 172.21.15.100:0/3949962665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28100310 0x7f1a28198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 >> 172.21.15.100:0/3949962665 conn(0x7f1a280771e0 msgr2=0x7f1a281065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 shutdown_connections 2024-03-12T05:12:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.862+0000 7f1a2eb6c640 1 -- 172.21.15.100:0/3949962665 wait complete. 2024-03-12T05:12:28.868 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:28 smithi100 ceph-mon[32210]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-12T05:12:28.869 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:28 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2144630743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:12:28.869 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:12:28.870 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:12:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 -- 172.21.15.100:0/227818288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084105330 msgr2=0x7f708410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 --2- 172.21.15.100:0/227818288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084105330 0x7f708410cf00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f707c009cd0 tx=0x7f707c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 -- 172.21.15.100:0/227818288 shutdown_connections 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 --2- 172.21.15.100:0/227818288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084105330 0x7f708410cf00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 --2- 172.21.15.100:0/227818288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7084100c10 0x7f7084104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 --2- 172.21.15.100:0/227818288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7084100260 0x7f7084100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 -- 172.21.15.100:0/227818288 >> 172.21.15.100:0/227818288 conn(0x7f7084077180 msgr2=0x7f7084077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 -- 172.21.15.100:0/227818288 shutdown_connections 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.977+0000 7f708c464640 1 -- 172.21.15.100:0/227818288 wait complete. 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 Processor -- start 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 -- start start 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7084100260 0x7f708419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7084105330 0x7f708419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 0x7f7084197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084198100 con 0x7f7084197630 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084198270 con 0x7f7084100260 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708c464640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084198430 con 0x7f7084105330 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 0x7f7084197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 0x7f7084197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41122/0 (socket says 172.21.15.100:41122) 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 -- 172.21.15.100:0/275166236 learned_addr learned my addr 172.21.15.100:0/275166236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f70899ff640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7084105330 0x7f708419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708a200640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7084100260 0x7f708419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 -- 172.21.15.100:0/275166236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7084105330 msgr2=0x7f708419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7084105330 0x7f708419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 -- 172.21.15.100:0/275166236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7084100260 msgr2=0x7f708419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7084100260 0x7f708419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.978+0000 7f708aa01640 1 -- 172.21.15.100:0/275166236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7080009490 con 0x7f7084197630 2024-03-12T05:12:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.979+0000 7f708aa01640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 0x7f7084197ab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f707c004af0 tx=0x7f707c032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.979+0000 7f707b7fe640 1 -- 172.21.15.100:0/275166236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0328b0 con 0x7f7084197630 2024-03-12T05:12:28.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.979+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c009900 con 0x7f7084197630 2024-03-12T05:12:28.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.979+0000 7f707b7fe640 1 -- 172.21.15.100:0/275166236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f707c003e50 con 0x7f7084197630 2024-03-12T05:12:28.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.979+0000 7f707b7fe640 1 -- 172.21.15.100:0/275166236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c004740 con 0x7f7084197630 2024-03-12T05:12:28.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.979+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70841a4660 con 0x7f7084197630 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.981+0000 7f707b7fe640 1 -- 172.21.15.100:0/275166236 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f707c00a040 con 0x7f7084197630 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.981+0000 7f708f03c740 1 Processor -- start 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 -- start start 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f707b7fe640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70700a1c30 0x7f70700a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x56310620b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620bb70 0x563106214210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 0x563106216b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631061c6960 con 0x563106214750 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310617a310 con 0x56310620b250 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708f03c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106177840 con 0x56310620bb70 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f70899ff640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70700a1c30 0x7f70700a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 0x563106216b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f707b7fe640 1 -- 172.21.15.100:0/275166236 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f707c0f0470 con 0x7f7084197630 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 0x563106216b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41138/0 (socket says 172.21.15.100:41138) 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 -- 172.21.15.100:0/1761488605 learned_addr learned my addr 172.21.15.100:0/1761488605 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708a200640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x56310620b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f70899ff640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620bb70 0x563106214210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 -- 172.21.15.100:0/1761488605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620bb70 msgr2=0x563106214210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620bb70 0x563106214210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 -- 172.21.15.100:0/1761488605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 msgr2=0x56310620b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x56310620b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 -- 172.21.15.100:0/1761488605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563106135060 con 0x563106214750 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f70899ff640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620bb70 0x563106214210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708a200640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x56310620b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f708aa01640 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 0x563106216b40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f707c0bcd30 tx=0x7f707c0b9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9444c73e0e03059 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.982+0000 7f70899ff640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70700a1c30 0x7f70700a40f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7084198e80 tx=0x7f7074013830 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f7078bfe640 1 -- 172.21.15.100:0/1761488605 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0951f0 con 0x563106214750 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f7078bfe640 1 -- 172.21.15.100:0/1761488605 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f707c034da0 con 0x563106214750 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f7078bfe640 1 -- 172.21.15.100:0/1761488605 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0c57d0 con 0x563106214750 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 -- 172.21.15.100:0/1761488605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 msgr2=0x563106216b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 0x563106216b40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f707c0bcd30 tx=0x7f707c0b9390 comp rx=0 tx=0).stop 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 -- 172.21.15.100:0/1761488605 shutdown_connections 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563106214750 0x563106216b40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620bb70 0x563106214210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 --2- 172.21.15.100:0/1761488605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x56310620b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 -- 172.21.15.100:0/1761488605 >> 172.21.15.100:0/1761488605 conn(0x563106205980 msgr2=0x563106205d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 -- 172.21.15.100:0/1761488605 shutdown_connections 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.983+0000 7f708f03c740 1 -- 172.21.15.100:0/1761488605 wait complete. 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 -- start start 2024-03-12T05:12:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 Processor -- start 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x563106211780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106211cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563106214750 0x563106212200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106173020 con 0x56310620bb70 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563105e46430 con 0x56310620b250 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563105e02e00 con 0x563106214750 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708f03c740 10 client.0 _ll_register_callbacks cb 0x56310611acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106211cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106211cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41152/0 (socket says 172.21.15.100:41152) 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 -- 172.21.15.100:0/2985587857 learned_addr learned my addr 172.21.15.100:0/2985587857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708a200640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x563106211780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f708aa01640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563106214750 0x563106212200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 -- 172.21.15.100:0/2985587857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563106214750 msgr2=0x563106212200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563106214750 0x563106212200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 -- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 msgr2=0x563106211780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x563106211780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.984+0000 7f70899ff640 1 -- 172.21.15.100:0/2985587857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70841a4660 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f70899ff640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106211cc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f70740041f0 tx=0x7f7074004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccee86cfcbf2892d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f706dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f706dffb640 1 -- 172.21.15.100:0/2985587857 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074018070 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f706dffb640 1 -- 172.21.15.100:0/2985587857 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7074005020 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563105e727e0 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f706dffb640 1 -- 172.21.15.100:0/2985587857 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70740051e0 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 10 client.15582 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5631061716a0 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f706dffb640 1 -- 172.21.15.100:0/2985587857 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563105e727e0 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f706dffb640 1 -- 172.21.15.100:0/2985587857 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7074020450 con 0x56310620bb70 2024-03-12T05:12:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 10 client.15582 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 20 client.15582 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563106224930 0x563106226d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 4 client.15582 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.985+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563105eaf720 con 0x563106224930 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.986+0000 7f708a200640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563106224930 0x563106226d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.986+0000 7f706dffb640 10 client.15582 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.986+0000 7f708a200640 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563106224930 0x563106226d50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bb6691cba865297 server_cookie=e9753c8f79e4b196 in_seq=0 out_seq=0 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f706dffb640 1 -- 172.21.15.100:0/2985587857 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563105eaf720 con 0x563106224930 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f706dffb640 10 client.15582 handle_command_reply: tid=0 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 client.15582 shutdown 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 msgr2=0x563106211cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106211cc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f70740041f0 tx=0x7f7074004730 comp rx=0 tx=0).stop 2024-03-12T05:12:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 shutdown_connections 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563106224930 0x563106226d50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563106214750 0x563106212200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f706dffb640 0 client.15582 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106211cc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 --2- 172.21.15.100:0/2985587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310620b250 0x563106211780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 >> 172.21.15.100:0/2985587857 conn(0x563106205980 msgr2=0x5631062076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 shutdown_connections 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 1 -- 172.21.15.100:0/2985587857 wait complete. 2024-03-12T05:12:28.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.987+0000 7f708f03c740 20 client.15582 trim_cache size 0 max 16384 2024-03-12T05:12:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 Processor -- start 2024-03-12T05:12:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 -- start start 2024-03-12T05:12:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x563106213eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106212020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310620e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631060f8130 con 0x56310620bb70 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106206780 con 0x563106214750 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708f03c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310620ace0 con 0x56310620b250 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f70899ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106212020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f70899ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106212020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41154/0 (socket says 172.21.15.100:41154) 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f70899ff640 1 -- 172.21.15.100:0/780068773 learned_addr learned my addr 172.21.15.100:0/780068773 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:28.999+0000 7f708a200640 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x563106213eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f708aa01640 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310620e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f70899ff640 1 -- 172.21.15.100:0/780068773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 msgr2=0x563106213eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f70899ff640 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x563106213eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f70899ff640 1 -- 172.21.15.100:0/780068773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 msgr2=0x56310620e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f70899ff640 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310620e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f70899ff640 1 -- 172.21.15.100:0/780068773 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56310610dc20 con 0x56310620bb70 2024-03-12T05:12:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f70899ff640 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106212020 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f70740207e0 tx=0x7f7074009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0471f7099aa567 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f706ffff640 1 -- 172.21.15.100:0/780068773 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074003a80 con 0x56310620bb70 2024-03-12T05:12:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f706ffff640 1 -- 172.21.15.100:0/780068773 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7074004030 con 0x56310620bb70 2024-03-12T05:12:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.000+0000 7f706ffff640 1 -- 172.21.15.100:0/780068773 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074020b40 con 0x56310620bb70 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- 172.21.15.100:0/780068773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 msgr2=0x563106212020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106212020 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f70740207e0 tx=0x7f7074009ab0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- 172.21.15.100:0/780068773 shutdown_connections 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310620e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106212020 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- 172.21.15.100:0/780068773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x563106213eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- 172.21.15.100:0/780068773 >> 172.21.15.100:0/780068773 conn(0x563106205980 msgr2=0x56310620ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- 172.21.15.100:0/780068773 shutdown_connections 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- 172.21.15.100:0/780068773 wait complete. 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- start start 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 Processor -- start 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x56310621b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106219c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310621a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310621aa80 con 0x56310620bb70 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310627ae80 con 0x563106214750 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310627afc0 con 0x56310620b250 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f708f03c740 10 client.0 _ll_register_callbacks cb 0x7f707c0eda90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f70899ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106219c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f70899ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106219c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41166/0 (socket says 172.21.15.100:41166) 2024-03-12T05:12:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.001+0000 7f70899ff640 1 -- 172.21.15.100:0/478608400 learned_addr learned my addr 172.21.15.100:0/478608400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708aa01640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310621a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f70899ff640 1 -- 172.21.15.100:0/478608400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 msgr2=0x56310621b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708a200640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x56310621b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f70899ff640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x56310621b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f70899ff640 1 -- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 msgr2=0x56310621a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f70899ff640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310621a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f70899ff640 1 -- 172.21.15.100:0/478608400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631061716a0 con 0x56310620bb70 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708aa01640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310621a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708a200640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x56310621b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f70899ff640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106219c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7074009250 tx=0x7f7074020c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52c77ba202190cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f706dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f706dffb640 1 -- 172.21.15.100:0/478608400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70740046b0 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f706dffb640 1 -- 172.21.15.100:0/478608400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7074004810 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f707c034da0 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f706dffb640 1 -- 172.21.15.100:0/478608400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70740049d0 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 10 client.15588 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5631061338c0 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f706dffb640 1 -- 172.21.15.100:0/478608400 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f707c034da0 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f706dffb640 1 -- 172.21.15.100:0/478608400 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f707403d460 con 0x56310620bb70 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 10 client.15588 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 20 client.15588 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56310627c440 0x56310627e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 4 client.15588 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.002+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563105efe890 con 0x56310627c440 2024-03-12T05:12:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708a200640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56310627c440 0x56310627e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f706dffb640 10 client.15588 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708a200640 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56310627c440 0x56310627e860 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115ec358835b148d server_cookie=c7fe64957d04f159 in_seq=0 out_seq=0 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f706dffb640 1 -- 172.21.15.100:0/478608400 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563105efe890 con 0x56310627c440 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f706dffb640 10 client.15588 handle_command_reply: tid=0 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 client.15588 shutdown 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 msgr2=0x563106219c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106219c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7074009250 tx=0x7f7074020c90 comp rx=0 tx=0).stop 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 shutdown_connections 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56310627c440 0x56310627e860 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563106214750 0x56310621a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56310620bb70 0x563106219c90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 --2- 172.21.15.100:0/478608400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56310620b250 0x56310621b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f706dffb640 0 client.15588 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:29.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.003+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 >> 172.21.15.100:0/478608400 conn(0x563106205980 msgr2=0x5631062042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:29.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 shutdown_connections 2024-03-12T05:12:29.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708f03c740 1 -- 172.21.15.100:0/478608400 wait complete. 2024-03-12T05:12:29.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708f03c740 20 client.15588 trim_cache size 0 max 16384 2024-03-12T05:12:29.010 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70700a1c30 msgr2=0x7f70700a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70700a1c30 0x7f70700a40f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7084198e80 tx=0x7f7074013830 comp rx=0 tx=0).stop 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 msgr2=0x7f7084197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 0x7f7084197ab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f707c004af0 tx=0x7f707c032570 comp rx=0 tx=0).stop 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 shutdown_connections 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70700a1c30 0x7f70700a40f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7084197630 0x7f7084197ab0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7084105330 0x7f708419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 --2- 172.21.15.100:0/275166236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7084100260 0x7f708419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:29.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 >> 172.21.15.100:0/275166236 conn(0x7f7084077180 msgr2=0x7f70840fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:29.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 shutdown_connections 2024-03-12T05:12:29.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:29.004+0000 7f708c464640 1 -- 172.21.15.100:0/275166236 wait complete. 2024-03-12T05:12:29.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:28 smithi156 ceph-mon[36270]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-12T05:12:29.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:28 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2144630743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:12:29.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:28 smithi161 ceph-mon[36043]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-12T05:12:29.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:28 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2144630743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:12:30.014 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:12:30.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:12:30.014 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:12:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 -- 172.21.15.100:0/2520989570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 msgr2=0x7f790010b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 --2- 172.21.15.100:0/2520989570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f790010b1f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009900 tx=0x7f78f002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 -- 172.21.15.100:0/2520989570 shutdown_connections 2024-03-12T05:12:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 --2- 172.21.15.100:0/2520989570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f790010b7c0 0x7f790010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 --2- 172.21.15.100:0/2520989570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f790010b1f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 --2- 172.21.15.100:0/2520989570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79001023f0 0x7f79001027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 -- 172.21.15.100:0/2520989570 >> 172.21.15.100:0/2520989570 conn(0x7f79000771e0 msgr2=0x7f79000775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 -- 172.21.15.100:0/2520989570 shutdown_connections 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.134+0000 7f7907ec9640 1 -- 172.21.15.100:0/2520989570 wait complete. 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 Processor -- start 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 -- start start 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79001023f0 0x7f7900100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f7900100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790010b7c0 0x7f7900100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79001a5ca0 con 0x7f7900102d10 2024-03-12T05:12:30.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79001a5e10 con 0x7f79001023f0 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7907ec9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79001a5fd0 con 0x7f790010b7c0 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f7900100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f7900100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41184/0 (socket says 172.21.15.100:41184) 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 -- 172.21.15.100:0/3675834375 learned_addr learned my addr 172.21.15.100:0/3675834375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7906466640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790010b7c0 0x7f7900100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905c65640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79001023f0 0x7f7900100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 -- 172.21.15.100:0/3675834375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790010b7c0 msgr2=0x7f7900100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790010b7c0 0x7f7900100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 -- 172.21.15.100:0/3675834375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79001023f0 msgr2=0x7f7900100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79001023f0 0x7f7900100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905464640 1 -- 172.21.15.100:0/3675834375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f0009560 con 0x7f7900102d10 2024-03-12T05:12:30.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7906466640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790010b7c0 0x7f7900100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.135+0000 7f7905c65640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79001023f0 0x7f7900100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.136+0000 7f7905464640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f7900100580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005940 tx=0x7f78f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.136+0000 7f78f6ffd640 1 -- 172.21.15.100:0/3675834375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0004440 con 0x7f7900102d10 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.136+0000 7f78f6ffd640 1 -- 172.21.15.100:0/3675834375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78f00052d0 con 0x7f7900102d10 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.136+0000 7f78f6ffd640 1 -- 172.21.15.100:0/3675834375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003da0 con 0x7f7900102d10 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.136+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79001a6250 con 0x7f7900102d10 2024-03-12T05:12:30.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.136+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79001a6760 con 0x7f7900102d10 2024-03-12T05:12:30.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.138+0000 7f78f6ffd640 1 -- 172.21.15.100:0/3675834375 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f78f0005440 con 0x7f7900102d10 2024-03-12T05:12:30.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.138+0000 7f790ab70740 1 Processor -- start 2024-03-12T05:12:30.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.138+0000 7f790ab70740 1 -- start start 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f790ab70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f790ab70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f78f6ffd640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78ec0a1c30 0x7f78ec0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f790ab70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f790ab70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48fee0f90 con 0x55d48ff72d30 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f790ab70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48fede4c0 con 0x55d48ff72380 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f790ab70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48fed9ca0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905464640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78ec0a1c30 0x7f78ec0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff7b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41200/0 (socket says 172.21.15.100:41200) 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 -- 172.21.15.100:0/3061292120 learned_addr learned my addr 172.21.15.100:0/3061292120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f78f6ffd640 1 -- 172.21.15.100:0/3675834375 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f78f00f0260 con 0x7f7900102d10 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7906466640 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7906466640 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 -- 172.21.15.100:0/3061292120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 msgr2=0x55d48ff7dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff7dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 -- 172.21.15.100:0/3061292120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 msgr2=0x55d48ff72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff72760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 -- 172.21.15.100:0/3061292120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d48fe9c0b0 con 0x55d48ff72d30 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905464640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78ec0a1c30 0x7f78ec0a40f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78f00eda50 tx=0x7f78f008f020 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.139+0000 7f7905c65640 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff7b2e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00aa10 tx=0x7f78fc00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facfde995f02b847 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f78ebfff640 1 -- 172.21.15.100:0/3061292120 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00cbe0 con 0x55d48ff72d30 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f78ebfff640 1 -- 172.21.15.100:0/3061292120 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78fc00cd40 con 0x55d48ff72d30 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f78ebfff640 1 -- 172.21.15.100:0/3061292120 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc018740 con 0x55d48ff72d30 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 -- 172.21.15.100:0/3061292120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 msgr2=0x55d48ff7b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff7b2e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00aa10 tx=0x7f78fc00aed0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 -- 172.21.15.100:0/3061292120 shutdown_connections 2024-03-12T05:12:30.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff7dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff7b2e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 --2- 172.21.15.100:0/3061292120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 -- 172.21.15.100:0/3061292120 >> 172.21.15.100:0/3061292120 conn(0x55d48ff6c840 msgr2=0x55d48ff6cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.140+0000 7f790ab70740 1 -- 172.21.15.100:0/3061292120 shutdown_connections 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 -- 172.21.15.100:0/3061292120 wait complete. 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 -- start start 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 Processor -- start 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 0x55d48ff78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff79200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48fe7ef40 con 0x55d48ff72380 2024-03-12T05:12:30.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48fbad4b0 con 0x55d48ff72d30 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48fe5f670 con 0x55d48ff7b8b0 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f790ab70740 10 client.0 _ll_register_callbacks cb 0x55d48feccf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f7906466640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 0x55d48ff78780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f7906466640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 0x55d48ff78780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41206/0 (socket says 172.21.15.100:41206) 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.141+0000 7f7906466640 1 -- 172.21.15.100:0/3475720231 learned_addr learned my addr 172.21.15.100:0/3475720231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7905c65640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff79200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 -- 172.21.15.100:0/3475720231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 msgr2=0x55d48ff79200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff79200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 -- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 msgr2=0x55d48ff78cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff78cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 -- 172.21.15.100:0/3475720231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc00a640 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f7906466640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 0x55d48ff78780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f78f8014bd0 tx=0x7f78f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfddf50d0a919067 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f78e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f78e97fa640 1 -- 172.21.15.100:0/3475720231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8004d20 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f78e97fa640 1 -- 172.21.15.100:0/3475720231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f78f8012e40 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d48feb3f40 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f78e97fa640 1 -- 172.21.15.100:0/3475720231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78f8004e80 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f790ab70740 10 client.15600 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d48fed8320 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f78e97fa640 1 -- 172.21.15.100:0/3475720231 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d48feb3f40 con 0x55d48ff72380 2024-03-12T05:12:30.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f78e97fa640 1 -- 172.21.15.100:0/3475720231 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78f8005300 con 0x55d48ff72380 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f790ab70740 10 client.15600 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f790ab70740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.142+0000 7f790ab70740 20 client.15600 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.143+0000 7f790ab70740 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ff8bcd0 0x55d48ff8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.143+0000 7f790ab70740 4 client.15600 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.143+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d48fed0310 con 0x55d48ff8bcd0 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.143+0000 7f7905c65640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ff8bcd0 0x55d48ff8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.143+0000 7f7905c65640 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ff8bcd0 0x55d48ff8e0f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f3579bf42e7b03 server_cookie=23ca5c2f65ab5a98 in_seq=0 out_seq=0 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.143+0000 7f78e97fa640 10 client.15600 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f78e97fa640 1 -- 172.21.15.100:0/3475720231 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d48fed0310 con 0x55d48ff8bcd0 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f78e97fa640 10 client.15600 handle_command_reply: tid=0 2024-03-12T05:12:30.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 client.15600 shutdown 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 msgr2=0x55d48ff78780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 0x55d48ff78780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f78f8014bd0 tx=0x7f78f8004ab0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 shutdown_connections 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ff8bcd0 0x55d48ff8e0f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff79200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff78cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f78e97fa640 0 client.15600 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 --2- 172.21.15.100:0/3475720231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72380 0x55d48ff78780 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 >> 172.21.15.100:0/3475720231 conn(0x55d48ff6c840 msgr2=0x55d48ff6e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 shutdown_connections 2024-03-12T05:12:30.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 1 -- 172.21.15.100:0/3475720231 wait complete. 2024-03-12T05:12:30.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.144+0000 7f790ab70740 20 client.15600 trim_cache size 0 max 16384 2024-03-12T05:12:30.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.159+0000 7f790ab70740 1 Processor -- start 2024-03-12T05:12:30.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.159+0000 7f790ab70740 1 -- start start 2024-03-12T05:12:30.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f790ab70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff7aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f790ab70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f790ab70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f790ab70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48ff71cd0 con 0x55d48ff72d30 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f790ab70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48ff68180 con 0x55d48ff72380 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f790ab70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48ff888d0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41218/0 (socket says 172.21.15.100:41218) 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 -- 172.21.15.100:0/2012807336 learned_addr learned my addr 172.21.15.100:0/2012807336 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7906466640 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff7aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 -- 172.21.15.100:0/2012807336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 msgr2=0x55d48ff75f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7906466640 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff75f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 -- 172.21.15.100:0/2012807336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 msgr2=0x55d48ff7aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff7aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 -- 172.21.15.100:0/2012807336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d48fe74c00 con 0x55d48ff72d30 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7906466640 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff75f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.160+0000 7f7905c65640 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff759e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78fc011f20 tx=0x7f78fc01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b393a167f447a574 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f78eb7fe640 1 -- 172.21.15.100:0/2012807336 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc006be0 con 0x55d48ff72d30 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f78eb7fe640 1 -- 172.21.15.100:0/2012807336 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78fc0118d0 con 0x55d48ff72d30 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f78eb7fe640 1 -- 172.21.15.100:0/2012807336 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc026740 con 0x55d48ff72d30 2024-03-12T05:12:30.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 -- 172.21.15.100:0/2012807336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 msgr2=0x55d48ff759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff759e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78fc011f20 tx=0x7f78fc01c040 comp rx=0 tx=0).stop 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 -- 172.21.15.100:0/2012807336 shutdown_connections 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff7b8b0 0x55d48ff75f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff72d30 0x55d48ff759e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 --2- 172.21.15.100:0/2012807336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72380 0x55d48ff7aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 -- 172.21.15.100:0/2012807336 >> 172.21.15.100:0/2012807336 conn(0x55d48ff6c840 msgr2=0x55d48ff77050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 -- 172.21.15.100:0/2012807336 shutdown_connections 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 -- 172.21.15.100:0/2012807336 wait complete. 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.161+0000 7f790ab70740 1 -- start start 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 Processor -- start 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff72380 0x55d48ff81cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 0x55d48ffdc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48ffdc860 con 0x55d48ff7b8b0 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48ffdc9d0 con 0x55d48ff72d30 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48ffdcb90 con 0x55d48ff72380 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f790ab70740 10 client.0 _ll_register_callbacks cb 0x7f78fc00a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 0x55d48ffdc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 0x55d48ffdc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41222/0 (socket says 172.21.15.100:41222) 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 -- 172.21.15.100:0/2268455688 learned_addr learned my addr 172.21.15.100:0/2268455688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff72380 0x55d48ff81cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7905c65640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 -- 172.21.15.100:0/2268455688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff72380 msgr2=0x55d48ff81cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff72380 0x55d48ff81cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 -- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 msgr2=0x55d48ff82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.162+0000 7f7906466640 1 -- 172.21.15.100:0/2268455688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d48fe9c0b0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f7906466640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 0x55d48ffdc320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f78f80094a0 tx=0x7f78f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aad62942fa8505 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f78e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f78e97fa640 1 -- 172.21.15.100:0/2268455688 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80118b0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f78e97fa640 1 -- 172.21.15.100:0/2268455688 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f78f8011a10 con 0x55d48ff7b8b0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d48ff785b0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f78e97fa640 1 -- 172.21.15.100:0/2268455688 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78f8011bd0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 10 client.15606 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d48fe9a910 con 0x55d48ff7b8b0 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f78e97fa640 1 -- 172.21.15.100:0/2268455688 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d48ff785b0 con 0x55d48ff7b8b0 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f78e97fa640 1 -- 172.21.15.100:0/2268455688 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78f8026430 con 0x55d48ff7b8b0 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 10 client.15606 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 20 client.15606 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ffe4270 0x55d48ffe6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 4 client.15606 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d48feff900 con 0x55d48ffe4270 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.163+0000 7f7905c65640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ffe4270 0x55d48ffe6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.164+0000 7f78e97fa640 10 client.15606 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.164+0000 7f7905c65640 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ffe4270 0x55d48ffe6690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a5fd8534b9af98 server_cookie=8dd385743c714c8d in_seq=0 out_seq=0 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f78e97fa640 1 -- 172.21.15.100:0/2268455688 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d48feff900 con 0x55d48ffe4270 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f78e97fa640 10 client.15606 handle_command_reply: tid=0 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 client.15606 shutdown 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 msgr2=0x55d48ffdc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 0x55d48ffdc320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f78f80094a0 tx=0x7f78f8004a40 comp rx=0 tx=0).stop 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 shutdown_connections 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d48ffe4270 0x55d48ffe6690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d48ff7b8b0 0x55d48ffdc320 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f78e97fa640 0 client.15606 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d48ff72d30 0x55d48ff82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 --2- 172.21.15.100:0/2268455688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48ff72380 0x55d48ff81cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 >> 172.21.15.100:0/2268455688 conn(0x55d48ff6c840 msgr2=0x55d48ff6b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 shutdown_connections 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 1 -- 172.21.15.100:0/2268455688 wait complete. 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.165+0000 7f790ab70740 20 client.15606 trim_cache size 0 max 16384 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:12:30.172 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "4b649554-c604-4558-a37f-d301a10d54c4", 2024-03-12T05:12:30.173 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:12:30.173 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:12:30.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78ec0a1c30 msgr2=0x7f78ec0a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78ec0a1c30 0x7f78ec0a40f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78f00eda50 tx=0x7f78f008f020 comp rx=0 tx=0).stop 2024-03-12T05:12:30.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 msgr2=0x7f7900100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f7900100580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005940 tx=0x7f78f0004040 comp rx=0 tx=0).stop 2024-03-12T05:12:30.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 shutdown_connections 2024-03-12T05:12:30.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78ec0a1c30 0x7f78ec0a40f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790010b7c0 0x7f7900100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7900102d10 0x7f7900100580 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 --2- 172.21.15.100:0/3675834375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79001023f0 0x7f7900100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 >> 172.21.15.100:0/3675834375 conn(0x7f79000771e0 msgr2=0x7f79000fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 shutdown_connections 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.166+0000 7f7907ec9640 1 -- 172.21.15.100:0/3675834375 wait complete. 2024-03-12T05:12:30.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:12:30.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:12:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 -- 172.21.15.100:0/2924259287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164106430 msgr2=0x7f916410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 --2- 172.21.15.100:0/2924259287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164106430 0x7f916410e000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f915c009cd0 tx=0x7f915c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 -- 172.21.15.100:0/2924259287 shutdown_connections 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 --2- 172.21.15.100:0/2924259287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164106430 0x7f916410e000 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 --2- 172.21.15.100:0/2924259287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9164101d10 0x7f9164105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 --2- 172.21.15.100:0/2924259287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164101360 0x7f9164101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 -- 172.21.15.100:0/2924259287 >> 172.21.15.100:0/2924259287 conn(0x7f91640fcc20 msgr2=0x7f91640ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 -- 172.21.15.100:0/2924259287 shutdown_connections 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 -- 172.21.15.100:0/2924259287 wait complete. 2024-03-12T05:12:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.282+0000 7f9169096640 1 Processor -- start 2024-03-12T05:12:30.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 -- start start 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 0x7f91641976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164106430 0x7f9164197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916419bf80 0x7f916419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916419c970 con 0x7f9164101360 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916419cae0 con 0x7f9164106430 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9169096640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916419cc50 con 0x7f916419bf80 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 0x7f91641976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 0x7f91641976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41234/0 (socket says 172.21.15.100:41234) 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 -- 172.21.15.100:0/1462008463 learned_addr learned my addr 172.21.15.100:0/1462008463 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f916359e640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916419bf80 0x7f916419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f916259c640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164106430 0x7f9164197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 -- 172.21.15.100:0/1462008463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916419bf80 msgr2=0x7f916419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916419bf80 0x7f916419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 -- 172.21.15.100:0/1462008463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164106430 msgr2=0x7f9164197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164106430 0x7f9164197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 -- 172.21.15.100:0/1462008463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c009900 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f916359e640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916419bf80 0x7f916419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f916259c640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164106430 0x7f9164197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.283+0000 7f9162d9d640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 0x7f91641976c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f915800c920 tx=0x7f915800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.284+0000 7f914bfff640 1 -- 172.21.15.100:0/1462008463 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158007c70 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.284+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91641a4230 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.284+0000 7f914bfff640 1 -- 172.21.15.100:0/1462008463 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9158007dd0 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.284+0000 7f914bfff640 1 -- 172.21.15.100:0/1462008463 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158018860 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.284+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91641a4730 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f914bfff640 1 -- 172.21.15.100:0/1462008463 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f915800fac0 con 0x7f9164101360 2024-03-12T05:12:30.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 Processor -- start 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 -- start start 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab83a280 con 0x5611ab8cbbd0 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab8377b0 con 0x5611ab8d47b0 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f916bc62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab832f90 con 0x5611ab8cb2b0 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f9162d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f9162d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41236/0 (socket says 172.21.15.100:41236) 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.286+0000 7f9162d9d640 1 -- 172.21.15.100:0/4122881252 learned_addr learned my addr 172.21.15.100:0/4122881252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f914bfff640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f914c0a1c30 0x7f914c0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f916259c640 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f916359e640 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f916359e640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f914c0a1c30 0x7f914c0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f9162d9d640 1 -- 172.21.15.100:0/4122881252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 msgr2=0x5611ab8cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f9162d9d640 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f9162d9d640 1 -- 172.21.15.100:0/4122881252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 msgr2=0x5611ab8d6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f9162d9d640 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f9162d9d640 1 -- 172.21.15.100:0/4122881252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ab7f1570 con 0x5611ab8cbbd0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f916259c640 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f914bfff640 1 -- 172.21.15.100:0/1462008463 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91580cdbc0 con 0x7f9164101360 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f9162d9d640 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d4270 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9158097040 tx=0x7f915809a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35509f8faaf5899d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f91497fa640 1 -- 172.21.15.100:0/4122881252 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580d11e0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f916359e640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f914c0a1c30 0x7f914c0a40f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f915c006010 tx=0x7f915c0095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f91497fa640 1 -- 172.21.15.100:0/4122881252 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9158099fd0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.287+0000 7f91497fa640 1 -- 172.21.15.100:0/4122881252 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915809e7c0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 -- 172.21.15.100:0/4122881252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 msgr2=0x5611ab8d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d4270 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9158097040 tx=0x7f915809a570 comp rx=0 tx=0).stop 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 -- 172.21.15.100:0/4122881252 shutdown_connections 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d4270 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 --2- 172.21.15.100:0/4122881252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 -- 172.21.15.100:0/4122881252 >> 172.21.15.100:0/4122881252 conn(0x5611ab8c5990 msgr2=0x5611ab8c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 -- 172.21.15.100:0/4122881252 shutdown_connections 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 -- 172.21.15.100:0/4122881252 wait complete. 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 -- start start 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.288+0000 7f916bc62740 1 Processor -- start 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab506430 con 0x5611ab8cbbd0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab4c2e00 con 0x5611ab8d47b0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab7b7fd0 con 0x5611ab8cb2b0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 10 client.0 _ll_register_callbacks cb 0x5611ab80dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41250/0 (socket says 172.21.15.100:41250) 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 -- 172.21.15.100:0/977625072 learned_addr learned my addr 172.21.15.100:0/977625072 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f9162d9d640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916359e640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 -- 172.21.15.100:0/977625072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 msgr2=0x5611ab8d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 -- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 msgr2=0x5611ab8d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 -- 172.21.15.100:0/977625072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91641a4730 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916259c640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d1b60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f915400bf00 tx=0x7f91540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d01809567435a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f9146ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f9146ff5640 1 -- 172.21.15.100:0/977625072 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154012650 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f9146ff5640 1 -- 172.21.15.100:0/977625072 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91540127b0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611ab5327e0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f9146ff5640 1 -- 172.21.15.100:0/977625072 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9154012970 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 10 client.15618 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.289+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5611ab831610 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f9146ff5640 1 -- 172.21.15.100:0/977625072 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9154027080 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f9146ff5640 1 -- 172.21.15.100:0/977625072 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9154023410 con 0x5611ab8cbbd0 2024-03-12T05:12:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f916bc62740 10 client.15618 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f916bc62740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f916bc62740 20 client.15618 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f916bc62740 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab8e4a90 0x5611ab8e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f916bc62740 4 client.15618 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611ab56f720 con 0x5611ab8e4a90 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f9162d9d640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab8e4a90 0x5611ab8e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f9146ff5640 10 client.15618 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.290+0000 7f9162d9d640 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab8e4a90 0x5611ab8e6eb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f0a112a5fdcc9c server_cookie=798f144fdac2940f in_seq=0 out_seq=0 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f9146ff5640 1 -- 172.21.15.100:0/977625072 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611ab56f720 con 0x5611ab8e4a90 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f9146ff5640 10 client.15618 handle_command_reply: tid=0 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 client.15618 shutdown 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 msgr2=0x5611ab8d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d1b60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f915400bf00 tx=0x7f91540117b0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 shutdown_connections 2024-03-12T05:12:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab8e4a90 0x5611ab8e6eb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8d1b60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f9146ff5640 0 client.15618 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 --2- 172.21.15.100:0/977625072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 >> 172.21.15.100:0/977625072 conn(0x5611ab8c5990 msgr2=0x5611ab8c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 shutdown_connections 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 1 -- 172.21.15.100:0/977625072 wait complete. 2024-03-12T05:12:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.291+0000 7f916bc62740 20 client.15618 trim_cache size 0 max 16384 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 Processor -- start 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 -- start start 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab8d0d10 con 0x5611ab8cbbd0 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab8c12b0 con 0x5611ab8d47b0 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916bc62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab8bfe20 con 0x5611ab8cb2b0 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41252/0 (socket says 172.21.15.100:41252) 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 -- 172.21.15.100:0/2106403638 learned_addr learned my addr 172.21.15.100:0/2106403638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 -- 172.21.15.100:0/2106403638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 msgr2=0x5611ab8d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 -- 172.21.15.100:0/2106403638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 msgr2=0x5611ab8d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 -- 172.21.15.100:0/2106403638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ab7cdac0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.306+0000 7f916259c640 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8c48d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9154001eb0 tx=0x7f9154011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1ebc08b91b572d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f9148ff9640 1 -- 172.21.15.100:0/2106403638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154023a00 con 0x5611ab8cbbd0 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f9148ff9640 1 -- 172.21.15.100:0/2106403638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611ab5327e0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f9148ff9640 1 -- 172.21.15.100:0/2106403638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154004680 con 0x5611ab8cbbd0 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 -- 172.21.15.100:0/2106403638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 msgr2=0x5611ab8c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8c48d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9154001eb0 tx=0x7f9154011de0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 -- 172.21.15.100:0/2106403638 shutdown_connections 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8d47b0 0x5611ab8d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8c48d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 --2- 172.21.15.100:0/2106403638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8cb2b0 0x5611ab8d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 -- 172.21.15.100:0/2106403638 >> 172.21.15.100:0/2106403638 conn(0x5611ab8c5990 msgr2=0x5611ab8c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 -- 172.21.15.100:0/2106403638 shutdown_connections 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 -- 172.21.15.100:0/2106403638 wait complete. 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.307+0000 7f916bc62740 1 -- start start 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 Processor -- start 2024-03-12T05:12:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8cb2b0 0x5611ab8da640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8d47b0 0x5611ab93afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab93b4e0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab93b620 con 0x5611ab8cb2b0 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ab93b760 con 0x5611ab8d47b0 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 10 client.0 _ll_register_callbacks cb 0x7f91580cb0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8dab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41264/0 (socket says 172.21.15.100:41264) 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 -- 172.21.15.100:0/4263411375 learned_addr learned my addr 172.21.15.100:0/4263411375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:12:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916359e640 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8d47b0 0x5611ab93afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 -- 172.21.15.100:0/4263411375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8d47b0 msgr2=0x5611ab93afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8d47b0 0x5611ab93afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 -- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8cb2b0 msgr2=0x5611ab8da640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8cb2b0 0x5611ab8da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 -- 172.21.15.100:0/4263411375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611ab831610 con 0x5611ab8cbbd0 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916259c640 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8dab80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f91540058c0 tx=0x7f9154023640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52c622a9512e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f9146ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f9146ff5640 1 -- 172.21.15.100:0/4263411375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154035070 con 0x5611ab8cbbd0 2024-03-12T05:12:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f9146ff5640 1 -- 172.21.15.100:0/4263411375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f915402a030 con 0x5611ab8cbbd0 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.308+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9158099fd0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f9146ff5640 1 -- 172.21.15.100:0/4263411375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9154005620 con 0x5611ab8cbbd0 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 10 client.15624 fetch_fsmap learned FSMap version 23 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5611ab7f36a0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f9146ff5640 1 -- 172.21.15.100:0/4263411375 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9158099fd0 con 0x5611ab8cbbd0 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f9146ff5640 1 -- 172.21.15.100:0/4263411375 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f915402fb20 con 0x5611ab8cbbd0 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 10 client.15624 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 20 client.15624 populate_metadata read hostname 'smithi100' 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab93cc10 0x5611ab93f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 4 client.15624 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5611ab5be890 con 0x5611ab93cc10 2024-03-12T05:12:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.309+0000 7f9162d9d640 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab93cc10 0x5611ab93f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f9146ff5640 10 client.15624 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f9162d9d640 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab93cc10 0x5611ab93f030 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523fd7e471efb084 server_cookie=7ec2e5dedff959bb in_seq=0 out_seq=0 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f9146ff5640 1 -- 172.21.15.100:0/4263411375 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5611ab5be890 con 0x5611ab93cc10 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f9146ff5640 10 client.15624 handle_command_reply: tid=0 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 client.15624 shutdown 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 msgr2=0x5611ab8dab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8dab80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f91540058c0 tx=0x7f9154023640 comp rx=0 tx=0).stop 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 shutdown_connections 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5611ab93cc10 0x5611ab93f030 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611ab8d47b0 0x5611ab93afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f9146ff5640 0 client.15624 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611ab8cbbd0 0x5611ab8dab80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 --2- 172.21.15.100:0/4263411375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611ab8cb2b0 0x5611ab8da640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 >> 172.21.15.100:0/4263411375 conn(0x5611ab8c5990 msgr2=0x5611ab8d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 shutdown_connections 2024-03-12T05:12:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 1 -- 172.21.15.100:0/4263411375 wait complete. 2024-03-12T05:12:30.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.310+0000 7f916bc62740 20 client.15624 trim_cache size 0 max 16384 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: "4b649554-c604-4558-a37f-d301a10d54c4": { 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "4b649554-c604-4558-a37f-d301a10d54c4", 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:12:30.319 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:12:30.320 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:12:30.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f914c0a1c30 msgr2=0x7f914c0a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f914c0a1c30 0x7f914c0a40f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f915c006010 tx=0x7f915c0095c0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 msgr2=0x7f91641976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 0x7f91641976c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f915800c920 tx=0x7f915800cde0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 shutdown_connections 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f914c0a1c30 0x7f914c0a40f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916419bf80 0x7f916419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9164106430 0x7f9164197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 --2- 172.21.15.100:0/1462008463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164101360 0x7f91641976c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 >> 172.21.15.100:0/1462008463 conn(0x7f91640fcc20 msgr2=0x7f91640fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 shutdown_connections 2024-03-12T05:12:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:12:30.311+0000 7f9169096640 1 -- 172.21.15.100:0/1462008463 wait complete. 2024-03-12T05:12:30.323 INFO:tasks.cephfs.filesystem:scrub status for tag:4b649554-c604-4558-a37f-d301a10d54c4 - {'path': '/', 'tag': '4b649554-c604-4558-a37f-d301a10d54c4', 'options': 'recursive,force'} 2024-03-12T05:12:31.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:30 smithi100 ceph-mon[32210]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:31.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:30 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:12:31.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:12:31.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:30 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:12:31.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:12:31.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:30 smithi156 ceph-mon[36270]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:31.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:30 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:12:31.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:12:31.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:30 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:12:31.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:12:31.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:30 smithi161 ceph-mon[36043]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:31.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:30 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:12:31.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:12:31.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:30 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:12:31.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:12:33.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:32 smithi100 ceph-mon[32210]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:33.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:32 smithi156 ceph-mon[36270]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:33.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:32 smithi161 ceph-mon[36043]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:35.042 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:34 smithi161 ceph-mon[36043]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-12T05:12:35.042 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:34 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:12:35.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:34 smithi100 ceph-mon[32210]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-12T05:12:35.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:34 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:12:35.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:34 smithi156 ceph-mon[36270]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-12T05:12:35.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:34 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:12:37.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:36 smithi100 ceph-mon[32210]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:37.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:36 smithi156 ceph-mon[36270]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:37.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:36 smithi161 ceph-mon[36043]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:39.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:38 smithi100 ceph-mon[32210]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-12T05:12:39.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:38 smithi156 ceph-mon[36270]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-12T05:12:39.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:38 smithi161 ceph-mon[36043]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-12T05:12:41.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:40 smithi100 ceph-mon[32210]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:41.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:40 smithi156 ceph-mon[36270]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:41.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:40 smithi161 ceph-mon[36043]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:43.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:42 smithi100 ceph-mon[32210]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:43.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:42 smithi156 ceph-mon[36270]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:43.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:42 smithi161 ceph-mon[36043]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-12T05:12:45.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:44 smithi100 ceph-mon[32210]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-12T05:12:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:44 smithi156 ceph-mon[36270]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-12T05:12:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:44 smithi161 ceph-mon[36043]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-12T05:12:47.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:46 smithi100 ceph-mon[32210]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:47.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:46 smithi156 ceph-mon[36270]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:47.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:46 smithi161 ceph-mon[36043]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:49.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:48 smithi100 ceph-mon[32210]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-12T05:12:49.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:48 smithi156 ceph-mon[36270]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-12T05:12:49.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:48 smithi161 ceph-mon[36043]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-12T05:12:51.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:50 smithi100 ceph-mon[32210]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:51.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:50 smithi156 ceph-mon[36270]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:51.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:50 smithi161 ceph-mon[36043]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:53.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:52 smithi100 ceph-mon[32210]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:53.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:52 smithi156 ceph-mon[36270]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:53.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:52 smithi161 ceph-mon[36043]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-12T05:12:55.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:54 smithi100 ceph-mon[32210]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-12T05:12:55.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:54 smithi156 ceph-mon[36270]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-12T05:12:55.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:54 smithi161 ceph-mon[36043]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-12T05:12:57.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:56 smithi100 ceph-mon[32210]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:57.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:56 smithi156 ceph-mon[36270]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:57.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:56 smithi161 ceph-mon[36043]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:12:59.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:12:58 smithi156 ceph-mon[36270]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-12T05:12:59.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:12:58 smithi161 ceph-mon[36043]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-12T05:12:59.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:12:58 smithi100 ceph-mon[32210]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-12T05:13:00.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:13:00.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 -- 172.21.15.100:0/663584680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100c20 msgr2=0x7f21ec104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 --2- 172.21.15.100:0/663584680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100c20 0x7f21ec104c60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009930 tx=0x7f21e002bc30 comp rx=0 tx=0).stop 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 -- 172.21.15.100:0/663584680 shutdown_connections 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 --2- 172.21.15.100:0/663584680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ec105340 0x7f21ec10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 --2- 172.21.15.100:0/663584680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100c20 0x7f21ec104c60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 --2- 172.21.15.100:0/663584680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ec100270 0x7f21ec100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.491+0000 7f21f1e64640 1 -- 172.21.15.100:0/663584680 >> 172.21.15.100:0/663584680 conn(0x7f21ec077230 msgr2=0x7f21ec077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.492+0000 7f21f1e64640 1 -- 172.21.15.100:0/663584680 shutdown_connections 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.492+0000 7f21f1e64640 1 -- 172.21.15.100:0/663584680 wait complete. 2024-03-12T05:13:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.492+0000 7f21f1e64640 1 Processor -- start 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.492+0000 7f21f1e64640 1 -- start start 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21f1e64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 0x7f21ec1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 0x7f21ec1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 0x7f21ec1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42362/0 (socket says 172.21.15.100:42362) 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 -- 172.21.15.100:0/910269737 learned_addr learned my addr 172.21.15.100:0/910269737 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ec105340 0x7f21ec198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ec199060 0x7f21ec1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec1a3cb0 con 0x7f21ec100270 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec1a3e20 con 0x7f21ec105340 2024-03-12T05:13:00.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec1a3fe0 con 0x7f21ec199060 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eaffd640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ec105340 0x7f21ec198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21ebfff640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ec199060 0x7f21ec1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 -- 172.21.15.100:0/910269737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ec199060 msgr2=0x7f21ec1a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ec199060 0x7f21ec1a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 -- 172.21.15.100:0/910269737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ec105340 msgr2=0x7f21ec198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ec105340 0x7f21ec198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.493+0000 7f21eb7fe640 1 -- 172.21.15.100:0/910269737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc009490 con 0x7f21ec100270 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.494+0000 7f21eb7fe640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 0x7f21ec1985e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00ea60 tx=0x7f21dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.494+0000 7f21e8ff9640 1 -- 172.21.15.100:0/910269737 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00cde0 con 0x7f21ec100270 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.494+0000 7f21e8ff9640 1 -- 172.21.15.100:0/910269737 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21dc0035f0 con 0x7f21ec100270 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.494+0000 7f21e8ff9640 1 -- 172.21.15.100:0/910269737 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc010690 con 0x7f21ec100270 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.494+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e0009560 con 0x7f21ec100270 2024-03-12T05:13:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.494+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ec1a45c0 con 0x7f21ec100270 2024-03-12T05:13:00.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.497+0000 7f21f4a37740 1 Processor -- start 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.497+0000 7f21f4a37740 1 -- start start 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae2c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae2cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21f4a37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ad9b280 con 0x56403ae2c6d0 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21f4a37740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ad987b0 con 0x56403ae3dee0 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21f4a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ad93f90 con 0x56403ae2bd20 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21e8ff9640 1 -- 172.21.15.100:0/910269737 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f21dc003af0 con 0x7f21ec100270 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae2cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae2c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37348/0 (socket says 172.21.15.100:37348) 2024-03-12T05:13:00.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 -- 172.21.15.100:0/2329804912 learned_addr learned my addr 172.21.15.100:0/2329804912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:00.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21eaffd640 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 -- 172.21.15.100:0/2329804912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 msgr2=0x56403ae402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 -- 172.21.15.100:0/2329804912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 msgr2=0x56403ae2cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae2cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21ebfff640 1 -- 172.21.15.100:0/2329804912 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403ad52570 con 0x56403ae2bd20 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.498+0000 7f21eaffd640 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae2c100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f21e4007dd0 tx=0x7f21e400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ddab4a82bc50b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21e8ff9640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f21d40a1cb0 0x7f21d40a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21d1ffb640 1 -- 172.21.15.100:0/2329804912 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4004d20 con 0x56403ae2bd20 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21e8ff9640 1 -- 172.21.15.100:0/910269737 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f21dc014070 con 0x7f21ec100270 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21eb7fe640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f21d40a1cb0 0x7f21d40a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21d1ffb640 1 -- 172.21.15.100:0/2329804912 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21e4004e80 con 0x56403ae2bd20 2024-03-12T05:13:00.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.499+0000 7f21d1ffb640 1 -- 172.21.15.100:0/2329804912 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4010990 con 0x56403ae2bd20 2024-03-12T05:13:00.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 -- 172.21.15.100:0/2329804912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 msgr2=0x56403ae2c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21eb7fe640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f21d40a1cb0 0x7f21d40a4170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cb530 tx=0x7f21dc0d16d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae2c100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f21e4007dd0 tx=0x7f21e400e720 comp rx=0 tx=0).stop 2024-03-12T05:13:00.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 -- 172.21.15.100:0/2329804912 shutdown_connections 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae2cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2329804912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae2c100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.500+0000 7f21f4a37740 1 -- 172.21.15.100:0/2329804912 >> 172.21.15.100:0/2329804912 conn(0x56403ae26990 msgr2=0x56403ae26d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.501+0000 7f21f4a37740 1 -- 172.21.15.100:0/2329804912 shutdown_connections 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.501+0000 7f21f4a37740 1 -- 172.21.15.100:0/2329804912 wait complete. 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.501+0000 7f21f4a37740 1 -- start start 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.501+0000 7f21f4a37740 1 Processor -- start 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.501+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.501+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae37a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae37a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae374e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37362/0 (socket says 172.21.15.100:37362) 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae37a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42378/0 (socket says 172.21.15.100:42378) 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21f4a37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403aa67430 con 0x56403ae2c6d0 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21ebfff640 1 -- 172.21.15.100:0/1347068848 learned_addr learned my addr 172.21.15.100:0/1347068848 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403aa23e00 con 0x56403ae3dee0 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ad18fd0 con 0x56403ae2bd20 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21ebfff640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21f4a37740 10 client.0 _ll_register_callbacks cb 0x56403ad6efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 -- 172.21.15.100:0/1347068848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 msgr2=0x56403ae374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 -- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 msgr2=0x56403ae3d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 -- 172.21.15.100:0/1347068848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e4007a00 con 0x56403ae2c6d0 2024-03-12T05:13:00.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21ebfff640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56403aad0720 con 0x56403ae2c6d0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21eaffd640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae37a20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f21e002c170 tx=0x7f21e002cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34236217ad6d1a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.502+0000 7f21caffd640 10 client.15642 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21caffd640 1 -- 172.21.15.100:0/1347068848 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004250 con 0x56403ae2c6d0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21caffd640 1 -- 172.21.15.100:0/1347068848 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f21e00043b0 con 0x56403ae2c6d0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21caffd640 1 -- 172.21.15.100:0/1347068848 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21e0004df0 con 0x56403ae2c6d0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 10 client.15642 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x56403ad92610 con 0x56403ae2c6d0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21caffd640 1 -- 172.21.15.100:0/1347068848 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21e003c040 con 0x56403ae2c6d0 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 10 client.15642 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 20 client.15642 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403ae58c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 4 client.15642 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.503+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56403ab1f890 con 0x56403ae56870 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.504+0000 7f21ebfff640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403ae58c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.504+0000 7f21ebfff640 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403ae58c60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86812efc7c652000 server_cookie=40fe18734ff73381 in_seq=0 out_seq=0 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.504+0000 7f21caffd640 10 client.15642 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21caffd640 1 -- 172.21.15.100:0/1347068848 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56403ab1f890 con 0x56403ae56870 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21caffd640 10 client.15642 handle_command_reply: tid=0 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 client.15642 shutdown 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 msgr2=0x56403ae37a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae37a20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f21e002c170 tx=0x7f21e002cb50 comp rx=0 tx=0).stop 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 shutdown_connections 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403ae58c60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21caffd640 0 client.15642 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:00.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403ae37a20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 --2- 172.21.15.100:0/1347068848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403ae374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 >> 172.21.15.100:0/1347068848 conn(0x56403ae26990 msgr2=0x56403ae28730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 shutdown_connections 2024-03-12T05:13:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 1 -- 172.21.15.100:0/1347068848 wait complete. 2024-03-12T05:13:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.505+0000 7f21f4a37740 20 client.15642 trim_cache size 0 max 16384 2024-03-12T05:13:00.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 Processor -- start 2024-03-12T05:13:00.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 -- start start 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 0x56403ae372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae2c6d0 0x56403ae37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 0x56403ae372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 0x56403ae372d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42386/0 (socket says 172.21.15.100:42386) 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae3dee0 0x56403ae2f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ae22320 con 0x56403ae2bd20 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ae20e20 con 0x56403ae2c6d0 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21f4a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ae2b3a0 con 0x56403ae3dee0 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 -- 172.21.15.100:0/3532860312 learned_addr learned my addr 172.21.15.100:0/3532860312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21eaffd640 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae2c6d0 0x56403ae37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 -- 172.21.15.100:0/3532860312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae3dee0 msgr2=0x56403ae2f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae3dee0 0x56403ae2f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 -- 172.21.15.100:0/3532860312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae2c6d0 msgr2=0x56403ae37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae2c6d0 0x56403ae37810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 -- 172.21.15.100:0/3532860312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403ad2eac0 con 0x56403ae2bd20 2024-03-12T05:13:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.534+0000 7f21ebfff640 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 0x56403ae372d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f21e4007da0 tx=0x7f21e400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65231b2fb310da37 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.535+0000 7f21d17fa640 1 -- 172.21.15.100:0/3532860312 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e401c040 con 0x56403ae2bd20 2024-03-12T05:13:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.535+0000 7f21d17fa640 1 -- 172.21.15.100:0/3532860312 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21e4003e50 con 0x56403ae2bd20 2024-03-12T05:13:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.535+0000 7f21d17fa640 1 -- 172.21.15.100:0/3532860312 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4004050 con 0x56403ae2bd20 2024-03-12T05:13:00.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 -- 172.21.15.100:0/3532860312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 msgr2=0x56403ae372d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 0x56403ae372d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f21e4007da0 tx=0x7f21e400e720 comp rx=0 tx=0).stop 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 -- 172.21.15.100:0/3532860312 shutdown_connections 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae3dee0 0x56403ae2f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae2c6d0 0x56403ae37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 --2- 172.21.15.100:0/3532860312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2bd20 0x56403ae372d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 -- 172.21.15.100:0/3532860312 >> 172.21.15.100:0/3532860312 conn(0x56403ae26990 msgr2=0x56403ae309e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 -- 172.21.15.100:0/3532860312 shutdown_connections 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 -- 172.21.15.100:0/3532860312 wait complete. 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 -- start start 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.536+0000 7f21f4a37740 1 Processor -- start 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403aea75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403aea7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ae3c950 con 0x56403ae2c6d0 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ae3cac0 con 0x56403ae3dee0 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ae3cc80 con 0x56403ae2bd20 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21f4a37740 10 client.0 _ll_register_callbacks cb 0x7f21e4007560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403aea7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403aea7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42390/0 (socket says 172.21.15.100:42390) 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 -- 172.21.15.100:0/2639516088 learned_addr learned my addr 172.21.15.100:0/2639516088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403aea75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 -- 172.21.15.100:0/2639516088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 msgr2=0x56403aea75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403aea75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 -- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 msgr2=0x56403ae3c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 -- 172.21.15.100:0/2639516088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56403ad52570 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21eaffd640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403aea7af0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009850 tx=0x7f21e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32b0858afb96e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.537+0000 7f21caffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21caffd640 1 -- 172.21.15.100:0/2639516088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004df0 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21caffd640 1 -- 172.21.15.100:0/2639516088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f21e0000e40 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56403aa937e0 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21caffd640 1 -- 172.21.15.100:0/2639516088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21e002c5d0 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 10 client.15648 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56403ad546a0 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21caffd640 1 -- 172.21.15.100:0/2639516088 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56403aad0720 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21caffd640 1 -- 172.21.15.100:0/2639516088 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21e003c040 con 0x56403ae2c6d0 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 10 client.15648 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:00.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 20 client.15648 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:00.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403aeb1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 4 client.15648 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:13:00.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.538+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56403aa937e0 con 0x56403ae56870 2024-03-12T05:13:00.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.539+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403aeb1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.539+0000 7f21ebfff640 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403aeb1200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e6eb7f2745c15c server_cookie=1c902c8f087a6d49 in_seq=0 out_seq=0 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.539+0000 7f21caffd640 10 client.15648 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.539+0000 7f21caffd640 1 -- 172.21.15.100:0/2639516088 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56403aa937e0 con 0x56403ae56870 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.539+0000 7f21caffd640 10 client.15648 handle_command_reply: tid=0 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.539+0000 7f21f4a37740 1 client.15648 shutdown 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 msgr2=0x56403aea7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403aea7af0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009850 tx=0x7f21e0009260 comp rx=0 tx=0).stop 2024-03-12T05:13:00.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 shutdown_connections 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56403ae56870 0x56403aeb1200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56403ae3dee0 0x56403ae3c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56403ae2c6d0 0x56403aea7af0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 --2- 172.21.15.100:0/2639516088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56403ae2bd20 0x56403aea75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 >> 172.21.15.100:0/2639516088 conn(0x56403ae26990 msgr2=0x56403ae251c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21caffd640 0 client.15648 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 shutdown_connections 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 1 -- 172.21.15.100:0/2639516088 wait complete. 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f4a37740 20 client.15648 trim_cache size 0 max 16384 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:13:00.541 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:13:00.542 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:13:00.542 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:13:00.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f21d40a1cb0 msgr2=0x7f21d40a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f21d40a1cb0 0x7f21d40a4170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cb530 tx=0x7f21dc0d16d0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 msgr2=0x7f21ec1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.540+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 0x7f21ec1985e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00ea60 tx=0x7f21dc00ef20 comp rx=0 tx=0).stop 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 shutdown_connections 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f21d40a1cb0 0x7f21d40a4170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ec199060 0x7f21ec1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ec105340 0x7f21ec198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 --2- 172.21.15.100:0/910269737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21ec100270 0x7f21ec1985e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 >> 172.21.15.100:0/910269737 conn(0x7f21ec077230 msgr2=0x7f21ec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 shutdown_connections 2024-03-12T05:13:00.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.541+0000 7f21f1e64640 1 -- 172.21.15.100:0/910269737 wait complete. 2024-03-12T05:13:00.551 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:13:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:13:00.227+0000 7fc4a409b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:13:00.551 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:13:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:13:00.227+0000 7fc4a409b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:13:00.551 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:13:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:13:00.227+0000 7fc4a409b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:13:00.551 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:13:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:13:00.227+0000 7fc4a409b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:13:00.551 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:13:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:13:00.227+0000 7fc4a409b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-12T05:13:00.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:13:00.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 -- 172.21.15.100:0/2085895152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff24102d10 msgr2=0x7eff2410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 --2- 172.21.15.100:0/2085895152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff24102d10 0x7eff2410b1f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff18009900 tx=0x7eff1802bc30 comp rx=0 tx=0).stop 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 -- 172.21.15.100:0/2085895152 shutdown_connections 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 --2- 172.21.15.100:0/2085895152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff2410b7c0 0x7eff2410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 --2- 172.21.15.100:0/2085895152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff24102d10 0x7eff2410b1f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 --2- 172.21.15.100:0/2085895152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff241023f0 0x7eff241027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 -- 172.21.15.100:0/2085895152 >> 172.21.15.100:0/2085895152 conn(0x7eff24077180 msgr2=0x7eff24077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 -- 172.21.15.100:0/2085895152 shutdown_connections 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.659+0000 7eff298d4640 1 -- 172.21.15.100:0/2085895152 wait complete. 2024-03-12T05:13:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 Processor -- start 2024-03-12T05:13:00.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 -- start start 2024-03-12T05:13:00.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff241023f0 0x7eff2419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24102d10 0x7eff2419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 0x7eff24197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24197de0 con 0x7eff2410b7c0 2024-03-12T05:13:00.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24197f50 con 0x7eff241023f0 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.660+0000 7eff298d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24198110 con 0x7eff24102d10 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 0x7eff24197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 0x7eff24197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42396/0 (socket says 172.21.15.100:42396) 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 -- 172.21.15.100:0/278549177 learned_addr learned my addr 172.21.15.100:0/278549177 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff227fc640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24102d10 0x7eff2419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff22ffd640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff241023f0 0x7eff2419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 -- 172.21.15.100:0/278549177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24102d10 msgr2=0x7eff2419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24102d10 0x7eff2419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 -- 172.21.15.100:0/278549177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff241023f0 msgr2=0x7eff2419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff241023f0 0x7eff2419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 -- 172.21.15.100:0/278549177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009560 con 0x7eff2410b7c0 2024-03-12T05:13:00.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff237fe640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 0x7eff24197650 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00eb10 tx=0x7eff1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c004b20 con 0x7eff2410b7c0 2024-03-12T05:13:00.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff241983f0 con 0x7eff2410b7c0 2024-03-12T05:13:00.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff1c004c80 con 0x7eff2410b7c0 2024-03-12T05:13:00.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.661+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff24100150 con 0x7eff2410b7c0 2024-03-12T05:13:00.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.662+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c010690 con 0x7eff2410b7c0 2024-03-12T05:13:00.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.663+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff24068b20 con 0x7eff2410b7c0 2024-03-12T05:13:00.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.664+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7eff1c0107f0 con 0x7eff2410b7c0 2024-03-12T05:13:00.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.664+0000 7eff288d2640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7eff0c0aa440 0x7eff0c0ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:00.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.665+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7eff1c014070 con 0x7eff2410b7c0 2024-03-12T05:13:00.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.665+0000 7eff22ffd640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7eff0c0aa440 0x7eff0c0ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:00.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.666+0000 7eff22ffd640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7eff0c0aa440 0x7eff0c0ac900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff14009b60 tx=0x7eff14005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:00.672+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff1c096fc0 con 0x7eff2410b7c0 2024-03-12T05:13:01.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.007+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff241068b0 con 0x7eff2410b7c0 2024-03-12T05:13:01.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.007+0000 7eff288d2640 1 -- 172.21.15.100:0/278549177 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7eff1c09a680 con 0x7eff2410b7c0 2024-03-12T05:13:01.004 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:13:01.004 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:11:51.088786+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:13:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 23 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7eff0c0aa440 msgr2=0x7eff0c0ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7eff0c0aa440 0x7eff0c0ac900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff14009b60 tx=0x7eff14005f90 comp rx=0 tx=0).stop 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 msgr2=0x7eff24197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 0x7eff24197650 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00eb10 tx=0x7eff1c00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 shutdown_connections 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7eff0c0aa440 0x7eff0c0ac900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2410b7c0 0x7eff24197650 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24102d10 0x7eff2419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 --2- 172.21.15.100:0/278549177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff241023f0 0x7eff2419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 >> 172.21.15.100:0/278549177 conn(0x7eff24077180 msgr2=0x7eff240fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 shutdown_connections 2024-03-12T05:13:01.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.010+0000 7eff298d4640 1 -- 172.21.15.100:0/278549177 wait complete. 2024-03-12T05:13:01.022 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:13:01.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 -- 172.21.15.100:0/183106061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 msgr2=0x7f719010e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 --2- 172.21.15.100:0/183106061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719010e070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7188009cd0 tx=0x7f718802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 -- 172.21.15.100:0/183106061 shutdown_connections 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 --2- 172.21.15.100:0/183106061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719010e070 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 --2- 172.21.15.100:0/183106061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7190068fa0 0x7f7190105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 --2- 172.21.15.100:0/183106061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71900685f0 0x7f71900689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 -- 172.21.15.100:0/183106061 >> 172.21.15.100:0/183106061 conn(0x7f7190077180 msgr2=0x7f7190077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 -- 172.21.15.100:0/183106061 shutdown_connections 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 -- 172.21.15.100:0/183106061 wait complete. 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.131+0000 7f7196565640 1 Processor -- start 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 -- start start 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71900685f0 0x7f719019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7190197600 0x7f7190197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190197ff0 con 0x7f71901064c0 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42416/0 (socket says 172.21.15.100:42416) 2024-03-12T05:13:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190198160 con 0x7f7190197600 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 -- 172.21.15.100:0/3616655550 learned_addr learned my addr 172.21.15.100:0/3616655550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190198320 con 0x7f71900685f0 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718ffff640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71900685f0 0x7f719019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 -- 172.21.15.100:0/3616655550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71900685f0 msgr2=0x7f719019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71900685f0 0x7f719019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 -- 172.21.15.100:0/3616655550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7190197600 msgr2=0x7f7190197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f7194b02640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7190197600 0x7f7190197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.132+0000 7f718f7fe640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7190197600 0x7f7190197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f718f7fe640 1 -- 172.21.15.100:0/3616655550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188009900 con 0x7f71901064c0 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f7194b02640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7190197600 0x7f7190197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f718f7fe640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719019d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7184009a00 tx=0x7f7184009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f718d7fa640 1 -- 172.21.15.100:0/3616655550 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184014070 con 0x7f71901064c0 2024-03-12T05:13:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71900fff70 con 0x7f71901064c0 2024-03-12T05:13:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f718d7fa640 1 -- 172.21.15.100:0/3616655550 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7184007da0 con 0x7f71901064c0 2024-03-12T05:13:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f718d7fa640 1 -- 172.21.15.100:0/3616655550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184004b60 con 0x7f71901064c0 2024-03-12T05:13:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.133+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7190100430 con 0x7f71901064c0 2024-03-12T05:13:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 Processor -- start 2024-03-12T05:13:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 -- start start 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 0x56278a172630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172b70 0x56278a17b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f718d7fa640 1 -- 172.21.15.100:0/3616655550 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f7184018070 con 0x7f71901064c0 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a17b750 0x56278a17db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a12d960 con 0x56278a172250 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a0e1310 con 0x56278a172b70 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7199214740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a0de840 con 0x56278a17b750 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7194b02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 0x56278a172630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7194b02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 0x56278a172630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42418/0 (socket says 172.21.15.100:42418) 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.136+0000 7f7194b02640 1 -- 172.21.15.100:0/3248838295 learned_addr learned my addr 172.21.15.100:0/3248838295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f718ffff640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172b70 0x56278a17b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f7194b02640 1 -- 172.21.15.100:0/3248838295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a17b750 msgr2=0x56278a17db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f718f7fe640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a17b750 0x56278a17db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f7194b02640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a17b750 0x56278a17db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f7194b02640 1 -- 172.21.15.100:0/3248838295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172b70 msgr2=0x56278a17b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f7194b02640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172b70 0x56278a17b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f7194b02640 1 -- 172.21.15.100:0/3248838295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56278a09c060 con 0x56278a172250 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f718f7fe640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a17b750 0x56278a17db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f718ffff640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172b70 0x56278a17b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f718d7fa640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f71780a1c30 0x7f71780a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f7194b02640 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 0x56278a172630 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7188004af0 tx=0x7f7188032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dfdb0e0bcf3b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f718d7fa640 1 -- 172.21.15.100:0/3616655550 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f71840d0d00 con 0x7f71901064c0 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f71767fc640 1 -- 172.21.15.100:0/3248838295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188034440 con 0x56278a172250 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f71767fc640 1 -- 172.21.15.100:0/3248838295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71880345a0 con 0x56278a172250 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.137+0000 7f71767fc640 1 -- 172.21.15.100:0/3248838295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188003bd0 con 0x56278a172250 2024-03-12T05:13:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f718ffff640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f71780a1c30 0x7f71780a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 -- 172.21.15.100:0/3248838295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 msgr2=0x56278a172630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f718ffff640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f71780a1c30 0x7f71780a40f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56278a171d20 tx=0x7f7180005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 0x56278a172630 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7188004af0 tx=0x7f7188032d70 comp rx=0 tx=0).stop 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 -- 172.21.15.100:0/3248838295 shutdown_connections 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a17b750 0x56278a17db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172b70 0x56278a17b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 --2- 172.21.15.100:0/3248838295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172250 0x56278a172630 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 -- 172.21.15.100:0/3248838295 >> 172.21.15.100:0/3248838295 conn(0x56278a16c980 msgr2=0x56278a16cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 -- 172.21.15.100:0/3248838295 shutdown_connections 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.138+0000 7f7199214740 1 -- 172.21.15.100:0/3248838295 wait complete. 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 -- start start 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 Processor -- start 2024-03-12T05:13:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a1786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a178bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a179130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a0da020 con 0x56278a172b70 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562789dad430 con 0x56278a17b750 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562789d69e00 con 0x56278a172250 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.139+0000 7f7199214740 10 client.0 _ll_register_callbacks cb 0x56278a081cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a178bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a178bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42430/0 (socket says 172.21.15.100:42430) 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 -- 172.21.15.100:0/781254892 learned_addr learned my addr 172.21.15.100:0/781254892 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718ffff640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a1786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f7194b02640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a179130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 -- 172.21.15.100:0/781254892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 msgr2=0x56278a1786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a1786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 -- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 msgr2=0x56278a179130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a179130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 -- 172.21.15.100:0/781254892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719010e880 con 0x56278a172b70 2024-03-12T05:13:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f718f7fe640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a178bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f718409d740 tx=0x7f718409d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87a66ecf60fd1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f716f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f716f7fe640 1 -- 172.21.15.100:0/781254892 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840ce830 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f716f7fe640 1 -- 172.21.15.100:0/781254892 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f718409d7b0 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562789dd97e0 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f716f7fe640 1 -- 172.21.15.100:0/781254892 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71840a1900 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f7199214740 10 client.15666 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.140+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56278a0d86a0 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f716f7fe640 1 -- 172.21.15.100:0/781254892 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562789dd97e0 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f716f7fe640 1 -- 172.21.15.100:0/781254892 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f71840880e0 con 0x56278a172b70 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f7199214740 10 client.15666 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f7199214740 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f7199214740 20 client.15666 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f7199214740 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a18b8d0 0x56278a18dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f7199214740 4 client.15666 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.141+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562789e16720 con 0x56278a18b8d0 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.142+0000 7f718ffff640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a18b8d0 0x56278a18dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.142+0000 7f718ffff640 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a18b8d0 0x56278a18dcf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab1b493cec601f56 server_cookie=a4102e07343d9880 in_seq=0 out_seq=0 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.142+0000 7f716f7fe640 10 client.15666 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f716f7fe640 1 -- 172.21.15.100:0/781254892 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562789e16720 con 0x56278a18b8d0 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f716f7fe640 10 client.15666 handle_command_reply: tid=0 2024-03-12T05:13:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 client.15666 shutdown 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 msgr2=0x56278a178bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a178bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f718409d740 tx=0x7f718409d2f0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 shutdown_connections 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a18b8d0 0x56278a18dcf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a179130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a178bf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 --2- 172.21.15.100:0/781254892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a1786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f716f7fe640 0 client.15666 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 >> 172.21.15.100:0/781254892 conn(0x56278a16c980 msgr2=0x56278a16e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 shutdown_connections 2024-03-12T05:13:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 1 -- 172.21.15.100:0/781254892 wait complete. 2024-03-12T05:13:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.143+0000 7f7199214740 20 client.15666 trim_cache size 0 max 16384 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 Processor -- start 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 -- start start 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a17ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a173950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a173e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a05f130 con 0x56278a172b70 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562789dd97e0 con 0x56278a17b750 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7199214740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a15ea40 con 0x56278a172250 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a173950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a173950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42440/0 (socket says 172.21.15.100:42440) 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 -- 172.21.15.100:0/111725114 learned_addr learned my addr 172.21.15.100:0/111725114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718ffff640 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a17ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f7194b02640 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 -- 172.21.15.100:0/111725114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 msgr2=0x56278a17ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a17ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 -- 172.21.15.100:0/111725114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 msgr2=0x56278a173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.155+0000 7f718f7fe640 1 -- 172.21.15.100:0/111725114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56278a074c20 con 0x56278a172b70 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.156+0000 7f7194b02640 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a173e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.156+0000 7f718f7fe640 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a173950 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f718409a180 tx=0x7f718409d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9544dda56355a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.156+0000 7f7175ffb640 1 -- 172.21.15.100:0/111725114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184099a50 con 0x56278a172b70 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.156+0000 7f7175ffb640 1 -- 172.21.15.100:0/111725114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71840a1900 con 0x56278a172b70 2024-03-12T05:13:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.156+0000 7f7175ffb640 1 -- 172.21.15.100:0/111725114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840cedb0 con 0x56278a172b70 2024-03-12T05:13:01.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 -- 172.21.15.100:0/111725114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 msgr2=0x56278a173950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a173950 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f718409a180 tx=0x7f718409d420 comp rx=0 tx=0).stop 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 -- 172.21.15.100:0/111725114 shutdown_connections 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a17b750 0x56278a173e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a172b70 0x56278a173950 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 --2- 172.21.15.100:0/111725114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172250 0x56278a17ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 -- 172.21.15.100:0/111725114 >> 172.21.15.100:0/111725114 conn(0x56278a16c980 msgr2=0x56278a167a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 -- 172.21.15.100:0/111725114 shutdown_connections 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 -- 172.21.15.100:0/111725114 wait complete. 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.157+0000 7f7199214740 1 -- start start 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 Processor -- start 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172250 0x56278a181710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172b70 0x56278a181c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 0x56278a1e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 0x56278a1e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 0x56278a1e1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42448/0 (socket says 172.21.15.100:42448) 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 -- 172.21.15.100:0/4029964265 learned_addr learned my addr 172.21.15.100:0/4029964265 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a1e2320 con 0x56278a17b750 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a1e2460 con 0x56278a172250 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278a1e25a0 con 0x56278a172b70 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f718f7fe640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172b70 0x56278a181c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f718ffff640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172250 0x56278a181710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7199214740 10 client.0 _ll_register_callbacks cb 0x7f7188037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 -- 172.21.15.100:0/4029964265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172b70 msgr2=0x56278a181c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172b70 0x56278a181c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 -- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172250 msgr2=0x56278a181710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172250 0x56278a181710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f7194b02640 1 -- 172.21.15.100:0/4029964265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56278a0d86a0 con 0x56278a17b750 2024-03-12T05:13:01.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.158+0000 7f718f7fe640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172b70 0x56278a181c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7194b02640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 0x56278a1e1de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7188004af0 tx=0x7f7188034530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1315e54fb8e8d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f716f7fe640 10 client.15672 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56278a1784e0 con 0x56278a17b750 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f716f7fe640 1 -- 172.21.15.100:0/4029964265 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71880328b0 con 0x56278a17b750 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f716f7fe640 1 -- 172.21.15.100:0/4029964265 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7188034e20 con 0x56278a17b750 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f716f7fe640 1 -- 172.21.15.100:0/4029964265 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7188032a10 con 0x56278a17b750 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 10 client.15672 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x56278a09a8c0 con 0x56278a17b750 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f716f7fe640 1 -- 172.21.15.100:0/4029964265 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7188003970 con 0x56278a17b750 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 10 client.15672 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 10 client.15672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 20 client.15672 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a1e37b0 0x56278a1e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 4 client.15672 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.159+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56278a1784e0 con 0x56278a1e37b0 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.160+0000 7f718ffff640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a1e37b0 0x56278a1e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.160+0000 7f716f7fe640 10 client.15672 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.160+0000 7f718ffff640 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a1e37b0 0x56278a1e5bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22afb68af1344a69 server_cookie=b67afa842dc1002c in_seq=0 out_seq=0 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.160+0000 7f716f7fe640 1 -- 172.21.15.100:0/4029964265 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56278a1784e0 con 0x56278a1e37b0 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.160+0000 7f716f7fe640 10 client.15672 handle_command_reply: tid=0 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.160+0000 7f7199214740 1 client.15672 shutdown 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 msgr2=0x56278a1e1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 0x56278a1e1de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7188004af0 tx=0x7f7188034530 comp rx=0 tx=0).stop 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 shutdown_connections 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56278a1e37b0 0x56278a1e5bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f716f7fe640 0 client.15672 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56278a17b750 0x56278a1e1de0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56278a172b70 0x56278a181c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 --2- 172.21.15.100:0/4029964265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56278a172250 0x56278a181710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 >> 172.21.15.100:0/4029964265 conn(0x56278a16c980 msgr2=0x56278a173950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 shutdown_connections 2024-03-12T05:13:01.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 1 -- 172.21.15.100:0/4029964265 wait complete. 2024-03-12T05:13:01.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7199214740 20 client.15672 trim_cache size 0 max 16384 2024-03-12T05:13:01.164 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f71780a1c30 msgr2=0x7f71780a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7196565640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f71780a1c30 0x7f71780a40f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56278a171d20 tx=0x7f7180005f90 comp rx=0 tx=0).stop 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 msgr2=0x7f719019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.161+0000 7f7196565640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719019d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7184009a00 tx=0x7f7184009ec0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 shutdown_connections 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f71780a1c30 0x7f71780a40f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7190197600 0x7f7190197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71901064c0 0x7f719019d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 --2- 172.21.15.100:0/3616655550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71900685f0 0x7f719019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 >> 172.21.15.100:0/3616655550 conn(0x7f7190077180 msgr2=0x7f71900fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 shutdown_connections 2024-03-12T05:13:01.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.162+0000 7f7196565640 1 -- 172.21.15.100:0/3616655550 wait complete. 2024-03-12T05:13:01.169 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:00 smithi100 ceph-mon[32210]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:13:01.170 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:13:01.170 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:13:01.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:00 smithi156 ceph-mon[36270]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:13:01.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:00 smithi161 ceph-mon[36043]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:13:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 -- 172.21.15.100:0/3711414767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4441064c0 msgr2=0x7ff44410e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 --2- 172.21.15.100:0/3711414767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4441064c0 0x7ff44410e070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009cd0 tx=0x7ff43c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 -- 172.21.15.100:0/3711414767 shutdown_connections 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 --2- 172.21.15.100:0/3711414767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4441064c0 0x7ff44410e070 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 --2- 172.21.15.100:0/3711414767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff444068fa0 0x7ff444105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 --2- 172.21.15.100:0/3711414767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 0x7ff4440689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 -- 172.21.15.100:0/3711414767 >> 172.21.15.100:0/3711414767 conn(0x7ff444077180 msgr2=0x7ff444077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 -- 172.21.15.100:0/3711414767 shutdown_connections 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.278+0000 7ff44b484640 1 -- 172.21.15.100:0/3711414767 wait complete. 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 Processor -- start 2024-03-12T05:13:01.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 -- start start 2024-03-12T05:13:01.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 0x7ff44419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4441064c0 0x7ff44419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 0x7ff444197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444197ff0 con 0x7ff444197600 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444198160 con 0x7ff4440685f0 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff44b484640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444198320 con 0x7ff4441064c0 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 0x7ff444197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 0x7ff444197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42468/0 (socket says 172.21.15.100:42468) 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 -- 172.21.15.100:0/3856919039 learned_addr learned my addr 172.21.15.100:0/3856919039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449220640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 0x7ff44419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff448a1f640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4441064c0 0x7ff44419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 -- 172.21.15.100:0/3856919039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4441064c0 msgr2=0x7ff44419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4441064c0 0x7ff44419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 -- 172.21.15.100:0/3856919039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 msgr2=0x7ff44419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 0x7ff44419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449a21640 1 -- 172.21.15.100:0/3856919039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c009900 con 0x7ff444197600 2024-03-12T05:13:01.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.279+0000 7ff449220640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 0x7ff44419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:01.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.280+0000 7ff449a21640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 0x7ff444197ab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff43c004af0 tx=0x7ff43c034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.280+0000 7ff43a7fc640 1 -- 172.21.15.100:0/3856919039 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0328c0 con 0x7ff444197600 2024-03-12T05:13:01.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.280+0000 7ff43a7fc640 1 -- 172.21.15.100:0/3856919039 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff43c003e50 con 0x7ff444197600 2024-03-12T05:13:01.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.280+0000 7ff43a7fc640 1 -- 172.21.15.100:0/3856919039 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c004860 con 0x7ff444197600 2024-03-12T05:13:01.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.280+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4441a4310 con 0x7ff444197600 2024-03-12T05:13:01.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.280+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441a4740 con 0x7ff444197600 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.282+0000 7ff43a7fc640 1 -- 172.21.15.100:0/3856919039 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7ff43c00a040 con 0x7ff444197600 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.282+0000 7ff44e058740 1 Processor -- start 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.282+0000 7ff44e058740 1 -- start start 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff44e058740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f1404360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff43a7fc640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4300a1cb0 0x7ff4300a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff44e058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f1411620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff44e058740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f1413f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff44e058740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f13bf960 con 0x55b6f1411b60 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff44e058740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f1373310 con 0x55b6f1403f80 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff44e058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f1370840 con 0x55b6f14048a0 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f1413f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449220640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4300a1cb0 0x7ff4300a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f1413f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42470/0 (socket says 172.21.15.100:42470) 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 -- 172.21.15.100:0/3818295071 learned_addr learned my addr 172.21.15.100:0/3818295071 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff43a7fc640 1 -- 172.21.15.100:0/3856919039 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff43c0f0670 con 0x7ff444197600 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff448a1f640 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f1404360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff448a1f640 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f1411620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 -- 172.21.15.100:0/3818295071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 msgr2=0x55b6f1411620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f1411620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 -- 172.21.15.100:0/3818295071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 msgr2=0x55b6f1404360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f1404360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449a21640 1 -- 172.21.15.100:0/3818295071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6f132e060 con 0x55b6f1411b60 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.283+0000 7ff449220640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4300a1cb0 0x7ff4300a4170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff4400058c0 tx=0x7ff440005850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.284+0000 7ff449a21640 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f1413f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0b9750 tx=0x7ff43c03bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0af51aaa97b937 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.284+0000 7ff42f7fe640 1 -- 172.21.15.100:0/3818295071 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0bce30 con 0x55b6f1411b60 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.284+0000 7ff42f7fe640 1 -- 172.21.15.100:0/3818295071 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff43c0d7070 con 0x55b6f1411b60 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.284+0000 7ff42f7fe640 1 -- 172.21.15.100:0/3818295071 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0c5a90 con 0x55b6f1411b60 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.284+0000 7ff44e058740 1 -- 172.21.15.100:0/3818295071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 msgr2=0x55b6f1413f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.284+0000 7ff44e058740 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f1413f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0b9750 tx=0x7ff43c03bcc0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- 172.21.15.100:0/3818295071 shutdown_connections 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f1413f50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f1411620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 --2- 172.21.15.100:0/3818295071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f1404360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- 172.21.15.100:0/3818295071 >> 172.21.15.100:0/3818295071 conn(0x55b6f13fe980 msgr2=0x55b6f13fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- 172.21.15.100:0/3818295071 shutdown_connections 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- 172.21.15.100:0/3818295071 wait complete. 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- start start 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 Processor -- start 2024-03-12T05:13:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f140eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f140f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f140f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f136c020 con 0x55b6f1411b60 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f103f430 con 0x55b6f1403f80 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.285+0000 7ff44e058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f0ffbe00 con 0x55b6f14048a0 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff44e058740 10 client.0 _ll_register_callbacks cb 0x55b6f1313cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f140f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f140f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42480/0 (socket says 172.21.15.100:42480) 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f140f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff449a21640 1 -- 172.21.15.100:0/577623974 learned_addr learned my addr 172.21.15.100:0/577623974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff449220640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f140eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 -- 172.21.15.100:0/577623974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 msgr2=0x55b6f140eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f140eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 -- 172.21.15.100:0/577623974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 msgr2=0x55b6f140f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f140f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 -- 172.21.15.100:0/577623974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441a4740 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff449a21640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f140f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff448a1f640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f140f0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff4440ffb10 tx=0x7ff434011c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe678fdc88bb50b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff42cff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff42cff9640 1 -- 172.21.15.100:0/577623974 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434017030 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff42cff9640 1 -- 172.21.15.100:0/577623974 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff434010920 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.286+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6f106b7e0 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff42cff9640 1 -- 172.21.15.100:0/577623974 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff434004520 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 10 client.25207 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b6f136a6a0 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff42cff9640 1 -- 172.21.15.100:0/577623974 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4340035f0 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff42cff9640 1 -- 172.21.15.100:0/577623974 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff43401e440 con 0x55b6f14048a0 2024-03-12T05:13:01.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 10 client.25207 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 10 client.25207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 20 client.25207 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1421cf0 0x55b6f1424170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 4 client.25207 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6f106b7e0 con 0x55b6f1421cf0 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.287+0000 7ff449220640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1421cf0 0x55b6f1424170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.288+0000 7ff42cff9640 10 client.25207 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.288+0000 7ff449220640 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1421cf0 0x55b6f1424170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=deeb9f40829d7f4b server_cookie=6e929430f835a2d4 in_seq=0 out_seq=0 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.288+0000 7ff42cff9640 1 -- 172.21.15.100:0/577623974 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6f106b7e0 con 0x55b6f1421cf0 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.288+0000 7ff42cff9640 10 client.25207 handle_command_reply: tid=0 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.288+0000 7ff44e058740 1 client.25207 shutdown 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 msgr2=0x55b6f140f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f140f0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff4440ffb10 tx=0x7ff434011c40 comp rx=0 tx=0).stop 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 shutdown_connections 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f140f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f14048a0 0x55b6f140f0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1421cf0 0x55b6f1424170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 --2- 172.21.15.100:0/577623974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1403f80 0x55b6f140eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 >> 172.21.15.100:0/577623974 conn(0x55b6f13fe980 msgr2=0x55b6f14006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff42cff9640 0 client.25207 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 shutdown_connections 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 1 -- 172.21.15.100:0/577623974 wait complete. 2024-03-12T05:13:01.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.289+0000 7ff44e058740 20 client.25207 trim_cache size 0 max 16384 2024-03-12T05:13:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 Processor -- start 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 -- start start 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1428b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 0x55b6f140f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1411b60 0x55b6f14004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f12f1130 con 0x55b6f14048a0 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f1404da0 con 0x55b6f1411b60 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff44e058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f13ff780 con 0x55b6f1403f80 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 0x55b6f140f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 0x55b6f140f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42488/0 (socket says 172.21.15.100:42488) 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 -- 172.21.15.100:0/3075792163 learned_addr learned my addr 172.21.15.100:0/3075792163 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff449a21640 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1411b60 0x55b6f14004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff449220640 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1428b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 -- 172.21.15.100:0/3075792163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 msgr2=0x55b6f1428b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1428b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 -- 172.21.15.100:0/3075792163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1411b60 msgr2=0x55b6f14004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1411b60 0x55b6f14004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff448a1f640 1 -- 172.21.15.100:0/3075792163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6f1306c20 con 0x55b6f14048a0 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.301+0000 7ff449a21640 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1411b60 0x55b6f14004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff448a1f640 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 0x55b6f140f400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff434006010 tx=0x7ff434009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aa1378d722ff66 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff42effd640 1 -- 172.21.15.100:0/3075792163 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434017040 con 0x55b6f14048a0 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff42effd640 1 -- 172.21.15.100:0/3075792163 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff434005500 con 0x55b6f14048a0 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff42effd640 1 -- 172.21.15.100:0/3075792163 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4340047a0 con 0x55b6f14048a0 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 -- 172.21.15.100:0/3075792163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 msgr2=0x55b6f140f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 0x55b6f140f400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff434006010 tx=0x7ff434009fb0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 -- 172.21.15.100:0/3075792163 shutdown_connections 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f1411b60 0x55b6f14004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f14048a0 0x55b6f140f400 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 --2- 172.21.15.100:0/3075792163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1428b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.302+0000 7ff44e058740 1 -- 172.21.15.100:0/3075792163 >> 172.21.15.100:0/3075792163 conn(0x55b6f13fe980 msgr2=0x55b6f13ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 -- 172.21.15.100:0/3075792163 shutdown_connections 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 -- 172.21.15.100:0/3075792163 wait complete. 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 -- start start 2024-03-12T05:13:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 Processor -- start 2024-03-12T05:13:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1417570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f14048a0 0x55b6f1417ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f14782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f14182f0 con 0x55b6f1411b60 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f14787e0 con 0x55b6f14048a0 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f1478920 con 0x55b6f1403f80 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff44e058740 10 client.0 _ll_register_callbacks cb 0x7ff43c0edc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.303+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f14782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449220640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1417570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f14782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42496/0 (socket says 172.21.15.100:42496) 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449220640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1417570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37540/0 (socket says 172.21.15.100:37540) 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 -- 172.21.15.100:0/2280106215 learned_addr learned my addr 172.21.15.100:0/2280106215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff448a1f640 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f14048a0 0x55b6f1417ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 -- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 msgr2=0x55b6f1417570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1417570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 -- 172.21.15.100:0/2280106215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f14048a0 msgr2=0x55b6f1417ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f14048a0 0x55b6f1417ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 -- 172.21.15.100:0/2280106215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6f136a6a0 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff449a21640 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f14782a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0bcff0 tx=0x7ff43c004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40925b438f41022 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff42cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff42cff9640 1 -- 172.21.15.100:0/2280106215 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0bce30 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff42cff9640 1 -- 172.21.15.100:0/2280106215 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff43c0bd9f0 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff43c0d7070 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff42cff9640 1 -- 172.21.15.100:0/2280106215 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff43c0bcc50 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff44e058740 10 client.15690 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.304+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b6f132c8c0 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff42cff9640 1 -- 172.21.15.100:0/2280106215 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff43c0d7070 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff42cff9640 1 -- 172.21.15.100:0/2280106215 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff43c0da070 con 0x55b6f1411b60 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff44e058740 10 client.15690 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff44e058740 10 client.15690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff44e058740 20 client.15690 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff44e058740 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1479d70 0x55b6f147c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff44e058740 4 client.15690 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6f10a8720 con 0x55b6f1479d70 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff449220640 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1479d70 0x55b6f147c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff42cff9640 10 client.15690 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.305+0000 7ff449220640 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1479d70 0x55b6f147c160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b94ec64e60424976 server_cookie=f52463fdfbb3af03 in_seq=0 out_seq=0 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff42cff9640 1 -- 172.21.15.100:0/2280106215 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6f10a8720 con 0x55b6f1479d70 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff42cff9640 10 client.15690 handle_command_reply: tid=0 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 client.15690 shutdown 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 msgr2=0x55b6f14782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f14782a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0bcff0 tx=0x7ff43c004300 comp rx=0 tx=0).stop 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 shutdown_connections 2024-03-12T05:13:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f1411b60 0x55b6f14782a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b6f1479d70 0x55b6f147c160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6f14048a0 0x55b6f1417ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 --2- 172.21.15.100:0/2280106215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6f1403f80 0x55b6f1417570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 >> 172.21.15.100:0/2280106215 conn(0x55b6f13fe980 msgr2=0x55b6f13fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff42cff9640 0 client.15690 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 shutdown_connections 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 1 -- 172.21.15.100:0/2280106215 wait complete. 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44e058740 20 client.15690 trim_cache size 0 max 16384 2024-03-12T05:13:01.309 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:13:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.306+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4300a1cb0 msgr2=0x7ff4300a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4300a1cb0 0x7ff4300a4170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff4400058c0 tx=0x7ff440005850 comp rx=0 tx=0).stop 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 msgr2=0x7ff444197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 0x7ff444197ab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff43c004af0 tx=0x7ff43c034cc0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 shutdown_connections 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4300a1cb0 0x7ff4300a4170 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff444197600 0x7ff444197ab0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4441064c0 0x7ff44419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 --2- 172.21.15.100:0/3856919039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4440685f0 0x7ff44419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 >> 172.21.15.100:0/3856919039 conn(0x7ff444077180 msgr2=0x7ff4440fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 shutdown_connections 2024-03-12T05:13:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.307+0000 7ff44b484640 1 -- 172.21.15.100:0/3856919039 wait complete. 2024-03-12T05:13:01.314 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:13:01.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:13:01.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 -- 172.21.15.100:0/4075179160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4104ef0 msgr2=0x7f06f41052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 --2- 172.21.15.100:0/4075179160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4104ef0 0x7f06f41052d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009930 tx=0x7f06e402bc30 comp rx=0 tx=0).stop 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 -- 172.21.15.100:0/4075179160 shutdown_connections 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 --2- 172.21.15.100:0/4075179160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f4101420 0x7f06f410cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 --2- 172.21.15.100:0/4075179160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4100a60 0x7f06f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 --2- 172.21.15.100:0/4075179160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4104ef0 0x7f06f41052d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 -- 172.21.15.100:0/4075179160 >> 172.21.15.100:0/4075179160 conn(0x7f06f40752a0 msgr2=0x7f06f40756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 -- 172.21.15.100:0/4075179160 shutdown_connections 2024-03-12T05:13:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.426+0000 7f06f359e640 1 -- 172.21.15.100:0/4075179160 wait complete. 2024-03-12T05:13:01.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.427+0000 7f06f359e640 1 Processor -- start 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.427+0000 7f06f359e640 1 -- start start 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f359e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4100a60 0x7f06f4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 0x7f06f4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f4104ef0 0x7f06f419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f359e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e4d0 con 0x7f06f4101420 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f359e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e640 con 0x7f06f4100a60 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f359e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e800 con 0x7f06f4104ef0 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 0x7f06f4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 0x7f06f4199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42510/0 (socket says 172.21.15.100:42510) 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 -- 172.21.15.100:0/272775523 learned_addr learned my addr 172.21.15.100:0/272775523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f259c640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4100a60 0x7f06f4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 -- 172.21.15.100:0/272775523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f4104ef0 msgr2=0x7f06f419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f4104ef0 0x7f06f419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 -- 172.21.15.100:0/272775523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4100a60 msgr2=0x7f06f4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4100a60 0x7f06f4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 -- 172.21.15.100:0/272775523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e4009560 con 0x7f06f4101420 2024-03-12T05:13:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 0x7f06f4199ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f06e800e9f0 tx=0x7f06e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.428+0000 7f06e37fe640 1 -- 172.21.15.100:0/272775523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8009700 con 0x7f06f4101420 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.429+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f419eae0 con 0x7f06f4101420 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.429+0000 7f06e37fe640 1 -- 172.21.15.100:0/272775523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e800cd80 con 0x7f06f4101420 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.429+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f41a6360 con 0x7f06f4101420 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.429+0000 7f06e37fe640 1 -- 172.21.15.100:0/272775523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8018920 con 0x7f06f4101420 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.431+0000 7f06e37fe640 1 -- 172.21.15.100:0/272775523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f06e8018a80 con 0x7f06f4101420 2024-03-12T05:13:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.431+0000 7f06e37fe640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06dc0a1c20 0x7f06dc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06e37fe640 1 -- 172.21.15.100:0/272775523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f06e8014070 con 0x7f06f4101420 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06f259c640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06dc0a1c20 0x7f06dc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 Processor -- start 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 -- start start 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7440 0x557c057a7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057b0940 0x557c057b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c05762960 con 0x557c057a7440 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c05716310 con 0x557c057b0940 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.432+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c05713840 con 0x557c057a7d60 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f259c640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06dc0a1c20 0x7f06dc0a40e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009900 tx=0x7f06e4000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37558/0 (socket says 172.21.15.100:37558) 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 -- 172.21.15.100:0/429515674 learned_addr learned my addr 172.21.15.100:0/429515674 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f259c640 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7440 0x557c057a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057b0940 0x557c057b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 -- 172.21.15.100:0/429515674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057b0940 msgr2=0x557c057b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057b0940 0x557c057b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 -- 172.21.15.100:0/429515674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7440 msgr2=0x557c057a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7440 0x557c057a7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 -- 172.21.15.100:0/429515674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c056d1060 con 0x557c057a7d60 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b0400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000c00 tx=0x7f06e809b060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a8229f36551db1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06e0ff9640 1 -- 172.21.15.100:0/429515674 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80a3100 con 0x557c057a7d60 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.433+0000 7f06e0ff9640 1 -- 172.21.15.100:0/429515674 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e8010b80 con 0x557c057a7d60 2024-03-12T05:13:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.434+0000 7f06e0ff9640 1 -- 172.21.15.100:0/429515674 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e809a010 con 0x557c057a7d60 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.434+0000 7f06fb3d3740 1 -- 172.21.15.100:0/429515674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 msgr2=0x557c057b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.434+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b0400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000c00 tx=0x7f06e809b060 comp rx=0 tx=0).stop 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 -- 172.21.15.100:0/429515674 shutdown_connections 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057b0940 0x557c057b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b0400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/429515674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7440 0x557c057a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 -- 172.21.15.100:0/429515674 >> 172.21.15.100:0/429515674 conn(0x557c057a1b70 msgr2=0x557c057a1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 -- 172.21.15.100:0/429515674 shutdown_connections 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 -- 172.21.15.100:0/429515674 wait complete. 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 -- start start 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 Processor -- start 2024-03-12T05:13:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7440 0x557c057ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.435+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7d60 0x557c057adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0570f020 con 0x557c057b0940 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c053e2430 con 0x557c057a7d60 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0539ee00 con 0x557c057a7440 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42518/0 (socket says 172.21.15.100:42518) 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 -- 172.21.15.100:0/1518174150 learned_addr learned my addr 172.21.15.100:0/1518174150 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06fb3d3740 10 client.0 _ll_register_callbacks cb 0x557c056b6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7d60 0x557c057adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 -- 172.21.15.100:0/1518174150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7440 msgr2=0x557c057ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f259c640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7440 0x557c057ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7440 0x557c057ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 -- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7d60 msgr2=0x557c057adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7d60 0x557c057adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 -- 172.21.15.100:0/1518174150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f41a6360 con 0x557c057b0940 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7d60 0x557c057adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f259c640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7440 0x557c057ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057ae510 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011970 tx=0x7f06ec011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faeb74c904fd299 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.436+0000 7f06d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06d9ffb640 1 -- 172.21.15.100:0/1518174150 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec003e40 con 0x557c057b0940 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06d9ffb640 1 -- 172.21.15.100:0/1518174150 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f06ec004b20 con 0x557c057b0940 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c0540e7e0 con 0x557c057b0940 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06d9ffb640 1 -- 172.21.15.100:0/1518174150 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06ec004ce0 con 0x557c057b0940 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06fb3d3740 10 client.15702 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557c0570d6a0 con 0x557c057b0940 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06d9ffb640 1 -- 172.21.15.100:0/1518174150 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557c0540e7e0 con 0x557c057b0940 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06d9ffb640 1 -- 172.21.15.100:0/1518174150 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06ec019430 con 0x557c057b0940 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06fb3d3740 10 client.15702 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.437+0000 7f06fb3d3740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.438+0000 7f06fb3d3740 20 client.15702 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.438+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c057c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.438+0000 7f06fb3d3740 4 client.15702 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.438+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c0544b720 con 0x557c057c0ce0 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.438+0000 7f06f259c640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c057c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06d9ffb640 10 client.15702 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06f259c640 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c057c3100 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf5135db1c33127 server_cookie=144fcdc0355b51a2 in_seq=0 out_seq=0 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06d9ffb640 1 -- 172.21.15.100:0/1518174150 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c0544b720 con 0x557c057c0ce0 2024-03-12T05:13:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06d9ffb640 10 client.15702 handle_command_reply: tid=0 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06fb3d3740 1 client.15702 shutdown 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 msgr2=0x557c057ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.439+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057ae510 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011970 tx=0x7f06ec011e30 comp rx=0 tx=0).stop 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 shutdown_connections 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c057c3100 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06d9ffb640 0 client.15702 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057ae510 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7d60 0x557c057adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/1518174150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7440 0x557c057ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 >> 172.21.15.100:0/1518174150 conn(0x557c057a1b70 msgr2=0x557c057a38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 shutdown_connections 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 1 -- 172.21.15.100:0/1518174150 wait complete. 2024-03-12T05:13:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.440+0000 7f06fb3d3740 20 client.15702 trim_cache size 0 max 16384 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.451+0000 7f06fb3d3740 1 Processor -- start 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.451+0000 7f06fb3d3740 1 -- start start 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 0x557c057ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057b0940 0x557c057aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c05694130 con 0x557c057a7d60 2024-03-12T05:13:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c057b07c0 con 0x557c057a7440 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c057a70a0 con 0x557c057b0940 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 0x557c057ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 0x557c057ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42530/0 (socket says 172.21.15.100:42530) 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 -- 172.21.15.100:0/2534988898 learned_addr learned my addr 172.21.15.100:0/2534988898 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f259c640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057b0940 0x557c057aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 -- 172.21.15.100:0/2534988898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057b0940 msgr2=0x557c057aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057b0940 0x557c057aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 -- 172.21.15.100:0/2534988898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 msgr2=0x557c057b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f1d9b640 1 -- 172.21.15.100:0/2534988898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c056a9c20 con 0x557c057a7d60 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057b0940 0x557c057aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.452+0000 7f06f259c640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.453+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 0x557c057ae350 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f06e809a660 tx=0x7f06e8010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1374733e36ca1f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.453+0000 7f06dbfff640 1 -- 172.21.15.100:0/2534988898 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80cb470 con 0x557c057a7d60 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.453+0000 7f06dbfff640 1 -- 172.21.15.100:0/2534988898 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e80a3100 con 0x557c057a7d60 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.453+0000 7f06dbfff640 1 -- 172.21.15.100:0/2534988898 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8099ff0 con 0x557c057a7d60 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.453+0000 7f06fb3d3740 1 -- 172.21.15.100:0/2534988898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 msgr2=0x557c057ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.453+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 0x557c057ae350 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f06e809a660 tx=0x7f06e8010f50 comp rx=0 tx=0).stop 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- 172.21.15.100:0/2534988898 shutdown_connections 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057b0940 0x557c057aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057a7d60 0x557c057ae350 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/2534988898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- 172.21.15.100:0/2534988898 >> 172.21.15.100:0/2534988898 conn(0x557c057a1b70 msgr2=0x557c057c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- 172.21.15.100:0/2534988898 shutdown_connections 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- 172.21.15.100:0/2534988898 wait complete. 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- start start 2024-03-12T05:13:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 Processor -- start 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c057b6770 con 0x557c057b0940 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c057b68e0 con 0x557c057a7440 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c057b6a50 con 0x557c057a7d60 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06fb3d3740 10 client.0 _ll_register_callbacks cb 0x7f06e80cb3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06f2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.454+0000 7f06f2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42536/0 (socket says 172.21.15.100:42536) 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 -- 172.21.15.100:0/3270427716 learned_addr learned my addr 172.21.15.100:0/3270427716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f259c640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f1d9b640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 -- 172.21.15.100:0/3270427716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 msgr2=0x557c057b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 -- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 msgr2=0x557c057b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 -- 172.21.15.100:0/3270427716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c056d1060 con 0x557c057b0940 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f259c640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06f2d9d640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057b6230 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000f80 tx=0x7f06ec009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573891d328fc0a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06d9ffb640 1 -- 172.21.15.100:0/3270427716 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec011020 con 0x557c057b0940 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06d9ffb640 1 -- 172.21.15.100:0/3270427716 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f06ec011180 con 0x557c057b0940 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06e8010b80 con 0x557c057b0940 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06d9ffb640 1 -- 172.21.15.100:0/3270427716 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06ec00f960 con 0x557c057b0940 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06fb3d3740 10 client.15708 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.455+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557c056cf8c0 con 0x557c057b0940 2024-03-12T05:13:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06d9ffb640 1 -- 172.21.15.100:0/3270427716 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06ec02c450 con 0x557c057b0940 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06d9ffb640 1 -- 172.21.15.100:0/3270427716 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06ec02a070 con 0x557c057b0940 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06fb3d3740 10 client.15708 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06fb3d3740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06fb3d3740 20 client.15708 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c0581a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06fb3d3740 4 client.15708 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f06e8010b80 con 0x557c057c0ce0 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.456+0000 7f06f259c640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c0581a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06d9ffb640 10 client.15708 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06f259c640 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c0581a700 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe6dd7314113de2 server_cookie=1a33146eb639119d in_seq=0 out_seq=0 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06d9ffb640 1 -- 172.21.15.100:0/3270427716 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f06e8010b80 con 0x557c057c0ce0 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06d9ffb640 10 client.15708 handle_command_reply: tid=0 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 client.15708 shutdown 2024-03-12T05:13:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 msgr2=0x557c057b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057b6230 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000f80 tx=0x7f06ec009610 comp rx=0 tx=0).stop 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 shutdown_connections 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557c057c0ce0 0x557c0581a700 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c057b0940 0x557c057b6230 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c057a7d60 0x557c057b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06d9ffb640 0 client.15708 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 --2- 172.21.15.100:0/3270427716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c057a7440 0x557c057b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 >> 172.21.15.100:0/3270427716 conn(0x557c057a1b70 msgr2=0x557c057a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 shutdown_connections 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 1 -- 172.21.15.100:0/3270427716 wait complete. 2024-03-12T05:13:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.457+0000 7f06fb3d3740 20 client.15708 trim_cache size 0 max 16384 2024-03-12T05:13:01.463 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06dc0a1c20 msgr2=0x7f06dc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06dc0a1c20 0x7f06dc0a40e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009900 tx=0x7f06e4000bc0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 msgr2=0x7f06f4199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 0x7f06f4199ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f06e800e9f0 tx=0x7f06e800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 shutdown_connections 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06dc0a1c20 0x7f06dc0a40e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06f4104ef0 0x7f06f419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4101420 0x7f06f4199ba0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.458+0000 7f06f359e640 1 --2- 172.21.15.100:0/272775523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4100a60 0x7f06f4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.459+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 >> 172.21.15.100:0/272775523 conn(0x7f06f40752a0 msgr2=0x7f06f40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.459+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 shutdown_connections 2024-03-12T05:13:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:01.459+0000 7f06f359e640 1 -- 172.21.15.100:0/272775523 wait complete. 2024-03-12T05:13:02.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:01 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/278549177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:13:02.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:01 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/278549177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:13:02.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:01 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/278549177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:13:02.470 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:13:02.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:13:02.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.590+0000 7fef09b1e640 1 -- 172.21.15.100:0/222412937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 msgr2=0x7fef0410d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.590+0000 7fef09b1e640 1 --2- 172.21.15.100:0/222412937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0410d5e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feef8009900 tx=0x7feef802bc40 comp rx=0 tx=0).stop 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 -- 172.21.15.100:0/222412937 shutdown_connections 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 --2- 172.21.15.100:0/222412937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef0410db20 0x7fef0410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 --2- 172.21.15.100:0/222412937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0410d5e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 --2- 172.21.15.100:0/222412937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef04068360 0x7fef04068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 -- 172.21.15.100:0/222412937 >> 172.21.15.100:0/222412937 conn(0x7fef04077180 msgr2=0x7fef04077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 -- 172.21.15.100:0/222412937 shutdown_connections 2024-03-12T05:13:02.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 -- 172.21.15.100:0/222412937 wait complete. 2024-03-12T05:13:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.591+0000 7fef09b1e640 1 Processor -- start 2024-03-12T05:13:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 -- start start 2024-03-12T05:13:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef04068360 0x7fef0419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef0410db20 0x7fef041996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef04199e70 con 0x7fef04068c80 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef04199fe0 con 0x7fef0410db20 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef09b1e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0419a1a0 con 0x7fef04068360 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42548/0 (socket says 172.21.15.100:42548) 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef02ffd640 1 -- 172.21.15.100:0/2885690000 learned_addr learned my addr 172.21.15.100:0/2885690000 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef037fe640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef04068360 0x7fef0419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.592+0000 7fef03fff640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef0410db20 0x7fef041996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef02ffd640 1 -- 172.21.15.100:0/2885690000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef04068360 msgr2=0x7fef0419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef04068360 0x7fef0419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef02ffd640 1 -- 172.21.15.100:0/2885690000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef0410db20 msgr2=0x7fef041996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef0410db20 0x7fef041996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef02ffd640 1 -- 172.21.15.100:0/2885690000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8009560 con 0x7fef04068c80 2024-03-12T05:13:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0419fb60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feef8005940 tx=0x7feef80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef00ff9640 1 -- 172.21.15.100:0/2885690000 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80044e0 con 0x7fef04068c80 2024-03-12T05:13:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef00ff9640 1 -- 172.21.15.100:0/2885690000 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feef802cbd0 con 0x7fef04068c80 2024-03-12T05:13:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef00ff9640 1 -- 172.21.15.100:0/2885690000 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8003d10 con 0x7fef04068c80 2024-03-12T05:13:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0419a420 con 0x7fef04068c80 2024-03-12T05:13:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.593+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef041a65a0 con 0x7fef04068c80 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.596+0000 7fef0c7cd740 1 Processor -- start 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.596+0000 7fef0c7cd740 1 -- start start 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd8b90 0x55ba8edd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8ede1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8ede20c0 0x55ba8ede4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ed3ef90 con 0x55ba8edd9540 2024-03-12T05:13:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ed3c4c0 con 0x55ba8edd8b90 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ed37ca0 con 0x55ba8ede20c0 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8ede1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8ede1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42552/0 (socket says 172.21.15.100:42552) 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 -- 172.21.15.100:0/2375770645 learned_addr learned my addr 172.21.15.100:0/2375770645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef00ff9640 1 -- 172.21.15.100:0/2885690000 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7feef8005000 con 0x7fef04068c80 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef03fff640 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd8b90 0x55ba8edd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 -- 172.21.15.100:0/2375770645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8ede20c0 msgr2=0x55ba8ede4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8ede20c0 0x55ba8ede4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8ede20c0 0x55ba8ede4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 -- 172.21.15.100:0/2375770645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd8b90 msgr2=0x55ba8edd8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd8b90 0x55ba8edd8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef037fe640 1 -- 172.21.15.100:0/2375770645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8ecfa0b0 con 0x55ba8edd9540 2024-03-12T05:13:02.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.597+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8ede20c0 0x55ba8ede4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7fef037fe640 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8ede1af0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feef400ad00 tx=0x7feef400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc5d7df18ab9b8d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7feee9ffb640 1 -- 172.21.15.100:0/2375770645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400a800 con 0x55ba8edd9540 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7fef00ff9640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feeec0a1d30 0x7feeec0a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7feee9ffb640 1 -- 172.21.15.100:0/2375770645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feef4018b20 con 0x55ba8edd9540 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7feee9ffb640 1 -- 172.21.15.100:0/2375770645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4010d80 con 0x55ba8edd9540 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7fef00ff9640 1 -- 172.21.15.100:0/2885690000 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feef80efb70 con 0x7fef04068c80 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.598+0000 7fef03fff640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feeec0a1d30 0x7feeec0a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2375770645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 msgr2=0x55ba8ede1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8ede1af0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feef400ad00 tx=0x7feef400f790 comp rx=0 tx=0).stop 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef03fff640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feeec0a1d30 0x7feeec0a41f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef0406a6b0 tx=0x7feefc007520 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2375770645 shutdown_connections 2024-03-12T05:13:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8ede20c0 0x55ba8ede4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8ede1af0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2375770645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd8b90 0x55ba8edd8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2375770645 >> 172.21.15.100:0/2375770645 conn(0x55ba8edd3050 msgr2=0x55ba8edd3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2375770645 shutdown_connections 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2375770645 wait complete. 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 -- start start 2024-03-12T05:13:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.599+0000 7fef0c7cd740 1 Processor -- start 2024-03-12T05:13:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 0x55ba8eddf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ede20c0 0x55ba8eddfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ecdcf40 con 0x55ba8edd8b90 2024-03-12T05:13:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ea0b4b0 con 0x55ba8ede20c0 2024-03-12T05:13:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ecbd670 con 0x55ba8edd9540 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef0c7cd740 10 client.0 _ll_register_callbacks cb 0x55ba8ed2af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 0x55ba8eddf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 0x55ba8eddf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37632/0 (socket says 172.21.15.100:37632) 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef02ffd640 1 -- 172.21.15.100:0/880277046 learned_addr learned my addr 172.21.15.100:0/880277046 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42554/0 (socket says 172.21.15.100:42554) 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef03fff640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ede20c0 0x55ba8eddfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 -- 172.21.15.100:0/880277046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 msgr2=0x55ba8eddf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 0x55ba8eddf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 -- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ede20c0 msgr2=0x55ba8eddfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ede20c0 0x55ba8eddfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef037fe640 1 -- 172.21.15.100:0/880277046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef400a960 con 0x55ba8edd8b90 2024-03-12T05:13:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef02ffd640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 0x55ba8eddf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.600+0000 7fef03fff640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ede20c0 0x55ba8eddfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7fef037fe640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55ba8edd42d0 tx=0x7feef4005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac8a09c682edfdd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7feee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7feee2ffd640 1 -- 172.21.15.100:0/880277046 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40048c0 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba8ed11f40 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7feee2ffd640 1 -- 172.21.15.100:0/880277046 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feef4011e10 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7feee2ffd640 1 -- 172.21.15.100:0/880277046 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef400a7e0 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7fef0c7cd740 10 client.15720 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ba8ed36320 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.601+0000 7feee2ffd640 1 -- 172.21.15.100:0/880277046 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ba8ed11f40 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7feee2ffd640 1 -- 172.21.15.100:0/880277046 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feef4018900 con 0x55ba8edd8b90 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef0c7cd740 10 client.15720 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef0c7cd740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef0c7cd740 20 client.15720 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8edf2470 0x55ba8edf4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef0c7cd740 4 client.15720 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba8ed2e310 con 0x55ba8edf2470 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.602+0000 7fef02ffd640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8edf2470 0x55ba8edf4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.603+0000 7fef02ffd640 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8edf2470 0x55ba8edf4860 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebaf3d0ccd037a7 server_cookie=4687a43d4c41ff93 in_seq=0 out_seq=0 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.603+0000 7feee2ffd640 10 client.15720 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.603+0000 7feee2ffd640 1 -- 172.21.15.100:0/880277046 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba8ed2e310 con 0x55ba8edf2470 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7feee2ffd640 10 client.15720 handle_command_reply: tid=0 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 client.15720 shutdown 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 msgr2=0x55ba8eddf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55ba8edd42d0 tx=0x7feef4005530 comp rx=0 tx=0).stop 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 shutdown_connections 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8edf2470 0x55ba8edf4860 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ede20c0 0x55ba8eddfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd9540 0x55ba8eddf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7feee2ffd640 0 client.15720 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/880277046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf050 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 >> 172.21.15.100:0/880277046 conn(0x55ba8edd3050 msgr2=0x55ba8edd4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 shutdown_connections 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 1 -- 172.21.15.100:0/880277046 wait complete. 2024-03-12T05:13:02.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.604+0000 7fef0c7cd740 20 client.15720 trim_cache size 0 max 16384 2024-03-12T05:13:02.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 Processor -- start 2024-03-12T05:13:02.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 -- start start 2024-03-12T05:13:02.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd3050 0x55ba8edde1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8edc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ede4bd0 con 0x55ba8edd9540 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8edde6f0 con 0x55ba8edd3050 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8edf2220 con 0x55ba8edd8b90 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8edc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8edc9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42562/0 (socket says 172.21.15.100:42562) 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef03fff640 1 -- 172.21.15.100:0/3595928805 learned_addr learned my addr 172.21.15.100:0/3595928805 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef037fe640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd3050 0x55ba8edde1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef02ffd640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef03fff640 1 -- 172.21.15.100:0/3595928805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 msgr2=0x55ba8eddf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.633+0000 7fef03fff640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7fef03fff640 1 -- 172.21.15.100:0/3595928805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd3050 msgr2=0x55ba8edde1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7fef03fff640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd3050 0x55ba8edde1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7fef03fff640 1 -- 172.21.15.100:0/3595928805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8ecd2c00 con 0x55ba8edd9540 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7fef037fe640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd3050 0x55ba8edde1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7fef02ffd640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7fef03fff640 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8edc9760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feefc01e700 tx=0x7feefc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92ea2eb24dd09d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7feee97fa640 1 -- 172.21.15.100:0/3595928805 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc023070 con 0x55ba8edd9540 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7feee97fa640 1 -- 172.21.15.100:0/3595928805 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feefc003e50 con 0x55ba8edd9540 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.634+0000 7feee97fa640 1 -- 172.21.15.100:0/3595928805 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc015070 con 0x55ba8edd9540 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 -- 172.21.15.100:0/3595928805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 msgr2=0x55ba8edc9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8edc9760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feefc01e700 tx=0x7feefc015000 comp rx=0 tx=0).stop 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 -- 172.21.15.100:0/3595928805 shutdown_connections 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd9540 0x55ba8edc9760 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8eddf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/3595928805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd3050 0x55ba8edde1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 -- 172.21.15.100:0/3595928805 >> 172.21.15.100:0/3595928805 conn(0x55ba8edd9c90 msgr2=0x55ba8edc9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 -- 172.21.15.100:0/3595928805 shutdown_connections 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.635+0000 7fef0c7cd740 1 -- 172.21.15.100:0/3595928805 wait complete. 2024-03-12T05:13:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 -- start start 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 Processor -- start 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 0x55ba8ee42e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8ee433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd9540 0x55ba8ede84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ede8b90 con 0x55ba8edd3050 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ede8d00 con 0x55ba8edd9540 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.636+0000 7fef037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 0x55ba8ee42e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef0c7cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ede8ec0 con 0x55ba8edd8b90 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 0x55ba8ee42e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42572/0 (socket says 172.21.15.100:42572) 2024-03-12T05:13:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 -- 172.21.15.100:0/2582865889 learned_addr learned my addr 172.21.15.100:0/2582865889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8ee433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef0c7cd740 10 client.0 _ll_register_callbacks cb 0x7feef400a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef03fff640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd9540 0x55ba8ede84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 -- 172.21.15.100:0/2582865889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 msgr2=0x55ba8ee433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8ee433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 -- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd9540 msgr2=0x55ba8ede84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd9540 0x55ba8ede84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 -- 172.21.15.100:0/2582865889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba8ed36320 con 0x55ba8edd3050 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef037fe640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 0x55ba8ee42e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feef4018c90 tx=0x7feef4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d378ac05e1109f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7feee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7feee2ffd640 1 -- 172.21.15.100:0/2582865889 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400fd40 con 0x55ba8edd3050 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7feee2ffd640 1 -- 172.21.15.100:0/2582865889 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feef4000e40 con 0x55ba8edd3050 2024-03-12T05:13:02.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.637+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba8ed5d900 con 0x55ba8edd3050 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7feee2ffd640 1 -- 172.21.15.100:0/2582865889 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef400a7e0 con 0x55ba8edd3050 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 10 client.15726 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7feefc01e430 con 0x55ba8edd3050 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7feee2ffd640 1 -- 172.21.15.100:0/2582865889 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feef40055a0 con 0x55ba8edd3050 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7feee2ffd640 1 -- 172.21.15.100:0/2582865889 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feef4004660 con 0x55ba8edd3050 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 10 client.15726 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 20 client.15726 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8ede20c0 0x55ba8ee4c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 4 client.15726 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.638+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ba8ed5d900 con 0x55ba8ede20c0 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.639+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8ede20c0 0x55ba8ee4c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.639+0000 7feee2ffd640 10 client.15726 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.639+0000 7fef02ffd640 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8ede20c0 0x55ba8ee4c8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6546ed1494837b server_cookie=e9a87dba430e4793 in_seq=0 out_seq=0 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.640+0000 7feee2ffd640 1 -- 172.21.15.100:0/2582865889 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ba8ed5d900 con 0x55ba8ede20c0 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.640+0000 7feee2ffd640 10 client.15726 handle_command_reply: tid=0 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.640+0000 7fef0c7cd740 1 client.15726 shutdown 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 msgr2=0x55ba8ee42e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 0x55ba8ee42e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feef4018c90 tx=0x7feef4011da0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 shutdown_connections 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ba8ede20c0 0x55ba8ee4c8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7feee2ffd640 0 client.15726 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8edd9540 0x55ba8ede84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba8edd8b90 0x55ba8ee433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 --2- 172.21.15.100:0/2582865889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8edd3050 0x55ba8ee42e60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 >> 172.21.15.100:0/2582865889 conn(0x55ba8edd9c90 msgr2=0x55ba8edc98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 shutdown_connections 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 1 -- 172.21.15.100:0/2582865889 wait complete. 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.641+0000 7fef0c7cd740 20 client.15726 trim_cache size 0 max 16384 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "d42a319d-60c3-4b82-ab65-b7eefa71595d", 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:13:02.643 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:13:02.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feeec0a1d30 msgr2=0x7feeec0a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feeec0a1d30 0x7feeec0a41f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef0406a6b0 tx=0x7feefc007520 comp rx=0 tx=0).stop 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 msgr2=0x7fef0419fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0419fb60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feef8005940 tx=0x7feef80042f0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 shutdown_connections 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feeec0a1d30 0x7feeec0a41f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef0410db20 0x7fef041996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef04068c80 0x7fef0419fb60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 --2- 172.21.15.100:0/2885690000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef04068360 0x7fef0419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.642+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 >> 172.21.15.100:0/2885690000 conn(0x7fef04077180 msgr2=0x7fef0410ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.643+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 shutdown_connections 2024-03-12T05:13:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.643+0000 7fef09b1e640 1 -- 172.21.15.100:0/2885690000 wait complete. 2024-03-12T05:13:02.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:13:02.651 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.778+0000 7fcb6c713640 1 -- 172.21.15.100:0/2702813422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb641023a0 msgr2=0x7fcb64102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.778+0000 7fcb6c713640 1 --2- 172.21.15.100:0/2702813422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb641023a0 0x7fcb64102780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009860 tx=0x7fcb6002bc00 comp rx=0 tx=0).stop 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 -- 172.21.15.100:0/2702813422 shutdown_connections 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 --2- 172.21.15.100:0/2702813422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6410b6e0 0x7fcb6410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 --2- 172.21.15.100:0/2702813422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb64102cc0 0x7fcb6410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 --2- 172.21.15.100:0/2702813422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb641023a0 0x7fcb64102780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 -- 172.21.15.100:0/2702813422 >> 172.21.15.100:0/2702813422 conn(0x7fcb64077230 msgr2=0x7fcb64077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 -- 172.21.15.100:0/2702813422 shutdown_connections 2024-03-12T05:13:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 -- 172.21.15.100:0/2702813422 wait complete. 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 Processor -- start 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.779+0000 7fcb6c713640 1 -- start start 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6c713640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb641023a0 0x7fcb6419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6c713640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 0x7fcb6419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6c713640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb6410b6e0 0x7fcb64197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6c713640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64197c70 con 0x7fcb64102cc0 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6c713640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64197de0 con 0x7fcb6410b6e0 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6c713640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64197fa0 con 0x7fcb641023a0 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 0x7fcb6419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 0x7fcb6419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42586/0 (socket says 172.21.15.100:42586) 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 -- 172.21.15.100:0/3309162834 learned_addr learned my addr 172.21.15.100:0/3309162834 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb641023a0 0x7fcb6419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb6410b6e0 0x7fcb64197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 -- 172.21.15.100:0/3309162834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb641023a0 msgr2=0x7fcb6419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb641023a0 0x7fcb6419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 -- 172.21.15.100:0/3309162834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb6410b6e0 msgr2=0x7fcb64197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb6410b6e0 0x7fcb64197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.780+0000 7fcb69cae640 1 -- 172.21.15.100:0/3309162834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009900 con 0x7fcb64102cc0 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.781+0000 7fcb69cae640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 0x7fcb6419d9b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d770 tx=0x7fcb5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.781+0000 7fcb5b7fe640 1 -- 172.21.15.100:0/3309162834 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54009c30 con 0x7fcb64102cc0 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.781+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb60009490 con 0x7fcb64102cc0 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.781+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb641a4330 con 0x7fcb64102cc0 2024-03-12T05:13:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.781+0000 7fcb5b7fe640 1 -- 172.21.15.100:0/3309162834 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb5400de80 con 0x7fcb64102cc0 2024-03-12T05:13:02.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.781+0000 7fcb5b7fe640 1 -- 172.21.15.100:0/3309162834 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54004e80 con 0x7fcb64102cc0 2024-03-12T05:13:02.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.783+0000 7fcb5b7fe640 1 -- 172.21.15.100:0/3309162834 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7fcb54005020 con 0x7fcb64102cc0 2024-03-12T05:13:02.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.784+0000 7fcb5b7fe640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcb500a1c30 0x7fcb500a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.784+0000 7fcb5b7fe640 1 -- 172.21.15.100:0/3309162834 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcb540cd360 con 0x7fcb64102cc0 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.784+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcb500a1c30 0x7fcb500a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.784+0000 7fcb6f3ca740 1 Processor -- start 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 -- start start 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8480720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 0x55b4a847ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8487020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a83ea280 con 0x55b4a8480340 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a83e77b0 con 0x55b4a847a8e0 2024-03-12T05:13:02.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a83e2f90 con 0x55b4a847b2a0 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcb500a1c30 0x7fcb500a40f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb6002c210 tx=0x7fcb600058d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 0x55b4a847ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 0x55b4a847ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38716/0 (socket says 172.21.15.100:38716) 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 -- 172.21.15.100:0/295961995 learned_addr learned my addr 172.21.15.100:0/295961995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8487020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8480720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 -- 172.21.15.100:0/295961995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 msgr2=0x55b4a8487020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8487020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 -- 172.21.15.100:0/295961995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 msgr2=0x55b4a8480720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8480720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.785+0000 7fcb69cae640 1 -- 172.21.15.100:0/295961995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4a83a1570 con 0x55b4a847a8e0 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.786+0000 7fcb69cae640 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 0x55b4a847ad60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d740 tx=0x7fcb54099b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae00525e7644c34 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.786+0000 7fcb58ff9640 1 -- 172.21.15.100:0/295961995 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540981c0 con 0x55b4a847a8e0 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.786+0000 7fcb58ff9640 1 -- 172.21.15.100:0/295961995 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb5409c070 con 0x55b4a847a8e0 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.786+0000 7fcb58ff9640 1 -- 172.21.15.100:0/295961995 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54099570 con 0x55b4a847a8e0 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/295961995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 msgr2=0x55b4a847ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 0x55b4a847ad60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d740 tx=0x7fcb54099b10 comp rx=0 tx=0).stop 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/295961995 shutdown_connections 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8487020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847a8e0 0x55b4a847ad60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/295961995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8480720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/295961995 >> 172.21.15.100:0/295961995 conn(0x55b4a8475bf0 msgr2=0x55b4a8475fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/295961995 shutdown_connections 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/295961995 wait complete. 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- start start 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 Processor -- start 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8486f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847b2a0 0x55b4a84842e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a84848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a80b6430 con 0x55b4a847b2a0 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a8072e00 con 0x55b4a8480340 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a8367fd0 con 0x55b4a847a8e0 2024-03-12T05:13:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.787+0000 7fcb6f3ca740 10 client.0 _ll_register_callbacks cb 0x55b4a83bdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a84848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a84848f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38730/0 (socket says 172.21.15.100:38730) 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 -- 172.21.15.100:0/2421227410 learned_addr learned my addr 172.21.15.100:0/2421227410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb69cae640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847b2a0 0x55b4a84842e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8486f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 -- 172.21.15.100:0/2421227410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 msgr2=0x55b4a8486f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8486f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 -- 172.21.15.100:0/2421227410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847b2a0 msgr2=0x55b4a84842e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847b2a0 0x55b4a84842e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 -- 172.21.15.100:0/2421227410 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb641a4330 con 0x55b4a8480340 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb69cae640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847b2a0 0x55b4a84842e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a84848f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00eec0 tx=0x7fcb5c00d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bc1c6b0ceb070b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2421227410 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c013040 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2421227410 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcb5c003b70 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.788+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4a80e27e0 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2421227410 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5c003d30 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 10 client.25118 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b4a83e1610 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2421227410 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb5c003e90 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2421227410 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb5c01f060 con 0x55b4a8480340 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 10 client.25118 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 10 client.25118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 20 client.25118 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a8496e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 4 client.25118 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.789+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4a80e27e0 con 0x55b4a8494a10 2024-03-12T05:13:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.790+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a8496e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.790+0000 7fcb4dffb640 10 client.25118 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.790+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a8496e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c076569c30aa82c server_cookie=68c16a0bc12e3d85 in_seq=0 out_seq=0 2024-03-12T05:13:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2421227410 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4a80e27e0 con 0x55b4a8494a10 2024-03-12T05:13:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb4dffb640 10 client.25118 handle_command_reply: tid=0 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 client.25118 shutdown 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 msgr2=0x55b4a84848f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a84848f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00eec0 tx=0x7fcb5c00d7b0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 shutdown_connections 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a8496e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a84848f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb4dffb640 0 client.25118 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847b2a0 0x55b4a84842e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2421227410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8486f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 >> 172.21.15.100:0/2421227410 conn(0x55b4a8475bf0 msgr2=0x55b4a847a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 shutdown_connections 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2421227410 wait complete. 2024-03-12T05:13:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.791+0000 7fcb6f3ca740 20 client.25118 trim_cache size 0 max 16384 2024-03-12T05:13:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.820+0000 7fcb6f3ca740 1 Processor -- start 2024-03-12T05:13:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.820+0000 7fcb6f3ca740 1 -- start start 2024-03-12T05:13:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8496e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847b2a0 0x55b4a8486a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8486f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a847b880 con 0x55b4a8480340 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a8482830 con 0x55b4a847b2a0 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6f3ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a846fe20 con 0x55b4a847a8e0 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6acb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8486f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6acb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8486f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42612/0 (socket says 172.21.15.100:42612) 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6acb0640 1 -- 172.21.15.100:0/323863436 learned_addr learned my addr 172.21.15.100:0/323863436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb69cae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847b2a0 0x55b4a8486a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8496e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb69cae640 1 -- 172.21.15.100:0/323863436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 msgr2=0x55b4a8496e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb69cae640 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8496e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb69cae640 1 -- 172.21.15.100:0/323863436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 msgr2=0x55b4a8486f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb69cae640 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8486f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb69cae640 1 -- 172.21.15.100:0/323863436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4a837dac0 con 0x55b4a847b2a0 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.821+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8496e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.822+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8486f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.822+0000 7fcb69cae640 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847b2a0 0x55b4a8486a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55b4a847b800 tx=0x7fcb54096290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce607d3907fbc1a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.822+0000 7fcb4ffff640 1 -- 172.21.15.100:0/323863436 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540ca920 con 0x55b4a847b2a0 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.822+0000 7fcb4ffff640 1 -- 172.21.15.100:0/323863436 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb54009d90 con 0x55b4a847b2a0 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.822+0000 7fcb4ffff640 1 -- 172.21.15.100:0/323863436 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540cac90 con 0x55b4a847b2a0 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/323863436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847b2a0 msgr2=0x55b4a8486a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847b2a0 0x55b4a8486a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55b4a847b800 tx=0x7fcb54096290 comp rx=0 tx=0).stop 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/323863436 shutdown_connections 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a8480340 0x55b4a8486f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a847b2a0 0x55b4a8486a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/323863436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847a8e0 0x55b4a8496e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/323863436 >> 172.21.15.100:0/323863436 conn(0x55b4a8475bf0 msgr2=0x55b4a8482bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/323863436 shutdown_connections 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.823+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/323863436 wait complete. 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 -- start start 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 Processor -- start 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 0x55b4a848bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8489f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a848a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6a4af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 0x55b4a848bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6a4af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 0x55b4a848bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42614/0 (socket says 172.21.15.100:42614) 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6a4af640 1 -- 172.21.15.100:0/2374091761 learned_addr learned my addr 172.21.15.100:0/2374091761 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a848ac50 con 0x55b4a847a8e0 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a848adc0 con 0x55b4a8480340 2024-03-12T05:13:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4a84eaf80 con 0x55b4a847b2a0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.824+0000 7fcb69cae640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8489f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6acb0640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a848a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6a4af640 1 -- 172.21.15.100:0/2374091761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 msgr2=0x55b4a8489f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6f3ca740 10 client.0 _ll_register_callbacks cb 0x7fcb540ca870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8489f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6a4af640 1 -- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 msgr2=0x55b4a848a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a848a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6a4af640 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6410e800 con 0x55b4a847a8e0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb69cae640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8489f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb5409c070 con 0x55b4a847a8e0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6a4af640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 0x55b4a848bbb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009860 tx=0x7fcb6002bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52490653278c6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb4dffb640 10 client.15744 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2374091761 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60053040 con 0x55b4a847a8e0 2024-03-12T05:13:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2374091761 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcb600556c0 con 0x55b4a847a8e0 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2374091761 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb600558a0 con 0x55b4a847a8e0 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6f3ca740 10 client.15744 fetch_fsmap learned FSMap version 23 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.825+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x55b4a83e1610 con 0x55b4a847a8e0 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2374091761 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb6002c5a0 con 0x55b4a847a8e0 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb6f3ca740 10 client.15744 fetch_fsmap finished waiting for FSMap version 23 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb6f3ca740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb6f3ca740 20 client.15744 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a84ee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb6f3ca740 4 client.15744 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b4a811f720 con 0x55b4a8494a10 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.826+0000 7fcb69cae640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a84ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb4dffb640 10 client.15744 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb69cae640 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a84ee570 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aaccd1602e7e426 server_cookie=1e5daf6337d4fe10 in_seq=0 out_seq=0 2024-03-12T05:13:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb4dffb640 1 -- 172.21.15.100:0/2374091761 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b4a811f720 con 0x55b4a8494a10 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb4dffb640 10 client.15744 handle_command_reply: tid=0 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 client.15744 shutdown 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 msgr2=0x55b4a848bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 0x55b4a848bbb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009860 tx=0x7fcb6002bc00 comp rx=0 tx=0).stop 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 shutdown_connections 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4a8494a10 0x55b4a84ee570 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4a8480340 0x55b4a848a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a847b2a0 0x55b4a8489f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 --2- 172.21.15.100:0/2374091761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4a847a8e0 0x55b4a848bbb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 >> 172.21.15.100:0/2374091761 conn(0x55b4a8475bf0 msgr2=0x55b4a8483b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb4dffb640 0 client.15744 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 shutdown_connections 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 1 -- 172.21.15.100:0/2374091761 wait complete. 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.827+0000 7fcb6f3ca740 20 client.15744 trim_cache size 0 max 16384 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: "d42a319d-60c3-4b82-ab65-b7eefa71595d": { 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "d42a319d-60c3-4b82-ab65-b7eefa71595d", 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:13:02.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:13:02.832 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:13:02.832 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:13:02.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcb500a1c30 msgr2=0x7fcb500a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcb500a1c30 0x7fcb500a40f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb6002c210 tx=0x7fcb600058d0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 msgr2=0x7fcb6419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 0x7fcb6419d9b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d770 tx=0x7fcb5400dc30 comp rx=0 tx=0).stop 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 shutdown_connections 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcb500a1c30 0x7fcb500a40f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb6410b6e0 0x7fcb64197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb64102cc0 0x7fcb6419d9b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 --2- 172.21.15.100:0/3309162834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb641023a0 0x7fcb6419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 >> 172.21.15.100:0/3309162834 conn(0x7fcb64077230 msgr2=0x7fcb640fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 shutdown_connections 2024-03-12T05:13:02.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:02.828+0000 7fcb6c713640 1 -- 172.21.15.100:0/3309162834 wait complete. 2024-03-12T05:13:02.848 INFO:tasks.cephfs.filesystem:scrub status for tag:d42a319d-60c3-4b82-ab65-b7eefa71595d - {'path': '/', 'tag': 'd42a319d-60c3-4b82-ab65-b7eefa71595d', 'options': 'recursive,force'} 2024-03-12T05:13:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:02 smithi156 ceph-mon[36270]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:13:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:02 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:13:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:02 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:02 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:13:03.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:02 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:03.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:02 smithi161 ceph-mon[36043]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:13:03.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:02 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:13:03.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:02 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:03.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:02 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:13:03.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:02 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:03.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:02 smithi100 ceph-mon[32210]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-12T05:13:03.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:02 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:13:03.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:02 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:03.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:02 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:13:03.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:02 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:04.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:03 smithi156 ceph-mon[36270]: mgrmap e25: x(active, since 57s), standbys: y, z 2024-03-12T05:13:04.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:03 smithi161 ceph-mon[36043]: mgrmap e25: x(active, since 57s), standbys: y, z 2024-03-12T05:13:04.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:03 smithi100 ceph-mon[32210]: mgrmap e25: x(active, since 57s), standbys: y, z 2024-03-12T05:13:05.124 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:04 smithi161 ceph-mon[36043]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-12T05:13:05.124 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:04 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:13:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:04 smithi156 ceph-mon[36270]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-12T05:13:05.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:04 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:13:05.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:04 smithi100 ceph-mon[32210]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-12T05:13:05.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:04 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:13:07.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:06 smithi156 ceph-mon[36270]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:07.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:06 smithi161 ceph-mon[36043]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:07.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:06 smithi100 ceph-mon[32210]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:09.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:08 smithi156 ceph-mon[36270]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T05:13:09.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:08 smithi161 ceph-mon[36043]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T05:13:09.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:08 smithi100 ceph-mon[32210]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T05:13:11.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:10 smithi156 ceph-mon[36270]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:11.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:10 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:13:11.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:10 smithi161 ceph-mon[36043]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:11.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:10 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:13:11.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:10 smithi100 ceph-mon[32210]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:11.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:10 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:13:12.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:11 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:11 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:11 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:13:12.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:11 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:13:12.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:11 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:11 smithi100 ceph-mon[32210]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:12.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:11 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:11 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:11 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:13:12.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:11 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:13:12.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:11 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:11 smithi156 ceph-mon[36270]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:12.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:11 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:11 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:11 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:13:12.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:11 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:13:12.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:11 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:13:12.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:11 smithi161 ceph-mon[36043]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-12T05:13:15.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:14 smithi100 ceph-mon[32210]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T05:13:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:14 smithi156 ceph-mon[36270]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T05:13:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:14 smithi161 ceph-mon[36043]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-12T05:13:15.615 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'b6a4997be2b8bb74411268cc42753f29ae9a78ad'. 2024-03-12T05:13:15.615 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.615 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-03-12T05:13:15.616 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.617 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-12T05:13:15.617 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-03-12T05:13:15.617 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at b6a4997be2b Merge pull request #56060 from rhcs-dashboard/wip-64808-squid 2024-03-12T05:13:15.619 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-12T05:13:15.646 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-12T05:13:15.648 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-12T05:13:15.648 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-12T05:13:16.397 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-12T05:13:16.460 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-12T05:13:16.500 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-12T05:13:16.502 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-12T05:13:16.502 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-12T05:13:16.541 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-12T05:13:16.544 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:13:16.544 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-12T05:13:16.600 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-12T05:13:16.601 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-12T05:13:16.602 DEBUG:teuthology.orchestra.run.smithi100:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-12T05:13:16.661 INFO:tasks.workunit.client.0.smithi100.stderr:+ dbench 1 2024-03-12T05:13:16.672 INFO:tasks.workunit.client.0.smithi100.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-12T05:13:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-03-12T05:13:16.673 INFO:tasks.workunit.client.0.smithi100.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-12T05:13:16.680 INFO:tasks.workunit.client.0.smithi100.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-12T05:13:16.780 INFO:tasks.workunit.client.0.smithi100.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-12T05:13:16.780 INFO:tasks.workunit.client.0.smithi100.stdout:releasing clients 2024-03-12T05:13:17.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:16 smithi100 ceph-mon[32210]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:13:17.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:16 smithi156 ceph-mon[36270]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:13:17.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:16 smithi161 ceph-mon[36043]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-12T05:13:17.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 3788 39.76 MB/sec warmup 1 sec latency 5.584 ms 2024-03-12T05:13:18.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 7180 28.87 MB/sec warmup 2 sec latency 6.205 ms 2024-03-12T05:13:19.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:18 smithi100 ceph-mon[32210]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-12T05:13:19.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:18 smithi156 ceph-mon[36270]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-12T05:13:19.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:18 smithi161 ceph-mon[36043]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-12T05:13:19.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10471 24.56 MB/sec warmup 3 sec latency 12.205 ms 2024-03-12T05:13:20.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10805 19.25 MB/sec warmup 4 sec latency 837.291 ms 2024-03-12T05:13:21.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:20 smithi156 ceph-mon[36270]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-12T05:13:21.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:20 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:13:21.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:20 smithi161 ceph-mon[36043]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-12T05:13:21.224 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:20 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:13:21.320 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:20 smithi100 ceph-mon[32210]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-12T05:13:21.321 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:20 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:13:21.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10805 15.40 MB/sec warmup 5 sec latency 1837.379 ms 2024-03-12T05:13:22.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10805 12.84 MB/sec warmup 6 sec latency 2837.495 ms 2024-03-12T05:13:23.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:22 smithi156 ceph-mon[36270]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-12T05:13:23.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:22 smithi161 ceph-mon[36043]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-12T05:13:23.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:22 smithi100 ceph-mon[32210]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-12T05:13:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 11232 11.19 MB/sec warmup 7 sec latency 3504.107 ms 2024-03-12T05:13:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 14466 12.14 MB/sec warmup 8 sec latency 19.891 ms 2024-03-12T05:13:25.118 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:24 smithi100 ceph-mon[32210]: pgmap v42: 97 pgs: 97 active+clean; 6.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 551 KiB/s wr, 73 op/s 2024-03-12T05:13:25.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:24 smithi156 ceph-mon[36270]: pgmap v42: 97 pgs: 97 active+clean; 6.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 551 KiB/s wr, 73 op/s 2024-03-12T05:13:25.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:24 smithi161 ceph-mon[36043]: pgmap v42: 97 pgs: 97 active+clean; 6.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 551 KiB/s wr, 73 op/s 2024-03-12T05:13:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 17497 12.43 MB/sec warmup 9 sec latency 6.156 ms 2024-03-12T05:13:26.780 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20232 12.38 MB/sec warmup 10 sec latency 8.200 ms 2024-03-12T05:13:27.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:26 smithi156 ceph-mon[36270]: pgmap v43: 97 pgs: 97 active+clean; 6.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 551 KiB/s wr, 70 op/s 2024-03-12T05:13:27.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:26 smithi161 ceph-mon[36043]: pgmap v43: 97 pgs: 97 active+clean; 6.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 551 KiB/s wr, 70 op/s 2024-03-12T05:13:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:26 smithi100 ceph-mon[32210]: pgmap v43: 97 pgs: 97 active+clean; 6.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 551 KiB/s wr, 70 op/s 2024-03-12T05:13:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 22724 12.80 MB/sec warmup 11 sec latency 16.966 ms 2024-03-12T05:13:28.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25616 12.93 MB/sec warmup 12 sec latency 6.526 ms 2024-03-12T05:13:29.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:28 smithi156 ceph-mon[36270]: pgmap v44: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-03-12T05:13:29.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:28 smithi161 ceph-mon[36043]: pgmap v44: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-03-12T05:13:29.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:28 smithi100 ceph-mon[32210]: pgmap v44: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-03-12T05:13:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 28834 13.38 MB/sec warmup 13 sec latency 8.487 ms 2024-03-12T05:13:30.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 32057 13.69 MB/sec warmup 14 sec latency 19.487 ms 2024-03-12T05:13:31.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:30 smithi156 ceph-mon[36270]: pgmap v45: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-03-12T05:13:31.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:30 smithi161 ceph-mon[36043]: pgmap v45: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-03-12T05:13:31.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:30 smithi100 ceph-mon[32210]: pgmap v45: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-03-12T05:13:31.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 34858 13.66 MB/sec warmup 15 sec latency 7.975 ms 2024-03-12T05:13:32.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 37402 13.60 MB/sec warmup 16 sec latency 7.421 ms 2024-03-12T05:13:32.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:13:32.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:13:32.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3803459962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd678102cc0 msgr2=0x7fd67810b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3803459962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd678102cc0 0x7fd67810b1a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009900 tx=0x7fd66c02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3803459962 shutdown_connections 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3803459962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd67810b6e0 0x7fd67810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3803459962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd678102cc0 0x7fd67810b1a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3803459962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6781023a0 0x7fd678102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3803459962 >> 172.21.15.100:0/3803459962 conn(0x7fd678077230 msgr2=0x7fd678077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3803459962 shutdown_connections 2024-03-12T05:13:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3803459962 wait complete. 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 Processor -- start 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.962+0000 7fd67cd3a640 1 -- start start 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67cd3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6781023a0 0x7fd67819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67cd3a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd678102cc0 0x7fd67819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67cd3a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd67810b6e0 0x7fd6781975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67cd3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd678197c30 con 0x7fd6781023a0 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67cd3a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd678197da0 con 0x7fd678102cc0 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67cd3a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd678197f60 con 0x7fd67810b6e0 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6781023a0 0x7fd67819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd678102cc0 0x7fd67819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6781023a0 0x7fd67819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47842/0 (socket says 172.21.15.100:47842) 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd67659c640 1 -- 172.21.15.100:0/3669703824 learned_addr learned my addr 172.21.15.100:0/3669703824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd676d9d640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd67810b6e0 0x7fd6781975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 -- 172.21.15.100:0/3669703824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd67810b6e0 msgr2=0x7fd6781975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd67810b6e0 0x7fd6781975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 -- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6781023a0 msgr2=0x7fd67819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6781023a0 0x7fd67819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 -- 172.21.15.100:0/3669703824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c009560 con 0x7fd678102cc0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd676d9d640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd67810b6e0 0x7fd6781975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.963+0000 7fd675d9b640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd678102cc0 0x7fd67819d970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd66c036040 tx=0x7fd66c004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.964+0000 7fd6677fe640 1 -- 172.21.15.100:0/3669703824 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0043d0 con 0x7fd678102cc0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.964+0000 7fd6677fe640 1 -- 172.21.15.100:0/3669703824 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd66c0052d0 con 0x7fd678102cc0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.964+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6781981e0 con 0x7fd678102cc0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.964+0000 7fd6677fe640 1 -- 172.21.15.100:0/3669703824 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c003d30 con 0x7fd678102cc0 2024-03-12T05:13:32.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.964+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6781a44d0 con 0x7fd678102cc0 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 Processor -- start 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd6677fe640 1 -- 172.21.15.100:0/3669703824 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7fd66c005440 con 0x7fd678102cc0 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 -- start start 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db199690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db108280 con 0x5598db1992b0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db1057b0 con 0x5598db199bd0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.966+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db100f90 con 0x5598db1a27b0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd676d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd676d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47848/0 (socket says 172.21.15.100:47848) 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd676d9d640 1 -- 172.21.15.100:0/1149455889 learned_addr learned my addr 172.21.15.100:0/1149455889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37920/0 (socket says 172.21.15.100:37920) 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd675d9b640 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd6677fe640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6600a1cc0 0x7fd6600a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd676d9d640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6600a1cc0 0x7fd6600a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 -- 172.21.15.100:0/1149455889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 msgr2=0x5598db1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 -- 172.21.15.100:0/1149455889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 msgr2=0x5598db199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 -- 172.21.15.100:0/1149455889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598db0bf570 con 0x5598db199bd0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd676d9d640 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd6677fe640 1 -- 172.21.15.100:0/3669703824 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd66c0f0170 con 0x7fd678102cc0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd67659c640 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a2270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fd0 tx=0x7fd670011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc44ebe8b0a73aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd676d9d640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6600a1cc0 0x7fd6600a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd6680095f0 tx=0x7fd668004730 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd664ff9640 1 -- 172.21.15.100:0/1149455889 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670004d20 con 0x5598db199bd0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd664ff9640 1 -- 172.21.15.100:0/1149455889 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd670004e80 con 0x5598db199bd0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.967+0000 7fd664ff9640 1 -- 172.21.15.100:0/1149455889 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670005580 con 0x5598db199bd0 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1149455889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 msgr2=0x5598db1a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a2270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fd0 tx=0x7fd670011870 comp rx=0 tx=0).stop 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1149455889 shutdown_connections 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a2270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1149455889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1149455889 >> 172.21.15.100:0/1149455889 conn(0x5598db193990 msgr2=0x5598db193d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1149455889 shutdown_connections 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1149455889 wait complete. 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.968+0000 7fd67f9e3740 1 -- start start 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 Processor -- start 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1a2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db19f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 0x5598db19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dadd4430 con 0x5598db1a27b0 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dad90e00 con 0x5598db199bd0 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db085fd0 con 0x5598db1992b0 2024-03-12T05:13:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67f9e3740 10 client.0 _ll_register_callbacks cb 0x5598db0dbfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 0x5598db19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd675d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 0x5598db19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47862/0 (socket says 172.21.15.100:47862) 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 -- 172.21.15.100:0/1767790115 learned_addr learned my addr 172.21.15.100:0/1767790115 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd67659c640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 -- 172.21.15.100:0/1767790115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 msgr2=0x5598db1a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 -- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 msgr2=0x5598db19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 -- 172.21.15.100:0/1767790115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd670009c30 con 0x5598db1a27b0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd675d9b640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd676d9d640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 0x5598db19fc00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd6680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd90bc8a9c5514f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.969+0000 7fd65dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd65dffb640 1 -- 172.21.15.100:0/1767790115 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680042b0 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd65dffb640 1 -- 172.21.15.100:0/1767790115 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd668004410 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598dae3d720 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd65dffb640 1 -- 172.21.15.100:0/1767790115 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd668003af0 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 10 client.15756 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5598db0ff610 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd65dffb640 1 -- 172.21.15.100:0/1767790115 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5598dae3d720 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd65dffb640 1 -- 172.21.15.100:0/1767790115 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd668010400 con 0x5598db1a27b0 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 10 client.15756 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 20 client.15756 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db1b2a90 0x5598db1b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 4 client.15756 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598dae8c890 con 0x5598db1b2a90 2024-03-12T05:13:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.970+0000 7fd67659c640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db1b2a90 0x5598db1b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.971+0000 7fd65dffb640 10 client.15756 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.971+0000 7fd67659c640 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db1b2a90 0x5598db1b4eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc426d45094b555 server_cookie=e6aeb23a13b6f186 in_seq=0 out_seq=0 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.971+0000 7fd65dffb640 1 -- 172.21.15.100:0/1767790115 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598dae8c890 con 0x5598db1b2a90 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.971+0000 7fd65dffb640 10 client.15756 handle_command_reply: tid=0 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 client.15756 shutdown 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 msgr2=0x5598db19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 0x5598db19fc00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd6680046c0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 shutdown_connections 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db1b2a90 0x5598db1b4eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1a27b0 0x5598db19fc00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd65dffb640 0 client.15756 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:32.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1767790115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 >> 172.21.15.100:0/1767790115 conn(0x5598db193990 msgr2=0x5598db195730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:32.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 shutdown_connections 2024-03-12T05:13:32.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1767790115 wait complete. 2024-03-12T05:13:32.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.972+0000 7fd67f9e3740 20 client.15756 trim_cache size 0 max 16384 2024-03-12T05:13:32.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 Processor -- start 2024-03-12T05:13:32.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 -- start start 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 0x5598db19de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db1a27b0 0x5598db19e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db193240 con 0x5598db199bd0 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db18f320 con 0x5598db1a27b0 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db18f0d0 con 0x5598db1992b0 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 0x5598db19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 0x5598db19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47868/0 (socket says 172.21.15.100:47868) 2024-03-12T05:13:32.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 -- 172.21.15.100:0/1850522358 learned_addr learned my addr 172.21.15.100:0/1850522358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd676d9d640 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db1a27b0 0x5598db19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 -- 172.21.15.100:0/1850522358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 msgr2=0x5598db1b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1b9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 -- 172.21.15.100:0/1850522358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db1a27b0 msgr2=0x5598db19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db1a27b0 0x5598db19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 -- 172.21.15.100:0/1850522358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598db09bac0 con 0x5598db199bd0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd67659c640 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd675d9b640 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 0x5598db19de50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0bcd00 tx=0x7fd66c0f2640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d507a81e7f4b1c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd65ffff640 1 -- 172.21.15.100:0/1850522358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0035f0 con 0x5598db199bd0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.988+0000 7fd65ffff640 1 -- 172.21.15.100:0/1850522358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd66c0bd470 con 0x5598db199bd0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd65ffff640 1 -- 172.21.15.100:0/1850522358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0bd1f0 con 0x5598db199bd0 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1850522358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 msgr2=0x5598db19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 0x5598db19de50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0bcd00 tx=0x7fd66c0f2640 comp rx=0 tx=0).stop 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1850522358 shutdown_connections 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db1a27b0 0x5598db19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db199bd0 0x5598db19de50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/1850522358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1992b0 0x5598db1b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1850522358 >> 172.21.15.100:0/1850522358 conn(0x5598db193990 msgr2=0x5598db19a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1850522358 shutdown_connections 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 -- 172.21.15.100:0/1850522358 wait complete. 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.989+0000 7fd67f9e3740 1 -- start start 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 Processor -- start 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db1a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db1a9f90 con 0x5598db1992b0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db1a8510 con 0x5598db199bd0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598db1a86d0 con 0x5598db1a27b0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67f9e3740 10 client.0 _ll_register_callbacks cb 0x7fd670009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db1a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db1a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47882/0 (socket says 172.21.15.100:47882) 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 -- 172.21.15.100:0/2125255917 learned_addr learned my addr 172.21.15.100:0/2125255917 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd675d9b640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd676d9d640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 -- 172.21.15.100:0/2125255917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 msgr2=0x5598db1a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 -- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 msgr2=0x5598db1a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 -- 172.21.15.100:0/2125255917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598db0bf570 con 0x5598db1992b0 2024-03-12T05:13:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd67659c640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db1a96c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5598db198d80 tx=0x7fd670005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e43311501c226c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.990+0000 7fd65dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd65dffb640 1 -- 172.21.15.100:0/2125255917 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67002a410 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd65dffb640 1 -- 172.21.15.100:0/2125255917 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd67002a570 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598dae007e0 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd65dffb640 1 -- 172.21.15.100:0/2125255917 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd67002a750 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 10 client.15762 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fd6781a44d0 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd65dffb640 1 -- 172.21.15.100:0/2125255917 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5598dae007e0 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd65dffb640 1 -- 172.21.15.100:0/2125255917 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd67000c070 con 0x5598db1992b0 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 10 client.15762 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 20 client.15762 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db20a3d0 0x5598db20c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 4 client.15762 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598dadef4e0 con 0x5598db20a3d0 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.991+0000 7fd675d9b640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db20a3d0 0x5598db20c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd65dffb640 10 client.15762 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd675d9b640 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db20a3d0 0x5598db20c7f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb227b5f3d250d5 server_cookie=538a4cc66b45212b in_seq=0 out_seq=0 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd65dffb640 1 -- 172.21.15.100:0/2125255917 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5598dadef4e0 con 0x5598db20a3d0 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd65dffb640 10 client.15762 handle_command_reply: tid=0 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 client.15762 shutdown 2024-03-12T05:13:32.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 msgr2=0x5598db1a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db1a96c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5598db198d80 tx=0x7fd670005f30 comp rx=0 tx=0).stop 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 shutdown_connections 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598db20a3d0 0x5598db20c7f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598db1a27b0 0x5598db1a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd65dffb640 0 client.15762 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598db199bd0 0x5598db1a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 --2- 172.21.15.100:0/2125255917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598db1992b0 0x5598db1a96c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 >> 172.21.15.100:0/2125255917 conn(0x5598db193990 msgr2=0x5598db1b9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:32.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 shutdown_connections 2024-03-12T05:13:32.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 1 -- 172.21.15.100:0/2125255917 wait complete. 2024-03-12T05:13:32.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.992+0000 7fd67f9e3740 20 client.15762 trim_cache size 0 max 16384 2024-03-12T05:13:32.998 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:13:32.998 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:13:32.998 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:13:32.998 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6600a1cc0 msgr2=0x7fd6600a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6600a1cc0 0x7fd6600a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd6680095f0 tx=0x7fd668004730 comp rx=0 tx=0).stop 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd678102cc0 msgr2=0x7fd67819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd678102cc0 0x7fd67819d970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd66c036040 tx=0x7fd66c004c00 comp rx=0 tx=0).stop 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 shutdown_connections 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6600a1cc0 0x7fd6600a4180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd67810b6e0 0x7fd6781975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd678102cc0 0x7fd67819d970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 --2- 172.21.15.100:0/3669703824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6781023a0 0x7fd67819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 >> 172.21.15.100:0/3669703824 conn(0x7fd678077230 msgr2=0x7fd6780fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 shutdown_connections 2024-03-12T05:13:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:32.993+0000 7fd67cd3a640 1 -- 172.21.15.100:0/3669703824 wait complete. 2024-03-12T05:13:33.002 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:13:33.002 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:13:33.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 -- 172.21.15.100:0/1564194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 msgr2=0x7f4b08107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/1564194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08107700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009930 tx=0x7f4b0402bcb0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 -- 172.21.15.100:0/1564194 shutdown_connections 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/1564194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 0x7f4b0810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/1564194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b08101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/1564194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08107700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.124+0000 7f4b0fe47640 1 -- 172.21.15.100:0/1564194 >> 172.21.15.100:0/1564194 conn(0x7f4b08079170 msgr2=0x7f4b08079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 -- 172.21.15.100:0/1564194 shutdown_connections 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 -- 172.21.15.100:0/1564194 wait complete. 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 Processor -- start 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 -- start start 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b0819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 0x7f4b0819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08197dd0 con 0x7f4b08101410 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08197f40 con 0x7f4b08101dd0 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.125+0000 7f4b0fe47640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08198100 con 0x7f4b08107320 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b0819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0e3e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0d3e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 0x7f4b0819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b0819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47890/0 (socket says 172.21.15.100:47890) 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0e3e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58586/0 (socket says 172.21.15.100:58586) 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 -- 172.21.15.100:0/3196478621 learned_addr learned my addr 172.21.15.100:0/3196478621 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 -- 172.21.15.100:0/3196478621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 msgr2=0x7f4b08197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 -- 172.21.15.100:0/3196478621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 msgr2=0x7f4b0819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 0x7f4b0819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 -- 172.21.15.100:0/3196478621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04009560 con 0x7f4b08101410 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0e3e4640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0d3e2640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 0x7f4b0819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0dbe3640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b0819d580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009230 tx=0x7f4b0402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0402dea0 con 0x7f4b08101410 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08198380 con 0x7f4b08101410 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b04004350 con 0x7f4b08101410 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04003b20 con 0x7f4b08101410 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.126+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b081a4500 con 0x7f4b08101410 2024-03-12T05:13:33.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.127+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08102900 con 0x7f4b08101410 2024-03-12T05:13:33.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.128+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f4b04005460 con 0x7f4b08101410 2024-03-12T05:13:33.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.129+0000 7f4afeffd640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4af40a1cc0 0x7f4af40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.129+0000 7f4b0d3e2640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4af40a1cc0 0x7f4af40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.129+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4b040eff60 con 0x7f4b08101410 2024-03-12T05:13:33.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.130+0000 7f4b0d3e2640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4af40a1cc0 0x7f4af40a4180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b08198e90 tx=0x7f4af800b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.134+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b040b8ce0 con 0x7f4b08101410 2024-03-12T05:13:33.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:32 smithi156 ceph-mon[36270]: pgmap v46: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-03-12T05:13:33.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:32 smithi161 ceph-mon[36043]: pgmap v46: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-03-12T05:13:33.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:32 smithi100 ceph-mon[32210]: pgmap v46: 97 pgs: 97 active+clean; 12 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-03-12T05:13:33.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.367+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b08198570 con 0x7f4b08101410 2024-03-12T05:13:33.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.368+0000 7f4afeffd640 1 -- 172.21.15.100:0/3196478621 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7f4b040bc3a0 con 0x7f4b08101410 2024-03-12T05:13:33.365 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:13:33.365 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:13:19.700942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:13:33.365 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 24 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4af40a1cc0 msgr2=0x7f4af40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4af40a1cc0 0x7f4af40a4180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b08198e90 tx=0x7f4af800b410 comp rx=0 tx=0).stop 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 msgr2=0x7f4b0819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b0819d580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009230 tx=0x7f4b0402c740 comp rx=0 tx=0).stop 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 shutdown_connections 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4af40a1cc0 0x7f4af40a4180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b08107320 0x7f4b08197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b08101dd0 0x7f4b0819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 --2- 172.21.15.100:0/3196478621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b08101410 0x7f4b0819d580 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 >> 172.21.15.100:0/3196478621 conn(0x7f4b08079170 msgr2=0x7f4b08100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 shutdown_connections 2024-03-12T05:13:33.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.370+0000 7f4b0fe47640 1 -- 172.21.15.100:0/3196478621 wait complete. 2024-03-12T05:13:33.378 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:13:33.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:13:33.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.490+0000 7fd2deedf640 1 -- 172.21.15.100:0/1497258947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 msgr2=0x7fd2d80ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.490+0000 7fd2deedf640 1 --2- 172.21.15.100:0/1497258947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d80ffcf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009900 tx=0x7fd2cc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:13:33.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- 172.21.15.100:0/1497258947 shutdown_connections 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 --2- 172.21.15.100:0/1497258947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d8106520 0x7fd2d810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 --2- 172.21.15.100:0/1497258947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d81002c0 0x7fd2d8105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 --2- 172.21.15.100:0/1497258947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d80ffcf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- 172.21.15.100:0/1497258947 >> 172.21.15.100:0/1497258947 conn(0x7fd2d8076640 msgr2=0x7fd2d80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- 172.21.15.100:0/1497258947 shutdown_connections 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- 172.21.15.100:0/1497258947 wait complete. 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 Processor -- start 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- start start 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d8106520 0x7fd2d819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d81975d0 0x7fd2d8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8197fc0 con 0x7fd2d80ff910 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8198130 con 0x7fd2d81975d0 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2deedf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81982f0 con 0x7fd2d8106520 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.491+0000 7fd2dcc7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47908/0 (socket says 172.21.15.100:47908) 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/3175820526 learned_addr learned my addr 172.21.15.100:0/3175820526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d8106520 0x7fd2d819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d81975d0 0x7fd2d8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/3175820526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d8106520 msgr2=0x7fd2d819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d8106520 0x7fd2d819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/3175820526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d81975d0 msgr2=0x7fd2d8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d81975d0 0x7fd2d8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/3175820526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc009560 con 0x7fd2d80ff910 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d8106520 0x7fd2d819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d819d420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005940 tx=0x7fd2cc02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2d5ffb640 1 -- 172.21.15.100:0/3175820526 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004350 con 0x7fd2d80ff910 2024-03-12T05:13:33.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d81a42e0 con 0x7fd2d80ff910 2024-03-12T05:13:33.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2d5ffb640 1 -- 172.21.15.100:0/3175820526 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc0044b0 con 0x7fd2d80ff910 2024-03-12T05:13:33.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2d5ffb640 1 -- 172.21.15.100:0/3175820526 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc003d30 con 0x7fd2d80ff910 2024-03-12T05:13:33.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.492+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d81a47e0 con 0x7fd2d80ff910 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.494+0000 7fd2d5ffb640 1 -- 172.21.15.100:0/3175820526 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7fd2cc004df0 con 0x7fd2d80ff910 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2d5ffb640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2c80a1cc0 0x7fd2c80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 Processor -- start 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2c80a1cc0 0x7fd2c80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2d5ffb640 1 -- 172.21.15.100:0/3175820526 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd2cc0ef9f0 con 0x7fd2d80ff910 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 -- start start 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb373630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb32e960 con 0x5626eb373b70 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb2e2310 con 0x5626eb37c750 2024-03-12T05:13:33.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb2df840 con 0x5626eb373250 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2c80a1cc0 0x7fd2c80a4180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80fec20 tx=0x7fd2c4005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.495+0000 7fd2d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37976/0 (socket says 172.21.15.100:37976) 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 -- 172.21.15.100:0/1287743900 learned_addr learned my addr 172.21.15.100:0/1287743900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb373630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 -- 172.21.15.100:0/1287743900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 msgr2=0x5626eb373630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb373630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 -- 172.21.15.100:0/1287743900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 msgr2=0x5626eb37c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 -- 172.21.15.100:0/1287743900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626eb29d060 con 0x5626eb37c750 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37eb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009fd0 tx=0x7fd2d00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8ece37c0cea115 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2c2ffd640 1 -- 172.21.15.100:0/1287743900 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0004d20 con 0x5626eb37c750 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2c2ffd640 1 -- 172.21.15.100:0/1287743900 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2d0004e80 con 0x5626eb37c750 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.496+0000 7fd2c2ffd640 1 -- 172.21.15.100:0/1287743900 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0005280 con 0x5626eb37c750 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1287743900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 msgr2=0x5626eb37eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37eb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009fd0 tx=0x7fd2d00118d0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1287743900 shutdown_connections 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37eb40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1287743900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb373630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1287743900 >> 172.21.15.100:0/1287743900 conn(0x5626eb36d980 msgr2=0x5626eb36dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1287743900 shutdown_connections 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1287743900 wait complete. 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 -- start start 2024-03-12T05:13:33.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.497+0000 7fd2e1b8e740 1 Processor -- start 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb379770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373b70 0x5626eb379cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb2db020 con 0x5626eb373250 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eafae430 con 0x5626eb37c750 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eaf6ae00 con 0x5626eb373b70 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 10 client.0 _ll_register_callbacks cb 0x5626eb282cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb379770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb379770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47924/0 (socket says 172.21.15.100:47924) 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1521671120 learned_addr learned my addr 172.21.15.100:0/1521671120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373b70 0x5626eb379cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1521671120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373b70 msgr2=0x5626eb379cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373b70 0x5626eb379cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 msgr2=0x5626eb37a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1521671120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d0009c80 con 0x5626eb373250 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb379770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0b8930 tx=0x7fd2cc02cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75090da0f69913d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1521671120 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0bc4d0 con 0x5626eb373250 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626eb017720 con 0x5626eb373250 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.498+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1521671120 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd2cc0c0620 con 0x5626eb373250 2024-03-12T05:13:33.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1521671120 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc0c4c70 con 0x5626eb373250 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 10 client.15780 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5626eb2d96a0 con 0x5626eb373250 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1521671120 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626eb017720 con 0x5626eb373250 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1521671120 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd2cc03c070 con 0x5626eb373250 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 10 client.15780 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 10 client.15780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 20 client.15780 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb38c8d0 0x5626eb38ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 4 client.15780 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626eb066890 con 0x5626eb38c8d0 2024-03-12T05:13:33.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.499+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb38c8d0 0x5626eb38ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.500+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb38c8d0 0x5626eb38ecc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53b3aa45a212250b server_cookie=6ae3829851993dd2 in_seq=0 out_seq=0 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.500+0000 7fd2b7fff640 10 client.15780 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1521671120 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626eb066890 con 0x5626eb38c8d0 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2b7fff640 10 client.15780 handle_command_reply: tid=0 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 client.15780 shutdown 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 msgr2=0x5626eb379770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb379770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0b8930 tx=0x7fd2cc02cda0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 shutdown_connections 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb38c8d0 0x5626eb38ecc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb37a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373b70 0x5626eb379cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2b7fff640 0 client.15780 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:33.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1521671120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb379770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 >> 172.21.15.100:0/1521671120 conn(0x5626eb36d980 msgr2=0x5626eb36f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 shutdown_connections 2024-03-12T05:13:33.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1521671120 wait complete. 2024-03-12T05:13:33.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.501+0000 7fd2e1b8e740 20 client.15780 trim_cache size 0 max 16384 2024-03-12T05:13:33.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.513+0000 7fd2e1b8e740 1 Processor -- start 2024-03-12T05:13:33.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.513+0000 7fd2e1b8e740 1 -- start start 2024-03-12T05:13:33.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb37bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb3769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb260130 con 0x5626eb373b70 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb367f90 con 0x5626eb37c750 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb3692b0 con 0x5626eb373250 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47930/0 (socket says 172.21.15.100:47930) 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 -- 172.21.15.100:0/695790206 learned_addr learned my addr 172.21.15.100:0/695790206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb3769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb37bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 -- 172.21.15.100:0/695790206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 msgr2=0x5626eb37bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb37bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 -- 172.21.15.100:0/695790206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 msgr2=0x5626eb3769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb3769d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 -- 172.21.15.100:0/695790206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626eb275c20 con 0x5626eb373b70 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37a010 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009e80 tx=0x7fd2c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55eb3920346f925 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2c27fc640 1 -- 172.21.15.100:0/695790206 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4024040 con 0x5626eb373b70 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2c27fc640 1 -- 172.21.15.100:0/695790206 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2c4029600 con 0x5626eb373b70 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.514+0000 7fd2c27fc640 1 -- 172.21.15.100:0/695790206 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c401e3f0 con 0x5626eb373b70 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/695790206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 msgr2=0x5626eb37a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37a010 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009e80 tx=0x7fd2c4014000 comp rx=0 tx=0).stop 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/695790206 shutdown_connections 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb37c750 0x5626eb3769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373b70 0x5626eb37a010 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/695790206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb373250 0x5626eb37bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/695790206 >> 172.21.15.100:0/695790206 conn(0x5626eb36d980 msgr2=0x5626eb374a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/695790206 shutdown_connections 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/695790206 wait complete. 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 -- start start 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.515+0000 7fd2e1b8e740 1 Processor -- start 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb3826b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb373b70 0x5626eb382bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb37c750 0x5626eb3e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb3832f0 con 0x5626eb373250 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb3e32c0 con 0x5626eb373b70 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eb3e3430 con 0x5626eb37c750 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2e1b8e740 10 client.0 _ll_register_callbacks cb 0x7fd2d0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb3826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb373b70 0x5626eb382bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb3826b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47936/0 (socket says 172.21.15.100:47936) 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1162629950 learned_addr learned my addr 172.21.15.100:0/1162629950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb37c750 0x5626eb3e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1162629950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb37c750 msgr2=0x5626eb3e2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb37c750 0x5626eb3e2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb373b70 msgr2=0x5626eb382bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb373b70 0x5626eb382bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 -- 172.21.15.100:0/1162629950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c4004180 con 0x5626eb373250 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb373b70 0x5626eb382bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dd47c640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb37c750 0x5626eb3e2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2dcc7b640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb3826b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0baa20 tx=0x7fd2cc0bbf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22522277f39e06f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.516+0000 7fd2b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1162629950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0b8770 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1162629950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd2cc0c49f0 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626eafda7e0 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1162629950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc0ed000 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 10 client.15786 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5626eb29b8c0 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1162629950 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626eafda7e0 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1162629950 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd2cc03c040 con 0x5626eb373250 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 10 client.15786 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 20 client.15786 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb3e4700 0x5626eb3e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 4 client.15786 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:13:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626eaf86700 con 0x5626eb3e4700 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.517+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb3e4700 0x5626eb3e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.518+0000 7fd2d7fff640 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb3e4700 0x5626eb3e6af0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bac2d07dad1e36a server_cookie=53cf06842a2ecab8 in_seq=0 out_seq=0 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.518+0000 7fd2b7fff640 10 client.15786 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2b7fff640 1 -- 172.21.15.100:0/1162629950 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626eaf86700 con 0x5626eb3e4700 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2b7fff640 10 client.15786 handle_command_reply: tid=0 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 client.15786 shutdown 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 msgr2=0x5626eb3826b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb3826b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0baa20 tx=0x7fd2cc0bbf60 comp rx=0 tx=0).stop 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 shutdown_connections 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5626eb3e4700 0x5626eb3e6af0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626eb37c750 0x5626eb3e2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626eb373b70 0x5626eb382bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2b7fff640 0 client.15786 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:13:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 --2- 172.21.15.100:0/1162629950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626eb373250 0x5626eb3826b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 >> 172.21.15.100:0/1162629950 conn(0x5626eb36d980 msgr2=0x5626eb36c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 shutdown_connections 2024-03-12T05:13:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 1 -- 172.21.15.100:0/1162629950 wait complete. 2024-03-12T05:13:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.519+0000 7fd2e1b8e740 20 client.15786 trim_cache size 0 max 16384 2024-03-12T05:13:33.523 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:13:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2c80a1cc0 msgr2=0x7fd2c80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2c80a1cc0 0x7fd2c80a4180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80fec20 tx=0x7fd2c4005f90 comp rx=0 tx=0).stop 2024-03-12T05:13:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 msgr2=0x7fd2d819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d819d420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005940 tx=0x7fd2cc02c670 comp rx=0 tx=0).stop 2024-03-12T05:13:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 shutdown_connections 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2c80a1cc0 0x7fd2c80a4180 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d81975d0 0x7fd2d8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d8106520 0x7fd2d819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 --2- 172.21.15.100:0/3175820526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d80ff910 0x7fd2d819d420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 >> 172.21.15.100:0/3175820526 conn(0x7fd2d8076640 msgr2=0x7fd2d80fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 shutdown_connections 2024-03-12T05:13:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.520+0000 7fd2deedf640 1 -- 172.21.15.100:0/3175820526 wait complete. 2024-03-12T05:13:33.527 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:13:33.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/737387178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c80ff790 msgr2=0x7fb2c80ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/737387178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c80ff790 0x7fb2c80ffc10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00aa40 tx=0x7fb2bc02cf50 comp rx=0 tx=0).stop 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/737387178 shutdown_connections 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/737387178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c8100150 0x7fb2c810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/737387178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c80ff790 0x7fb2c80ffc10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/737387178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c81057a0 0x7fb2c8105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/737387178 >> 172.21.15.100:0/737387178 conn(0x7fb2c80765a0 msgr2=0x7fb2c80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/737387178 shutdown_connections 2024-03-12T05:13:33.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/737387178 wait complete. 2024-03-12T05:13:33.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 Processor -- start 2024-03-12T05:13:33.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.640+0000 7fb2ce4d0640 1 -- start start 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c80ff790 0x7fb2c80fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 0x7fb2c80fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81057a0 0x7fb2c80fc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81a5b60 con 0x7fb2c8100150 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81a5cd0 con 0x7fb2c80ff790 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81a5e90 con 0x7fb2c81057a0 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 0x7fb2c80fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 0x7fb2c80fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47960/0 (socket says 172.21.15.100:47960) 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3086255506 learned_addr learned my addr 172.21.15.100:0/3086255506 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c80ff790 0x7fb2c80fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81057a0 0x7fb2c80fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3086255506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81057a0 msgr2=0x7fb2c80fc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81057a0 0x7fb2c80fc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3086255506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c80ff790 msgr2=0x7fb2c80fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c80ff790 0x7fb2c80fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3086255506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc00a670 con 0x7fb2c8100150 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 0x7fb2c80fbf30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00a340 tx=0x7fb2bc02dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2c57fa640 1 -- 172.21.15.100:0/3086255506 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc042070 con 0x7fb2c8100150 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c81a6110 con 0x7fb2c8100150 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.641+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c81a6610 con 0x7fb2c8100150 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.642+0000 7fb2c57fa640 1 -- 172.21.15.100:0/3086255506 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2bc035e90 con 0x7fb2c8100150 2024-03-12T05:13:33.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.642+0000 7fb2c57fa640 1 -- 172.21.15.100:0/3086255506 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc004d30 con 0x7fb2c8100150 2024-03-12T05:13:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 Processor -- start 2024-03-12T05:13:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2c57fa640 1 -- 172.21.15.100:0/3086255506 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7fb2bc004a50 con 0x7fb2c8100150 2024-03-12T05:13:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 -- start start 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 0x559d38dcc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dccb70 0x559d38dd5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38d87960 con 0x559d38dcc250 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38d3b310 con 0x559d38dd5750 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2d1177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38d38840 con 0x559d38dccb70 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2cca6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 0x559d38dcc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2cca6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 0x559d38dcc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47970/0 (socket says 172.21.15.100:47970) 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2cca6d640 1 -- 172.21.15.100:0/1198141794 learned_addr learned my addr 172.21.15.100:0/1198141794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dccb70 0x559d38dd5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.644+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2c57fa640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2b00a1cf0 0x7fb2b00a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 -- 172.21.15.100:0/1198141794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dccb70 msgr2=0x559d38dd5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dccb70 0x559d38dd5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 -- 172.21.15.100:0/1198141794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 msgr2=0x559d38dd7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 -- 172.21.15.100:0/1198141794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d38cf6060 con 0x559d38dcc250 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2c57fa640 1 -- 172.21.15.100:0/3086255506 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb2bc0ef7b0 con 0x7fb2c8100150 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2b00a1cf0 0x7fb2b00a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 0x559d38dcc630 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009ee0 tx=0x7fb2c0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97ba28ec1eea3ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2ae7fc640 1 -- 172.21.15.100:0/1198141794 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012ab0 con 0x559d38dcc250 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2ae7fc640 1 -- 172.21.15.100:0/1198141794 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2c00099c0 con 0x559d38dcc250 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.645+0000 7fb2ae7fc640 1 -- 172.21.15.100:0/1198141794 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0011e60 con 0x559d38dcc250 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 -- 172.21.15.100:0/1198141794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 msgr2=0x559d38dcc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2b00a1cf0 0x7fb2b00a41b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2c001e000 tx=0x7fb2c0021000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 0x559d38dcc630 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009ee0 tx=0x7fb2c0010740 comp rx=0 tx=0).stop 2024-03-12T05:13:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 -- 172.21.15.100:0/1198141794 shutdown_connections 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dccb70 0x559d38dd5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 --2- 172.21.15.100:0/1198141794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dcc250 0x559d38dcc630 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 -- 172.21.15.100:0/1198141794 >> 172.21.15.100:0/1198141794 conn(0x559d38dc6980 msgr2=0x559d38dc6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 -- 172.21.15.100:0/1198141794 shutdown_connections 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 -- 172.21.15.100:0/1198141794 wait complete. 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.646+0000 7fb2d1177740 1 -- start start 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 Processor -- start 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38dd2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38d34020 con 0x559d38dccb70 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38a07430 con 0x559d38dd5750 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d389c3e00 con 0x559d38dcc250 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2d1177740 10 client.0 _ll_register_callbacks cb 0x559d38cdbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38dd2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38dd2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47982/0 (socket says 172.21.15.100:47982) 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3773285961 learned_addr learned my addr 172.21.15.100:0/3773285961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 msgr2=0x559d38dd2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3773285961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 msgr2=0x559d38dd3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.647+0000 7fb2c77fe640 1 -- 172.21.15.100:0/3773285961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c0009b40 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38dd2c40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00a540 tx=0x7fb2bc0f5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e626e1446c86e984 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2a77fe640 1 -- 172.21.15.100:0/3773285961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc003920 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2a77fe640 1 -- 172.21.15.100:0/3773285961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb2bc0bc990 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d38a70720 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2a77fe640 1 -- 172.21.15.100:0/3773285961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2bc0bbdd0 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 10 client.15798 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d38d326a0 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2a77fe640 1 -- 172.21.15.100:0/3773285961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d38a70720 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2a77fe640 1 -- 172.21.15.100:0/3773285961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb2bc0c45a0 con 0x559d38dccb70 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 10 client.15798 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:13:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 20 client.15798 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38de5740 0x559d38de7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 4 client.15798 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d38abf890 con 0x559d38de5740 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.648+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38de5740 0x559d38de7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.649+0000 7fb2a77fe640 10 client.15798 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.649+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38de5740 0x559d38de7b60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0e2b88305f71116 server_cookie=e83596fe8bc7aea in_seq=0 out_seq=0 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.649+0000 7fb2a77fe640 1 -- 172.21.15.100:0/3773285961 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d38abf890 con 0x559d38de5740 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.649+0000 7fb2a77fe640 10 client.15798 handle_command_reply: tid=0 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.649+0000 7fb2d1177740 1 client.15798 shutdown 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 msgr2=0x559d38dd2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38dd2c40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00a540 tx=0x7fb2bc0f5750 comp rx=0 tx=0).stop 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 shutdown_connections 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38de5740 0x559d38de7b60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38dd3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38dd2c40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 --2- 172.21.15.100:0/3773285961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2a77fe640 0 client.15798 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 >> 172.21.15.100:0/3773285961 conn(0x559d38dc6980 msgr2=0x559d38dc86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 shutdown_connections 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 1 -- 172.21.15.100:0/3773285961 wait complete. 2024-03-12T05:13:33.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.650+0000 7fb2d1177740 20 client.15798 trim_cache size 0 max 16384 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 Processor -- start 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 -- start start 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dccb70 0x559d38dd2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 0x559d38dcf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38cb9130 con 0x559d38dd5750 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38dc67f0 con 0x559d38dccb70 2024-03-12T05:13:33.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2d1177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38de2610 con 0x559d38dcc250 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 0x559d38dcf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 0x559d38dcf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47990/0 (socket says 172.21.15.100:47990) 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 -- 172.21.15.100:0/13235645 learned_addr learned my addr 172.21.15.100:0/13235645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dccb70 0x559d38dd2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 -- 172.21.15.100:0/13235645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 msgr2=0x559d38dd4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 -- 172.21.15.100:0/13235645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dccb70 msgr2=0x559d38dd2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dccb70 0x559d38dd2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.663+0000 7fb2cca6d640 1 -- 172.21.15.100:0/13235645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d38ccec20 con 0x559d38dd5750 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2cca6d640 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 0x559d38dcf980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005c40 tx=0x7fb2c0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a619541df960623d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2adffb640 1 -- 172.21.15.100:0/13235645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c002c040 con 0x559d38dd5750 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2adffb640 1 -- 172.21.15.100:0/13235645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2c0012bf0 con 0x559d38dd5750 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2adffb640 1 -- 172.21.15.100:0/13235645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0026b10 con 0x559d38dd5750 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 -- 172.21.15.100:0/13235645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 msgr2=0x559d38dcf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 0x559d38dcf980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005c40 tx=0x7fb2c0027000 comp rx=0 tx=0).stop 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 -- 172.21.15.100:0/13235645 shutdown_connections 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dd5750 0x559d38dcf980 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dccb70 0x559d38dd2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 --2- 172.21.15.100:0/13235645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38dd4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 -- 172.21.15.100:0/13235645 >> 172.21.15.100:0/13235645 conn(0x559d38dc6980 msgr2=0x559d38dd0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 -- 172.21.15.100:0/13235645 shutdown_connections 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.664+0000 7fb2d1177740 1 -- 172.21.15.100:0/13235645 wait complete. 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 -- start start 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 Processor -- start 2024-03-12T05:13:33.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38ddcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38ddae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38ddb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38ddbc70 con 0x559d38dccb70 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38e3bc50 con 0x559d38dd5750 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d38e3bd90 con 0x559d38dcc250 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2d1177740 10 client.0 _ll_register_callbacks cb 0x7fb2c0009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38ddae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2cca6d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38ddb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38ddae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48006/0 (socket says 172.21.15.100:48006) 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2c77fe640 1 -- 172.21.15.100:0/582080064 learned_addr learned my addr 172.21.15.100:0/582080064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38ddcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.665+0000 7fb2c77fe640 1 -- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 msgr2=0x559d38ddcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38ddcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2c77fe640 1 -- 172.21.15.100:0/582080064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 msgr2=0x559d38ddb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38ddb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2c77fe640 1 -- 172.21.15.100:0/582080064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d38cf6060 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2c77fe640 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38ddae60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0bbf90 tx=0x7fb2bc0b85a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b793738784e4ada server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2a77fe640 1 -- 172.21.15.100:0/582080064 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0ecf90 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d38a337e0 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2a77fe640 1 -- 172.21.15.100:0/582080064 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb2bc0c03e0 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2a77fe640 1 -- 172.21.15.100:0/582080064 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2bc003920 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 10 client.15804 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d38cf48c0 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2a77fe640 1 -- 172.21.15.100:0/582080064 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d38a337e0 con 0x559d38dccb70 2024-03-12T05:13:33.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2a77fe640 1 -- 172.21.15.100:0/582080064 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb2bc0bb6e0 con 0x559d38dccb70 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 10 client.15804 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 10 client.15804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 20 client.15804 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38e3cf70 0x559d38e3f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 4 client.15804 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.666+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d389df700 con 0x559d38e3cf70 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38e3cf70 0x559d38e3f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2c7fff640 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38e3cf70 0x559d38e3f360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ca7557229190c1d server_cookie=be6c13b059559ea in_seq=0 out_seq=0 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2a77fe640 10 client.15804 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2a77fe640 1 -- 172.21.15.100:0/582080064 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d389df700 con 0x559d38e3cf70 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2a77fe640 10 client.15804 handle_command_reply: tid=0 2024-03-12T05:13:33.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 client.15804 shutdown 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 msgr2=0x559d38ddae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38ddae60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0bbf90 tx=0x7fb2bc0b85a0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 shutdown_connections 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559d38e3cf70 0x559d38e3f360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d38dd5750 0x559d38ddb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2a77fe640 0 client.15804 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d38dccb70 0x559d38ddae60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 --2- 172.21.15.100:0/582080064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d38dcc250 0x559d38ddcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 >> 172.21.15.100:0/582080064 conn(0x559d38dc6980 msgr2=0x559d38dd1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 shutdown_connections 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 1 -- 172.21.15.100:0/582080064 wait complete. 2024-03-12T05:13:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.667+0000 7fb2d1177740 20 client.15804 trim_cache size 0 max 16384 2024-03-12T05:13:33.672 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2b00a1cf0 msgr2=0x7fb2b00a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2b00a1cf0 0x7fb2b00a41b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2c001e000 tx=0x7fb2c0021000 comp rx=0 tx=0).stop 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 msgr2=0x7fb2c80fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 0x7fb2c80fbf30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00a340 tx=0x7fb2bc02dda0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 shutdown_connections 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2b00a1cf0 0x7fb2b00a41b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81057a0 0x7fb2c80fc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2c8100150 0x7fb2c80fbf30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 --2- 172.21.15.100:0/3086255506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c80ff790 0x7fb2c80fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 >> 172.21.15.100:0/3086255506 conn(0x7fb2c80765a0 msgr2=0x7fb2c80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 shutdown_connections 2024-03-12T05:13:33.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.668+0000 7fb2ce4d0640 1 -- 172.21.15.100:0/3086255506 wait complete. 2024-03-12T05:13:33.675 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:13:33.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:13:33.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 40124 13.72 MB/sec warmup 17 sec latency 16.176 ms 2024-03-12T05:13:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 -- 172.21.15.100:0/805900941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00400ffcf0 msgr2=0x7f00401000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 --2- 172.21.15.100:0/805900941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00400ffcf0 0x7f00401000d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f003c009930 tx=0x7f003c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:13:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 -- 172.21.15.100:0/805900941 shutdown_connections 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 --2- 172.21.15.100:0/805900941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00401064b0 0x7f004010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 --2- 172.21.15.100:0/805900941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401006a0 0x7f0040105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 --2- 172.21.15.100:0/805900941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00400ffcf0 0x7f00401000d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 -- 172.21.15.100:0/805900941 >> 172.21.15.100:0/805900941 conn(0x7f00400752e0 msgr2=0x7f00400756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.786+0000 7f00478a3640 1 -- 172.21.15.100:0/805900941 shutdown_connections 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 -- 172.21.15.100:0/805900941 wait complete. 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 Processor -- start 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 -- start start 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00400ffcf0 0x7f004019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 0x7f004019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401998b0 0x7f0040199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004019a2a0 con 0x7f00401064b0 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004019a410 con 0x7f00400ffcf0 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f00478a3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004019a5d0 con 0x7f00401998b0 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f0044e3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 0x7f004019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f004563f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00400ffcf0 0x7f004019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f0044e3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 0x7f004019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48020/0 (socket says 172.21.15.100:48020) 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f0044e3e640 1 -- 172.21.15.100:0/3442654570 learned_addr learned my addr 172.21.15.100:0/3442654570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.787+0000 7f0045e40640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401998b0 0x7f0040199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0044e3e640 1 -- 172.21.15.100:0/3442654570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401998b0 msgr2=0x7f0040199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0044e3e640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401998b0 0x7f0040199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0044e3e640 1 -- 172.21.15.100:0/3442654570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00400ffcf0 msgr2=0x7f004019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0044e3e640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00400ffcf0 0x7f004019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0044e3e640 1 -- 172.21.15.100:0/3442654570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009560 con 0x7f00401064b0 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0045e40640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401998b0 0x7f0040199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f004563f640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00400ffcf0 0x7f004019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f0044e3e640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 0x7f004019fc40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f003000c970 tx=0x7f003000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f00367fc640 1 -- 172.21.15.100:0/3442654570 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030007bb0 con 0x7f00401064b0 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00401a65c0 con 0x7f00401064b0 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f00367fc640 1 -- 172.21.15.100:0/3442654570 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0030007d10 con 0x7f00401064b0 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f00367fc640 1 -- 172.21.15.100:0/3442654570 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030018730 con 0x7f00401064b0 2024-03-12T05:13:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.788+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00401a6a50 con 0x7f00401064b0 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 Processor -- start 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f00367fc640 1 -- 172.21.15.100:0/3442654570 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f003000f450 con 0x7f00401064b0 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 -- start start 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ad6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778a91960 con 0x55b778ad6b70 2024-03-12T05:13:33.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778a45310 con 0x55b778ad6250 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.791+0000 7f004a470740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778a42840 con 0x55b778adf750 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ad6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ad6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38078/0 (socket says 172.21.15.100:38078) 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f004563f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 -- 172.21.15.100:0/2837292345 learned_addr learned my addr 172.21.15.100:0/2837292345 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f00367fc640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f002c0a1cc0 0x7f002c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 -- 172.21.15.100:0/2837292345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 msgr2=0x55b778ae1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 -- 172.21.15.100:0/2837292345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 msgr2=0x55b778adf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 -- 172.21.15.100:0/2837292345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b778a00060 con 0x55b778ad6250 2024-03-12T05:13:33.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f004563f640 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f00367fc640 1 -- 172.21.15.100:0/3442654570 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00300cdc00 con 0x7f00401064b0 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f002c0a1cc0 0x7f002c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ad6630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0038009fd0 tx=0x7f0038010a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1fadd8bb1e5202a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f002b7fe640 1 -- 172.21.15.100:0/2837292345 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800e650 con 0x55b778ad6250 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f0045e40640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f002c0a1cc0 0x7f002c0a4180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0038021000 tx=0x7f003801f000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f002b7fe640 1 -- 172.21.15.100:0/2837292345 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f003800ee80 con 0x55b778ad6250 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.792+0000 7f002b7fe640 1 -- 172.21.15.100:0/2837292345 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003801d850 con 0x55b778ad6250 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 -- 172.21.15.100:0/2837292345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 msgr2=0x55b778ad6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ad6630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0038009fd0 tx=0x7f0038010a00 comp rx=0 tx=0).stop 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 -- 172.21.15.100:0/2837292345 shutdown_connections 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 --2- 172.21.15.100:0/2837292345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ad6630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 -- 172.21.15.100:0/2837292345 >> 172.21.15.100:0/2837292345 conn(0x55b778ad0980 msgr2=0x55b778ad0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 -- 172.21.15.100:0/2837292345 shutdown_connections 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 -- 172.21.15.100:0/2837292345 wait complete. 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 -- start start 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.793+0000 7f004a470740 1 Processor -- start 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778adc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778add200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778a3e020 con 0x55b778ad6b70 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778711430 con 0x55b778ad6250 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7786cde00 con 0x55b778adf750 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004a470740 10 client.0 _ll_register_callbacks cb 0x55b7789e5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778adc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778adc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38088/0 (socket says 172.21.15.100:38088) 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 -- 172.21.15.100:0/2086483180 learned_addr learned my addr 172.21.15.100:0/2086483180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f0045e40640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778add200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 -- 172.21.15.100:0/2086483180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 msgr2=0x55b778add200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778add200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 -- 172.21.15.100:0/2086483180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 msgr2=0x55b778adccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.794+0000 7f004563f640 1 -- 172.21.15.100:0/2086483180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0038009c80 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004563f640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778adc780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f003c009900 tx=0x7f003c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c561fcc132ad2dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f0028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f0028ff9640 1 -- 172.21.15.100:0/2086483180 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c004df0 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b77877a720 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f0028ff9640 1 -- 172.21.15.100:0/2086483180 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f003c004f50 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f0028ff9640 1 -- 172.21.15.100:0/2086483180 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f003c005110 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 10 client.25196 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b778a3c6a0 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f0028ff9640 1 -- 172.21.15.100:0/2086483180 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b77877a720 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f0028ff9640 1 -- 172.21.15.100:0/2086483180 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f003c03c040 con 0x55b778ad6250 2024-03-12T05:13:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 10 client.25196 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 10 client.25196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 20 client.25196 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778aefb40 0x55b778af1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 4 client.25196 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.795+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7787c9890 con 0x55b778aefb40 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.796+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778aefb40 0x55b778af1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.796+0000 7f0028ff9640 10 client.25196 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.796+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778aefb40 0x55b778af1f30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8987b94c37f4a7 server_cookie=6a00bcce84d5f42b in_seq=0 out_seq=0 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f0028ff9640 1 -- 172.21.15.100:0/2086483180 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7787c9890 con 0x55b778aefb40 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f0028ff9640 10 client.25196 handle_command_reply: tid=0 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 client.25196 shutdown 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 msgr2=0x55b778adc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778adc780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f003c009900 tx=0x7f003c004270 comp rx=0 tx=0).stop 2024-03-12T05:13:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 shutdown_connections 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778aefb40 0x55b778af1f30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778add200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778adccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f0028ff9640 0 client.25196 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 --2- 172.21.15.100:0/2086483180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778adc780 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 >> 172.21.15.100:0/2086483180 conn(0x55b778ad0980 msgr2=0x55b778ad26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 shutdown_connections 2024-03-12T05:13:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 1 -- 172.21.15.100:0/2086483180 wait complete. 2024-03-12T05:13:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.797+0000 7f004a470740 20 client.25196 trim_cache size 0 max 16384 2024-03-12T05:13:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 Processor -- start 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 -- start start 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 0x55b778adeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778ad6b70 0x55b778add020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778adf750 0x55b778af68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7789c3130 con 0x55b778ad6250 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778ad1780 con 0x55b778adf750 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004a470740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778acc5c0 con 0x55b778ad6b70 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 0x55b778adeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 0x55b778adeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48044/0 (socket says 172.21.15.100:48044) 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 -- 172.21.15.100:0/1980762231 learned_addr learned my addr 172.21.15.100:0/1980762231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f0045e40640 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778adf750 0x55b778af68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f0044e3e640 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778ad6b70 0x55b778add020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 -- 172.21.15.100:0/1980762231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778ad6b70 msgr2=0x55b778add020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778ad6b70 0x55b778add020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 -- 172.21.15.100:0/1980762231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778adf750 msgr2=0x55b778af68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778adf750 0x55b778af68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 -- 172.21.15.100:0/1980762231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7789d8c20 con 0x55b778ad6250 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.810+0000 7f004563f640 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 0x55b778adeeb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f003c009850 tx=0x7f003c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5aa17eb6d6c2a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f002affd640 1 -- 172.21.15.100:0/1980762231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0349f0 con 0x55b778ad6250 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f002affd640 1 -- 172.21.15.100:0/1980762231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f003c034590 con 0x55b778ad6250 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f002affd640 1 -- 172.21.15.100:0/1980762231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c034e40 con 0x55b778ad6250 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 -- 172.21.15.100:0/1980762231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 msgr2=0x55b778adeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 0x55b778adeeb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f003c009850 tx=0x7f003c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 -- 172.21.15.100:0/1980762231 shutdown_connections 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778adf750 0x55b778af68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778ad6b70 0x55b778add020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 --2- 172.21.15.100:0/1980762231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6250 0x55b778adeeb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 -- 172.21.15.100:0/1980762231 >> 172.21.15.100:0/1980762231 conn(0x55b778ad0980 msgr2=0x55b778acba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 -- 172.21.15.100:0/1980762231 shutdown_connections 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.811+0000 7f004a470740 1 -- 172.21.15.100:0/1980762231 wait complete. 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 -- start start 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 Processor -- start 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ae6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778ae4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778ae5b40 con 0x55b778ad6b70 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778ae5cb0 con 0x55b778ad6250 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b778b46090 con 0x55b778adf750 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004a470740 10 client.0 _ll_register_callbacks cb 0x7f0038009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f0044e3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778ae4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004563f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ae6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f0044e3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778ae4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48058/0 (socket says 172.21.15.100:48058) 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f0044e3e640 1 -- 172.21.15.100:0/2556481823 learned_addr learned my addr 172.21.15.100:0/2556481823 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f0045e40640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004563f640 1 -- 172.21.15.100:0/2556481823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 msgr2=0x55b778ae5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004563f640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004563f640 1 -- 172.21.15.100:0/2556481823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 msgr2=0x55b778ae4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004563f640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778ae4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f004563f640 1 -- 172.21.15.100:0/2556481823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b778a3c6a0 con 0x55b778ad6250 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.812+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778ae4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004563f640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ae6aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f003c034510 tx=0x7f003c034e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2335700985c7e6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f0028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f0028ff9640 1 -- 172.21.15.100:0/2556481823 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c061070 con 0x55b778ad6250 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b77873d7e0 con 0x55b778ad6250 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f0028ff9640 1 -- 172.21.15.100:0/2556481823 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f003c000e40 con 0x55b778ad6250 2024-03-12T05:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f0028ff9640 1 -- 172.21.15.100:0/2556481823 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f003c004250 con 0x55b778ad6250 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 10 client.25202 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b7789fe8c0 con 0x55b778ad6250 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f0028ff9640 1 -- 172.21.15.100:0/2556481823 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b77873d7e0 con 0x55b778ad6250 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f0028ff9640 1 -- 172.21.15.100:0/2556481823 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f003c064020 con 0x55b778ad6250 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 10 client.25202 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 10 client.25202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 20 client.25202 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778b47380 0x55b778b49770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 4 client.25202 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.813+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7786e9700 con 0x55b778b47380 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778b47380 0x55b778b49770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f0028ff9640 10 client.25202 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f0044e3e640 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778b47380 0x55b778b49770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09a95f967716d16 server_cookie=c2fc22f2e2c78a17 in_seq=0 out_seq=0 2024-03-12T05:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f0028ff9640 1 -- 172.21.15.100:0/2556481823 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7786e9700 con 0x55b778b47380 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f0028ff9640 10 client.25202 handle_command_reply: tid=0 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 client.25202 shutdown 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 msgr2=0x55b778ae6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ae6aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f003c034510 tx=0x7f003c034e70 comp rx=0 tx=0).stop 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 shutdown_connections 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b778b47380 0x55b778b49770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b778adf750 0x55b778ae5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b778ad6b70 0x55b778ae4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f0028ff9640 0 client.25202 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 --2- 172.21.15.100:0/2556481823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b778ad6250 0x55b778ae6aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 >> 172.21.15.100:0/2556481823 conn(0x55b778ad0980 msgr2=0x55b778adac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 shutdown_connections 2024-03-12T05:13:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.814+0000 7f004a470740 1 -- 172.21.15.100:0/2556481823 wait complete. 2024-03-12T05:13:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f004a470740 20 client.25202 trim_cache size 0 max 16384 2024-03-12T05:13:33.820 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f002c0a1cc0 msgr2=0x7f002c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f002c0a1cc0 0x7f002c0a4180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0038021000 tx=0x7f003801f000 comp rx=0 tx=0).stop 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 msgr2=0x7f004019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 0x7f004019fc40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f003000c970 tx=0x7f003000ce30 comp rx=0 tx=0).stop 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 shutdown_connections 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f002c0a1cc0 0x7f002c0a4180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00401998b0 0x7f0040199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00401064b0 0x7f004019fc40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 --2- 172.21.15.100:0/3442654570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00400ffcf0 0x7f004019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 >> 172.21.15.100:0/3442654570 conn(0x7f00400752e0 msgr2=0x7f00400fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 shutdown_connections 2024-03-12T05:13:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:33.815+0000 7f00478a3640 1 -- 172.21.15.100:0/3442654570 wait complete. 2024-03-12T05:13:34.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:33 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3196478621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:13:34.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:33 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3196478621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:13:34.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:33 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3196478621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:13:34.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 43204 13.91 MB/sec warmup 18 sec latency 6.287 ms 2024-03-12T05:13:34.823 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:13:34.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:13:34.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:13:34.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 -- 172.21.15.100:0/3070588806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 msgr2=0x7f8d5010cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3070588806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d5010cf80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009cd0 tx=0x7f8d4802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 -- 172.21.15.100:0/3070588806 shutdown_connections 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3070588806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d5010cf80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3070588806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d50101770 0x7f8d50104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3070588806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 0x7f8d501011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 -- 172.21.15.100:0/3070588806 >> 172.21.15.100:0/3070588806 conn(0x7f8d500753c0 msgr2=0x7f8d500757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 -- 172.21.15.100:0/3070588806 shutdown_connections 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.942+0000 7f8d54d77640 1 -- 172.21.15.100:0/3070588806 wait complete. 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 Processor -- start 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 -- start start 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 0x7f8d501985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d50198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d501990b0 0x7f8d501a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501a3d30 con 0x7f8d50105440 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501a3ea0 con 0x7f8d50100dc0 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.943+0000 7f8d54d77640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501a4060 con 0x7f8d501990b0 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d50198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d50198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48072/0 (socket says 172.21.15.100:48072) 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d4e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 0x7f8d501985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 -- 172.21.15.100:0/3590550912 learned_addr learned my addr 172.21.15.100:0/3590550912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d501990b0 0x7f8d501a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 -- 172.21.15.100:0/3590550912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d501990b0 msgr2=0x7f8d501a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d501990b0 0x7f8d501a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 -- 172.21.15.100:0/3590550912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 msgr2=0x7f8d501985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 0x7f8d501985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 -- 172.21.15.100:0/3590550912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009900 con 0x7f8d50105440 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 0x7f8d501985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d47fff640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d50198b20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9c0 tx=0x7f8d3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d477fe640 1 -- 172.21.15.100:0/3590550912 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380096c0 con 0x7f8d50105440 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d501a4340 con 0x7f8d50105440 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d477fe640 1 -- 172.21.15.100:0/3590550912 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d38005500 con 0x7f8d50105440 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d477fe640 1 -- 172.21.15.100:0/3590550912 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38010430 con 0x7f8d50105440 2024-03-12T05:13:34.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.944+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d501a4800 con 0x7f8d50105440 2024-03-12T05:13:34.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.947+0000 7f8d477fe640 1 -- 172.21.15.100:0/3590550912 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f8d38005020 con 0x7f8d50105440 2024-03-12T05:13:34.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.947+0000 7f8d57a29740 1 Processor -- start 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.947+0000 7f8d57a29740 1 -- start start 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ad5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470ade2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ade8b0 0x55a470ae0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d57a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470a43f90 con 0x55a470ade8b0 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d57a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470a414c0 con 0x55a470ad5380 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d57a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470a3cca0 con 0x55a470ad5d30 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d477fe640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8d3c0a1cc0 0x7f8d3c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ade8b0 0x55a470ae0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ad5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ad5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38122/0 (socket says 172.21.15.100:38122) 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3697678062 learned_addr learned my addr 172.21.15.100:0/3697678062 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:34.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8d3c0a1cc0 0x7f8d3c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470ade2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d477fe640 1 -- 172.21.15.100:0/3590550912 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8d38014070 con 0x7f8d50105440 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3697678062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 msgr2=0x55a470ade2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470ade2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3697678062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ade8b0 msgr2=0x55a470ae0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ade8b0 0x55a470ae0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3697678062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4709ff0b0 con 0x55a470ad5380 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8d3c0a1cc0 0x7f8d3c0a4180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d4802c700 tx=0x7f8d4803c000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.948+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ad5760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d48004af0 tx=0x7f8d48034810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edcc8b1e68787f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d44ff9640 1 -- 172.21.15.100:0/3697678062 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48048070 con 0x55a470ad5380 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d44ff9640 1 -- 172.21.15.100:0/3697678062 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d48032df0 con 0x55a470ad5380 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d44ff9640 1 -- 172.21.15.100:0/3697678062 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004030 con 0x55a470ad5380 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 -- 172.21.15.100:0/3697678062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 msgr2=0x55a470ad5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ad5760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d48004af0 tx=0x7f8d48034810 comp rx=0 tx=0).stop 2024-03-12T05:13:34.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 -- 172.21.15.100:0/3697678062 shutdown_connections 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ade8b0 0x55a470ae0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470ade2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3697678062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ad5760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 -- 172.21.15.100:0/3697678062 >> 172.21.15.100:0/3697678062 conn(0x55a470acf840 msgr2=0x55a470acfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 -- 172.21.15.100:0/3697678062 shutdown_connections 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.949+0000 7f8d57a29740 1 -- 172.21.15.100:0/3697678062 wait complete. 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 -- start start 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 Processor -- start 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 0x55a470adb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470adbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ade8b0 0x55a470adc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4709e1f40 con 0x55a470ad5380 2024-03-12T05:13:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4707104b0 con 0x55a470ade8b0 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4709c2670 con 0x55a470ad5d30 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d57a29740 10 client.0 _ll_register_callbacks cb 0x55a470a2ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 0x55a470adb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 0x55a470adb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48088/0 (socket says 172.21.15.100:48088) 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 -- 172.21.15.100:0/1787869346 learned_addr learned my addr 172.21.15.100:0/1787869346 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ade8b0 0x55a470adc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 -- 172.21.15.100:0/1787869346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 msgr2=0x55a470adbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470adbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 -- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ade8b0 msgr2=0x55a470adc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ade8b0 0x55a470adc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.950+0000 7f8d4e59c640 1 -- 172.21.15.100:0/1787869346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5010d790 con 0x55a470ad5380 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 0x55a470adb850 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8d40012d50 tx=0x7f8d4000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f54c1cd2dacf22a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:34.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d35ffb640 1 -- 172.21.15.100:0/1787869346 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40017070 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d35ffb640 1 -- 172.21.15.100:0/1787869346 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8d40012110 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a470a16f40 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d35ffb640 1 -- 172.21.15.100:0/1787869346 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d4000fe80 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 10 client.15834 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a470a3b320 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d35ffb640 1 -- 172.21.15.100:0/1787869346 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a470a16f40 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d35ffb640 1 -- 172.21.15.100:0/1787869346 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f8d40010430 con 0x55a470ad5380 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 10 client.15834 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 20 client.15834 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470aeef50 0x55a470af1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 4 client.15834 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a470a33310 con 0x55a470aeef50 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.951+0000 7f8d47fff640 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470aeef50 0x55a470af1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.952+0000 7f8d47fff640 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470aeef50 0x55a470af1370 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff6abb01729655f server_cookie=652492e673c01a1f in_seq=0 out_seq=0 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.952+0000 7f8d35ffb640 10 client.15834 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d35ffb640 1 -- 172.21.15.100:0/1787869346 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a470a33310 con 0x55a470aeef50 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d35ffb640 10 client.15834 handle_command_reply: tid=0 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 client.15834 shutdown 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 msgr2=0x55a470adb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 0x55a470adb850 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8d40012d50 tx=0x7f8d4000f790 comp rx=0 tx=0).stop 2024-03-12T05:13:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 shutdown_connections 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470aeef50 0x55a470af1370 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ade8b0 0x55a470adc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d35ffb640 0 client.15834 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5d30 0x55a470adbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 --2- 172.21.15.100:0/1787869346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5380 0x55a470adb850 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 >> 172.21.15.100:0/1787869346 conn(0x55a470acf840 msgr2=0x55a470ad17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 shutdown_connections 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 1 -- 172.21.15.100:0/1787869346 wait complete. 2024-03-12T05:13:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.953+0000 7f8d57a29740 20 client.15834 trim_cache size 0 max 16384 2024-03-12T05:13:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 Processor -- start 2024-03-12T05:13:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 -- start start 2024-03-12T05:13:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470acf840 0x55a470addf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5380 0x55a470adc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470ad8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470ad4640 con 0x55a470ad5d30 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470adcba0 con 0x55a470acf840 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d57a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470af65a0 con 0x55a470ad5380 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470ad8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470ad8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48100/0 (socket says 172.21.15.100:48100) 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3636380042 learned_addr learned my addr 172.21.15.100:0/3636380042 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470acf840 0x55a470addf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d47fff640 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5380 0x55a470adc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3636380042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5380 msgr2=0x55a470adc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5380 0x55a470adc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3636380042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470acf840 msgr2=0x55a470addf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470acf840 0x55a470addf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/3636380042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4709d7c00 con 0x55a470ad5d30 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.969+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470acf840 0x55a470addf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470ad8ab0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8d48004af0 tx=0x7f8d48034f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcc5fe45e276cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d37fff640 1 -- 172.21.15.100:0/3636380042 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48048040 con 0x55a470ad5d30 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d37fff640 1 -- 172.21.15.100:0/3636380042 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d48032510 con 0x55a470ad5d30 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d37fff640 1 -- 172.21.15.100:0/3636380042 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48003970 con 0x55a470ad5d30 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 -- 172.21.15.100:0/3636380042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 msgr2=0x55a470ad8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470ad8ab0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8d48004af0 tx=0x7f8d48034f90 comp rx=0 tx=0).stop 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 -- 172.21.15.100:0/3636380042 shutdown_connections 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470ad8ab0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470ad5380 0x55a470adc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 --2- 172.21.15.100:0/3636380042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470acf840 0x55a470addf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.970+0000 7f8d57a29740 1 -- 172.21.15.100:0/3636380042 >> 172.21.15.100:0/3636380042 conn(0x55a470ad6480 msgr2=0x55a470ad6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 -- 172.21.15.100:0/3636380042 shutdown_connections 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 -- 172.21.15.100:0/3636380042 wait complete. 2024-03-12T05:13:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 -- start start 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 Processor -- start 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470acf840 0x55a470ae4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ae50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470b3f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470b3fa00 con 0x55a470ad5d30 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470b3fb70 con 0x55a470ad5380 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a470b3fd30 con 0x55a470acf840 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d57a29740 10 client.0 _ll_register_callbacks cb 0x7f8d40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ae50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d4ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470b3f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ae50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38150/0 (socket says 172.21.15.100:38150) 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d4ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470b3f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48112/0 (socket says 172.21.15.100:48112) 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d47fff640 1 -- 172.21.15.100:0/2034396985 learned_addr learned my addr 172.21.15.100:0/2034396985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:34.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.971+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470acf840 0x55a470ae4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/2034396985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470acf840 msgr2=0x55a470ae4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470acf840 0x55a470ae4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 msgr2=0x55a470ae50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ae50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d4ed9d640 1 -- 172.21.15.100:0/2034396985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a470a3b320 con 0x55a470ad5d30 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d47fff640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ae50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d4ed9d640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470b3f4c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009430 tx=0x7f8d48005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdaf899c28620a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d35ffb640 1 -- 172.21.15.100:0/2034396985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48048070 con 0x55a470ad5d30 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d35ffb640 1 -- 172.21.15.100:0/2034396985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8d48003970 con 0x55a470ad5d30 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a470a62900 con 0x55a470ad5d30 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d35ffb640 1 -- 172.21.15.100:0/2034396985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d48003b70 con 0x55a470ad5d30 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d57a29740 10 client.15840 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a4709ff0b0 con 0x55a470ad5d30 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.972+0000 7f8d35ffb640 1 -- 172.21.15.100:0/2034396985 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a470a62900 con 0x55a470ad5d30 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d35ffb640 1 -- 172.21.15.100:0/2034396985 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f8d48057420 con 0x55a470ad5d30 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d57a29740 10 client.15840 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d57a29740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d57a29740 20 client.15840 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d57a29740 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470ade8b0 0x55a470b491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d57a29740 4 client.15840 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a47073c860 con 0x55a470ade8b0 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470ade8b0 0x55a470b491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d35ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.973+0000 7f8d4e59c640 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470ade8b0 0x55a470b491d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ee4edcf53e9b73 server_cookie=ffb0a4874d44be14 in_seq=0 out_seq=0 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.974+0000 7f8d35ffb640 1 -- 172.21.15.100:0/2034396985 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a47073c860 con 0x55a470ade8b0 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.974+0000 7f8d35ffb640 10 client.15840 handle_command_reply: tid=0 2024-03-12T05:13:34.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.974+0000 7f8d57a29740 1 client.15840 shutdown 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.974+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 msgr2=0x55a470b3f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.974+0000 7f8d57a29740 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470b3f4c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009430 tx=0x7f8d48005ef0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 shutdown_connections 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a470ade8b0 0x55a470b491d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d35ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a470ad5d30 0x55a470b3f4c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a470ad5380 0x55a470ae50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 --2- 172.21.15.100:0/2034396985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a470acf840 0x55a470ae4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 >> 172.21.15.100:0/2034396985 conn(0x55a470ad6480 msgr2=0x55a470ad0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 shutdown_connections 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 1 -- 172.21.15.100:0/2034396985 wait complete. 2024-03-12T05:13:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d57a29740 20 client.15840 trim_cache size 0 max 16384 2024-03-12T05:13:34.979 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:13:34.979 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:13:34.979 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "ddcc190e-d946-44cf-8c50-92cfc5fa8680", 2024-03-12T05:13:34.979 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:13:34.979 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8d3c0a1cc0 msgr2=0x7f8d3c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8d3c0a1cc0 0x7f8d3c0a4180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d4802c700 tx=0x7f8d4803c000 comp rx=0 tx=0).stop 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 msgr2=0x7f8d50198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d50198b20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9c0 tx=0x7f8d3800ee80 comp rx=0 tx=0).stop 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 shutdown_connections 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8d3c0a1cc0 0x7f8d3c0a4180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d501990b0 0x7f8d501a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d50105440 0x7f8d50198b20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 --2- 172.21.15.100:0/3590550912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d50100dc0 0x7f8d501985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:34.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 >> 172.21.15.100:0/3590550912 conn(0x7f8d500753c0 msgr2=0x7f8d500fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:34.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 shutdown_connections 2024-03-12T05:13:34.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:34.975+0000 7f8d54d77640 1 -- 172.21.15.100:0/3590550912 wait complete. 2024-03-12T05:13:34.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:13:34.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:13:35.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 -- 172.21.15.100:0/2254203996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de41023c0 msgr2=0x7f3de41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 --2- 172.21.15.100:0/2254203996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de41023c0 0x7f3de41027a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 -- 172.21.15.100:0/2254203996 shutdown_connections 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 --2- 172.21.15.100:0/2254203996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de410b700 0x7f3de410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 --2- 172.21.15.100:0/2254203996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3de4102ce0 0x7f3de410b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 --2- 172.21.15.100:0/2254203996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de41023c0 0x7f3de41027a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 -- 172.21.15.100:0/2254203996 >> 172.21.15.100:0/2254203996 conn(0x7f3de4077230 msgr2=0x7f3de4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 -- 172.21.15.100:0/2254203996 shutdown_connections 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 -- 172.21.15.100:0/2254203996 wait complete. 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 Processor -- start 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.094+0000 7f3dec38a640 1 -- start start 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de41023c0 0x7f3de4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 0x7f3de4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3de410b700 0x7f3de419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de419c5e0 con 0x7f3de4102ce0 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de419c750 con 0x7f3de410b700 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de419c910 con 0x7f3de41023c0 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 0x7f3de4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 0x7f3de4197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48130/0 (socket says 172.21.15.100:48130) 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 -- 172.21.15.100:0/814439299 learned_addr learned my addr 172.21.15.100:0/814439299 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dea927640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3de410b700 0x7f3de419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dea126640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de41023c0 0x7f3de4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 -- 172.21.15.100:0/814439299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de41023c0 msgr2=0x7f3de4197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de41023c0 0x7f3de4197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 -- 172.21.15.100:0/814439299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3de410b700 msgr2=0x7f3de419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3de410b700 0x7f3de419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 -- 172.21.15.100:0/814439299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0009560 con 0x7f3de4102ce0 2024-03-12T05:13:35.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dea126640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de41023c0 0x7f3de4197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:13:35.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3de9925640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 0x7f3de4197cb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400e9f0 tx=0x7f3dd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:35.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3ddb7fe640 1 -- 172.21.15.100:0/814439299 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4009700 con 0x7f3de4102ce0 2024-03-12T05:13:35.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de419cbf0 con 0x7f3de4102ce0 2024-03-12T05:13:35.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3ddb7fe640 1 -- 172.21.15.100:0/814439299 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dd400cd80 con 0x7f3de4102ce0 2024-03-12T05:13:35.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3ddb7fe640 1 -- 172.21.15.100:0/814439299 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4018920 con 0x7f3de4102ce0 2024-03-12T05:13:35.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.095+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de41a4470 con 0x7f3de4102ce0 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.098+0000 7f3ddb7fe640 1 -- 172.21.15.100:0/814439299 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f3dd4018a80 con 0x7f3de4102ce0 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.098+0000 7f3ddb7fe640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3dd00a1cc0 0x7f3dd00a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.098+0000 7f3def037740 1 Processor -- start 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3dea126640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3dd00a1cc0 0x7f3dd00a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3ddb7fe640 1 -- 172.21.15.100:0/814439299 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3dd4014070 con 0x7f3de4102ce0 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 -- start start 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974a86280 con 0x55d974b17900 2024-03-12T05:13:35.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974a837b0 con 0x55d974b24bc0 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3def037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974a7ef90 con 0x55d974b16fe0 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3dea126640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3dd00a1cc0 0x7f3dd00a4180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005ec0 tx=0x7f3de0000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48144/0 (socket says 172.21.15.100:48144) 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3dea927640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 -- 172.21.15.100:0/3614965517 learned_addr learned my addr 172.21.15.100:0/3614965517 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3dea126640 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 -- 172.21.15.100:0/3614965517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 msgr2=0x55d974b173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 -- 172.21.15.100:0/3614965517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 msgr2=0x55d974b26fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b26fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3de9925640 1 -- 172.21.15.100:0/3614965517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d974a3d570 con 0x55d974b17900 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.099+0000 7f3dea927640 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.100+0000 7f3de9925640 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b24680 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400e9c0 tx=0x7f3dd409a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b628e1c593e3c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.100+0000 7f3dd8bfe640 1 -- 172.21.15.100:0/3614965517 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40a3050 con 0x55d974b17900 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.100+0000 7f3dd8bfe640 1 -- 172.21.15.100:0/3614965517 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4010b80 con 0x55d974b17900 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.100+0000 7f3dd8bfe640 1 -- 172.21.15.100:0/3614965517 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd409be40 con 0x55d974b17900 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.100+0000 7f3def037740 1 -- 172.21.15.100:0/3614965517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 msgr2=0x55d974b24680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.100+0000 7f3def037740 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b24680 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400e9c0 tx=0x7f3dd409a2d0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- 172.21.15.100:0/3614965517 shutdown_connections 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b24680 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 --2- 172.21.15.100:0/3614965517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- 172.21.15.100:0/3614965517 >> 172.21.15.100:0/3614965517 conn(0x55d974b11990 msgr2=0x55d974b11d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- 172.21.15.100:0/3614965517 shutdown_connections 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- 172.21.15.100:0/3614965517 wait complete. 2024-03-12T05:13:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- start start 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 Processor -- start 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b21a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b21f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974752430 con 0x55d974b17900 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97470ee00 con 0x55d974b24bc0 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974a03fd0 con 0x55d974b16fe0 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3def037740 10 client.0 _ll_register_callbacks cb 0x55d974a59fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3de9925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b21f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3de9925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b21f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48156/0 (socket says 172.21.15.100:48156) 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3de9925640 1 -- 172.21.15.100:0/1852505377 learned_addr learned my addr 172.21.15.100:0/1852505377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3dea927640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.101+0000 7f3dea126640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b21a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3de9925640 1 -- 172.21.15.100:0/1852505377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 msgr2=0x55d974b21a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3de9925640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b21a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3de9925640 1 -- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 msgr2=0x55d974b224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3de9925640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3de9925640 1 -- 172.21.15.100:0/1852505377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de41a4470 con 0x55d974b17900 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3de9925640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b21f80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40cbc20 tx=0x7f3dd409a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901a0dd57c9e9f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3dcdffb640 1 -- 172.21.15.100:0/1852505377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40b33d0 con 0x55d974b17900 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3dcdffb640 1 -- 172.21.15.100:0/1852505377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3dd40a3050 con 0x55d974b17900 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d97477e7e0 con 0x55d974b17900 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3dcdffb640 1 -- 172.21.15.100:0/1852505377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd40cb410 con 0x55d974b17900 2024-03-12T05:13:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 10 client.15852 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d974a7d610 con 0x55d974b17900 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3dcdffb640 1 -- 172.21.15.100:0/1852505377 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d97477e7e0 con 0x55d974b17900 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3dcdffb640 1 -- 172.21.15.100:0/1852505377 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3dd4099770 con 0x55d974b17900 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 10 client.15852 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 20 client.15852 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b34c10 0x55d974b37030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 4 client.15852 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.102+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9747bb720 con 0x55d974b34c10 2024-03-12T05:13:35.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.103+0000 7f3dea126640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b34c10 0x55d974b37030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.103+0000 7f3dcdffb640 10 client.15852 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.103+0000 7f3dea126640 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b34c10 0x55d974b37030 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0040ed3b4c5e97f server_cookie=c9196ee6251d1f40 in_seq=0 out_seq=0 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3dcdffb640 1 -- 172.21.15.100:0/1852505377 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9747bb720 con 0x55d974b34c10 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3dcdffb640 10 client.15852 handle_command_reply: tid=0 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 client.15852 shutdown 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 msgr2=0x55d974b21f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b21f80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40cbc20 tx=0x7f3dd409a340 comp rx=0 tx=0).stop 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 shutdown_connections 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b34c10 0x55d974b37030 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3dcdffb640 0 client.15852 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b17900 0x55d974b21f80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 --2- 172.21.15.100:0/1852505377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b16fe0 0x55d974b21a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 >> 172.21.15.100:0/1852505377 conn(0x55d974b11990 msgr2=0x55d974b13730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:35.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 shutdown_connections 2024-03-12T05:13:35.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 1 -- 172.21.15.100:0/1852505377 wait complete. 2024-03-12T05:13:35.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.104+0000 7f3def037740 20 client.15852 trim_cache size 0 max 16384 2024-03-12T05:13:35.126 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:34 smithi161 ceph-mon[36043]: pgmap v47: 97 pgs: 97 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-03-12T05:13:35.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 Processor -- start 2024-03-12T05:13:35.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 -- start start 2024-03-12T05:13:35.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b24170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b17900 0x55d974b222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b24bc0 0x55d974b208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974b16620 con 0x55d974b16fe0 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974b16c10 con 0x55d974b17900 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3def037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974b0be20 con 0x55d974b24bc0 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b24170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b24170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48172/0 (socket says 172.21.15.100:48172) 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 -- 172.21.15.100:0/8526554 learned_addr learned my addr 172.21.15.100:0/8526554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3de9925640 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b17900 0x55d974b222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea927640 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b24bc0 0x55d974b208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 -- 172.21.15.100:0/8526554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b24bc0 msgr2=0x55d974b208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b24bc0 0x55d974b208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 -- 172.21.15.100:0/8526554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b17900 msgr2=0x55d974b222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b17900 0x55d974b222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.120+0000 7f3dea126640 1 -- 172.21.15.100:0/8526554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d974a19ac0 con 0x55d974b16fe0 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3dea126640 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b24170 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6efd1d239cb8da server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:35.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3dcffff640 1 -- 172.21.15.100:0/8526554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0058070 con 0x55d974b16fe0 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3dcffff640 1 -- 172.21.15.100:0/8526554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de00049f0 con 0x55d974b16fe0 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3dcffff640 1 -- 172.21.15.100:0/8526554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0005030 con 0x55d974b16fe0 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3def037740 1 -- 172.21.15.100:0/8526554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 msgr2=0x55d974b24170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3def037740 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b24170 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de003d000 comp rx=0 tx=0).stop 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3def037740 1 -- 172.21.15.100:0/8526554 shutdown_connections 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3def037740 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b24bc0 0x55d974b208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3def037740 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b17900 0x55d974b222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.121+0000 7f3def037740 1 --2- 172.21.15.100:0/8526554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b24170 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- 172.21.15.100:0/8526554 >> 172.21.15.100:0/8526554 conn(0x55d974b11990 msgr2=0x55d974b201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- 172.21.15.100:0/8526554 shutdown_connections 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- 172.21.15.100:0/8526554 wait complete. 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- start start 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 Processor -- start 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b2a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b17900 0x55d974b2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b2b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974b8b2a0 con 0x55d974b16fe0 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974b8b410 con 0x55d974b24bc0 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d974b8b5d0 con 0x55d974b17900 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3def037740 10 client.0 _ll_register_callbacks cb 0x7f3dd4096430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b2a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b2a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48188/0 (socket says 172.21.15.100:48188) 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 -- 172.21.15.100:0/152842230 learned_addr learned my addr 172.21.15.100:0/152842230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea927640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b2b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3de9925640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b17900 0x55d974b2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 -- 172.21.15.100:0/152842230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b17900 msgr2=0x55d974b2abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b17900 0x55d974b2abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 -- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 msgr2=0x55d974b2b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b2b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3dea126640 1 -- 172.21.15.100:0/152842230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de410e800 con 0x55d974b16fe0 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.122+0000 7f3de9925640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b17900 0x55d974b2abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dea126640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b2a6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005ab0 tx=0x7f3de00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bda97db892438d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dcdffb640 1 -- 172.21.15.100:0/152842230 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0058070 con 0x55d974b16fe0 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd4010b80 con 0x55d974b16fe0 2024-03-12T05:13:35.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dcdffb640 1 -- 172.21.15.100:0/152842230 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3de00035f0 con 0x55d974b16fe0 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dcdffb640 1 -- 172.21.15.100:0/152842230 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de00037b0 con 0x55d974b16fe0 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3def037740 10 client.15858 fetch_fsmap learned FSMap version 24 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d974a3f6a0 con 0x55d974b16fe0 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dcdffb640 1 -- 172.21.15.100:0/152842230 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4010b80 con 0x55d974b16fe0 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.123+0000 7f3dcdffb640 1 -- 172.21.15.100:0/152842230 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3de003c040 con 0x55d974b16fe0 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 10 client.15858 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 20 client.15858 populate_metadata read hostname 'smithi100' 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b8cae0 0x55d974b8ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 4 client.15858 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d97480a890 con 0x55d974b8cae0 2024-03-12T05:13:35.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3de9925640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b8cae0 0x55d974b8ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3dcdffb640 10 client.15858 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3de9925640 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b8cae0 0x55d974b8ef00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c906108c38f6d9 server_cookie=4f058e4daf6e228e in_seq=0 out_seq=0 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3dcdffb640 1 -- 172.21.15.100:0/152842230 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d97480a890 con 0x55d974b8cae0 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3dcdffb640 10 client.15858 handle_command_reply: tid=0 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 client.15858 shutdown 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 msgr2=0x55d974b2a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b2a6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005ab0 tx=0x7f3de00048f0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 shutdown_connections 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d974b8cae0 0x55d974b8ef00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d974b24bc0 0x55d974b2b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.124+0000 7f3def037740 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d974b17900 0x55d974b2abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3def037740 1 --2- 172.21.15.100:0/152842230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d974b16fe0 0x55d974b2a6b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 >> 172.21.15.100:0/152842230 conn(0x55d974b11990 msgr2=0x55d974b20a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dcdffb640 0 client.15858 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 shutdown_connections 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3def037740 1 -- 172.21.15.100:0/152842230 wait complete. 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3def037740 20 client.15858 trim_cache size 0 max 16384 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:13:35.137 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: "ddcc190e-d946-44cf-8c50-92cfc5fa8680": { 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "ddcc190e-d946-44cf-8c50-92cfc5fa8680", 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:13:35.138 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3dd00a1cc0 msgr2=0x7f3dd00a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3dd00a1cc0 0x7f3dd00a4180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005ec0 tx=0x7f3de0000bc0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 msgr2=0x7f3de4197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 0x7f3de4197cb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400e9f0 tx=0x7f3dd400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 shutdown_connections 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3dd00a1cc0 0x7f3dd00a4180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3de410b700 0x7f3de419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de4102ce0 0x7f3de4197cb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 --2- 172.21.15.100:0/814439299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3de41023c0 0x7f3de4197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:13:35.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 >> 172.21.15.100:0/814439299 conn(0x7f3de4077230 msgr2=0x7f3de40fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:13:35.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 shutdown_connections 2024-03-12T05:13:35.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:13:35.125+0000 7f3dec38a640 1 -- 172.21.15.100:0/814439299 wait complete. 2024-03-12T05:13:35.141 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:34 smithi100 ceph-mon[32210]: pgmap v47: 97 pgs: 97 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-03-12T05:13:35.142 INFO:tasks.cephfs.filesystem:scrub status for tag:ddcc190e-d946-44cf-8c50-92cfc5fa8680 - {'path': '/', 'tag': 'ddcc190e-d946-44cf-8c50-92cfc5fa8680', 'options': 'recursive,force'} 2024-03-12T05:13:35.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:34 smithi156 ceph-mon[36270]: pgmap v47: 97 pgs: 97 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-03-12T05:13:35.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 46515 14.19 MB/sec warmup 19 sec latency 9.846 ms 2024-03-12T05:13:36.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:35 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:13:36.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:35 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:35 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:13:36.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:35 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:13:36.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:35 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:35 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:35 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:13:36.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:35 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:35 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:13:36.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:35 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:13:36.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:35 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:35 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:35 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:13:36.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:35 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:35 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:13:36.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:35 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:13:36.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:35 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:35 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:13:36.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 49696 14.35 MB/sec warmup 20 sec latency 30.604 ms 2024-03-12T05:13:37.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:36 smithi156 ceph-mon[36270]: pgmap v48: 97 pgs: 97 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-03-12T05:13:37.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:36 smithi161 ceph-mon[36043]: pgmap v48: 97 pgs: 97 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-03-12T05:13:37.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:36 smithi100 ceph-mon[32210]: pgmap v48: 97 pgs: 97 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-03-12T05:13:37.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 52589 14.31 MB/sec warmup 21 sec latency 10.037 ms 2024-03-12T05:13:38.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:37 smithi156 ceph-mon[36270]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-03-12T05:13:38.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:37 smithi161 ceph-mon[36043]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-03-12T05:13:38.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:37 smithi100 ceph-mon[32210]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-03-12T05:13:38.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 55162 14.24 MB/sec warmup 22 sec latency 15.689 ms 2024-03-12T05:13:39.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:38 smithi156 ceph-mon[36270]: pgmap v49: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-03-12T05:13:39.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:38 smithi161 ceph-mon[36043]: pgmap v49: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-03-12T05:13:39.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:38 smithi100 ceph-mon[32210]: pgmap v49: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-03-12T05:13:39.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 57707 14.25 MB/sec warmup 23 sec latency 30.969 ms 2024-03-12T05:13:40.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 60716 14.42 MB/sec warmup 24 sec latency 44.894 ms 2024-03-12T05:13:41.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:40 smithi156 ceph-mon[36270]: pgmap v50: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:41.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:40 smithi161 ceph-mon[36043]: pgmap v50: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:41.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:40 smithi100 ceph-mon[32210]: pgmap v50: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:41.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 63979 14.55 MB/sec warmup 25 sec latency 20.307 ms 2024-03-12T05:13:42.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 66474 14.35 MB/sec warmup 26 sec latency 6.796 ms 2024-03-12T05:13:43.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:42 smithi156 ceph-mon[36270]: pgmap v51: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:43.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:42 smithi161 ceph-mon[36043]: pgmap v51: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:43.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:42 smithi100 ceph-mon[32210]: pgmap v51: 97 pgs: 97 active+clean; 25 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:43.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 69294 14.43 MB/sec warmup 27 sec latency 8.316 ms 2024-03-12T05:13:44.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 71952 14.44 MB/sec warmup 28 sec latency 21.245 ms 2024-03-12T05:13:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:44 smithi156 ceph-mon[36270]: pgmap v52: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-03-12T05:13:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:44 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:13:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:44 smithi161 ceph-mon[36043]: pgmap v52: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-03-12T05:13:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:44 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:13:45.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:44 smithi100 ceph-mon[32210]: pgmap v52: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-03-12T05:13:45.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:44 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:13:45.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 75083 14.57 MB/sec warmup 29 sec latency 9.835 ms 2024-03-12T05:13:46.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:45 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:13:46.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:45 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:13:46.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:45 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:13:46.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 78290 14.71 MB/sec warmup 30 sec latency 38.031 ms 2024-03-12T05:13:47.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:46 smithi156 ceph-mon[36270]: pgmap v53: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:47.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:46 smithi161 ceph-mon[36043]: pgmap v53: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:47.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:46 smithi100 ceph-mon[32210]: pgmap v53: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-03-12T05:13:47.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 80852 14.60 MB/sec warmup 31 sec latency 38.847 ms 2024-03-12T05:13:48.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 83083 14.56 MB/sec warmup 32 sec latency 39.825 ms 2024-03-12T05:13:49.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:48 smithi156 ceph-mon[36270]: pgmap v54: 97 pgs: 97 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-03-12T05:13:49.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:48 smithi161 ceph-mon[36043]: pgmap v54: 97 pgs: 97 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-03-12T05:13:49.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:48 smithi100 ceph-mon[32210]: pgmap v54: 97 pgs: 97 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-03-12T05:13:49.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 85756 14.54 MB/sec warmup 33 sec latency 8.001 ms 2024-03-12T05:13:50.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 88882 14.64 MB/sec warmup 34 sec latency 12.450 ms 2024-03-12T05:13:51.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:50 smithi156 ceph-mon[36270]: pgmap v55: 97 pgs: 97 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-03-12T05:13:51.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:50 smithi161 ceph-mon[36043]: pgmap v55: 97 pgs: 97 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-03-12T05:13:51.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:50 smithi100 ceph-mon[32210]: pgmap v55: 97 pgs: 97 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-03-12T05:13:51.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1 91333 14.49 MB/sec warmup 35 sec latency 22.796 ms 2024-03-12T05:13:52.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 93721 14.53 MB/sec warmup 36 sec latency 8.113 ms 2024-03-12T05:13:53.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:52 smithi156 ceph-mon[36270]: pgmap v56: 97 pgs: 97 active+clean; 42 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-03-12T05:13:53.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:52 smithi161 ceph-mon[36043]: pgmap v56: 97 pgs: 97 active+clean; 42 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-03-12T05:13:53.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:52 smithi100 ceph-mon[32210]: pgmap v56: 97 pgs: 97 active+clean; 42 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-03-12T05:13:53.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 96361 14.52 MB/sec warmup 37 sec latency 21.614 ms 2024-03-12T05:13:54.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 99480 14.61 MB/sec warmup 38 sec latency 8.986 ms 2024-03-12T05:13:55.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:54 smithi156 ceph-mon[36270]: pgmap v57: 97 pgs: 97 active+clean; 63 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 285 op/s 2024-03-12T05:13:55.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:54 smithi161 ceph-mon[36043]: pgmap v57: 97 pgs: 97 active+clean; 63 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 285 op/s 2024-03-12T05:13:55.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:54 smithi100 ceph-mon[32210]: pgmap v57: 97 pgs: 97 active+clean; 63 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 285 op/s 2024-03-12T05:13:55.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 101931 14.47 MB/sec warmup 39 sec latency 13.298 ms 2024-03-12T05:13:56.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 103998 14.44 MB/sec warmup 40 sec latency 14.396 ms 2024-03-12T05:13:57.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:56 smithi156 ceph-mon[36270]: pgmap v58: 97 pgs: 97 active+clean; 63 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-03-12T05:13:57.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:56 smithi161 ceph-mon[36043]: pgmap v58: 97 pgs: 97 active+clean; 63 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-03-12T05:13:57.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:56 smithi100 ceph-mon[32210]: pgmap v58: 97 pgs: 97 active+clean; 63 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-03-12T05:13:57.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 106885 14.50 MB/sec warmup 41 sec latency 12.544 ms 2024-03-12T05:13:58.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 109425 14.46 MB/sec warmup 42 sec latency 82.701 ms 2024-03-12T05:13:59.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:13:58 smithi156 ceph-mon[36270]: pgmap v59: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 305 op/s 2024-03-12T05:13:59.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:13:58 smithi161 ceph-mon[36043]: pgmap v59: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 305 op/s 2024-03-12T05:13:59.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:13:58 smithi100 ceph-mon[32210]: pgmap v59: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 305 op/s 2024-03-12T05:13:59.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 111695 14.40 MB/sec warmup 43 sec latency 17.460 ms 2024-03-12T05:14:00.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 113923 14.38 MB/sec warmup 44 sec latency 23.176 ms 2024-03-12T05:14:01.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:00 smithi156 ceph-mon[36270]: pgmap v60: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 214 op/s 2024-03-12T05:14:01.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:00 smithi161 ceph-mon[36043]: pgmap v60: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 214 op/s 2024-03-12T05:14:01.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:00 smithi100 ceph-mon[32210]: pgmap v60: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 214 op/s 2024-03-12T05:14:01.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 116749 14.36 MB/sec warmup 45 sec latency 14.668 ms 2024-03-12T05:14:02.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 118866 14.29 MB/sec warmup 46 sec latency 16.073 ms 2024-03-12T05:14:03.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:02 smithi156 ceph-mon[36270]: pgmap v61: 97 pgs: 97 active+clean; 72 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-03-12T05:14:03.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:02 smithi161 ceph-mon[36043]: pgmap v61: 97 pgs: 97 active+clean; 72 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-03-12T05:14:03.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:02 smithi100 ceph-mon[32210]: pgmap v61: 97 pgs: 97 active+clean; 72 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-03-12T05:14:03.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 121139 14.28 MB/sec warmup 47 sec latency 59.098 ms 2024-03-12T05:14:04.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 124189 14.35 MB/sec warmup 48 sec latency 19.077 ms 2024-03-12T05:14:05.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:14:05.143 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:14:05.167 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:04 smithi100 ceph-mon[32210]: pgmap v62: 97 pgs: 97 active+clean; 73 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 358 op/s 2024-03-12T05:14:05.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:04 smithi156 ceph-mon[36270]: pgmap v62: 97 pgs: 97 active+clean; 73 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 358 op/s 2024-03-12T05:14:05.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:04 smithi161 ceph-mon[36043]: pgmap v62: 97 pgs: 97 active+clean; 73 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 358 op/s 2024-03-12T05:14:05.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- 172.21.15.100:0/2883114793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 msgr2=0x7f08e810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/2883114793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e810cf10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009930 tx=0x7f08d802bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:05.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- 172.21.15.100:0/2883114793 shutdown_connections 2024-03-12T05:14:05.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/2883114793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e810cf10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/2883114793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100c20 0x7f08e8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/2883114793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8100270 0x7f08e8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- 172.21.15.100:0/2883114793 >> 172.21.15.100:0/2883114793 conn(0x7f08e8077230 msgr2=0x7f08e8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- 172.21.15.100:0/2883114793 shutdown_connections 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- 172.21.15.100:0/2883114793 wait complete. 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 Processor -- start 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- start start 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100270 0x7f08e81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e819bfa0 0x7f08e819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e819c990 con 0x7f08e8105340 2024-03-12T05:14:05.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e819cb00 con 0x7f08e8100270 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.260+0000 7f08ef7f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e819cc70 con 0x7f08e819bfa0 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ed590640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100270 0x7f08e81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41750/0 (socket says 172.21.15.100:41750) 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 -- 172.21.15.100:0/3867176265 learned_addr learned my addr 172.21.15.100:0/3867176265 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08edd91640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e819bfa0 0x7f08e819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 -- 172.21.15.100:0/3867176265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e819bfa0 msgr2=0x7f08e819c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e819bfa0 0x7f08e819c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 -- 172.21.15.100:0/3867176265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100270 msgr2=0x7f08e81976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100270 0x7f08e81976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 -- 172.21.15.100:0/3867176265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8009560 con 0x7f08e8105340 2024-03-12T05:14:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ed590640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100270 0x7f08e81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e8197c20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e9c0 tx=0x7f08dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08e67fc640 1 -- 172.21.15.100:0/3867176265 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc009700 con 0x7f08e8105340 2024-03-12T05:14:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08e67fc640 1 -- 172.21.15.100:0/3867176265 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08dc00cd80 con 0x7f08e8105340 2024-03-12T05:14:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e81a4250 con 0x7f08e8105340 2024-03-12T05:14:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08e67fc640 1 -- 172.21.15.100:0/3867176265 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc010430 con 0x7f08e8105340 2024-03-12T05:14:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.261+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e81a4700 con 0x7f08e8105340 2024-03-12T05:14:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.263+0000 7f08e67fc640 1 -- 172.21.15.100:0/3867176265 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f08dc005000 con 0x7f08e8105340 2024-03-12T05:14:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 Processor -- start 2024-03-12T05:14:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 -- start start 2024-03-12T05:14:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08e67fc640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f08d00a1ba0 0x7f08d00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 0x56220b5c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b534280 con 0x56220b5d2bc0 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b5317b0 con 0x56220b5c4fe0 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08f23c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b52cf90 con 0x56220b5c5900 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08e67fc640 1 -- 172.21.15.100:0/3867176265 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f08dc014070 con 0x7f08e8105340 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08edd91640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f08d00a1ba0 0x7f08d00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08ed590640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 0x56220b5c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08edd91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08ed590640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 0x56220b5c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48368/0 (socket says 172.21.15.100:48368) 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08ed590640 1 -- 172.21.15.100:0/387973871 learned_addr learned my addr 172.21.15.100:0/387973871 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.264+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08ed590640 1 -- 172.21.15.100:0/387973871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 msgr2=0x56220b5d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08ed590640 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08ed590640 1 -- 172.21.15.100:0/387973871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 msgr2=0x56220b5d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08ed590640 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08ed590640 1 -- 172.21.15.100:0/387973871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56220b4eb570 con 0x56220b5c4fe0 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08edd91640 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08edd91640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f08d00a1ba0 0x7f08d00a4060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006010 tx=0x7f08d80047d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08ed590640 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 0x56220b5c53c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f08e000f6e0 tx=0x7f08e000fba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c734d6821f0de467 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08cf7fe640 1 -- 172.21.15.100:0/387973871 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0007e70 con 0x56220b5c4fe0 2024-03-12T05:14:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08cf7fe640 1 -- 172.21.15.100:0/387973871 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08e000de00 con 0x56220b5c4fe0 2024-03-12T05:14:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.265+0000 7f08cf7fe640 1 -- 172.21.15.100:0/387973871 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00058c0 con 0x56220b5c4fe0 2024-03-12T05:14:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 -- 172.21.15.100:0/387973871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 msgr2=0x56220b5c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 0x56220b5c53c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f08e000f6e0 tx=0x7f08e000fba0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 -- 172.21.15.100:0/387973871 shutdown_connections 2024-03-12T05:14:05.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 --2- 172.21.15.100:0/387973871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c4fe0 0x56220b5c53c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 -- 172.21.15.100:0/387973871 >> 172.21.15.100:0/387973871 conn(0x56220b5bf990 msgr2=0x56220b5bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 -- 172.21.15.100:0/387973871 shutdown_connections 2024-03-12T05:14:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 -- 172.21.15.100:0/387973871 wait complete. 2024-03-12T05:14:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 -- start start 2024-03-12T05:14:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 Processor -- start 2024-03-12T05:14:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.266+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 0x56220b5d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 0x56220b5cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b200430 con 0x56220b5c4fe0 2024-03-12T05:14:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b1bce00 con 0x56220b5d2bc0 2024-03-12T05:14:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b4b1fd0 con 0x56220b5c5900 2024-03-12T05:14:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 10 client.0 _ll_register_callbacks cb 0x56220b507fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ed590640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 0x56220b5d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 0x56220b5cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ed590640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 0x56220b5d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41762/0 (socket says 172.21.15.100:41762) 2024-03-12T05:14:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 0x56220b5cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48370/0 (socket says 172.21.15.100:48370) 2024-03-12T05:14:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ed590640 1 -- 172.21.15.100:0/2146971171 learned_addr learned my addr 172.21.15.100:0/2146971171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 -- 172.21.15.100:0/2146971171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 msgr2=0x56220b5cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 -- 172.21.15.100:0/2146971171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 msgr2=0x56220b5d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 0x56220b5d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 -- 172.21.15.100:0/2146971171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e000f3c0 con 0x56220b5d2bc0 2024-03-12T05:14:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ed590640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 0x56220b5d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08edd91640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 0x56220b5cff40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f08d802c170 tx=0x7f08d8046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a415a3cb0114ef2e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:14:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ccff9640 1 -- 172.21.15.100:0/2146971171 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80044f0 con 0x56220b5d2bc0 2024-03-12T05:14:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56220b22c7e0 con 0x56220b5d2bc0 2024-03-12T05:14:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ccff9640 1 -- 172.21.15.100:0/2146971171 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f08d80035f0 con 0x56220b5d2bc0 2024-03-12T05:14:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08ccff9640 1 -- 172.21.15.100:0/2146971171 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d80037b0 con 0x56220b5d2bc0 2024-03-12T05:14:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 10 client.25250 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.267+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56220b52b610 con 0x56220b5d2bc0 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08ccff9640 1 -- 172.21.15.100:0/2146971171 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08d805f080 con 0x56220b5d2bc0 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08ccff9640 1 -- 172.21.15.100:0/2146971171 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f08d8043a30 con 0x56220b5d2bc0 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08f23c6740 10 client.25250 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08f23c6740 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08f23c6740 20 client.25250 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b5e2d20 0x56220b5e5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08f23c6740 4 client.25250 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56220b269720 con 0x56220b5e2d20 2024-03-12T05:14:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08ed590640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b5e2d20 0x56220b5e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08ccff9640 10 client.25250 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.268+0000 7f08ed590640 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b5e2d20 0x56220b5e5110 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bec95c3430e339 server_cookie=a9100b3d1466313d in_seq=0 out_seq=0 2024-03-12T05:14:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08ccff9640 1 -- 172.21.15.100:0/2146971171 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56220b269720 con 0x56220b5e2d20 2024-03-12T05:14:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08ccff9640 10 client.25250 handle_command_reply: tid=0 2024-03-12T05:14:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 client.25250 shutdown 2024-03-12T05:14:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 msgr2=0x56220b5cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 0x56220b5cff40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f08d802c170 tx=0x7f08d8046040 comp rx=0 tx=0).stop 2024-03-12T05:14:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 shutdown_connections 2024-03-12T05:14:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b5e2d20 0x56220b5e5110 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5d2bc0 0x56220b5cff40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c5900 0x56220b5cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2146971171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5c4fe0 0x56220b5d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08ccff9640 0 client.25250 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.269+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 >> 172.21.15.100:0/2146971171 conn(0x56220b5bf990 msgr2=0x56220b5c1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.270+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 shutdown_connections 2024-03-12T05:14:05.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.270+0000 7f08f23c6740 1 -- 172.21.15.100:0/2146971171 wait complete. 2024-03-12T05:14:05.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.270+0000 7f08f23c6740 20 client.25250 trim_cache size 0 max 16384 2024-03-12T05:14:05.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.285+0000 7f08f23c6740 1 Processor -- start 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.285+0000 7f08f23c6740 1 -- start start 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5c1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08f23c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b269720 con 0x56220b5d2bc0 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08f23c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b5c5e00 con 0x56220b5c5900 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08f23c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b5b9eb0 con 0x56220b5c4fe0 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41772/0 (socket says 172.21.15.100:41772) 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 -- 172.21.15.100:0/2102253884 learned_addr learned my addr 172.21.15.100:0/2102253884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08ed590640 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 -- 172.21.15.100:0/2102253884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 msgr2=0x56220b5ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 -- 172.21.15.100:0/2102253884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 msgr2=0x56220b5c1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5c1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 -- 172.21.15.100:0/2102253884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56220b4c7ac0 con 0x56220b5d2bc0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08ed590640 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08edd91640 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d2060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f08d8005cf0 tx=0x7f08d8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c9f3f77c387ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08ceffd640 1 -- 172.21.15.100:0/2102253884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d802c8e0 con 0x56220b5d2bc0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08ceffd640 1 -- 172.21.15.100:0/2102253884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56220b22c7e0 con 0x56220b5d2bc0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.286+0000 7f08ceffd640 1 -- 172.21.15.100:0/2102253884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8003830 con 0x56220b5d2bc0 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 -- 172.21.15.100:0/2102253884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 msgr2=0x56220b5d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d2060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f08d8005cf0 tx=0x7f08d8005a40 comp rx=0 tx=0).stop 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 -- 172.21.15.100:0/2102253884 shutdown_connections 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d2060 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 --2- 172.21.15.100:0/2102253884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 -- 172.21.15.100:0/2102253884 >> 172.21.15.100:0/2102253884 conn(0x56220b5bf990 msgr2=0x56220b5c0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 -- 172.21.15.100:0/2102253884 shutdown_connections 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 -- 172.21.15.100:0/2102253884 wait complete. 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 -- start start 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.287+0000 7f08f23c6740 1 Processor -- start 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5d66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b639330 con 0x56220b5d2bc0 2024-03-12T05:14:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b6394a0 con 0x56220b5c5900 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220b639660 con 0x56220b5c4fe0 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08f23c6740 10 client.0 _ll_register_callbacks cb 0x7f08e0007dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41774/0 (socket says 172.21.15.100:41774) 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 -- 172.21.15.100:0/190619331 learned_addr learned my addr 172.21.15.100:0/190619331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08ed590640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5d66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08ecd8f640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 -- 172.21.15.100:0/190619331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 msgr2=0x56220b5d66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5d66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 -- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 msgr2=0x56220b5d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5d6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 -- 172.21.15.100:0/190619331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56220b52b610 con 0x56220b5d2bc0 2024-03-12T05:14:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08edd91640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d7140 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f08d80437c0 tx=0x7f08d802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98df4ca7f4c888ea server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.288+0000 7f08ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08ccff9640 1 -- 172.21.15.100:0/190619331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80051c0 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08ccff9640 1 -- 172.21.15.100:0/190619331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f08d802c8e0 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08e000de00 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08ccff9640 1 -- 172.21.15.100:0/190619331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d80438a0 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 10 client.15876 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56220b4ed6a0 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08ccff9640 1 -- 172.21.15.100:0/190619331 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08e000de00 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08ccff9640 1 -- 172.21.15.100:0/190619331 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f08d8034440 con 0x56220b5d2bc0 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 10 client.15876 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 20 client.15876 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b63a900 0x56220b63cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 4 client.15876 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56220b2b8890 con 0x56220b63a900 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.289+0000 7f08ed590640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b63a900 0x56220b63cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08ccff9640 10 client.15876 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08ed590640 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b63a900 0x56220b63cd20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b401f1a6073a91d1 server_cookie=415ed7c4edd535f3 in_seq=0 out_seq=0 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08ccff9640 1 -- 172.21.15.100:0/190619331 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56220b2b8890 con 0x56220b63a900 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08ccff9640 10 client.15876 handle_command_reply: tid=0 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 client.15876 shutdown 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 msgr2=0x56220b5d7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d7140 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f08d80437c0 tx=0x7f08d802bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 shutdown_connections 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56220b63a900 0x56220b63cd20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56220b5d2bc0 0x56220b5d7140 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56220b5c5900 0x56220b5d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 --2- 172.21.15.100:0/190619331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56220b5c4fe0 0x56220b5d66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08ccff9640 0 client.15876 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 >> 172.21.15.100:0/190619331 conn(0x56220b5bf990 msgr2=0x56220b5c0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 shutdown_connections 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 1 -- 172.21.15.100:0/190619331 wait complete. 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.290+0000 7f08f23c6740 20 client.15876 trim_cache size 0 max 16384 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:14:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:14:05.301 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:14:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f08d00a1ba0 msgr2=0x7f08d00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f08d00a1ba0 0x7f08d00a4060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006010 tx=0x7f08d80047d0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 msgr2=0x7f08e8197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e8197c20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e9c0 tx=0x7f08dc00ee80 comp rx=0 tx=0).stop 2024-03-12T05:14:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 shutdown_connections 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f08d00a1ba0 0x7f08d00a4060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e819bfa0 0x7f08e819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08e8105340 0x7f08e8197c20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 --2- 172.21.15.100:0/3867176265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e8100270 0x7f08e81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 >> 172.21.15.100:0/3867176265 conn(0x7f08e8077230 msgr2=0x7f08e80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 shutdown_connections 2024-03-12T05:14:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.291+0000 7f08ef7f4640 1 -- 172.21.15.100:0/3867176265 wait complete. 2024-03-12T05:14:05.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:14:05.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:14:05.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.456+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/3744210898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c8102380 msgr2=0x7fd1c8102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.456+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/3744210898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c8102380 0x7fd1c8102760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009900 tx=0x7fd1b802bcd0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/3744210898 shutdown_connections 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/3744210898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c810b750 0x7fd1c810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/3744210898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c8102ca0 0x7fd1c810b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/3744210898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c8102380 0x7fd1c8102760 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/3744210898 >> 172.21.15.100:0/3744210898 conn(0x7fd1c8076e30 msgr2=0x7fd1c80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/3744210898 shutdown_connections 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/3744210898 wait complete. 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 Processor -- start 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- start start 2024-03-12T05:14:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c8102380 0x7fd1c819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c8102ca0 0x7fd1c819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 0x7fd1c8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8197c50 con 0x7fd1c810b750 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8197dc0 con 0x7fd1c8102380 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.457+0000 7fd1cd2f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8197f80 con 0x7fd1c8102ca0 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 0x7fd1c8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 0x7fd1c8197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41792/0 (socket says 172.21.15.100:41792) 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 -- 172.21.15.100:0/2992285404 learned_addr learned my addr 172.21.15.100:0/2992285404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c6ffd640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c8102380 0x7fd1c819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1be7fc640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c8102ca0 0x7fd1c819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 -- 172.21.15.100:0/2992285404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c8102ca0 msgr2=0x7fd1c819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c8102ca0 0x7fd1c819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 -- 172.21.15.100:0/2992285404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c8102380 msgr2=0x7fd1c819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c8102380 0x7fd1c819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 -- 172.21.15.100:0/2992285404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009560 con 0x7fd1c810b750 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c77fe640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 0x7fd1c8197600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000eb10 tx=0x7fd1c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0004b20 con 0x7fd1c810b750 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1c0004c80 con 0x7fd1c810b750 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c8198260 con 0x7fd1c810b750 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0010650 con 0x7fd1c810b750 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.458+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c81a44f0 con 0x7fd1c810b750 2024-03-12T05:14:05.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.459+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c80676e0 con 0x7fd1c810b750 2024-03-12T05:14:05.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.460+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd1c0005020 con 0x7fd1c810b750 2024-03-12T05:14:05.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.461+0000 7fd1c4ff9640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1ba0 0x7fd1b40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.461+0000 7fd1c6ffd640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1ba0 0x7fd1b40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.461+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd1c0014070 con 0x7fd1c810b750 2024-03-12T05:14:05.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.462+0000 7fd1c6ffd640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1ba0 0x7fd1b40a4060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b8005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.466+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1c0096c00 con 0x7fd1c810b750 2024-03-12T05:14:05.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.698+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1c8198b30 con 0x7fd1c810b750 2024-03-12T05:14:05.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.699+0000 7fd1c4ff9640 1 -- 172.21.15.100:0/2992285404 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fd1c009a2c0 con 0x7fd1c810b750 2024-03-12T05:14:05.696 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:14:05.696 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:13:19.700942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:14:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 24 2024-03-12T05:14:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1ba0 msgr2=0x7fd1b40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1ba0 0x7fd1b40a4060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b8005e40 comp rx=0 tx=0).stop 2024-03-12T05:14:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 msgr2=0x7fd1c8197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 0x7fd1c8197600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000eb10 tx=0x7fd1c000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 shutdown_connections 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1ba0 0x7fd1b40a4060 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c810b750 0x7fd1c8197600 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c8102ca0 0x7fd1c819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 --2- 172.21.15.100:0/2992285404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c8102380 0x7fd1c819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 >> 172.21.15.100:0/2992285404 conn(0x7fd1c8076e30 msgr2=0x7fd1c80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 shutdown_connections 2024-03-12T05:14:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.701+0000 7fd1cd2f1640 1 -- 172.21.15.100:0/2992285404 wait complete. 2024-03-12T05:14:05.709 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:14:05.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:14:05.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 126564 14.23 MB/sec warmup 49 sec latency 29.110 ms 2024-03-12T05:14:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 -- 172.21.15.100:0/4249741531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 msgr2=0x7fd574100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/4249741531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd574100640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd570009930 tx=0x7fd57002bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 -- 172.21.15.100:0/4249741531 shutdown_connections 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/4249741531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105330 0x7fd57410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/4249741531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd574100c10 0x7fd574104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/4249741531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd574100640 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 -- 172.21.15.100:0/4249741531 >> 172.21.15.100:0/4249741531 conn(0x7fd574077180 msgr2=0x7fd574077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 -- 172.21.15.100:0/4249741531 shutdown_connections 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.821+0000 7fd57c5c9640 1 -- 172.21.15.100:0/4249741531 wait complete. 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 Processor -- start 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 -- start start 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd57419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105330 0x7fd57419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5741975d0 0x7fd574197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574197fc0 con 0x7fd574100260 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574198130 con 0x7fd5741975d0 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57c5c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5741982f0 con 0x7fd574105330 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd57419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd57419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41820/0 (socket says 172.21.15.100:41820) 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 -- 172.21.15.100:0/1572014329 learned_addr learned my addr 172.21.15.100:0/1572014329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd579b64640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105330 0x7fd57419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57ab66640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5741975d0 0x7fd574197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 -- 172.21.15.100:0/1572014329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105330 msgr2=0x7fd57419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105330 0x7fd57419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 -- 172.21.15.100:0/1572014329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5741975d0 msgr2=0x7fd574197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5741975d0 0x7fd574197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.822+0000 7fd57a365640 1 -- 172.21.15.100:0/1572014329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564009490 con 0x7fd574100260 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.823+0000 7fd57a365640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd57419d420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd570009230 tx=0x7fd57002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.823+0000 7fd56b7fe640 1 -- 172.21.15.100:0/1572014329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570004350 con 0x7fd574100260 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.823+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd570009560 con 0x7fd574100260 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.823+0000 7fd56b7fe640 1 -- 172.21.15.100:0/1572014329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5700044b0 con 0x7fd574100260 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.823+0000 7fd56b7fe640 1 -- 172.21.15.100:0/1572014329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570003d00 con 0x7fd574100260 2024-03-12T05:14:05.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.823+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5741a4600 con 0x7fd574100260 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.825+0000 7fd56b7fe640 1 -- 172.21.15.100:0/1572014329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd570004df0 con 0x7fd574100260 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.825+0000 7fd56b7fe640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd5600a1ba0 0x7fd5600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.825+0000 7fd57f19f740 1 Processor -- start 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd579b64640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd5600a1ba0 0x7fd5600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd56b7fe640 1 -- 172.21.15.100:0/1572014329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd5700ef670 con 0x7fd574100260 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 -- start start 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 0x55de3fc31630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fbec960 con 0x55de3fc31250 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fba0310 con 0x55de3fc3a750 2024-03-12T05:14:05.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57f19f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fb9d840 con 0x55de3fc31b70 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57a365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 0x55de3fc31630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57a365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 0x55de3fc31630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41826/0 (socket says 172.21.15.100:41826) 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57a365640 1 -- 172.21.15.100:0/974077476 learned_addr learned my addr 172.21.15.100:0/974077476 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd579b64640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd5600a1ba0 0x7fd5600a4060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5740ffb70 tx=0x7fd564005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57ab66640 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd579b64640 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57ab66640 1 -- 172.21.15.100:0/974077476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 msgr2=0x55de3fc3a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57ab66640 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc3a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57ab66640 1 -- 172.21.15.100:0/974077476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 msgr2=0x55de3fc31630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57ab66640 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 0x55de3fc31630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57ab66640 1 -- 172.21.15.100:0/974077476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de3fb5b060 con 0x55de3fc3a750 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.826+0000 7fd57a365640 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 0x55de3fc31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.827+0000 7fd57ab66640 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc3cb40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009f00 tx=0x7fd56c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15f704c1c101d19 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.827+0000 7fd568bfe640 1 -- 172.21.15.100:0/974077476 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004d20 con 0x55de3fc3a750 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.827+0000 7fd568bfe640 1 -- 172.21.15.100:0/974077476 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd56c004e80 con 0x55de3fc3a750 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.827+0000 7fd568bfe640 1 -- 172.21.15.100:0/974077476 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c005280 con 0x55de3fc3a750 2024-03-12T05:14:05.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- 172.21.15.100:0/974077476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 msgr2=0x55de3fc3cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc3cb40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009f00 tx=0x7fd56c011810 comp rx=0 tx=0).stop 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- 172.21.15.100:0/974077476 shutdown_connections 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc3cb40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- 172.21.15.100:0/974077476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31250 0x55de3fc31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- 172.21.15.100:0/974077476 >> 172.21.15.100:0/974077476 conn(0x55de3fc2b980 msgr2=0x55de3fc2bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- 172.21.15.100:0/974077476 shutdown_connections 2024-03-12T05:14:05.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- 172.21.15.100:0/974077476 wait complete. 2024-03-12T05:14:05.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- start start 2024-03-12T05:14:05.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 Processor -- start 2024-03-12T05:14:05.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31250 0x55de3fc37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fb99020 con 0x55de3fc31b70 2024-03-12T05:14:05.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f86c430 con 0x55de3fc3a750 2024-03-12T05:14:05.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f828e00 con 0x55de3fc31250 2024-03-12T05:14:05.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd57f19f740 10 client.0 _ll_register_callbacks cb 0x55de3fb40cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:05.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd579b64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.828+0000 7fd579b64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc37ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41836/0 (socket says 172.21.15.100:41836) 2024-03-12T05:14:05.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 -- 172.21.15.100:0/1684155895 learned_addr learned my addr 172.21.15.100:0/1684155895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd57ab66640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd57a365640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31250 0x55de3fc37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 -- 172.21.15.100:0/1684155895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31250 msgr2=0x55de3fc37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31250 0x55de3fc37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 -- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 msgr2=0x55de3fc381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 -- 172.21.15.100:0/1684155895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c009be0 con 0x55de3fc31b70 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd57ab66640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd579b64640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc37ca0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd56401db70 tx=0x7fd564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1bfa13250fbbae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd55dffb640 1 -- 172.21.15.100:0/1684155895 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564022070 con 0x55de3fc31b70 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd55dffb640 1 -- 172.21.15.100:0/1684155895 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd564004030 con 0x55de3fc31b70 2024-03-12T05:14:05.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de3f8d5720 con 0x55de3fc31b70 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd55dffb640 1 -- 172.21.15.100:0/1684155895 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5640041f0 con 0x55de3fc31b70 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd57f19f740 10 client.15894 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.829+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55de3fb976a0 con 0x55de3fc31b70 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd55dffb640 1 -- 172.21.15.100:0/1684155895 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55de3f8d5720 con 0x55de3fc31b70 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd55dffb640 1 -- 172.21.15.100:0/1684155895 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd5640038f0 con 0x55de3fc31b70 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57f19f740 10 client.15894 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57f19f740 10 client.15894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:14:05.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57f19f740 20 client.15894 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57f19f740 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fc4ac20 0x55de3fc4d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57f19f740 4 client.15894 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de3f924890 con 0x55de3fc4ac20 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57a365640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fc4ac20 0x55de3fc4d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd57a365640 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fc4ac20 0x55de3fc4d040 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2401a50b3feefe69 server_cookie=855296b5cd00c543 in_seq=0 out_seq=0 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.830+0000 7fd55dffb640 10 client.15894 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd55dffb640 1 -- 172.21.15.100:0/1684155895 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de3f924890 con 0x55de3fc4ac20 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd55dffb640 10 client.15894 handle_command_reply: tid=0 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 client.15894 shutdown 2024-03-12T05:14:05.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 msgr2=0x55de3fc37ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc37ca0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd56401db70 tx=0x7fd564004730 comp rx=0 tx=0).stop 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 shutdown_connections 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fc4ac20 0x55de3fc4d040 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc3a750 0x55de3fc381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd55dffb640 0 client.15894 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc37ca0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 --2- 172.21.15.100:0/1684155895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31250 0x55de3fc37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 >> 172.21.15.100:0/1684155895 conn(0x55de3fc2b980 msgr2=0x55de3fc2d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 shutdown_connections 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 1 -- 172.21.15.100:0/1684155895 wait complete. 2024-03-12T05:14:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.831+0000 7fd57f19f740 20 client.15894 trim_cache size 0 max 16384 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.843+0000 7fd57f19f740 1 Processor -- start 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 -- start start 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc39e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc38000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc3a750 0x55de3fc349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fb1e130 con 0x55de3fc31b70 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fc30c90 con 0x55de3fc31250 2024-03-12T05:14:05.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57f19f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fc2c580 con 0x55de3fc3a750 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc38000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57ab66640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc3a750 0x55de3fc349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc38000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41850/0 (socket says 172.21.15.100:41850) 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd57a365640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc39e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 -- 172.21.15.100:0/3189088094 learned_addr learned my addr 172.21.15.100:0/3189088094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 -- 172.21.15.100:0/3189088094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc3a750 msgr2=0x55de3fc349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc3a750 0x55de3fc349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 -- 172.21.15.100:0/3189088094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 msgr2=0x55de3fc39e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc39e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 -- 172.21.15.100:0/3189088094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de3fb33c20 con 0x55de3fc31b70 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.844+0000 7fd579b64640 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc38000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd564016040 tx=0x7fd56401efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7488357e62c0f1d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd55ffff640 1 -- 172.21.15.100:0/3189088094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564022070 con 0x55de3fc31b70 2024-03-12T05:14:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd55ffff640 1 -- 172.21.15.100:0/3189088094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd56401d720 con 0x55de3fc31b70 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd55ffff640 1 -- 172.21.15.100:0/3189088094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56401cd00 con 0x55de3fc31b70 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 -- 172.21.15.100:0/3189088094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 msgr2=0x55de3fc38000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc38000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd564016040 tx=0x7fd56401efc0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 -- 172.21.15.100:0/3189088094 shutdown_connections 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc3a750 0x55de3fc349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc31b70 0x55de3fc38000 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 --2- 172.21.15.100:0/3189088094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc39e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 -- 172.21.15.100:0/3189088094 >> 172.21.15.100:0/3189088094 conn(0x55de3fc2b980 msgr2=0x55de3fc32a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 -- 172.21.15.100:0/3189088094 shutdown_connections 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.845+0000 7fd57f19f740 1 -- 172.21.15.100:0/3189088094 wait complete. 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 -- start start 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 Processor -- start 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc40160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 0x55de3fc40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fca11b0 con 0x55de3fc3a750 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fca1320 con 0x55de3fc31250 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3fca14e0 con 0x55de3fc31b70 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57f19f740 10 client.0 _ll_register_callbacks cb 0x7fd56c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 0x55de3fc40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 0x55de3fc40be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41864/0 (socket says 172.21.15.100:41864) 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 -- 172.21.15.100:0/995478649 learned_addr learned my addr 172.21.15.100:0/995478649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57a365640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc40160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd579b64640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 -- 172.21.15.100:0/995478649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 msgr2=0x55de3fc406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 -- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 msgr2=0x55de3fc40160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc40160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.846+0000 7fd57ab66640 1 -- 172.21.15.100:0/995478649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de3fb976a0 con 0x55de3fc3a750 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57ab66640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 0x55de3fc40be0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55de3fc2cb10 tx=0x7fd56c004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309daa6cbbc97e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd55dffb640 1 -- 172.21.15.100:0/995478649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004bf0 con 0x55de3fc3a750 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd55dffb640 1 -- 172.21.15.100:0/995478649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd56c005000 con 0x55de3fc3a750 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de3f8987e0 con 0x55de3fc3a750 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd55dffb640 1 -- 172.21.15.100:0/995478649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd56c0051c0 con 0x55de3fc3a750 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 10 client.15900 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55de3fb598c0 con 0x55de3fc3a750 2024-03-12T05:14:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd55dffb640 1 -- 172.21.15.100:0/995478649 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55de3f8987e0 con 0x55de3fc3a750 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd55dffb640 1 -- 172.21.15.100:0/995478649 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd56c021040 con 0x55de3fc3a750 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 10 client.15900 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 10 client.15900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 20 client.15900 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fca2cd0 0x55de3fca50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 4 client.15900 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de3f844700 con 0x55de3fca2cd0 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.847+0000 7fd57a365640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fca2cd0 0x55de3fca50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd55dffb640 10 client.15900 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57a365640 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fca2cd0 0x55de3fca50f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=494e972868158f3 server_cookie=43424ff6e0178c57 in_seq=0 out_seq=0 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd55dffb640 1 -- 172.21.15.100:0/995478649 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de3f844700 con 0x55de3fca2cd0 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd55dffb640 10 client.15900 handle_command_reply: tid=0 2024-03-12T05:14:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 client.15900 shutdown 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 msgr2=0x55de3fc40be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 0x55de3fc40be0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55de3fc2cb10 tx=0x7fd56c004900 comp rx=0 tx=0).stop 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 shutdown_connections 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55de3fca2cd0 0x55de3fca50f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de3fc3a750 0x55de3fc40be0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de3fc31b70 0x55de3fc406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 --2- 172.21.15.100:0/995478649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de3fc31250 0x55de3fc40160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 >> 172.21.15.100:0/995478649 conn(0x55de3fc2b980 msgr2=0x55de3fc2ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd55dffb640 0 client.15900 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 shutdown_connections 2024-03-12T05:14:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 1 -- 172.21.15.100:0/995478649 wait complete. 2024-03-12T05:14:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.848+0000 7fd57f19f740 20 client.15900 trim_cache size 0 max 16384 2024-03-12T05:14:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd5600a1ba0 msgr2=0x7fd5600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd5600a1ba0 0x7fd5600a4060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5740ffb70 tx=0x7fd564005f90 comp rx=0 tx=0).stop 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 msgr2=0x7fd57419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd57419d420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd570009230 tx=0x7fd57002c650 comp rx=0 tx=0).stop 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 shutdown_connections 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd5600a1ba0 0x7fd5600a4060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5741975d0 0x7fd574197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105330 0x7fd57419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 --2- 172.21.15.100:0/1572014329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd574100260 0x7fd57419d420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 >> 172.21.15.100:0/1572014329 conn(0x7fd574077180 msgr2=0x7fd5740fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 shutdown_connections 2024-03-12T05:14:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.849+0000 7fd57c5c9640 1 -- 172.21.15.100:0/1572014329 wait complete. 2024-03-12T05:14:05.859 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:14:05.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:14:05.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 -- 172.21.15.100:0/3882539486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62900685d0 msgr2=0x7f62900689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 --2- 172.21.15.100:0/3882539486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62900685d0 0x7f62900689b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6288009cd0 tx=0x7f628802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 -- 172.21.15.100:0/3882539486 shutdown_connections 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 --2- 172.21.15.100:0/3882539486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6290106430 0x7f629010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 --2- 172.21.15.100:0/3882539486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6290068f80 0x7f6290105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 --2- 172.21.15.100:0/3882539486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62900685d0 0x7f62900689b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 -- 172.21.15.100:0/3882539486 >> 172.21.15.100:0/3882539486 conn(0x7f6290077180 msgr2=0x7f6290077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 -- 172.21.15.100:0/3882539486 shutdown_connections 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 -- 172.21.15.100:0/3882539486 wait complete. 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 Processor -- start 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.973+0000 7f62986fb640 1 -- start start 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62900685d0 0x7f629019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6290106430 0x7f629019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 0x7f6290199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019a2b0 con 0x7f62901998c0 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019a420 con 0x7f62900685d0 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019a5e0 con 0x7f6290106430 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 0x7f6290199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 0x7f6290199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41876/0 (socket says 172.21.15.100:41876) 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296497640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62900685d0 0x7f629019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 -- 172.21.15.100:0/1975446304 learned_addr learned my addr 172.21.15.100:0/1975446304 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6295c96640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6290106430 0x7f629019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 -- 172.21.15.100:0/1975446304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6290106430 msgr2=0x7f629019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6290106430 0x7f629019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 -- 172.21.15.100:0/1975446304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62900685d0 msgr2=0x7f629019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62900685d0 0x7f629019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 -- 172.21.15.100:0/1975446304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009900 con 0x7f62901998c0 2024-03-12T05:14:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f6296c98640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 0x7f6290199d70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f628000e970 tx=0x7f628000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f628f7fe640 1 -- 172.21.15.100:0/1975446304 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280009720 con 0x7f62901998c0 2024-03-12T05:14:05.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f628f7fe640 1 -- 172.21.15.100:0/1975446304 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6280005500 con 0x7f62901998c0 2024-03-12T05:14:05.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62901a65d0 con 0x7f62901998c0 2024-03-12T05:14:05.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f628f7fe640 1 -- 172.21.15.100:0/1975446304 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280010430 con 0x7f62901998c0 2024-03-12T05:14:05.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.974+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901a6a60 con 0x7f62901998c0 2024-03-12T05:14:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.976+0000 7f628f7fe640 1 -- 172.21.15.100:0/1975446304 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6280005020 con 0x7f62901998c0 2024-03-12T05:14:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 Processor -- start 2024-03-12T05:14:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 -- start start 2024-03-12T05:14:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6a9b70 0x562fea6b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f628f7fe640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62780a1ba0 0x7f62780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 0x562fea6b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea664960 con 0x562fea6b2750 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea618310 con 0x562fea6a9250 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f629b3ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea615840 con 0x562fea6a9b70 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f6296c98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 0x562fea6b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f6296497640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62780a1ba0 0x7f62780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f6295c96640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f6296c98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 0x562fea6b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41878/0 (socket says 172.21.15.100:41878) 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f6296c98640 1 -- 172.21.15.100:0/546853995 learned_addr learned my addr 172.21.15.100:0/546853995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.977+0000 7f6295c96640 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6a9b70 0x562fea6b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f628f7fe640 1 -- 172.21.15.100:0/1975446304 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6280014070 con 0x7f62901998c0 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296c98640 1 -- 172.21.15.100:0/546853995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6a9b70 msgr2=0x562fea6b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296c98640 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6a9b70 0x562fea6b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296c98640 1 -- 172.21.15.100:0/546853995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 msgr2=0x562fea6a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296c98640 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296c98640 1 -- 172.21.15.100:0/546853995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fea5d3060 con 0x562fea6b2750 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6295c96640 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296c98640 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 0x562fea6b4b40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f62800cb300 tx=0x7f628000ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86518bdc0e2656ea server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f6296497640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62780a1ba0 0x7f62780a4060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6288009ca0 tx=0x7f6288005f10 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f628cff9640 1 -- 172.21.15.100:0/546853995 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628009a0e0 con 0x562fea6b2750 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f628cff9640 1 -- 172.21.15.100:0/546853995 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f628009b8b0 con 0x562fea6b2750 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.978+0000 7f628cff9640 1 -- 172.21.15.100:0/546853995 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628009b110 con 0x562fea6b2750 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 -- 172.21.15.100:0/546853995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 msgr2=0x562fea6b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 0x562fea6b4b40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f62800cb300 tx=0x7f628000ae70 comp rx=0 tx=0).stop 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 -- 172.21.15.100:0/546853995 shutdown_connections 2024-03-12T05:14:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6b2750 0x562fea6b4b40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6a9b70 0x562fea6b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 --2- 172.21.15.100:0/546853995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 -- 172.21.15.100:0/546853995 >> 172.21.15.100:0/546853995 conn(0x562fea6a3980 msgr2=0x562fea6a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 -- 172.21.15.100:0/546853995 shutdown_connections 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 -- 172.21.15.100:0/546853995 wait complete. 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 -- start start 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.979+0000 7f629b3ab740 1 Processor -- start 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea611020 con 0x562fea6a9250 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea2e4430 con 0x562fea6a9b70 2024-03-12T05:14:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea2a0e00 con 0x562fea6b2750 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 10 client.0 _ll_register_callbacks cb 0x562fea5b8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41880/0 (socket says 172.21.15.100:41880) 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 -- 172.21.15.100:0/976219310 learned_addr learned my addr 172.21.15.100:0/976219310 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296c98640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6295c96640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 -- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 msgr2=0x562fea6b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 -- 172.21.15.100:0/976219310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 msgr2=0x562fea6afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 -- 172.21.15.100:0/976219310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901a6a60 con 0x562fea6a9250 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6296497640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6af790 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6288036000 tx=0x7f62880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf65d435cf3af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6275ffb640 1 -- 172.21.15.100:0/976219310 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288032ac0 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f6275ffb640 1 -- 172.21.15.100:0/976219310 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6288047050 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.980+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fea3107e0 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f6275ffb640 1 -- 172.21.15.100:0/976219310 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6288003970 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 10 client.15912 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562fea60f6a0 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f6275ffb640 1 -- 172.21.15.100:0/976219310 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562fea3107e0 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f6275ffb640 1 -- 172.21.15.100:0/976219310 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f628804c420 con 0x562fea6a9250 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 10 client.15912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 20 client.15912 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea6c2b30 0x562fea6c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 4 client.15912 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fea34d720 con 0x562fea6c2b30 2024-03-12T05:14:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.981+0000 7f6295c96640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea6c2b30 0x562fea6c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f6295c96640 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea6c2b30 0x562fea6c4f50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a4b4d69c2631245 server_cookie=afab7e5089387627 in_seq=0 out_seq=0 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f6275ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f6275ffb640 1 -- 172.21.15.100:0/976219310 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562fea34d720 con 0x562fea6c2b30 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f6275ffb640 10 client.15912 handle_command_reply: tid=0 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 client.15912 shutdown 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 msgr2=0x562fea6af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6af790 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6288036000 tx=0x7f62880095c0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 shutdown_connections 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea6c2b30 0x562fea6c4f50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 --2- 172.21.15.100:0/976219310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6af790 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f6275ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.982+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 >> 172.21.15.100:0/976219310 conn(0x562fea6a3980 msgr2=0x562fea6a56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.983+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 shutdown_connections 2024-03-12T05:14:05.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.983+0000 7f629b3ab740 1 -- 172.21.15.100:0/976219310 wait complete. 2024-03-12T05:14:05.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.983+0000 7f629b3ab740 20 client.15912 trim_cache size 0 max 16384 2024-03-12T05:14:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 Processor -- start 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 -- start start 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea596130 con 0x562fea6a9250 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea6bf620 con 0x562fea6a9b70 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f629b3ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea6ca1a0 con 0x562fea6b2750 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41888/0 (socket says 172.21.15.100:41888) 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 -- 172.21.15.100:0/1571810047 learned_addr learned my addr 172.21.15.100:0/1571810047 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6295c96640 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296c98640 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 -- 172.21.15.100:0/1571810047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 msgr2=0x562fea6b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 -- 172.21.15.100:0/1571810047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 msgr2=0x562fea6afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 -- 172.21.15.100:0/1571810047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fea5abc20 con 0x562fea6a9250 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.995+0000 7f6296497640 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6b1ec0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6288033000 tx=0x7f6288009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b89ce5d7a7c2c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f6277fff640 1 -- 172.21.15.100:0/1571810047 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628804c9a0 con 0x562fea6a9250 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f6277fff640 1 -- 172.21.15.100:0/1571810047 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6288003e50 con 0x562fea6a9250 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f6277fff640 1 -- 172.21.15.100:0/1571810047 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880045a0 con 0x562fea6a9250 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 -- 172.21.15.100:0/1571810047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 msgr2=0x562fea6b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6b1ec0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6288033000 tx=0x7f6288009b20 comp rx=0 tx=0).stop 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 -- 172.21.15.100:0/1571810047 shutdown_connections 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9b70 0x562fea6afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 --2- 172.21.15.100:0/1571810047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9250 0x562fea6b1ec0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 -- 172.21.15.100:0/1571810047 >> 172.21.15.100:0/1571810047 conn(0x562fea6a3980 msgr2=0x562fea6aaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 -- 172.21.15.100:0/1571810047 shutdown_connections 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.996+0000 7f629b3ab740 1 -- 172.21.15.100:0/1571810047 wait complete. 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 -- start start 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 Processor -- start 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 0x562fea6b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea6b84e0 con 0x562fea6a9b70 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea6b8650 con 0x562fea6a9250 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fea6b8810 con 0x562fea6b2750 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f629b3ab740 10 client.0 _ll_register_callbacks cb 0x7f62800048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 0x562fea6b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 0x562fea6b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41894/0 (socket says 172.21.15.100:41894) 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 -- 172.21.15.100:0/3030845443 learned_addr learned my addr 172.21.15.100:0/3030845443 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6296c98640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6296497640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 -- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 msgr2=0x562fea6b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 -- 172.21.15.100:0/3030845443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 msgr2=0x562fea6b96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6b96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.997+0000 7f6295c96640 1 -- 172.21.15.100:0/3030845443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fea60f6a0 con 0x562fea6a9b70 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6295c96640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 0x562fea6b7a60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6284004700 tx=0x7f6284022410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a4a175e239d923 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6275ffb640 1 -- 172.21.15.100:0/3030845443 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284016040 con 0x562fea6a9b70 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6275ffb640 1 -- 172.21.15.100:0/3030845443 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6284005020 con 0x562fea6a9b70 2024-03-12T05:14:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f628009b8b0 con 0x562fea6a9b70 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6275ffb640 1 -- 172.21.15.100:0/3030845443 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62840051e0 con 0x562fea6a9b70 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 10 client.15918 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562fea5d18c0 con 0x562fea6a9b70 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6275ffb640 1 -- 172.21.15.100:0/3030845443 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6284036080 con 0x562fea6a9b70 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f6275ffb640 1 -- 172.21.15.100:0/3030845443 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f62840035f0 con 0x562fea6a9b70 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 10 client.15918 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 10 client.15918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 20 client.15918 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea71a690 0x562fea71cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 4 client.15918 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.998+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562fea39c890 con 0x562fea71a690 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f6296497640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea71a690 0x562fea71cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f6275ffb640 10 client.15918 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f6296497640 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea71a690 0x562fea71cab0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6913cdc48206bc54 server_cookie=8a2865021b5f91d6 in_seq=0 out_seq=0 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f6275ffb640 1 -- 172.21.15.100:0/3030845443 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562fea39c890 con 0x562fea71a690 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f6275ffb640 10 client.15918 handle_command_reply: tid=0 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 client.15918 shutdown 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 msgr2=0x562fea6b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 0x562fea6b7a60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6284004700 tx=0x7f6284022410 comp rx=0 tx=0).stop 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 shutdown_connections 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fea6b2750 0x562fea6b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fea6a9b70 0x562fea6b7a60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562fea71a690 0x562fea71cab0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 --2- 172.21.15.100:0/3030845443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fea6a9250 0x562fea6b96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 >> 172.21.15.100:0/3030845443 conn(0x562fea6a3980 msgr2=0x562fea6a22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f6275ffb640 0 client.15918 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 shutdown_connections 2024-03-12T05:14:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 1 -- 172.21.15.100:0/3030845443 wait complete. 2024-03-12T05:14:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:05.999+0000 7f629b3ab740 20 client.15918 trim_cache size 0 max 16384 2024-03-12T05:14:06.003 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62780a1ba0 msgr2=0x7f62780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62780a1ba0 0x7f62780a4060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6288009ca0 tx=0x7f6288005f10 comp rx=0 tx=0).stop 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 msgr2=0x7f6290199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 0x7f6290199d70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f628000e970 tx=0x7f628000ed70 comp rx=0 tx=0).stop 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 shutdown_connections 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62780a1ba0 0x7f62780a4060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62901998c0 0x7f6290199d70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6290106430 0x7f629019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 --2- 172.21.15.100:0/1975446304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62900685d0 0x7f629019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 >> 172.21.15.100:0/1975446304 conn(0x7f6290077180 msgr2=0x7f62900fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 shutdown_connections 2024-03-12T05:14:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.000+0000 7f62986fb640 1 -- 172.21.15.100:0/1975446304 wait complete. 2024-03-12T05:14:06.006 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:05 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2992285404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:14:06.007 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:14:06.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:14:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 -- 172.21.15.100:0/4052946435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110068fa0 msgr2=0x7fe110105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 --2- 172.21.15.100:0/4052946435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110068fa0 0x7fe110105de0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe104009930 tx=0x7fe10402bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 -- 172.21.15.100:0/4052946435 shutdown_connections 2024-03-12T05:14:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 --2- 172.21.15.100:0/4052946435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1101064c0 0x7fe11010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 --2- 172.21.15.100:0/4052946435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110068fa0 0x7fe110105de0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 --2- 172.21.15.100:0/4052946435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1100685f0 0x7fe1100689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 -- 172.21.15.100:0/4052946435 >> 172.21.15.100:0/4052946435 conn(0x7fe110077180 msgr2=0x7fe110077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 -- 172.21.15.100:0/4052946435 shutdown_connections 2024-03-12T05:14:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.120+0000 7fe1148c0640 1 -- 172.21.15.100:0/4052946435 wait complete. 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 Processor -- start 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 -- start start 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1100685f0 0x7fe11019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1101064c0 0x7fe11019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 0x7fe110197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110198000 con 0x7fe110197610 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110198170 con 0x7fe1101064c0 2024-03-12T05:14:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe1148c0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110198330 con 0x7fe1100685f0 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 0x7fe110197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 0x7fe110197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41914/0 (socket says 172.21.15.100:41914) 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1793073698 learned_addr learned my addr 172.21.15.100:0/1793073698 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1101064c0 0x7fe11019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.121+0000 7fe10e59c640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1100685f0 0x7fe11019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1793073698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1100685f0 msgr2=0x7fe11019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1100685f0 0x7fe11019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1793073698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1101064c0 msgr2=0x7fe11019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1101064c0 0x7fe11019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1793073698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104009560 con 0x7fe110197610 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 0x7fe110197ac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe108009fd0 tx=0x7fe10800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe0ff7fe640 1 -- 172.21.15.100:0/1793073698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800bea0 con 0x7fe110197610 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe0ff7fe640 1 -- 172.21.15.100:0/1793073698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe108009940 con 0x7fe110197610 2024-03-12T05:14:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1101a4320 con 0x7fe110197610 2024-03-12T05:14:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe0ff7fe640 1 -- 172.21.15.100:0/1793073698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080106c0 con 0x7fe110197610 2024-03-12T05:14:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.122+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1101a47b0 con 0x7fe110197610 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.124+0000 7fe0ff7fe640 1 -- 172.21.15.100:0/1793073698 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe108004240 con 0x7fe110197610 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 Processor -- start 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe0ff7fe640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0f80a1ba0 0x7fe0f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 -- start start 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 0x5568547e52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0f80a1ba0 0x7fe0f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568547a0960 con 0x5568547f2b60 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe0ff7fe640 1 -- 172.21.15.100:0/1793073698 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe1080cd490 con 0x7fe110197610 2024-03-12T05:14:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556854754310 con 0x5568547e4f00 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe117569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556854751840 con 0x5568547e5820 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 0x5568547e52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41916/0 (socket says 172.21.15.100:41916) 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 0x5568547e52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48522/0 (socket says 172.21.15.100:48522) 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10ed9d640 1 -- 172.21.15.100:0/3554659300 learned_addr learned my addr 172.21.15.100:0/3554659300 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.125+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10ed9d640 1 -- 172.21.15.100:0/3554659300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 msgr2=0x5568547f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10ed9d640 1 -- 172.21.15.100:0/3554659300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 msgr2=0x5568547e52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 0x5568547e52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10ed9d640 1 -- 172.21.15.100:0/3554659300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55685470f060 con 0x5568547f2b60 2024-03-12T05:14:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10e59c640 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 0x5568547e52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0f80a1ba0 0x7fe0f80a4060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe104036040 tx=0x7fe104004ad0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f4f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe108096b30 tx=0x7fe10809a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789c5fa162250d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe0fcff9640 1 -- 172.21.15.100:0/3554659300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108098460 con 0x5568547f2b60 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe0fcff9640 1 -- 172.21.15.100:0/3554659300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1080cae50 con 0x5568547f2b60 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.126+0000 7fe0fcff9640 1 -- 172.21.15.100:0/3554659300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108099810 con 0x5568547f2b60 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- 172.21.15.100:0/3554659300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 msgr2=0x5568547f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f4f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe108096b30 tx=0x7fe10809a160 comp rx=0 tx=0).stop 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- 172.21.15.100:0/3554659300 shutdown_connections 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f4f50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- 172.21.15.100:0/3554659300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e4f00 0x5568547e52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- 172.21.15.100:0/3554659300 >> 172.21.15.100:0/3554659300 conn(0x5568547df980 msgr2=0x5568547dfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- 172.21.15.100:0/3554659300 shutdown_connections 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- 172.21.15.100:0/3554659300 wait complete. 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- start start 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 Processor -- start 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 0x5568547efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685474d020 con 0x5568547f2b60 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556854420430 con 0x5568547e5820 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568543dce00 con 0x5568547e4f00 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.127+0000 7fe117569740 10 client.0 _ll_register_callbacks cb 0x5568546f4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 0x5568547efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 0x5568547efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:47454/0 (socket says 172.21.15.100:47454) 2024-03-12T05:14:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41928/0 (socket says 172.21.15.100:41928) 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10e59c640 1 -- 172.21.15.100:0/1342668417 learned_addr learned my addr 172.21.15.100:0/1342668417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1342668417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 msgr2=0x5568547efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 0x5568547efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 msgr2=0x5568547f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 -- 172.21.15.100:0/1342668417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1101a47b0 con 0x5568547f2b60 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10e59c640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 0x5568547efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f05a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe10809a0f0 tx=0x7fe1080cab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21fb12f94d9ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/1342668417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080b8070 con 0x5568547f2b60 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/1342668417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe1080cafc0 con 0x5568547f2b60 2024-03-12T05:14:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55685444c7e0 con 0x5568547f2b60 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/1342668417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe108014070 con 0x5568547f2b60 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe117569740 10 client.15930 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55685474b6a0 con 0x5568547f2b60 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.128+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/1342668417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe108081020 con 0x5568547f2b60 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/1342668417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe10807c070 con 0x5568547f2b60 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe117569740 10 client.15930 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe117569740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe117569740 20 client.15930 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe117569740 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556854802bb0 0x556854804fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe117569740 4 client.15930 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556854489720 con 0x556854802bb0 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe10e59c640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556854802bb0 0x556854804fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe10e59c640 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556854802bb0 0x556854804fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f5fd5811c2ab4b server_cookie=42a8b4f4195e7581 in_seq=0 out_seq=0 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.129+0000 7fe0f5ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/1342668417 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556854489720 con 0x556854802bb0 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe0f5ffb640 10 client.15930 handle_command_reply: tid=0 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 client.15930 shutdown 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 msgr2=0x5568547f05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f05a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe10809a0f0 tx=0x7fe1080cab70 comp rx=0 tx=0).stop 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 shutdown_connections 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556854802bb0 0x556854804fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547f2b60 0x5568547f05a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe0f5ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 --2- 172.21.15.100:0/1342668417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e4f00 0x5568547efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 >> 172.21.15.100:0/1342668417 conn(0x5568547df980 msgr2=0x5568547e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 shutdown_connections 2024-03-12T05:14:06.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 1 -- 172.21.15.100:0/1342668417 wait complete. 2024-03-12T05:14:06.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.130+0000 7fe117569740 20 client.15930 trim_cache size 0 max 16384 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.142+0000 7fe117569740 1 Processor -- start 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.142+0000 7fe117569740 1 -- start start 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe117569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe117569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe117569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547f2b60 0x5568547eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe117569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568546d2130 con 0x5568547e4f00 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe117569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685480a170 con 0x5568547f2b60 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe117569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568547db2b0 con 0x5568547e5820 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41936/0 (socket says 172.21.15.100:41936) 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 -- 172.21.15.100:0/536668443 learned_addr learned my addr 172.21.15.100:0/536668443 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547f2b60 0x5568547eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 -- 172.21.15.100:0/536668443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 msgr2=0x5568547f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 -- 172.21.15.100:0/536668443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547f2b60 msgr2=0x5568547eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547f2b60 0x5568547eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 -- 172.21.15.100:0/536668443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568546e7c20 con 0x5568547e4f00 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe10e59c640 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f2250 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe100010720 tx=0x7fe100004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c163076d93942a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe0f7fff640 1 -- 172.21.15.100:0/536668443 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100016070 con 0x5568547e4f00 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe0f7fff640 1 -- 172.21.15.100:0/536668443 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe100018040 con 0x5568547e4f00 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.143+0000 7fe0f7fff640 1 -- 172.21.15.100:0/536668443 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000035f0 con 0x5568547e4f00 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 -- 172.21.15.100:0/536668443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 msgr2=0x5568547f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f2250 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe100010720 tx=0x7fe100004410 comp rx=0 tx=0).stop 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 -- 172.21.15.100:0/536668443 shutdown_connections 2024-03-12T05:14:06.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547f2b60 0x5568547eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547e5820 0x5568547f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 --2- 172.21.15.100:0/536668443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f2250 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 -- 172.21.15.100:0/536668443 >> 172.21.15.100:0/536668443 conn(0x5568547df980 msgr2=0x5568547dfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 -- 172.21.15.100:0/536668443 shutdown_connections 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 -- 172.21.15.100:0/536668443 wait complete. 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 -- start start 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.144+0000 7fe117569740 1 Processor -- start 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547f2b60 0x5568547f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556854859140 con 0x5568547e4f00 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568548592b0 con 0x5568547e5820 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556854859470 con 0x5568547f2b60 2024-03-12T05:14:06.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 10 client.0 _ll_register_callbacks cb 0x7fe1080964a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41950/0 (socket says 172.21.15.100:41950) 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 -- 172.21.15.100:0/62956722 learned_addr learned my addr 172.21.15.100:0/62956722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10ed9d640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547f2b60 0x5568547f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 -- 172.21.15.100:0/62956722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547f2b60 msgr2=0x5568547f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547f2b60 0x5568547f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 -- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 msgr2=0x5568547f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 -- 172.21.15.100:0/62956722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe104049050 con 0x5568547e4f00 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe10e59c640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f86b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe100005890 tx=0x7fe100011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be298bc43a8ac48d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/62956722 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100016050 con 0x5568547e4f00 2024-03-12T05:14:06.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/62956722 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe100004c30 con 0x5568547e4f00 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.145+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1080cae50 con 0x5568547e4f00 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/62956722 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10002c070 con 0x5568547e4f00 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 10 client.15936 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fe100005c50 con 0x5568547e4f00 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/62956722 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe10003d020 con 0x5568547e4f00 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/62956722 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe10001f2e0 con 0x5568547e4f00 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 10 client.15936 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 20 client.15936 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55685485ab90 0x55685485cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 4 client.15936 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568544d8890 con 0x55685485ab90 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55685485ab90 0x55685485cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:06.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe0f5ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.146+0000 7fe10dd9b640 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55685485ab90 0x55685485cfb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44577e8487544d server_cookie=e8cedb6e261878bc in_seq=0 out_seq=0 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe0f5ffb640 1 -- 172.21.15.100:0/62956722 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568544d8890 con 0x55685485ab90 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe0f5ffb640 10 client.15936 handle_command_reply: tid=0 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 client.15936 shutdown 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 msgr2=0x5568547f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f86b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe100005890 tx=0x7fe100011d70 comp rx=0 tx=0).stop 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 shutdown_connections 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55685485ab90 0x55685485cfb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568547f2b60 0x5568547f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe0f5ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568547e5820 0x5568547f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 --2- 172.21.15.100:0/62956722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5568547e4f00 0x5568547f86b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 >> 172.21.15.100:0/62956722 conn(0x5568547df980 msgr2=0x5568547eae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 shutdown_connections 2024-03-12T05:14:06.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 1 -- 172.21.15.100:0/62956722 wait complete. 2024-03-12T05:14:06.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe117569740 20 client.15936 trim_cache size 0 max 16384 2024-03-12T05:14:06.151 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0f80a1ba0 msgr2=0x7fe0f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe1148c0640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0f80a1ba0 0x7fe0f80a4060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe104036040 tx=0x7fe104004ad0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 msgr2=0x7fe110197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.147+0000 7fe1148c0640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 0x7fe110197ac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe108009fd0 tx=0x7fe10800b260 comp rx=0 tx=0).stop 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 shutdown_connections 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0f80a1ba0 0x7fe0f80a4060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe110197610 0x7fe110197ac0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1101064c0 0x7fe11019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 --2- 172.21.15.100:0/1793073698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1100685f0 0x7fe11019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:06.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 >> 172.21.15.100:0/1793073698 conn(0x7fe110077180 msgr2=0x7fe1100fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:06.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 shutdown_connections 2024-03-12T05:14:06.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:06.148+0000 7fe1148c0640 1 -- 172.21.15.100:0/1793073698 wait complete. 2024-03-12T05:14:06.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:05 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2992285404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:14:06.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:05 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2992285404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:14:06.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 128851 14.23 MB/sec warmup 50 sec latency 17.373 ms 2024-03-12T05:14:07.156 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:14:07.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:14:07.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:14:07.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:06 smithi100 ceph-mon[32210]: pgmap v63: 97 pgs: 97 active+clean; 73 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-12T05:14:07.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:06 smithi156 ceph-mon[36270]: pgmap v63: 97 pgs: 97 active+clean; 73 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-12T05:14:07.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:06 smithi161 ceph-mon[36043]: pgmap v63: 97 pgs: 97 active+clean; 73 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 -- 172.21.15.100:0/1013784517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4100c30 msgr2=0x7f15f4104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/1013784517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4100c30 0x7f15f4104c70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009930 tx=0x7f15e802bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 -- 172.21.15.100:0/1013784517 shutdown_connections 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/1013784517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4105350 0x7f15f410cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/1013784517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4100c30 0x7f15f4104c70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/1013784517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f4100280 0x7f15f4100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 -- 172.21.15.100:0/1013784517 >> 172.21.15.100:0/1013784517 conn(0x7f15f4076e30 msgr2=0x7f15f40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 -- 172.21.15.100:0/1013784517 shutdown_connections 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 -- 172.21.15.100:0/1013784517 wait complete. 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.272+0000 7f15f9d8a640 1 Processor -- start 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 -- start start 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f4100280 0x7f15f40fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 0x7f15f40fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f40fcf60 0x7f15f41a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41a5f80 con 0x7f15f4105350 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41a60f0 con 0x7f15f40fcf60 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f9d8a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41a62b0 con 0x7f15f4100280 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 0x7f15f40fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 0x7f15f40fca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41972/0 (socket says 172.21.15.100:41972) 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 -- 172.21.15.100:0/2202559366 learned_addr learned my addr 172.21.15.100:0/2202559366 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f3fff640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f40fcf60 0x7f15f41a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f37fe640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f4100280 0x7f15f40fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 -- 172.21.15.100:0/2202559366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f4100280 msgr2=0x7f15f40fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f4100280 0x7f15f40fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 -- 172.21.15.100:0/2202559366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f40fcf60 msgr2=0x7f15f41a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f40fcf60 0x7f15f41a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 -- 172.21.15.100:0/2202559366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8009560 con 0x7f15f4105350 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.273+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 0x7f15f40fca20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009230 tx=0x7f15e802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.274+0000 7f15f0ff9640 1 -- 172.21.15.100:0/2202559366 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8004350 con 0x7f15f4105350 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.274+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f41a6530 con 0x7f15f4105350 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.274+0000 7f15f0ff9640 1 -- 172.21.15.100:0/2202559366 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15e80044b0 con 0x7f15f4105350 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.274+0000 7f15f0ff9640 1 -- 172.21.15.100:0/2202559366 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003da0 con 0x7f15f4105350 2024-03-12T05:14:07.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.274+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f41a6a90 con 0x7f15f4105350 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15f0ff9640 1 -- 172.21.15.100:0/2202559366 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f15e8004df0 con 0x7f15f4105350 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 Processor -- start 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 -- start start 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15f0ff9640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f15dc0a1ba0 0x7f15dc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc178760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc1812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc1818b0 0x55a1dc183d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc0e6f90 con 0x55a1dc1818b0 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc0e44c0 con 0x55a1dc178380 2024-03-12T05:14:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15fca3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc0dfca0 con 0x55a1dc178d30 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15f3fff640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f15dc0a1ba0 0x7f15dc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc1818b0 0x55a1dc183d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.276+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc178760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc1812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc178760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48572/0 (socket says 172.21.15.100:48572) 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 -- 172.21.15.100:0/3589630507 learned_addr learned my addr 172.21.15.100:0/3589630507 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f0ff9640 1 -- 172.21.15.100:0/2202559366 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f15e80ef850 con 0x7f15f4105350 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 -- 172.21.15.100:0/3589630507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 msgr2=0x55a1dc1812e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc1812e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 -- 172.21.15.100:0/3589630507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc1818b0 msgr2=0x55a1dc183d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc1818b0 0x55a1dc183d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 -- 172.21.15.100:0/3589630507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1dc0a20b0 con 0x55a1dc178380 2024-03-12T05:14:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f3fff640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f15dc0a1ba0 0x7f15dc0a4060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005c40 tx=0x7f15ec004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15f37fe640 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc178760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009c90 tx=0x7f15e40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733ef6f98896cf14 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15d9ffb640 1 -- 172.21.15.100:0/3589630507 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4009760 con 0x55a1dc178380 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15d9ffb640 1 -- 172.21.15.100:0/3589630507 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15e4012930 con 0x55a1dc178380 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.277+0000 7f15d9ffb640 1 -- 172.21.15.100:0/3589630507 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400fb40 con 0x55a1dc178380 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 -- 172.21.15.100:0/3589630507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 msgr2=0x55a1dc178760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc178760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009c90 tx=0x7f15e40117b0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 -- 172.21.15.100:0/3589630507 shutdown_connections 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc1818b0 0x55a1dc183d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc1812e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3589630507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc178760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 -- 172.21.15.100:0/3589630507 >> 172.21.15.100:0/3589630507 conn(0x55a1dc172840 msgr2=0x55a1dc172c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 -- 172.21.15.100:0/3589630507 shutdown_connections 2024-03-12T05:14:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 -- 172.21.15.100:0/3589630507 wait complete. 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.278+0000 7f15fca3a740 1 -- start start 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 Processor -- start 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc17e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc17eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc17f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc084f40 con 0x55a1dc178d30 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dbdb34b0 con 0x55a1dc178380 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc065670 con 0x55a1dc1818b0 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15fca3a740 10 client.0 _ll_register_callbacks cb 0x55a1dc0d2f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc17eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc17eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41984/0 (socket says 172.21.15.100:41984) 2024-03-12T05:14:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc17e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc17f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 -- 172.21.15.100:0/380682987 learned_addr learned my addr 172.21.15.100:0/380682987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 -- 172.21.15.100:0/380682987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 msgr2=0x55a1dc17f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc17f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 -- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 msgr2=0x55a1dc17e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc17e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f2ffd640 1 -- 172.21.15.100:0/380682987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e40098c0 con 0x55a1dc178d30 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.279+0000 7f15f37fe640 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc17e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc17eca0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b89e0 tx=0x7f15e8047f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c38c6afd6162f89 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15d2ffd640 1 -- 172.21.15.100:0/380682987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80bbee0 con 0x55a1dc178d30 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15d2ffd640 1 -- 172.21.15.100:0/380682987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f15e80bcc80 con 0x55a1dc178d30 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1dc0b9f40 con 0x55a1dc178d30 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15d2ffd640 1 -- 172.21.15.100:0/380682987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e80cebf0 con 0x55a1dc178d30 2024-03-12T05:14:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 10 client.15948 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a1dc0de320 con 0x55a1dc178d30 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15d2ffd640 1 -- 172.21.15.100:0/380682987 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a1dc0b9f40 con 0x55a1dc178d30 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15d2ffd640 1 -- 172.21.15.100:0/380682987 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f15e803c070 con 0x55a1dc178d30 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 10 client.15948 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 20 client.15948 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc191ab0 0x55a1dc193ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 4 client.15948 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1dc0d6310 con 0x55a1dc191ab0 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.280+0000 7f15f37fe640 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc191ab0 0x55a1dc193ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.281+0000 7f15f37fe640 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc191ab0 0x55a1dc193ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d06f976d5e958c server_cookie=7243559a7c468b9c in_seq=0 out_seq=0 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.281+0000 7f15d2ffd640 10 client.15948 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.281+0000 7f15d2ffd640 1 -- 172.21.15.100:0/380682987 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a1dc0d6310 con 0x55a1dc191ab0 2024-03-12T05:14:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.281+0000 7f15d2ffd640 10 client.15948 handle_command_reply: tid=0 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.281+0000 7f15fca3a740 1 client.15948 shutdown 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 msgr2=0x55a1dc17eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc17eca0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b89e0 tx=0x7f15e8047f90 comp rx=0 tx=0).stop 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 shutdown_connections 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc191ab0 0x55a1dc193ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc17f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15d2ffd640 0 client.15948 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc17eca0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 --2- 172.21.15.100:0/380682987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc17e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 >> 172.21.15.100:0/380682987 conn(0x55a1dc172840 msgr2=0x55a1dc1747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 shutdown_connections 2024-03-12T05:14:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 1 -- 172.21.15.100:0/380682987 wait complete. 2024-03-12T05:14:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.282+0000 7f15fca3a740 20 client.15948 trim_cache size 0 max 16384 2024-03-12T05:14:07.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 Processor -- start 2024-03-12T05:14:07.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 -- start start 2024-03-12T05:14:07.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 0x55a1dc17d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc17f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc1818b0 0x55a1dc171760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc1918b0 con 0x55a1dc178380 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc17de00 con 0x55a1dc1818b0 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15fca3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc172670 con 0x55a1dc178d30 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 0x55a1dc17d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 0x55a1dc17d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41998/0 (socket says 172.21.15.100:41998) 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 -- 172.21.15.100:0/2170066753 learned_addr learned my addr 172.21.15.100:0/2170066753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f3fff640 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc1818b0 0x55a1dc171760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc17f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 -- 172.21.15.100:0/2170066753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 msgr2=0x55a1dc17f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc17f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 -- 172.21.15.100:0/2170066753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc1818b0 msgr2=0x55a1dc171760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc1818b0 0x55a1dc171760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.298+0000 7f15f37fe640 1 -- 172.21.15.100:0/2170066753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1dc07ac00 con 0x55a1dc178380 2024-03-12T05:14:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15f37fe640 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 0x55a1dc17d8c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f15e401c040 tx=0x7f15e4011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812b9945260aecfc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15d97fa640 1 -- 172.21.15.100:0/2170066753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4009760 con 0x55a1dc178380 2024-03-12T05:14:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15d97fa640 1 -- 172.21.15.100:0/2170066753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15e4029070 con 0x55a1dc178380 2024-03-12T05:14:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15d97fa640 1 -- 172.21.15.100:0/2170066753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e40052a0 con 0x55a1dc178380 2024-03-12T05:14:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 -- 172.21.15.100:0/2170066753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 msgr2=0x55a1dc17d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 0x55a1dc17d8c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f15e401c040 tx=0x7f15e4011de0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 -- 172.21.15.100:0/2170066753 shutdown_connections 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc1818b0 0x55a1dc171760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc178d30 0x55a1dc17f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 --2- 172.21.15.100:0/2170066753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178380 0x55a1dc17d8c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 -- 172.21.15.100:0/2170066753 >> 172.21.15.100:0/2170066753 conn(0x55a1dc172840 msgr2=0x55a1dc171050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.299+0000 7f15fca3a740 1 -- 172.21.15.100:0/2170066753 shutdown_connections 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 -- 172.21.15.100:0/2170066753 wait complete. 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 -- start start 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 Processor -- start 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc187a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc187fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc1e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc1e8530 con 0x55a1dc178d30 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc1e8670 con 0x55a1dc178380 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1dc1e87b0 con 0x55a1dc1818b0 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15fca3a740 10 client.0 _ll_register_callbacks cb 0x7f15e40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc187fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc187fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42008/0 (socket says 172.21.15.100:42008) 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15f2ffd640 1 -- 172.21.15.100:0/3035612480 learned_addr learned my addr 172.21.15.100:0/3035612480 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15f37fe640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc187a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.300+0000 7f15f3fff640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc1e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15f2ffd640 1 -- 172.21.15.100:0/3035612480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 msgr2=0x55a1dc1e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc1e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15f2ffd640 1 -- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 msgr2=0x55a1dc187a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc187a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15f2ffd640 1 -- 172.21.15.100:0/3035612480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1dc0a20b0 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15f2ffd640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc187fb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f15e8048000 tx=0x7f15e80d0b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f362bdf661c3d88b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15d2ffd640 1 -- 172.21.15.100:0/3035612480 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80dc1d0 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15d2ffd640 1 -- 172.21.15.100:0/3035612480 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f15e8083060 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1dc105900 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15d2ffd640 1 -- 172.21.15.100:0/3035612480 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e80bbee0 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 10 client.15954 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a1dc0a0910 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15d2ffd640 1 -- 172.21.15.100:0/3035612480 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a1dc105900 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15d2ffd640 1 -- 172.21.15.100:0/3035612480 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f15e80d6030 con 0x55a1dc178d30 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 10 client.15954 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 20 client.15954 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:07.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.301+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc1e9a90 0x55a1dc1ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.302+0000 7f15fca3a740 4 client.15954 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.302+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a1dbddf860 con 0x55a1dc1e9a90 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.302+0000 7f15f37fe640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc1e9a90 0x55a1dc1ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.302+0000 7f15d2ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.302+0000 7f15f37fe640 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc1e9a90 0x55a1dc1ebe80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82308f7d7b7903dd server_cookie=22a84dbe246fdd2 in_seq=0 out_seq=0 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15d2ffd640 1 -- 172.21.15.100:0/3035612480 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a1dbddf860 con 0x55a1dc1e9a90 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15d2ffd640 10 client.15954 handle_command_reply: tid=0 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 client.15954 shutdown 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 msgr2=0x55a1dc187fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc187fb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f15e8048000 tx=0x7f15e80d0b70 comp rx=0 tx=0).stop 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 shutdown_connections 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a1dc1e9a90 0x55a1dc1ebe80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1dc1818b0 0x55a1dc1e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1dc178d30 0x55a1dc187fb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 --2- 172.21.15.100:0/3035612480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1dc178380 0x55a1dc187a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 >> 172.21.15.100:0/3035612480 conn(0x55a1dc172840 msgr2=0x55a1dc1718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15d2ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 shutdown_connections 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.303+0000 7f15fca3a740 1 -- 172.21.15.100:0/3035612480 wait complete. 2024-03-12T05:14:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15fca3a740 20 client.15954 trim_cache size 0 max 16384 2024-03-12T05:14:07.307 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:14:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:14:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "43a4bb7d-e71f-44c4-a807-310c4e182b1a", 2024-03-12T05:14:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:14:07.307 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:14:07.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f15dc0a1ba0 msgr2=0x7f15dc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f15dc0a1ba0 0x7f15dc0a4060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005c40 tx=0x7f15ec004ab0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 msgr2=0x7f15f40fca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 0x7f15f40fca20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009230 tx=0x7f15e802c680 comp rx=0 tx=0).stop 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 shutdown_connections 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f15dc0a1ba0 0x7f15dc0a4060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f40fcf60 0x7f15f41a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f4105350 0x7f15f40fca20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 --2- 172.21.15.100:0/2202559366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15f4100280 0x7f15f40fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 >> 172.21.15.100:0/2202559366 conn(0x7f15f4076e30 msgr2=0x7f15f40697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 shutdown_connections 2024-03-12T05:14:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.304+0000 7f15f9d8a640 1 -- 172.21.15.100:0/2202559366 wait complete. 2024-03-12T05:14:07.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:14:07.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:14:07.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 -- 172.21.15.100:0/923964844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68100c90 msgr2=0x7ffb6810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/923964844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68100c90 0x7ffb6810ce40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009ca0 tx=0x7ffb6002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 -- 172.21.15.100:0/923964844 shutdown_connections 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/923964844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68100c90 0x7ffb6810ce40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/923964844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb681002d0 0x7ffb68100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/923964844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb68106110 0x7ffb681064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 -- 172.21.15.100:0/923964844 >> 172.21.15.100:0/923964844 conn(0x7ffb68076e20 msgr2=0x7ffb68077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 -- 172.21.15.100:0/923964844 shutdown_connections 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 -- 172.21.15.100:0/923964844 wait complete. 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.466+0000 7ffb6fe81640 1 Processor -- start 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 -- start start 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb681002d0 0x7ffb68198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb68100c90 0x7ffb68198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 0x7ffb681a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681993d0 con 0x7ffb68106110 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681a3bd0 con 0x7ffb68100c90 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6fe81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681a3d10 con 0x7ffb681002d0 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 0x7ffb681a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb681002d0 0x7ffb68198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 0x7ffb681a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42026/0 (socket says 172.21.15.100:42026) 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb681002d0 0x7ffb68198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:47534/0 (socket says 172.21.15.100:47534) 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 -- 172.21.15.100:0/1915337006 learned_addr learned my addr 172.21.15.100:0/1915337006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb68100c90 0x7ffb68198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 -- 172.21.15.100:0/1915337006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb681002d0 msgr2=0x7ffb68198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb681002d0 0x7ffb68198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 -- 172.21.15.100:0/1915337006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb68100c90 msgr2=0x7ffb68198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb68100c90 0x7ffb68198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 -- 172.21.15.100:0/1915337006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009900 con 0x7ffb68106110 2024-03-12T05:14:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb6e41e640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 0x7ffb681a3690 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009bf0 tx=0x7ffb60034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.467+0000 7ffb5effd640 1 -- 172.21.15.100:0/1915337006 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600328c0 con 0x7ffb68106110 2024-03-12T05:14:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.468+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb68196400 con 0x7ffb68106110 2024-03-12T05:14:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.468+0000 7ffb5effd640 1 -- 172.21.15.100:0/1915337006 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb60003e50 con 0x7ffb68106110 2024-03-12T05:14:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.468+0000 7ffb5effd640 1 -- 172.21.15.100:0/1915337006 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600047a0 con 0x7ffb68106110 2024-03-12T05:14:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.468+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb68196850 con 0x7ffb68106110 2024-03-12T05:14:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.470+0000 7ffb5effd640 1 -- 172.21.15.100:0/1915337006 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ffb6000a040 con 0x7ffb68106110 2024-03-12T05:14:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.470+0000 7ffb72a52740 1 Processor -- start 2024-03-12T05:14:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.470+0000 7ffb72a52740 1 -- start start 2024-03-12T05:14:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de63690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 0x560d1de6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de6c7b0 0x560d1de6eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb72a52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1ddd2280 con 0x560d1de6c7b0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb72a52740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1ddcf7b0 con 0x560d1de632b0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb72a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1ddcaf90 con 0x560d1de63bd0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb5effd640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffb540a1ba0 0x7ffb540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6d41c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de6c7b0 0x560d1de6eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 0x560d1de6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 0x560d1de6c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:47550/0 (socket says 172.21.15.100:47550) 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6d41c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de6c7b0 0x560d1de6eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42036/0 (socket says 172.21.15.100:42036) 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1303856374 learned_addr learned my addr 172.21.15.100:0/1303856374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de63690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6e41e640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffb540a1ba0 0x7ffb540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1303856374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 msgr2=0x560d1de63690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de63690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1303856374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de6c7b0 msgr2=0x560d1de6eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de6c7b0 0x560d1de6eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1303856374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d1dd89570 con 0x560d1de63bd0 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb5effd640 1 -- 172.21.15.100:0/1915337006 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffb600f00c0 con 0x7ffb68106110 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 0x560d1de6c270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400b710 tx=0x7ffb64004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887bd73b52f1bf98 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb6e41e640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffb540a1ba0 0x7ffb540a4060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb600bcaf0 tx=0x7ffb600b9090 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.471+0000 7ffb53fff640 1 -- 172.21.15.100:0/1303856374 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64003b50 con 0x560d1de63bd0 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb53fff640 1 -- 172.21.15.100:0/1303856374 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb64005500 con 0x560d1de63bd0 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb53fff640 1 -- 172.21.15.100:0/1303856374 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64004ea0 con 0x560d1de63bd0 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 -- 172.21.15.100:0/1303856374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 msgr2=0x560d1de6c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 0x560d1de6c270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400b710 tx=0x7ffb64004610 comp rx=0 tx=0).stop 2024-03-12T05:14:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 -- 172.21.15.100:0/1303856374 shutdown_connections 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de6c7b0 0x560d1de6eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de63bd0 0x560d1de6c270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1303856374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de63690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 -- 172.21.15.100:0/1303856374 >> 172.21.15.100:0/1303856374 conn(0x560d1de5d990 msgr2=0x560d1de5dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 -- 172.21.15.100:0/1303856374 shutdown_connections 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.472+0000 7ffb72a52740 1 -- 172.21.15.100:0/1303856374 wait complete. 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 -- start start 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 Processor -- start 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de69530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de6c7b0 0x560d1de69b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1da9e430 con 0x560d1de63bd0 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1da5ae00 con 0x560d1de632b0 2024-03-12T05:14:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.473+0000 7ffb72a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1dd4ffd0 con 0x560d1de6c7b0 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de69530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6d41c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb72a52740 10 client.0 _ll_register_callbacks cb 0x560d1dda5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de69530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42052/0 (socket says 172.21.15.100:42052) 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6d41c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de6c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48656/0 (socket says 172.21.15.100:48656) 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/2776038431 learned_addr learned my addr 172.21.15.100:0/2776038431 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de6c7b0 0x560d1de69b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/2776038431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de6c7b0 msgr2=0x560d1de69b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de6c7b0 0x560d1de69b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 msgr2=0x560d1de6c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de6c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/2776038431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb640042f0 con 0x560d1de63bd0 2024-03-12T05:14:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de69530 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffb64004290 tx=0x7ffb640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c00af5711ece495 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb517fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb517fa640 1 -- 172.21.15.100:0/2776038431 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64004830 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb517fa640 1 -- 172.21.15.100:0/2776038431 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffb64004ba0 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d1daca7e0 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb517fa640 1 -- 172.21.15.100:0/2776038431 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb64004d60 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb72a52740 10 client.15966 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560d1ddc9610 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.474+0000 7ffb517fa640 1 -- 172.21.15.100:0/2776038431 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb64030020 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb517fa640 1 -- 172.21.15.100:0/2776038431 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ffb64011b70 con 0x560d1de63bd0 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb72a52740 10 client.15966 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb72a52740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb72a52740 20 client.15966 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb72a52740 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1de7c870 0x560d1de7ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb72a52740 4 client.15966 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d1db07720 con 0x560d1de7c870 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1de7c870 0x560d1de7ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb517fa640 10 client.15966 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.475+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1de7c870 0x560d1de7ec90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c077d354f63da9 server_cookie=9dd429a9b0a87694 in_seq=0 out_seq=0 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb517fa640 1 -- 172.21.15.100:0/2776038431 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d1db07720 con 0x560d1de7c870 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb517fa640 10 client.15966 handle_command_reply: tid=0 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 client.15966 shutdown 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 msgr2=0x560d1de69530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de69530 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffb64004290 tx=0x7ffb640046c0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 shutdown_connections 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1de7c870 0x560d1de7ec90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de6c7b0 0x560d1de69b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de69530 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb517fa640 0 client.15966 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 --2- 172.21.15.100:0/2776038431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de632b0 0x560d1de6c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 >> 172.21.15.100:0/2776038431 conn(0x560d1de5d990 msgr2=0x560d1de5f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 shutdown_connections 2024-03-12T05:14:07.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 1 -- 172.21.15.100:0/2776038431 wait complete. 2024-03-12T05:14:07.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.476+0000 7ffb72a52740 20 client.15966 trim_cache size 0 max 16384 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 Processor -- start 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 -- start start 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de6bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1de67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1de5d1d0 con 0x560d1de63bd0 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1de62c80 con 0x560d1de6c7b0 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb72a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1de64f60 con 0x560d1de632b0 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de6c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42068/0 (socket says 172.21.15.100:42068) 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1749626985 learned_addr learned my addr 172.21.15.100:0/1749626985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de6bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.491+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1de67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1749626985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 msgr2=0x560d1de6bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de6bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1749626985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 msgr2=0x560d1de67e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1de67e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb6dc1d640 1 -- 172.21.15.100:0/1749626985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d1dd65ac0 con 0x560d1de63bd0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de6c1a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffb640042c0 tx=0x7ffb64003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8c6e53930fadda server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb537fe640 1 -- 172.21.15.100:0/1749626985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64004930 con 0x560d1de63bd0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb537fe640 1 -- 172.21.15.100:0/1749626985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb64030030 con 0x560d1de63bd0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.492+0000 7ffb537fe640 1 -- 172.21.15.100:0/1749626985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64004d70 con 0x560d1de63bd0 2024-03-12T05:14:07.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- 172.21.15.100:0/1749626985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 msgr2=0x560d1de6c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de6c1a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffb640042c0 tx=0x7ffb64003ac0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- 172.21.15.100:0/1749626985 shutdown_connections 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1de67e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de6c1a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- 172.21.15.100:0/1749626985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de6bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- 172.21.15.100:0/1749626985 >> 172.21.15.100:0/1749626985 conn(0x560d1de5d990 msgr2=0x560d1de64850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- 172.21.15.100:0/1749626985 shutdown_connections 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- 172.21.15.100:0/1749626985 wait complete. 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- start start 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 Processor -- start 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de66500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de66a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1ded2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1de67310 con 0x560d1de63bd0 2024-03-12T05:14:07.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1ded32d0 con 0x560d1de6c7b0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1ded3410 con 0x560d1de632b0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb72a52740 10 client.0 _ll_register_callbacks cb 0x7ffb640040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb6dc1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de66a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.493+0000 7ffb6d41c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de66500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de66500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:47564/0 (socket says 172.21.15.100:47564) 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 -- 172.21.15.100:0/3091932415 learned_addr learned my addr 172.21.15.100:0/3091932415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1ded2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 -- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 msgr2=0x560d1ded2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1ded2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 -- 172.21.15.100:0/3091932415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 msgr2=0x560d1de66a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de66a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 -- 172.21.15.100:0/3091932415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d1ddc9610 con 0x560d1de632b0 2024-03-12T05:14:07.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb6d41c640 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de66500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb58010fd0 tx=0x7ffb640056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911ff0c0cb70bb32 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb517fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb517fa640 1 -- 172.21.15.100:0/3091932415 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58028040 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb64005500 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb517fa640 1 -- 172.21.15.100:0/3091932415 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffb58004230 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb517fa640 1 -- 172.21.15.100:0/3091932415 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb58015040 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb72a52740 10 client.25465 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.494+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560d1dd8b6a0 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb517fa640 1 -- 172.21.15.100:0/3091932415 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb5803b080 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb517fa640 1 -- 172.21.15.100:0/3091932415 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ffb58004be0 con 0x560d1de632b0 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb72a52740 10 client.25465 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb72a52740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb72a52740 20 client.25465 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb72a52740 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1ded4a90 0x560d1ded6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:07.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb72a52740 4 client.25465 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d1db56890 con 0x560d1ded4a90 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1ded4a90 0x560d1ded6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb517fa640 10 client.25465 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb6dc1d640 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1ded4a90 0x560d1ded6eb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8940855130e154be server_cookie=e71fffc1af23b1a5 in_seq=0 out_seq=0 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb517fa640 1 -- 172.21.15.100:0/3091932415 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560d1db56890 con 0x560d1ded4a90 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.495+0000 7ffb517fa640 10 client.25465 handle_command_reply: tid=0 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 client.25465 shutdown 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 msgr2=0x560d1de66500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de66500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb58010fd0 tx=0x7ffb640056d0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 shutdown_connections 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560d1ded4a90 0x560d1ded6eb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560d1de6c7b0 0x560d1ded2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d1de63bd0 0x560d1de66a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 --2- 172.21.15.100:0/3091932415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d1de632b0 0x560d1de66500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb517fa640 0 client.25465 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 >> 172.21.15.100:0/3091932415 conn(0x560d1de5d990 msgr2=0x560d1de67d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 shutdown_connections 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 1 -- 172.21.15.100:0/3091932415 wait complete. 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb72a52740 20 client.25465 trim_cache size 0 max 16384 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stdout: "43a4bb7d-e71f-44c4-a807-310c4e182b1a": { 2024-03-12T05:14:07.499 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:14:07.500 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "43a4bb7d-e71f-44c4-a807-310c4e182b1a", 2024-03-12T05:14:07.500 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:14:07.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:14:07.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:14:07.500 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffb540a1ba0 msgr2=0x7ffb540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffb540a1ba0 0x7ffb540a4060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb600bcaf0 tx=0x7ffb600b9090 comp rx=0 tx=0).stop 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 msgr2=0x7ffb681a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 0x7ffb681a3690 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009bf0 tx=0x7ffb60034cc0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 shutdown_connections 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffb540a1ba0 0x7ffb540a4060 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68106110 0x7ffb681a3690 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb68100c90 0x7ffb68198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 --2- 172.21.15.100:0/1915337006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb681002d0 0x7ffb68198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 >> 172.21.15.100:0/1915337006 conn(0x7ffb68076e20 msgr2=0x7ffb680fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.496+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 shutdown_connections 2024-03-12T05:14:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:07.497+0000 7ffb6fe81640 1 -- 172.21.15.100:0/1915337006 wait complete. 2024-03-12T05:14:07.504 INFO:tasks.cephfs.filesystem:scrub status for tag:43a4bb7d-e71f-44c4-a807-310c4e182b1a - {'path': '/', 'tag': '43a4bb7d-e71f-44c4-a807-310c4e182b1a', 'options': 'recursive,force'} 2024-03-12T05:14:07.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 131684 14.28 MB/sec warmup 51 sec latency 13.284 ms 2024-03-12T05:14:08.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:07 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:14:08.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:07 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:07 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:14:08.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:07 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:14:08.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:07 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:07 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:07 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:14:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:07 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:07 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:14:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:07 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:14:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:07 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:07 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:07 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:14:08.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:07 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:07 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:14:08.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:07 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:14:08.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:07 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:07 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:08.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 134155 14.23 MB/sec warmup 52 sec latency 124.529 ms 2024-03-12T05:14:09.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:08 smithi156 ceph-mon[36270]: pgmap v64: 97 pgs: 97 active+clean; 79 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 389 op/s 2024-03-12T05:14:09.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:08 smithi161 ceph-mon[36043]: pgmap v64: 97 pgs: 97 active+clean; 79 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 389 op/s 2024-03-12T05:14:09.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:08 smithi100 ceph-mon[32210]: pgmap v64: 97 pgs: 97 active+clean; 79 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 389 op/s 2024-03-12T05:14:09.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 136255 14.20 MB/sec warmup 53 sec latency 19.760 ms 2024-03-12T05:14:10.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:09 smithi161 ceph-mon[36043]: pgmap v65: 97 pgs: 97 active+clean; 79 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 278 op/s 2024-03-12T05:14:10.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:09 smithi100 ceph-mon[32210]: pgmap v65: 97 pgs: 97 active+clean; 79 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 278 op/s 2024-03-12T05:14:10.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:09 smithi156 ceph-mon[36270]: pgmap v65: 97 pgs: 97 active+clean; 79 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 278 op/s 2024-03-12T05:14:10.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 138850 14.20 MB/sec warmup 54 sec latency 37.301 ms 2024-03-12T05:14:11.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 141940 14.26 MB/sec warmup 55 sec latency 8.310 ms 2024-03-12T05:14:12.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 144257 14.16 MB/sec warmup 56 sec latency 22.712 ms 2024-03-12T05:14:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:12 smithi156 ceph-mon[36270]: pgmap v66: 97 pgs: 97 active+clean; 83 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 357 op/s 2024-03-12T05:14:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:12 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:14:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:12 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:12 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:12 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:12 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:12 smithi161 ceph-mon[36043]: pgmap v66: 97 pgs: 97 active+clean; 83 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 357 op/s 2024-03-12T05:14:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:12 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:14:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:12 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:12 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:12 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:12 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:12 smithi100 ceph-mon[32210]: pgmap v66: 97 pgs: 97 active+clean; 83 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 357 op/s 2024-03-12T05:14:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:12 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:14:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:12 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:12 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:12 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:12 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:13.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 146019 14.12 MB/sec warmup 57 sec latency 75.879 ms 2024-03-12T05:14:14.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:13 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:14:14.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:13 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:14:14.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:13 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:14.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:13 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:14:14.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:13 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:14:14.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:13 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:14.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:13 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:14:14.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:13 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:14:14.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:13 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:14:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 149010 14.18 MB/sec warmup 58 sec latency 28.252 ms 2024-03-12T05:14:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:14 smithi156 ceph-mon[36270]: pgmap v67: 97 pgs: 97 active+clean; 84 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-03-12T05:14:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:14 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:14:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:14 smithi161 ceph-mon[36043]: pgmap v67: 97 pgs: 97 active+clean; 84 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-03-12T05:14:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:14 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:14:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:14 smithi100 ceph-mon[32210]: pgmap v67: 97 pgs: 97 active+clean; 84 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-03-12T05:14:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:14 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:14:15.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 151486 14.10 MB/sec warmup 59 sec latency 12.285 ms 2024-03-12T05:14:16.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:15 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:14:16.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:15 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:14:16.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:15 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:14:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 153729 14.09 MB/sec warmup 60 sec latency 12.294 ms 2024-03-12T05:14:17.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:16 smithi156 ceph-mon[36270]: pgmap v68: 97 pgs: 97 active+clean; 84 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-03-12T05:14:17.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:16 smithi161 ceph-mon[36043]: pgmap v68: 97 pgs: 97 active+clean; 84 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-03-12T05:14:17.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:16 smithi100 ceph-mon[32210]: pgmap v68: 97 pgs: 97 active+clean; 84 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-03-12T05:14:17.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 156655 14.14 MB/sec warmup 61 sec latency 8.817 ms 2024-03-12T05:14:18.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 159387 14.14 MB/sec warmup 62 sec latency 79.975 ms 2024-03-12T05:14:19.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:18 smithi156 ceph-mon[36270]: pgmap v69: 97 pgs: 97 active+clean; 92 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 379 op/s 2024-03-12T05:14:19.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:18 smithi161 ceph-mon[36043]: pgmap v69: 97 pgs: 97 active+clean; 92 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 379 op/s 2024-03-12T05:14:19.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:18 smithi100 ceph-mon[32210]: pgmap v69: 97 pgs: 97 active+clean; 92 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 379 op/s 2024-03-12T05:14:19.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 161169 14.07 MB/sec warmup 63 sec latency 25.194 ms 2024-03-12T05:14:20.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1 163760 14.07 MB/sec warmup 64 sec latency 7.921 ms 2024-03-12T05:14:21.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:20 smithi156 ceph-mon[36270]: pgmap v70: 97 pgs: 97 active+clean; 92 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 254 op/s 2024-03-12T05:14:21.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:20 smithi161 ceph-mon[36043]: pgmap v70: 97 pgs: 97 active+clean; 92 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 254 op/s 2024-03-12T05:14:21.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:20 smithi100 ceph-mon[32210]: pgmap v70: 97 pgs: 97 active+clean; 92 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 254 op/s 2024-03-12T05:14:21.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 166972 14.15 MB/sec warmup 65 sec latency 10.439 ms 2024-03-12T05:14:22.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 169448 14.10 MB/sec warmup 66 sec latency 13.861 ms 2024-03-12T05:14:23.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:22 smithi156 ceph-mon[36270]: pgmap v71: 97 pgs: 97 active+clean; 98 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 366 op/s 2024-03-12T05:14:23.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:22 smithi161 ceph-mon[36043]: pgmap v71: 97 pgs: 97 active+clean; 98 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 366 op/s 2024-03-12T05:14:23.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:22 smithi100 ceph-mon[32210]: pgmap v71: 97 pgs: 97 active+clean; 98 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 366 op/s 2024-03-12T05:14:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 171366 14.04 MB/sec warmup 67 sec latency 88.019 ms 2024-03-12T05:14:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 174234 14.09 MB/sec warmup 68 sec latency 20.669 ms 2024-03-12T05:14:25.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:24 smithi100 ceph-mon[32210]: pgmap v72: 97 pgs: 97 active+clean; 98 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-03-12T05:14:25.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:24 smithi156 ceph-mon[36270]: pgmap v72: 97 pgs: 97 active+clean; 98 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-03-12T05:14:25.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:24 smithi161 ceph-mon[36043]: pgmap v72: 97 pgs: 97 active+clean; 98 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-03-12T05:14:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 177183 14.09 MB/sec warmup 69 sec latency 7.862 ms 2024-03-12T05:14:26.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 179701 14.06 MB/sec warmup 70 sec latency 34.923 ms 2024-03-12T05:14:27.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:26 smithi156 ceph-mon[36270]: pgmap v73: 97 pgs: 97 active+clean; 98 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 252 op/s 2024-03-12T05:14:27.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:26 smithi161 ceph-mon[36043]: pgmap v73: 97 pgs: 97 active+clean; 98 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 252 op/s 2024-03-12T05:14:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:26 smithi100 ceph-mon[32210]: pgmap v73: 97 pgs: 97 active+clean; 98 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 252 op/s 2024-03-12T05:14:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 181981 14.06 MB/sec warmup 71 sec latency 8.685 ms 2024-03-12T05:14:28.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 184854 14.10 MB/sec warmup 72 sec latency 44.145 ms 2024-03-12T05:14:29.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:28 smithi156 ceph-mon[36270]: pgmap v74: 97 pgs: 97 active+clean; 104 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-03-12T05:14:29.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:28 smithi161 ceph-mon[36043]: pgmap v74: 97 pgs: 97 active+clean; 104 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-03-12T05:14:29.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:28 smithi100 ceph-mon[32210]: pgmap v74: 97 pgs: 97 active+clean; 104 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-03-12T05:14:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 187449 14.08 MB/sec warmup 73 sec latency 76.058 ms 2024-03-12T05:14:30.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 189852 14.05 MB/sec warmup 74 sec latency 11.428 ms 2024-03-12T05:14:31.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:30 smithi156 ceph-mon[36270]: pgmap v75: 97 pgs: 97 active+clean; 104 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-03-12T05:14:31.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:30 smithi161 ceph-mon[36043]: pgmap v75: 97 pgs: 97 active+clean; 104 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-03-12T05:14:31.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:30 smithi100 ceph-mon[32210]: pgmap v75: 97 pgs: 97 active+clean; 104 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-03-12T05:14:31.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192450 14.06 MB/sec warmup 75 sec latency 11.648 ms 2024-03-12T05:14:32.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 195528 14.11 MB/sec warmup 76 sec latency 15.085 ms 2024-03-12T05:14:33.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:32 smithi156 ceph-mon[36270]: pgmap v76: 97 pgs: 97 active+clean; 107 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-03-12T05:14:33.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:32 smithi161 ceph-mon[36043]: pgmap v76: 97 pgs: 97 active+clean; 107 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-03-12T05:14:33.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:32 smithi100 ceph-mon[32210]: pgmap v76: 97 pgs: 97 active+clean; 107 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-03-12T05:14:33.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 198639 14.16 MB/sec warmup 77 sec latency 11.841 ms 2024-03-12T05:14:34.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 200596 14.07 MB/sec warmup 78 sec latency 102.985 ms 2024-03-12T05:14:35.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:34 smithi156 ceph-mon[36270]: pgmap v77: 97 pgs: 97 active+clean; 109 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-03-12T05:14:35.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:34 smithi161 ceph-mon[36043]: pgmap v77: 97 pgs: 97 active+clean; 109 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-03-12T05:14:35.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:34 smithi100 ceph-mon[32210]: pgmap v77: 97 pgs: 97 active+clean; 109 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-03-12T05:14:35.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 203099 14.07 MB/sec warmup 79 sec latency 10.033 ms 2024-03-12T05:14:36.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 206139 14.13 MB/sec warmup 80 sec latency 9.226 ms 2024-03-12T05:14:37.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:36 smithi156 ceph-mon[36270]: pgmap v78: 97 pgs: 97 active+clean; 109 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-03-12T05:14:37.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:36 smithi161 ceph-mon[36043]: pgmap v78: 97 pgs: 97 active+clean; 109 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-03-12T05:14:37.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:36 smithi100 ceph-mon[32210]: pgmap v78: 97 pgs: 97 active+clean; 109 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-03-12T05:14:37.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:14:37.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 -- 172.21.15.100:0/3880283048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101d10 msgr2=0x7fc894105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/3880283048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101d10 0x7fc894105d50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc884009930 tx=0x7fc88402bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 -- 172.21.15.100:0/3880283048 shutdown_connections 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/3880283048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc894106430 0x7fc89410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/3880283048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101d10 0x7fc894105d50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/3880283048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc894101360 0x7fc894101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 -- 172.21.15.100:0/3880283048 >> 172.21.15.100:0/3880283048 conn(0x7fc8940fcc00 msgr2=0x7fc8940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 -- 172.21.15.100:0/3880283048 shutdown_connections 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.618+0000 7fc89c7c7640 1 -- 172.21.15.100:0/3880283048 wait complete. 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 Processor -- start 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 -- start start 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 0x7fc894073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc894106430 0x7fc8940739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc894073f00 0x7fc8941a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8941a5f50 con 0x7fc894101360 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8941a60c0 con 0x7fc894073f00 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89c7c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8941a6280 con 0x7fc894106430 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 0x7fc894073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 0x7fc894073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58498/0 (socket says 172.21.15.100:58498) 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 -- 172.21.15.100:0/281021763 learned_addr learned my addr 172.21.15.100:0/281021763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89ad64640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc894073f00 0x7fc8941a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc899d62640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc894106430 0x7fc8940739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 -- 172.21.15.100:0/281021763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc894106430 msgr2=0x7fc8940739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc894106430 0x7fc8940739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 -- 172.21.15.100:0/281021763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc894073f00 msgr2=0x7fc8941a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc894073f00 0x7fc8941a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 -- 172.21.15.100:0/281021763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009560 con 0x7fc894101360 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.619+0000 7fc89a563640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 0x7fc894073480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9f0 tx=0x7fc89000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.620+0000 7fc88b7fe640 1 -- 172.21.15.100:0/281021763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890009720 con 0x7fc894101360 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.620+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8941a6560 con 0x7fc894101360 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.620+0000 7fc88b7fe640 1 -- 172.21.15.100:0/281021763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc89000cd80 con 0x7fc894101360 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.620+0000 7fc88b7fe640 1 -- 172.21.15.100:0/281021763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890010690 con 0x7fc894101360 2024-03-12T05:14:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.620+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8941a6a70 con 0x7fc894101360 2024-03-12T05:14:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc88b7fe640 1 -- 172.21.15.100:0/281021763 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc8900046c0 con 0x7fc894101360 2024-03-12T05:14:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 Processor -- start 2024-03-12T05:14:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 -- start start 2024-03-12T05:14:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc88b7fe640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 0x5616c90f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c90ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c9065280 con 0x5616c90ff7b0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c90627b0 con 0x5616c90f62b0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.622+0000 7fc89f398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c905df90 con 0x5616c90f6bd0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89ad64640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 0x5616c90f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc88b7fe640 1 -- 172.21.15.100:0/281021763 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc890014070 con 0x7fc894101360 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc899d62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c90ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89ad64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 0x5616c90f6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:54898/0 (socket says 172.21.15.100:54898) 2024-03-12T05:14:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc899d62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c90ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:45000/0 (socket says 172.21.15.100:45000) 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 -- 172.21.15.100:0/4170187735 learned_addr learned my addr 172.21.15.100:0/4170187735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 -- 172.21.15.100:0/4170187735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 msgr2=0x5616c90ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c90ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 -- 172.21.15.100:0/4170187735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 msgr2=0x5616c9101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 -- 172.21.15.100:0/4170187735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616c901c570 con 0x5616c90f62b0 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc899d62640 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c90ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89ad64640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc88c005fd0 tx=0x7fc88c009e90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc89a563640 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 0x5616c90f6690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc890097230 tx=0x7fc89009a3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82e4c12ee538733 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc888ff9640 1 -- 172.21.15.100:0/4170187735 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89006b180 con 0x5616c90f62b0 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc888ff9640 1 -- 172.21.15.100:0/4170187735 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc89009a860 con 0x5616c90f62b0 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.623+0000 7fc888ff9640 1 -- 172.21.15.100:0/4170187735 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89006adb0 con 0x5616c90f62b0 2024-03-12T05:14:37.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 -- 172.21.15.100:0/4170187735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 msgr2=0x5616c90f6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 0x5616c90f6690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc890097230 tx=0x7fc89009a3a0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 -- 172.21.15.100:0/4170187735 shutdown_connections 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c90ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 --2- 172.21.15.100:0/4170187735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f62b0 0x5616c90f6690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 -- 172.21.15.100:0/4170187735 >> 172.21.15.100:0/4170187735 conn(0x5616c90f0990 msgr2=0x5616c90f0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 -- 172.21.15.100:0/4170187735 shutdown_connections 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 -- 172.21.15.100:0/4170187735 wait complete. 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.624+0000 7fc89f398740 1 -- start start 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 Processor -- start 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c90ff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c90fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90ff7b0 0x5616c90fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c8d31430 con 0x5616c90f62b0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c8cede00 con 0x5616c90f6bd0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c8fe2fd0 con 0x5616c90ff7b0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89f398740 10 client.0 _ll_register_callbacks cb 0x5616c9038fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c90ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c90ff260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58514/0 (socket says 172.21.15.100:58514) 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 -- 172.21.15.100:0/2084984966 learned_addr learned my addr 172.21.15.100:0/2084984966 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc899d62640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c90fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89ad64640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90ff7b0 0x5616c90fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 -- 172.21.15.100:0/2084984966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90ff7b0 msgr2=0x5616c90fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90ff7b0 0x5616c90fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 -- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 msgr2=0x5616c90fc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c90fc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 -- 172.21.15.100:0/2084984966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8941a6a70 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc899d62640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c90fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89ad64640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90ff7b0 0x5616c90fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc89a563640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c90ff260 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc8900cb430 tx=0x7fc89009e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4020e1dd5d4d8a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc87dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.625+0000 7fc87dffb640 1 -- 172.21.15.100:0/2084984966 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900cb600 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc87dffb640 1 -- 172.21.15.100:0/2084984966 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc8900600d0 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616c8d5d7e0 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc87dffb640 1 -- 172.21.15.100:0/2084984966 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8900602d0 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 10 client.15984 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616c905c610 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc87dffb640 1 -- 172.21.15.100:0/2084984966 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616c8d5d7e0 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc87dffb640 1 -- 172.21.15.100:0/2084984966 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc89001f060 con 0x5616c90f62b0 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 10 client.15984 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:37.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 20 client.15984 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c910f5d0 0x5616c91119c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 4 client.15984 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616c8d9a720 con 0x5616c910f5d0 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.626+0000 7fc899d62640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c910f5d0 0x5616c91119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc87dffb640 10 client.15984 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc899d62640 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c910f5d0 0x5616c91119c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8bc845d1e78c72 server_cookie=a50e98a62e5adc5 in_seq=0 out_seq=0 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc87dffb640 1 -- 172.21.15.100:0/2084984966 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616c8d9a720 con 0x5616c910f5d0 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc87dffb640 10 client.15984 handle_command_reply: tid=0 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 client.15984 shutdown 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 msgr2=0x5616c90ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c90ff260 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc8900cb430 tx=0x7fc89009e9b0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 shutdown_connections 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c910f5d0 0x5616c91119c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90ff7b0 0x5616c90fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c90fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc87dffb640 0 client.15984 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 --2- 172.21.15.100:0/2084984966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c90ff260 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 >> 172.21.15.100:0/2084984966 conn(0x5616c90f0990 msgr2=0x5616c90f2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 shutdown_connections 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.627+0000 7fc89f398740 1 -- 172.21.15.100:0/2084984966 wait complete. 2024-03-12T05:14:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.628+0000 7fc89f398740 20 client.15984 trim_cache size 0 max 16384 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 Processor -- start 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 -- start start 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c9106600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c9106b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90ff7b0 0x5616c90fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c90fd5d0 con 0x5616c90f62b0 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c8d9a750 con 0x5616c90ff7b0 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89f398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c90ec0f0 con 0x5616c90f6bd0 2024-03-12T05:14:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89a563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c9106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc899d62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c9106b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89a563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c9106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58516/0 (socket says 172.21.15.100:58516) 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89a563640 1 -- 172.21.15.100:0/3066082428 learned_addr learned my addr 172.21.15.100:0/3066082428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.643+0000 7fc89ad64640 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90ff7b0 0x5616c90fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89a563640 1 -- 172.21.15.100:0/3066082428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 msgr2=0x5616c9106b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89a563640 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c9106b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89a563640 1 -- 172.21.15.100:0/3066082428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90ff7b0 msgr2=0x5616c90fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89a563640 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90ff7b0 0x5616c90fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89a563640 1 -- 172.21.15.100:0/3066082428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616c8ff8ac0 con 0x5616c90f62b0 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc899d62640 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c9106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89a563640 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c9106600 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9c0 tx=0x7fc89009a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e2a6e3ececae26 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc87ffff640 1 -- 172.21.15.100:0/3066082428 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89009ea20 con 0x5616c90f62b0 2024-03-12T05:14:37.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc87ffff640 1 -- 172.21.15.100:0/3066082428 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc89009a1c0 con 0x5616c90f62b0 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc87ffff640 1 -- 172.21.15.100:0/3066082428 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890060330 con 0x5616c90f62b0 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89f398740 1 -- 172.21.15.100:0/3066082428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 msgr2=0x5616c9106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.644+0000 7fc89f398740 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c9106600 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9c0 tx=0x7fc89009a3a0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- 172.21.15.100:0/3066082428 shutdown_connections 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90ff7b0 0x5616c90fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f6bd0 0x5616c9106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 --2- 172.21.15.100:0/3066082428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90f62b0 0x5616c9106600 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- 172.21.15.100:0/3066082428 >> 172.21.15.100:0/3066082428 conn(0x5616c90f0990 msgr2=0x5616c90faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- 172.21.15.100:0/3066082428 shutdown_connections 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- 172.21.15.100:0/3066082428 wait complete. 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- start start 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 Processor -- start 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f62b0 0x5616c91053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c9105900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9165ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c91061d0 con 0x5616c90ff7b0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c9166000 con 0x5616c90f6bd0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c9166140 con 0x5616c90f62b0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89f398740 10 client.0 _ll_register_callbacks cb 0x7fc890004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9165ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9165ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58530/0 (socket says 172.21.15.100:58530) 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc899d62640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c9105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89a563640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f62b0 0x5616c91053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 -- 172.21.15.100:0/1280399579 learned_addr learned my addr 172.21.15.100:0/1280399579 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 -- 172.21.15.100:0/1280399579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f62b0 msgr2=0x5616c91053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f62b0 0x5616c91053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 -- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 msgr2=0x5616c9105900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.645+0000 7fc89ad64640 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c9105900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89ad64640 1 -- 172.21.15.100:0/1280399579 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc884030070 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89ad64640 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9165ac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00fc40 tx=0x7fc88c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc1e80d69cd51bc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc87dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc87dffb640 1 -- 172.21.15.100:0/1280399579 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c025070 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc87dffb640 1 -- 172.21.15.100:0/1280399579 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc88c004b20 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc89009a860 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc87dffb640 1 -- 172.21.15.100:0/1280399579 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc88c004ce0 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 10 client.15990 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616c905c610 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc87dffb640 1 -- 172.21.15.100:0/1280399579 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc89009a860 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc87dffb640 1 -- 172.21.15.100:0/1280399579 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc88c01d870 con 0x5616c90ff7b0 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 10 client.15990 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 20 client.15990 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c9167950 0x5616c9169d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 4 client.15990 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616c8de9890 con 0x5616c9167950 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.646+0000 7fc89a563640 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c9167950 0x5616c9169d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc87dffb640 10 client.15990 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89a563640 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c9167950 0x5616c9169d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d11bad219bb2ed3 server_cookie=b922f365e6aa5894 in_seq=0 out_seq=0 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc87dffb640 1 -- 172.21.15.100:0/1280399579 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5616c8de9890 con 0x5616c9167950 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc87dffb640 10 client.15990 handle_command_reply: tid=0 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 client.15990 shutdown 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 msgr2=0x5616c9165ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9165ac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00fc40 tx=0x7fc88c009500 comp rx=0 tx=0).stop 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 shutdown_connections 2024-03-12T05:14:37.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616c9167950 0x5616c9169d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c90ff7b0 0x5616c9165ac0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc87dffb640 0 client.15990 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616c90f6bd0 0x5616c9105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 --2- 172.21.15.100:0/1280399579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616c90f62b0 0x5616c91053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 >> 172.21.15.100:0/1280399579 conn(0x5616c90f0990 msgr2=0x5616c90ef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 shutdown_connections 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 1 -- 172.21.15.100:0/1280399579 wait complete. 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.647+0000 7fc89f398740 20 client.15990 trim_cache size 0 max 16384 2024-03-12T05:14:37.651 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:14:37.652 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:14:37.652 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:14:37.652 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:14:37.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 msgr2=0x7fc8800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc88c005fd0 tx=0x7fc88c009e90 comp rx=0 tx=0).stop 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 msgr2=0x7fc894073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 0x7fc894073480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9f0 tx=0x7fc89000edf0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 shutdown_connections 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc894073f00 0x7fc8941a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc894106430 0x7fc8940739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 --2- 172.21.15.100:0/281021763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc894101360 0x7fc894073480 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 >> 172.21.15.100:0/281021763 conn(0x7fc8940fcc00 msgr2=0x7fc8940fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 shutdown_connections 2024-03-12T05:14:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.648+0000 7fc89c7c7640 1 -- 172.21.15.100:0/281021763 wait complete. 2024-03-12T05:14:37.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:14:37.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:14:37.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 -- 172.21.15.100:0/3759370623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 msgr2=0x7f461410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 --2- 172.21.15.100:0/3759370623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461410b1f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4608009900 tx=0x7f460802bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:37.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 -- 172.21.15.100:0/3759370623 shutdown_connections 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 --2- 172.21.15.100:0/3759370623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f461410b7c0 0x7f461410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 --2- 172.21.15.100:0/3759370623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461410b1f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 --2- 172.21.15.100:0/3759370623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f46141027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 -- 172.21.15.100:0/3759370623 >> 172.21.15.100:0/3759370623 conn(0x7f4614077180 msgr2=0x7f4614077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 -- 172.21.15.100:0/3759370623 shutdown_connections 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.763+0000 7f461a89d640 1 -- 172.21.15.100:0/3759370623 wait complete. 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 Processor -- start 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 -- start start 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f461419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f461410b7c0 0x7f4614197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614197df0 con 0x7f4614102d10 2024-03-12T05:14:37.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614197f60 con 0x7f46141023f0 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f461a89d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614198120 con 0x7f461410b7c0 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f46137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f4613fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f461419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f46137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58540/0 (socket says 172.21.15.100:58540) 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f4613fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f461419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:54946/0 (socket says 172.21.15.100:54946) 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f46137fe640 1 -- 172.21.15.100:0/2762532777 learned_addr learned my addr 172.21.15.100:0/2762532777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.764+0000 7f4618e3a640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f461410b7c0 0x7f4614197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46137fe640 1 -- 172.21.15.100:0/2762532777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f461410b7c0 msgr2=0x7f4614197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46137fe640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f461410b7c0 0x7f4614197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46137fe640 1 -- 172.21.15.100:0/2762532777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 msgr2=0x7f461419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46137fe640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f461419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46137fe640 1 -- 172.21.15.100:0/2762532777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608009560 con 0x7f4614102d10 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f4613fff640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f461419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46137fe640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461419dae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4608005940 tx=0x7f460802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080043d0 con 0x7f4614102d10 2024-03-12T05:14:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46080052d0 con 0x7f4614102d10 2024-03-12T05:14:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46141983a0 con 0x7f4614102d10 2024-03-12T05:14:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608003d30 con 0x7f4614102d10 2024-03-12T05:14:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.765+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46141a4520 con 0x7f4614102d10 2024-03-12T05:14:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.766+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4614068b20 con 0x7f4614102d10 2024-03-12T05:14:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.767+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4608005440 con 0x7f4614102d10 2024-03-12T05:14:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.767+0000 7f46117fa640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45fc0a1c20 0x7f45fc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.768+0000 7f4613fff640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45fc0a1c20 0x7f45fc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.768+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f46080ef7b0 con 0x7f4614102d10 2024-03-12T05:14:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.768+0000 7f4613fff640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45fc0a1c20 0x7f45fc0a40e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4604009fc0 tx=0x7f4604005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:37.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.772+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f46080b8780 con 0x7f4614102d10 2024-03-12T05:14:37.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 209160 14.14 MB/sec warmup 81 sec latency 12.992 ms 2024-03-12T05:14:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.993+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46141068b0 con 0x7f4614102d10 2024-03-12T05:14:37.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.994+0000 7f46117fa640 1 -- 172.21.15.100:0/2762532777 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7f46080bbe40 con 0x7f4614102d10 2024-03-12T05:14:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:14:37.990 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:13:19.700942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:14:37.991 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 24 2024-03-12T05:14:37.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45fc0a1c20 msgr2=0x7f45fc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45fc0a1c20 0x7f45fc0a40e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4604009fc0 tx=0x7f4604005f90 comp rx=0 tx=0).stop 2024-03-12T05:14:37.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 msgr2=0x7f461419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461419dae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4608005940 tx=0x7f460802c6e0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 shutdown_connections 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45fc0a1c20 0x7f45fc0a40e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f461410b7c0 0x7f4614197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614102d10 0x7f461419dae0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 --2- 172.21.15.100:0/2762532777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141023f0 0x7f461419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 >> 172.21.15.100:0/2762532777 conn(0x7f4614077180 msgr2=0x7f46140fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 shutdown_connections 2024-03-12T05:14:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:37.996+0000 7f461a89d640 1 -- 172.21.15.100:0/2762532777 wait complete. 2024-03-12T05:14:38.003 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:14:38.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:14:38.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 -- 172.21.15.100:0/1492133952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7941055b0 msgr2=0x7fb794105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 --2- 172.21.15.100:0/1492133952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7941055b0 0x7fb794105a60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009cd0 tx=0x7fb78c02c180 comp rx=0 tx=0).stop 2024-03-12T05:14:38.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 -- 172.21.15.100:0/1492133952 shutdown_connections 2024-03-12T05:14:38.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 --2- 172.21.15.100:0/1492133952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7941055b0 0x7fb794105a60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 --2- 172.21.15.100:0/1492133952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb794100ad0 0x7fb794104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 --2- 172.21.15.100:0/1492133952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7941001b0 0x7fb794100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 -- 172.21.15.100:0/1492133952 >> 172.21.15.100:0/1492133952 conn(0x7fb794076400 msgr2=0x7fb794076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 -- 172.21.15.100:0/1492133952 shutdown_connections 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.113+0000 7fb79b89a640 1 -- 172.21.15.100:0/1492133952 wait complete. 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 Processor -- start 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 -- start start 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7941001b0 0x7fb79419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 0x7fb79419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7941055b0 0x7fb7941a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7941a2ca0 con 0x7fb794100ad0 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7941a2e10 con 0x7fb7941001b0 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb79b89a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7941a2fd0 con 0x7fb7941055b0 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb798e35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 0x7fb79419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb798e35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 0x7fb79419e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58560/0 (socket says 172.21.15.100:58560) 2024-03-12T05:14:38.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb798e35640 1 -- 172.21.15.100:0/2141472818 learned_addr learned my addr 172.21.15.100:0/2141472818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.114+0000 7fb799636640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7941001b0 0x7fb79419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb799e37640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7941055b0 0x7fb7941a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb798e35640 1 -- 172.21.15.100:0/2141472818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7941055b0 msgr2=0x7fb7941a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb798e35640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7941055b0 0x7fb7941a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb798e35640 1 -- 172.21.15.100:0/2141472818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7941001b0 msgr2=0x7fb79419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb798e35640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7941001b0 0x7fb79419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb798e35640 1 -- 172.21.15.100:0/2141472818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c009900 con 0x7fb794100ad0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb798e35640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 0x7fb79419e320 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9f0 tx=0x7fb78400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb78a7fc640 1 -- 172.21.15.100:0/2141472818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784009700 con 0x7fb794100ad0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb78a7fc640 1 -- 172.21.15.100:0/2141472818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb78400cd80 con 0x7fb794100ad0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7941a32b0 con 0x7fb794100ad0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb78a7fc640 1 -- 172.21.15.100:0/2141472818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784018920 con 0x7fb794100ad0 2024-03-12T05:14:38.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.115+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7941aab30 con 0x7fb794100ad0 2024-03-12T05:14:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb78a7fc640 1 -- 172.21.15.100:0/2141472818 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb784018a80 con 0x7fb794100ad0 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 Processor -- start 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 -- start start 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd35630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 0x564c2cd3e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd3e750 0x564c2cd40b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ccf0960 con 0x564c2cd35b70 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cca4310 con 0x564c2cd3e750 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb79e46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cca1840 con 0x564c2cd35250 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb799636640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 0x564c2cd3e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb799e37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd35630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb798e35640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd3e750 0x564c2cd40b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb799e37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd35630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:45062/0 (socket says 172.21.15.100:45062) 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb78a7fc640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb7800a1ba0 0x7fb7800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb799e37640 1 -- 172.21.15.100:0/957396249 learned_addr learned my addr 172.21.15.100:0/957396249 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.118+0000 7fb799636640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 0x564c2cd3e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58574/0 (socket says 172.21.15.100:58574) 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799e37640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb7800a1ba0 0x7fb7800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb78a7fc640 1 -- 172.21.15.100:0/2141472818 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb784014070 con 0x7fb794100ad0 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799636640 1 -- 172.21.15.100:0/957396249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 msgr2=0x564c2cd35630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799636640 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd35630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799636640 1 -- 172.21.15.100:0/957396249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd3e750 msgr2=0x564c2cd40b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799636640 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd3e750 0x564c2cd40b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799636640 1 -- 172.21.15.100:0/957396249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2cc5f060 con 0x564c2cd35b70 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb798e35640 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd3e750 0x564c2cd40b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799636640 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 0x564c2cd3e210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb790009c20 tx=0x7fb790011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69faeb6e093a2f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb799e37640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb7800a1ba0 0x7fb7800a4060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb78c02c790 tx=0x7fb78c02c180 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb77f7fe640 1 -- 172.21.15.100:0/957396249 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790004050 con 0x564c2cd35b70 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb77f7fe640 1 -- 172.21.15.100:0/957396249 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb790005500 con 0x564c2cd35b70 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.119+0000 7fb77f7fe640 1 -- 172.21.15.100:0/957396249 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790004cd0 con 0x564c2cd35b70 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 -- 172.21.15.100:0/957396249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 msgr2=0x564c2cd3e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 0x564c2cd3e210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb790009c20 tx=0x7fb790011810 comp rx=0 tx=0).stop 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 -- 172.21.15.100:0/957396249 shutdown_connections 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd3e750 0x564c2cd40b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd35b70 0x564c2cd3e210 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 --2- 172.21.15.100:0/957396249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd35630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 -- 172.21.15.100:0/957396249 >> 172.21.15.100:0/957396249 conn(0x564c2cd2f980 msgr2=0x564c2cd2fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 -- 172.21.15.100:0/957396249 shutdown_connections 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 -- 172.21.15.100:0/957396249 wait complete. 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 -- start start 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.120+0000 7fb79e46f740 1 Processor -- start 2024-03-12T05:14:38.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35b70 0x564c2cd3bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cc9d020 con 0x564c2cd3e750 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2c970430 con 0x564c2cd35b70 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2c92ce00 con 0x564c2cd35250 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 10 client.0 _ll_register_callbacks cb 0x564c2cc44cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd3c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58576/0 (socket says 172.21.15.100:58576) 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 -- 172.21.15.100:0/2992547164 learned_addr learned my addr 172.21.15.100:0/2992547164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb798e35640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35b70 0x564c2cd3bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799636640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 -- 172.21.15.100:0/2992547164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 msgr2=0x564c2cd3b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd3b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 -- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35b70 msgr2=0x564c2cd3bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35b70 0x564c2cd3bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 -- 172.21.15.100:0/2992547164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb790009850 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb799e37640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd3c1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009dc0 tx=0x7fb78c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd347813cad63f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb77cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb77cff9640 1 -- 172.21.15.100:0/2992547164 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c048040 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb77cff9640 1 -- 172.21.15.100:0/2992547164 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb78c032ce0 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2c99c7e0 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.121+0000 7fb77cff9640 1 -- 172.21.15.100:0/2992547164 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb78c032ea0 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 10 client.16008 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564c2cc9b6a0 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb77cff9640 1 -- 172.21.15.100:0/2992547164 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564c2c99c7e0 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb77cff9640 1 -- 172.21.15.100:0/2992547164 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb78c03e070 con 0x564c2cd3e750 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 10 client.16008 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:38.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 10 client.16008 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 20 client.16008 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cd4ebf0 0x564c2cd50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 4 client.16008 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2c9d9720 con 0x564c2cd4ebf0 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb799636640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cd4ebf0 0x564c2cd50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb77cff9640 10 client.16008 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.122+0000 7fb799636640 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cd4ebf0 0x564c2cd50fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fd9880b0581a556 server_cookie=1e667f3fbcbee956 in_seq=0 out_seq=0 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb77cff9640 1 -- 172.21.15.100:0/2992547164 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c2c9d9720 con 0x564c2cd4ebf0 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb77cff9640 10 client.16008 handle_command_reply: tid=0 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 client.16008 shutdown 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 msgr2=0x564c2cd3c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd3c1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009dc0 tx=0x7fb78c005dc0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 shutdown_connections 2024-03-12T05:14:38.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cd4ebf0 0x564c2cd50fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd3c1c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb77cff9640 0 client.16008 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35b70 0x564c2cd3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 --2- 172.21.15.100:0/2992547164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35250 0x564c2cd3b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 >> 172.21.15.100:0/2992547164 conn(0x564c2cd2f980 msgr2=0x564c2cd316d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 shutdown_connections 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.123+0000 7fb79e46f740 1 -- 172.21.15.100:0/2992547164 wait complete. 2024-03-12T05:14:38.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.124+0000 7fb79e46f740 20 client.16008 trim_cache size 0 max 16384 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 Processor -- start 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 -- start start 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cc22130 con 0x564c2cd3e750 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cd34eb0 con 0x564c2cd35250 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.135+0000 7fb79e46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cd30980 con 0x564c2cd35b70 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd389a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58586/0 (socket says 172.21.15.100:58586) 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 -- 172.21.15.100:0/855542753 learned_addr learned my addr 172.21.15.100:0/855542753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799636640 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb798e35640 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 -- 172.21.15.100:0/855542753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 msgr2=0x564c2cd3bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd3bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 -- 172.21.15.100:0/855542753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 msgr2=0x564c2cd3de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd3de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 -- 172.21.15.100:0/855542753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2cc37c20 con 0x564c2cd3e750 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb799e37640 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd389a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb78c02c6c0 tx=0x7fb78c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb182c5373bf02f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb77effd640 1 -- 172.21.15.100:0/855542753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c04e670 con 0x564c2cd3e750 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb77effd640 1 -- 172.21.15.100:0/855542753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb78c034910 con 0x564c2cd3e750 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.136+0000 7fb77effd640 1 -- 172.21.15.100:0/855542753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c004420 con 0x564c2cd3e750 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- 172.21.15.100:0/855542753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 msgr2=0x564c2cd389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd389a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb78c02c6c0 tx=0x7fb78c009540 comp rx=0 tx=0).stop 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- 172.21.15.100:0/855542753 shutdown_connections 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd389a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd3bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- 172.21.15.100:0/855542753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd3de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- 172.21.15.100:0/855542753 >> 172.21.15.100:0/855542753 conn(0x564c2cd2f980 msgr2=0x564c2cd36ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- 172.21.15.100:0/855542753 shutdown_connections 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- 172.21.15.100:0/855542753 wait complete. 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- start start 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 Processor -- start 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd44200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd44740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd44c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cda5170 con 0x564c2cd3e750 2024-03-12T05:14:38.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cda52e0 con 0x564c2cd35250 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2cda54a0 con 0x564c2cd35b70 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.137+0000 7fb79e46f740 10 client.0 _ll_register_callbacks cb 0x7fb7900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd44c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb798e35640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd44740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd44c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58590/0 (socket says 172.21.15.100:58590) 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799636640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd44200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 -- 172.21.15.100:0/45054404 learned_addr learned my addr 172.21.15.100:0/45054404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 -- 172.21.15.100:0/45054404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 msgr2=0x564c2cd44740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd44740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 -- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 msgr2=0x564c2cd44200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd44200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 -- 172.21.15.100:0/45054404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2cc9b6a0 con 0x564c2cd3e750 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb799e37640 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd44c80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098d0 tx=0x7fb78c04e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14883c0da649bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb77cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb77cff9640 1 -- 172.21.15.100:0/45054404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c032ae0 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb77cff9640 1 -- 172.21.15.100:0/45054404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb78c032c40 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb790005500 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb77cff9640 1 -- 172.21.15.100:0/45054404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb78c032e00 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb79e46f740 10 client.16014 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564c2cc5d8c0 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.138+0000 7fb77cff9640 1 -- 172.21.15.100:0/45054404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb790005500 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb77cff9640 1 -- 172.21.15.100:0/45054404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb78c03e040 con 0x564c2cd3e750 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb79e46f740 10 client.16014 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb79e46f740 10 client.16014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb79e46f740 20 client.16014 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb79e46f740 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cda6850 0x564c2cda8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb79e46f740 4 client.16014 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:14:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c2ca28890 con 0x564c2cda6850 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb799636640 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cda6850 0x564c2cda8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb77cff9640 10 client.16014 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.139+0000 7fb799636640 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cda6850 0x564c2cda8c40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2bc364eda3598e6 server_cookie=207532b35ad2e555 in_seq=0 out_seq=0 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb77cff9640 1 -- 172.21.15.100:0/45054404 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c2ca28890 con 0x564c2cda6850 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb77cff9640 10 client.16014 handle_command_reply: tid=0 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 client.16014 shutdown 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 msgr2=0x564c2cd44c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd44c80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098d0 tx=0x7fb78c04e6a0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 shutdown_connections 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564c2cda6850 0x564c2cda8c40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2cd3e750 0x564c2cd44c80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c2cd35b70 0x564c2cd44740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb77cff9640 0 client.16014 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 --2- 172.21.15.100:0/45054404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c2cd35250 0x564c2cd44200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 >> 172.21.15.100:0/45054404 conn(0x564c2cd2f980 msgr2=0x564c2cd2e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 shutdown_connections 2024-03-12T05:14:38.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 1 -- 172.21.15.100:0/45054404 wait complete. 2024-03-12T05:14:38.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.140+0000 7fb79e46f740 20 client.16014 trim_cache size 0 max 16384 2024-03-12T05:14:38.143 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb7800a1ba0 msgr2=0x7fb7800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb7800a1ba0 0x7fb7800a4060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb78c02c790 tx=0x7fb78c02c180 comp rx=0 tx=0).stop 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 msgr2=0x7fb79419e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 0x7fb79419e320 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9f0 tx=0x7fb78400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 shutdown_connections 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb7800a1ba0 0x7fb7800a4060 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7941055b0 0x7fb7941a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb794100ad0 0x7fb79419e320 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 --2- 172.21.15.100:0/2141472818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7941001b0 0x7fb79419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 >> 172.21.15.100:0/2141472818 conn(0x7fb794076400 msgr2=0x7fb794102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 shutdown_connections 2024-03-12T05:14:38.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.141+0000 7fb79b89a640 1 -- 172.21.15.100:0/2141472818 wait complete. 2024-03-12T05:14:38.147 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:14:38.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:14:38.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 -- 172.21.15.100:0/2610936197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a181053e0 msgr2=0x7f7a1810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 --2- 172.21.15.100:0/2610936197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a181053e0 0x7f7a1810cfb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009cd0 tx=0x7f7a1002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 -- 172.21.15.100:0/2610936197 shutdown_connections 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 --2- 172.21.15.100:0/2610936197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a181053e0 0x7f7a1810cfb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 --2- 172.21.15.100:0/2610936197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a18100cc0 0x7f7a18104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 --2- 172.21.15.100:0/2610936197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a18100310 0x7f7a181006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 -- 172.21.15.100:0/2610936197 >> 172.21.15.100:0/2610936197 conn(0x7f7a180771e0 msgr2=0x7f7a180775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 -- 172.21.15.100:0/2610936197 shutdown_connections 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.260+0000 7f7a207c7640 1 -- 172.21.15.100:0/2610936197 wait complete. 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 Processor -- start 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 -- start start 2024-03-12T05:14:38.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a18100310 0x7f7a18196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a181053e0 0x7f7a18196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 0x7f7a1819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1819b9e0 con 0x7f7a1819aff0 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1819bb50 con 0x7f7a18100310 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a207c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1819bcc0 con 0x7f7a181053e0 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 0x7f7a1819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 0x7f7a1819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58606/0 (socket says 172.21.15.100:58606) 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 -- 172.21.15.100:0/1156466778 learned_addr learned my addr 172.21.15.100:0/1156466778 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a18100310 0x7f7a18196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a181053e0 0x7f7a18196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 -- 172.21.15.100:0/1156466778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a181053e0 msgr2=0x7f7a18196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a181053e0 0x7f7a18196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 -- 172.21.15.100:0/1156466778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a18100310 msgr2=0x7f7a18196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a18100310 0x7f7a18196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 -- 172.21.15.100:0/1156466778 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a08009490 con 0x7f7a1819aff0 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.261+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 0x7f7a1819b4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7a10004af0 tx=0x7f7a10032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.262+0000 7f7a0f7fe640 1 -- 172.21.15.100:0/1156466778 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100328b0 con 0x7f7a1819aff0 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.262+0000 7f7a0f7fe640 1 -- 172.21.15.100:0/1156466778 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a10003e50 con 0x7f7a1819aff0 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.262+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a10009900 con 0x7f7a1819aff0 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.262+0000 7f7a0f7fe640 1 -- 172.21.15.100:0/1156466778 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10004740 con 0x7f7a1819aff0 2024-03-12T05:14:38.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.262+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a181a34e0 con 0x7f7a1819aff0 2024-03-12T05:14:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.264+0000 7f7a0f7fe640 1 -- 172.21.15.100:0/1156466778 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7a1000a040 con 0x7f7a1819aff0 2024-03-12T05:14:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.264+0000 7f7a23398740 1 Processor -- start 2024-03-12T05:14:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.264+0000 7f7a23398740 1 -- start start 2024-03-12T05:14:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a0f7fe640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a040a1ba0 0x7f7a040a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a23398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a23398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f9b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a23398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a23398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6f4b960 con 0x555ef6f8ff00 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a23398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6eff310 con 0x555ef6f90820 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a23398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6efc840 con 0x555ef6f9b960 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a040a1ba0 0x7f7a040a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f902e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58608/0 (socket says 172.21.15.100:58608) 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 -- 172.21.15.100:0/1586981153 learned_addr learned my addr 172.21.15.100:0/1586981153 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a0f7fe640 1 -- 172.21.15.100:0/1156466778 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a100f02b0 con 0x7f7a1819aff0 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f9b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 -- 172.21.15.100:0/1586981153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 msgr2=0x555ef6f9dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 -- 172.21.15.100:0/1586981153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 msgr2=0x555ef6f9b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f9b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 -- 172.21.15.100:0/1586981153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ef6eba060 con 0x555ef6f8ff00 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a040a1ba0 0x7f7a040a4060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a18104d40 tx=0x7f7a08009bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a1e563640 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f902e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009cf0 tx=0x7f7a14011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1a51d70b80110c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.265+0000 7f7a0cff9640 1 -- 172.21.15.100:0/1586981153 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14009ea0 con 0x555ef6f8ff00 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a0cff9640 1 -- 172.21.15.100:0/1586981153 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a14004230 con 0x555ef6f8ff00 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a0cff9640 1 -- 172.21.15.100:0/1586981153 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14019800 con 0x555ef6f8ff00 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 -- 172.21.15.100:0/1586981153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 msgr2=0x555ef6f902e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f902e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009cf0 tx=0x7f7a14011810 comp rx=0 tx=0).stop 2024-03-12T05:14:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 -- 172.21.15.100:0/1586981153 shutdown_connections 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f9b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 --2- 172.21.15.100:0/1586981153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f902e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 -- 172.21.15.100:0/1586981153 >> 172.21.15.100:0/1586981153 conn(0x555ef6f8a980 msgr2=0x555ef6f8ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 -- 172.21.15.100:0/1586981153 shutdown_connections 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.266+0000 7f7a23398740 1 -- 172.21.15.100:0/1586981153 wait complete. 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 -- start start 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 Processor -- start 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f98850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f98d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6ef8020 con 0x555ef6f8ff00 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6bcb430 con 0x555ef6f90820 2024-03-12T05:14:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6b87e00 con 0x555ef6f9b960 2024-03-12T05:14:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a23398740 10 client.0 _ll_register_callbacks cb 0x555ef6e9fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f98850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1ed64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f98850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58622/0 (socket says 172.21.15.100:58622) 2024-03-12T05:14:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1dd62640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f98d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 -- 172.21.15.100:0/4120590945 learned_addr learned my addr 172.21.15.100:0/4120590945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 -- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 msgr2=0x555ef6f992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 -- 172.21.15.100:0/4120590945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 msgr2=0x555ef6f98d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f98d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1e563640 1 -- 172.21.15.100:0/4120590945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a14009950 con 0x555ef6f8ff00 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.267+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a1e563640 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f98850 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009560 tx=0x7f7a14003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168e174a3afe33d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a01ffb640 1 -- 172.21.15.100:0/4120590945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14005240 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a01ffb640 1 -- 172.21.15.100:0/4120590945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a140053a0 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ef6bf77e0 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a01ffb640 1 -- 172.21.15.100:0/4120590945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a14003a20 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 10 client.16026 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555ef6ef66a0 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a01ffb640 1 -- 172.21.15.100:0/4120590945 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555ef6bf77e0 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a01ffb640 1 -- 172.21.15.100:0/4120590945 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7a14021040 con 0x555ef6f8ff00 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 10 client.16026 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 10 client.16026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 20 client.16026 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef6fabcb0 0x555ef6fae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 4 client.16026 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.268+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ef6c34720 con 0x555ef6fabcb0 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.269+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef6fabcb0 0x555ef6fae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.269+0000 7f7a01ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.269+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef6fabcb0 0x555ef6fae0d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11f5ac9a6857483c server_cookie=6f705fbc377a9ae5 in_seq=0 out_seq=0 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.269+0000 7f7a01ffb640 1 -- 172.21.15.100:0/4120590945 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ef6c34720 con 0x555ef6fabcb0 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.269+0000 7f7a01ffb640 10 client.16026 handle_command_reply: tid=0 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.269+0000 7f7a23398740 1 client.16026 shutdown 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 msgr2=0x555ef6f98850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f98850 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009560 tx=0x7f7a14003be0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 shutdown_connections 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef6fabcb0 0x555ef6fae0d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f98d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 --2- 172.21.15.100:0/4120590945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f98850 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a01ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 >> 172.21.15.100:0/4120590945 conn(0x555ef6f8a980 msgr2=0x555ef6f8c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 shutdown_connections 2024-03-12T05:14:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 1 -- 172.21.15.100:0/4120590945 wait complete. 2024-03-12T05:14:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.270+0000 7f7a23398740 20 client.16026 trim_cache size 0 max 16384 2024-03-12T05:14:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 Processor -- start 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 -- start start 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f89a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6fa28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6e7d130 con 0x555ef6f8ff00 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6f8fa30 con 0x555ef6f90820 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.282+0000 7f7a23398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6fb3320 con 0x555ef6f9b960 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58632/0 (socket says 172.21.15.100:58632) 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 -- 172.21.15.100:0/1161737728 learned_addr learned my addr 172.21.15.100:0/1161737728 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f89a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6fa28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 -- 172.21.15.100:0/1161737728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 msgr2=0x555ef6fa28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6fa28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 -- 172.21.15.100:0/1161737728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 msgr2=0x555ef6f89a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f89a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 -- 172.21.15.100:0/1161737728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ef6e92c20 con 0x555ef6f8ff00 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6fa28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a1e563640 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f990f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009380 tx=0x7f7a140098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773393055926129b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a03fff640 1 -- 172.21.15.100:0/1161737728 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14003e40 con 0x555ef6f8ff00 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a03fff640 1 -- 172.21.15.100:0/1161737728 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a14009ea0 con 0x555ef6f8ff00 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.283+0000 7f7a03fff640 1 -- 172.21.15.100:0/1161737728 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14005610 con 0x555ef6f8ff00 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 -- 172.21.15.100:0/1161737728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 msgr2=0x555ef6f990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f990f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009380 tx=0x7f7a140098d0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 -- 172.21.15.100:0/1161737728 shutdown_connections 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6fa28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f90820 0x555ef6f89a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 --2- 172.21.15.100:0/1161737728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f8ff00 0x555ef6f990f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 -- 172.21.15.100:0/1161737728 >> 172.21.15.100:0/1161737728 conn(0x555ef6f8a980 msgr2=0x555ef6f892f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 -- 172.21.15.100:0/1161737728 shutdown_connections 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 -- 172.21.15.100:0/1161737728 wait complete. 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.284+0000 7f7a23398740 1 -- start start 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 Processor -- start 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f8ff00 0x555ef6fa2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 0x555ef6f9ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6f9fb30 con 0x555ef6f90820 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6f9fca0 con 0x555ef6f8ff00 2024-03-12T05:14:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef70001e0 con 0x555ef6f9b960 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a23398740 10 client.0 _ll_register_callbacks cb 0x7f7a140097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 0x555ef6f9ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 0x555ef6f9ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58634/0 (socket says 172.21.15.100:58634) 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 -- 172.21.15.100:0/10200359 learned_addr learned my addr 172.21.15.100:0/10200359 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1e563640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f8ff00 0x555ef6fa2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1ed64640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 -- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 msgr2=0x555ef6f9f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 -- 172.21.15.100:0/10200359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f8ff00 msgr2=0x555ef6fa2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f8ff00 0x555ef6fa2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.285+0000 7f7a1dd62640 1 -- 172.21.15.100:0/10200359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ef6ef66a0 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a1dd62640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 0x555ef6f9ef70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009460 tx=0x7f7a08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8fbbb78c9e8ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a01ffb640 1 -- 172.21.15.100:0/10200359 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08026040 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a01ffb640 1 -- 172.21.15.100:0/10200359 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a08032070 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a14004230 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a01ffb640 1 -- 172.21.15.100:0/10200359 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a08013550 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 10 client.16032 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555ef6eb88c0 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a01ffb640 1 -- 172.21.15.100:0/10200359 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a14004230 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a01ffb640 1 -- 172.21.15.100:0/10200359 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7a080128a0 con 0x555ef6f90820 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 10 client.16032 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 10 client.16032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:14:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 20 client.16032 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef70018f0 0x555ef7003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 4 client.16032 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.286+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ef6c83890 con 0x555ef70018f0 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a1e563640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef70018f0 0x555ef7003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a01ffb640 10 client.16032 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a1e563640 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef70018f0 0x555ef7003ce0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7c9588b2886cb21 server_cookie=abf6d7698597fe94 in_seq=0 out_seq=0 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a01ffb640 1 -- 172.21.15.100:0/10200359 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ef6c83890 con 0x555ef70018f0 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a01ffb640 10 client.16032 handle_command_reply: tid=0 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 client.16032 shutdown 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 msgr2=0x555ef6f9ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 0x555ef6f9ef70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009460 tx=0x7f7a08004730 comp rx=0 tx=0).stop 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 shutdown_connections 2024-03-12T05:14:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ef6f9b960 0x555ef6f9f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef6f90820 0x555ef6f9ef70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555ef70018f0 0x555ef7003ce0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 --2- 172.21.15.100:0/10200359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ef6f8ff00 0x555ef6fa2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a01ffb640 0 client.16032 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 >> 172.21.15.100:0/10200359 conn(0x555ef6f8a980 msgr2=0x555ef6f89b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 shutdown_connections 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 1 -- 172.21.15.100:0/10200359 wait complete. 2024-03-12T05:14:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.287+0000 7f7a23398740 20 client.16032 trim_cache size 0 max 16384 2024-03-12T05:14:38.300 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:14:38.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a040a1ba0 msgr2=0x7f7a040a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a040a1ba0 0x7f7a040a4060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a18104d40 tx=0x7f7a08009bc0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 msgr2=0x7f7a1819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 0x7f7a1819b4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7a10004af0 tx=0x7f7a10032570 comp rx=0 tx=0).stop 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 shutdown_connections 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a040a1ba0 0x7f7a040a4060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a1819aff0 0x7f7a1819b4a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a181053e0 0x7f7a18196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 --2- 172.21.15.100:0/1156466778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a18100310 0x7f7a18196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 >> 172.21.15.100:0/1156466778 conn(0x7f7a180771e0 msgr2=0x7f7a180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 shutdown_connections 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.288+0000 7f7a207c7640 1 -- 172.21.15.100:0/1156466778 wait complete. 2024-03-12T05:14:38.302 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:14:38.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:14:38.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.414+0000 7f6a5f743640 1 -- 172.21.15.100:0/1175313980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 msgr2=0x7f6a581064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.414+0000 7f6a5f743640 1 --2- 172.21.15.100:0/1175313980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 0x7f6a581064e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009900 tx=0x7f6a5402bcd0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 -- 172.21.15.100:0/1175313980 shutdown_connections 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 --2- 172.21.15.100:0/1175313980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a581009e0 0x7f6a5810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 --2- 172.21.15.100:0/1175313980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58100040 0x7f6a581004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 --2- 172.21.15.100:0/1175313980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 0x7f6a581064e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 -- 172.21.15.100:0/1175313980 >> 172.21.15.100:0/1175313980 conn(0x7f6a58077110 msgr2=0x7f6a58077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 -- 172.21.15.100:0/1175313980 shutdown_connections 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 -- 172.21.15.100:0/1175313980 wait complete. 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 Processor -- start 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 -- start start 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 0x7f6a5819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581009e0 0x7f6a5819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.415+0000 7f6a5f743640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 0x7f6a58197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5f743640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58197be0 con 0x7f6a58106100 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5f743640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58197d50 con 0x7f6a581009e0 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5f743640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58197f10 con 0x7f6a58100040 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 0x7f6a58197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 0x7f6a5819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5ccde640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581009e0 0x7f6a5819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 0x7f6a5819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:45146/0 (socket says 172.21.15.100:45146) 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 -- 172.21.15.100:0/9867489 learned_addr learned my addr 172.21.15.100:0/9867489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 -- 172.21.15.100:0/9867489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581009e0 msgr2=0x7f6a5819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581009e0 0x7f6a5819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 -- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 msgr2=0x7f6a58197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 0x7f6a58197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 -- 172.21.15.100:0/9867489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009560 con 0x7f6a58100040 2024-03-12T05:14:38.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5ccde640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581009e0 0x7f6a5819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:38.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 0x7f6a5819d3e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a54005940 tx=0x7f6a5402c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.416+0000 7f6a4e7fc640 1 -- 172.21.15.100:0/9867489 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540043d0 con 0x7f6a58100040 2024-03-12T05:14:38.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.417+0000 7f6a4e7fc640 1 -- 172.21.15.100:0/9867489 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a540052d0 con 0x7f6a58100040 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.417+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a58198190 con 0x7f6a58100040 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.417+0000 7f6a4e7fc640 1 -- 172.21.15.100:0/9867489 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54003d30 con 0x7f6a58100040 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.417+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581a4480 con 0x7f6a58100040 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a4e7fc640 1 -- 172.21.15.100:0/9867489 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6a54005440 con 0x7f6a58100040 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 Processor -- start 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 -- start start 2024-03-12T05:14:38.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea26250 0x55c42ea26630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a4e7fc640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a440a1ba0 0x7f6a440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea2f750 0x55c42ea31b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e9e1960 con 0x55c42ea26250 2024-03-12T05:14:38.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e995310 con 0x55c42ea26b70 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a623ea740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e992840 con 0x55c42ea2f750 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.419+0000 7f6a5ccde640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a440a1ba0 0x7f6a440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea26250 0x55c42ea26630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:55064/0 (socket says 172.21.15.100:55064) 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 -- 172.21.15.100:0/202816377 learned_addr learned my addr 172.21.15.100:0/202816377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a4e7fc640 1 -- 172.21.15.100:0/9867489 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6a540efe60 con 0x7f6a58100040 2024-03-12T05:14:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea2f750 0x55c42ea31b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 -- 172.21.15.100:0/202816377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea2f750 msgr2=0x55c42ea31b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea2f750 0x55c42ea31b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 -- 172.21.15.100:0/202816377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea26250 msgr2=0x55c42ea26630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea26250 0x55c42ea26630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 -- 172.21.15.100:0/202816377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42e950060 con 0x55c42ea26b70 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5ccde640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a440a1ba0 0x7f6a440a4060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a580777c0 tx=0x7f6a48005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2f210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a54049fd0 tx=0x7f6a5403bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9742e9a6f6ea65f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a437fe640 1 -- 172.21.15.100:0/202816377 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540bc6d0 con 0x55c42ea26b70 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a437fe640 1 -- 172.21.15.100:0/202816377 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a540bc4f0 con 0x55c42ea26b70 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.420+0000 7f6a437fe640 1 -- 172.21.15.100:0/202816377 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540c5330 con 0x55c42ea26b70 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 -- 172.21.15.100:0/202816377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 msgr2=0x55c42ea2f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2f210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a54049fd0 tx=0x7f6a5403bfa0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 -- 172.21.15.100:0/202816377 shutdown_connections 2024-03-12T05:14:38.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea2f750 0x55c42ea31b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2f210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 --2- 172.21.15.100:0/202816377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea26250 0x55c42ea26630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 -- 172.21.15.100:0/202816377 >> 172.21.15.100:0/202816377 conn(0x55c42ea20980 msgr2=0x55c42ea20d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 -- 172.21.15.100:0/202816377 shutdown_connections 2024-03-12T05:14:38.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 -- 172.21.15.100:0/202816377 wait complete. 2024-03-12T05:14:38.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 -- start start 2024-03-12T05:14:38.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.421+0000 7f6a623ea740 1 Processor -- start 2024-03-12T05:14:38.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea2c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea2cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e98e020 con 0x55c42ea2f750 2024-03-12T05:14:38.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e661430 con 0x55c42ea26250 2024-03-12T05:14:38.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e61de00 con 0x55c42ea26b70 2024-03-12T05:14:38.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 10 client.0 _ll_register_callbacks cb 0x55c42e935cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:38.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5d4df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea2c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58684/0 (socket says 172.21.15.100:58684) 2024-03-12T05:14:38.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 -- 172.21.15.100:0/2911735279 learned_addr learned my addr 172.21.15.100:0/2911735279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5ccde640 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea2cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 -- 172.21.15.100:0/2911735279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 msgr2=0x55c42ea2cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea2cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 -- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 msgr2=0x55c42ea2c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea2c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 -- 172.21.15.100:0/2911735279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581a4480 con 0x55c42ea2f750 2024-03-12T05:14:38.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2d0f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6a50012f40 tx=0x7f6a50011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa9ab5619ba43a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a40ff9640 1 -- 172.21.15.100:0/2911735279 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50004b20 con 0x55c42ea2f750 2024-03-12T05:14:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a40ff9640 1 -- 172.21.15.100:0/2911735279 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6a50004c80 con 0x55c42ea2f750 2024-03-12T05:14:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.422+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c42e68d7e0 con 0x55c42ea2f750 2024-03-12T05:14:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a40ff9640 1 -- 172.21.15.100:0/2911735279 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a50004e40 con 0x55c42ea2f750 2024-03-12T05:14:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 10 client.16044 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c42e98c6a0 con 0x55c42ea2f750 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a40ff9640 1 -- 172.21.15.100:0/2911735279 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c42e68d7e0 con 0x55c42ea2f750 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a40ff9640 1 -- 172.21.15.100:0/2911735279 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6a5001e740 con 0x55c42ea2f750 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 10 client.16044 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 20 client.16044 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea3f810 0x55c42ea41c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 4 client.16044 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c42e6ca720 con 0x55c42ea3f810 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.423+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea3f810 0x55c42ea41c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a40ff9640 10 client.16044 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea3f810 0x55c42ea41c30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61cef42c2cc6c54a server_cookie=f6d56d5c10ab7feb in_seq=0 out_seq=0 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a40ff9640 1 -- 172.21.15.100:0/2911735279 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c42e6ca720 con 0x55c42ea3f810 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a40ff9640 10 client.16044 handle_command_reply: tid=0 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 client.16044 shutdown 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 msgr2=0x55c42ea2d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2d0f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6a50012f40 tx=0x7f6a50011870 comp rx=0 tx=0).stop 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 shutdown_connections 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea3f810 0x55c42ea41c30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2d0f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea2cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 --2- 172.21.15.100:0/2911735279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea2c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a40ff9640 0 client.16044 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.424+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 >> 172.21.15.100:0/2911735279 conn(0x55c42ea20980 msgr2=0x55c42ea226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.425+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 shutdown_connections 2024-03-12T05:14:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.425+0000 7f6a623ea740 1 -- 172.21.15.100:0/2911735279 wait complete. 2024-03-12T05:14:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.425+0000 7f6a623ea740 20 client.16044 trim_cache size 0 max 16384 2024-03-12T05:14:38.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.436+0000 7f6a623ea740 1 Processor -- start 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.436+0000 7f6a623ea740 1 -- start start 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.436+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26250 0x55c42ea2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.436+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.436+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a623ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e913130 con 0x55c42ea2f750 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a623ea740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea25eb0 con 0x55c42ea26b70 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a623ea740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea1c5c0 con 0x55c42ea26250 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58694/0 (socket says 172.21.15.100:58694) 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1220148893 learned_addr learned my addr 172.21.15.100:0/1220148893 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5ccde640 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26250 0x55c42ea2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1220148893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26250 msgr2=0x55c42ea2eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26250 0x55c42ea2eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1220148893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 msgr2=0x55c42ea2cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1220148893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42e928c20 con 0x55c42ea2f750 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26250 0x55c42ea2eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2b650 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6a50012f10 tx=0x7f6a500021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd526e7e68a429 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a42ffd640 1 -- 172.21.15.100:0/1220148893 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50012b10 con 0x55c42ea2f750 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a42ffd640 1 -- 172.21.15.100:0/1220148893 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a50016040 con 0x55c42ea2f750 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.437+0000 7f6a42ffd640 1 -- 172.21.15.100:0/1220148893 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50010910 con 0x55c42ea2f750 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 -- 172.21.15.100:0/1220148893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 msgr2=0x55c42ea2b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2b650 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6a50012f10 tx=0x7f6a500021f0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 -- 172.21.15.100:0/1220148893 shutdown_connections 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea2b650 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26b70 0x55c42ea2cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1220148893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26250 0x55c42ea2eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 -- 172.21.15.100:0/1220148893 >> 172.21.15.100:0/1220148893 conn(0x55c42ea20980 msgr2=0x55c42ea2af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 -- 172.21.15.100:0/1220148893 shutdown_connections 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 -- 172.21.15.100:0/1220148893 wait complete. 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 -- start start 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.438+0000 7f6a623ea740 1 Processor -- start 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea34810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea34d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea35290 con 0x55c42ea2f750 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea35400 con 0x55c42ea26250 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea355c0 con 0x55c42ea26b70 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 10 client.0 _ll_register_callbacks cb 0x7f6a50012750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea34d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea34d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58706/0 (socket says 172.21.15.100:58706) 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1857841591 learned_addr learned my addr 172.21.15.100:0/1857841591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5ccde640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea34810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1857841591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 msgr2=0x55c42ea34810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea34810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 msgr2=0x55c42ea36470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea36470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 -- 172.21.15.100:0/1857841591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c42e98c6a0 con 0x55c42ea2f750 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea36470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a5dce0640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea34d50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6a5001ece0 tx=0x7f6a50013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15c7fb77a1a429a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a40ff9640 1 -- 172.21.15.100:0/1857841591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5001e590 con 0x55c42ea2f750 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a40ff9640 1 -- 172.21.15.100:0/1857841591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6a50009600 con 0x55c42ea2f750 2024-03-12T05:14:38.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.439+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a540bc4f0 con 0x55c42ea2f750 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a40ff9640 1 -- 172.21.15.100:0/1857841591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a50012b10 con 0x55c42ea2f750 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 10 client.16050 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c42e94e8c0 con 0x55c42ea2f750 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a40ff9640 1 -- 172.21.15.100:0/1857841591 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a540bc4f0 con 0x55c42ea2f750 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a40ff9640 1 -- 172.21.15.100:0/1857841591 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6a50010a60 con 0x55c42ea2f750 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 10 client.16050 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 20 client.16050 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea96e80 0x55c42ea992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 4 client.16050 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c42e719890 con 0x55c42ea96e80 2024-03-12T05:14:38.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea96e80 0x55c42ea992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a5d4df640 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea96e80 0x55c42ea992a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1fdf98bca50b66 server_cookie=9b9bfe1ece23bd1f in_seq=0 out_seq=0 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.440+0000 7f6a40ff9640 10 client.16050 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a40ff9640 1 -- 172.21.15.100:0/1857841591 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c42e719890 con 0x55c42ea96e80 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a40ff9640 10 client.16050 handle_command_reply: tid=0 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 client.16050 shutdown 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 msgr2=0x55c42ea34d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea34d50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6a5001ece0 tx=0x7f6a50013f50 comp rx=0 tx=0).stop 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 shutdown_connections 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c42ea96e80 0x55c42ea992a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c42ea2f750 0x55c42ea34d50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a40ff9640 0 client.16050 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c42ea26b70 0x55c42ea34810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 --2- 172.21.15.100:0/1857841591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c42ea26250 0x55c42ea36470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 >> 172.21.15.100:0/1857841591 conn(0x55c42ea20980 msgr2=0x55c42ea1f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 shutdown_connections 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 1 -- 172.21.15.100:0/1857841591 wait complete. 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a623ea740 20 client.16050 trim_cache size 0 max 16384 2024-03-12T05:14:38.461 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a440a1ba0 msgr2=0x7f6a440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a440a1ba0 0x7f6a440a4060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a580777c0 tx=0x7f6a48005f90 comp rx=0 tx=0).stop 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 msgr2=0x7f6a5819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 0x7f6a5819d3e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a54005940 tx=0x7f6a5402c780 comp rx=0 tx=0).stop 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 shutdown_connections 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a440a1ba0 0x7f6a440a4060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a58106100 0x7f6a58197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581009e0 0x7f6a5819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 --2- 172.21.15.100:0/9867489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a58100040 0x7f6a5819d3e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:38.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 >> 172.21.15.100:0/9867489 conn(0x7f6a58077110 msgr2=0x7f6a580fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:38.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.441+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 shutdown_connections 2024-03-12T05:14:38.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:38.442+0000 7f6a5f743640 1 -- 172.21.15.100:0/9867489 wait complete. 2024-03-12T05:14:38.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 211720 14.10 MB/sec warmup 82 sec latency 16.041 ms 2024-03-12T05:14:39.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:38 smithi156 ceph-mon[36270]: pgmap v79: 97 pgs: 97 active+clean; 115 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 360 op/s 2024-03-12T05:14:39.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:38 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2762532777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:14:39.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:38 smithi161 ceph-mon[36043]: pgmap v79: 97 pgs: 97 active+clean; 115 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 360 op/s 2024-03-12T05:14:39.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:38 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2762532777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:14:39.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:38 smithi100 ceph-mon[32210]: pgmap v79: 97 pgs: 97 active+clean; 115 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 360 op/s 2024-03-12T05:14:39.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:38 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2762532777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:14:39.464 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:14:39.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:14:39.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 -- 172.21.15.100:0/2573425322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 msgr2=0x7f23f4105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2573425322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4105400 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009930 tx=0x7f23e402bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 -- 172.21.15.100:0/2573425322 shutdown_connections 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2573425322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 0x7f23f410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2573425322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f4100b90 0x7f23f4101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2573425322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4105400 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 -- 172.21.15.100:0/2573425322 >> 172.21.15.100:0/2573425322 conn(0x7f23f40752e0 msgr2=0x7f23f40756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 -- 172.21.15.100:0/2573425322 shutdown_connections 2024-03-12T05:14:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.577+0000 7f23fa80b640 1 -- 172.21.15.100:0/2573425322 wait complete. 2024-03-12T05:14:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 Processor -- start 2024-03-12T05:14:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 -- start start 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f4100b90 0x7f23f419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 0x7f23f419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4197c60 con 0x7f23f4105020 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4197dd0 con 0x7f23f4101550 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23fa80b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4197f90 con 0x7f23f4100b90 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58720/0 (socket says 172.21.15.100:58720) 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 -- 172.21.15.100:0/2283755422 learned_addr learned my addr 172.21.15.100:0/2283755422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f37fe640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 0x7f23f419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f3fff640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f4100b90 0x7f23f419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 -- 172.21.15.100:0/2283755422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f4100b90 msgr2=0x7f23f419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f4100b90 0x7f23f419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 -- 172.21.15.100:0/2283755422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 msgr2=0x7f23f419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.578+0000 7f23f8da8640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 0x7f23f419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23f8da8640 1 -- 172.21.15.100:0/2283755422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4009560 con 0x7f23f4105020 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23f37fe640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 0x7f23f419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23f8da8640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4197610 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00eb10 tx=0x7f23ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23f17fa640 1 -- 172.21.15.100:0/2283755422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec004b20 con 0x7f23f4105020 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23f17fa640 1 -- 172.21.15.100:0/2283755422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23ec004c80 con 0x7f23f4105020 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23f17fa640 1 -- 172.21.15.100:0/2283755422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec010650 con 0x7f23f4105020 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f4198270 con 0x7f23f4105020 2024-03-12T05:14:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.579+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f41a4500 con 0x7f23f4105020 2024-03-12T05:14:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.581+0000 7f23f17fa640 1 -- 172.21.15.100:0/2283755422 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f23ec005020 con 0x7f23f4105020 2024-03-12T05:14:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.581+0000 7f23fd3d6740 1 Processor -- start 2024-03-12T05:14:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 -- start start 2024-03-12T05:14:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f17fa640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23dc0a1ba0 0x7f23dc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c80d380 0x55e14c80d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c8162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c8168b0 0x55e14c818d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c77bf90 con 0x55e14c80dd30 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c7794c0 con 0x55e14c8168b0 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c774ca0 con 0x55e14c80d380 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f37fe640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23dc0a1ba0 0x7f23dc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f17fa640 1 -- 172.21.15.100:0/2283755422 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f23ec014070 con 0x7f23f4105020 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c8162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c8162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58724/0 (socket says 172.21.15.100:58724) 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f37fe640 1 -- 172.21.15.100:0/3899145885 learned_addr learned my addr 172.21.15.100:0/3899145885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c8168b0 0x55e14c818d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f3fff640 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c80d380 0x55e14c80d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 -- 172.21.15.100:0/3899145885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c80d380 msgr2=0x55e14c80d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c80d380 0x55e14c80d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 -- 172.21.15.100:0/3899145885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 msgr2=0x55e14c8162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c8162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 -- 172.21.15.100:0/3899145885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e14c7370b0 con 0x55e14c8168b0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f37fe640 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c8162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f37fe640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23dc0a1ba0 0x7f23dc0a4060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f23e80058c0 tx=0x7f23e8009f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23f8da8640 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c8168b0 0x55e14c818d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0cb500 tx=0x7f23ec009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611c169571ae71ec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.582+0000 7f23da7fc640 1 -- 172.21.15.100:0/3899145885 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec09a9e0 con 0x55e14c8168b0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23da7fc640 1 -- 172.21.15.100:0/3899145885 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23ec09eb30 con 0x55e14c8168b0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23da7fc640 1 -- 172.21.15.100:0/3899145885 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec097050 con 0x55e14c8168b0 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 -- 172.21.15.100:0/3899145885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c8168b0 msgr2=0x55e14c818d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c8168b0 0x55e14c818d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0cb500 tx=0x7f23ec009a70 comp rx=0 tx=0).stop 2024-03-12T05:14:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 -- 172.21.15.100:0/3899145885 shutdown_connections 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c8168b0 0x55e14c818d00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c8162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/3899145885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c80d380 0x55e14c80d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 -- 172.21.15.100:0/3899145885 >> 172.21.15.100:0/3899145885 conn(0x55e14c807840 msgr2=0x55e14c807c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 -- 172.21.15.100:0/3899145885 shutdown_connections 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.583+0000 7f23fd3d6740 1 -- 172.21.15.100:0/3899145885 wait complete. 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 -- start start 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 Processor -- start 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c813870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c813db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c8142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c719f40 con 0x55e14c80d380 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c4484b0 con 0x55e14c80dd30 2024-03-12T05:14:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c6fa670 con 0x55e14c8168b0 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23fd3d6740 10 client.0 _ll_register_callbacks cb 0x55e14c767f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c813870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c813870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58728/0 (socket says 172.21.15.100:58728) 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 -- 172.21.15.100:0/830610483 learned_addr learned my addr 172.21.15.100:0/830610483 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f37fe640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c813db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f8da8640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c8142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 -- 172.21.15.100:0/830610483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 msgr2=0x55e14c8142f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c8142f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 -- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 msgr2=0x55e14c813db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c813db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f3fff640 1 -- 172.21.15.100:0/830610483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f41a4500 con 0x55e14c80d380 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.584+0000 7f23f8da8640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c8142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23f3fff640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c813870 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009900 tx=0x7f23e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53eeaeb7e66805e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23d37fe640 1 -- 172.21.15.100:0/830610483 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004a40 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23d37fe640 1 -- 172.21.15.100:0/830610483 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f23e4004050 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e14c74ef40 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23d37fe640 1 -- 172.21.15.100:0/830610483 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23e4004210 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 10 client.16062 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e14c773320 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23d37fe640 1 -- 172.21.15.100:0/830610483 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e14c74ef40 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23d37fe640 1 -- 172.21.15.100:0/830610483 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f23e403c040 con 0x55e14c80d380 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 10 client.16062 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 20 client.16062 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c826f10 0x55e14c829330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 4 client.16062 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.585+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e14c76b310 con 0x55e14c826f10 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.586+0000 7f23f37fe640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c826f10 0x55e14c829330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.586+0000 7f23d37fe640 10 client.16062 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.586+0000 7f23f37fe640 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c826f10 0x55e14c829330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97038de4f8a599a4 server_cookie=63762ffa3aba462 in_seq=0 out_seq=0 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.586+0000 7f23d37fe640 1 -- 172.21.15.100:0/830610483 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e14c76b310 con 0x55e14c826f10 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.586+0000 7f23d37fe640 10 client.16062 handle_command_reply: tid=0 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 client.16062 shutdown 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 msgr2=0x55e14c813870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c813870 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009900 tx=0x7f23e40046c0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 shutdown_connections 2024-03-12T05:14:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c826f10 0x55e14c829330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c8142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23d37fe640 0 client.16062 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c813db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/830610483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c813870 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 >> 172.21.15.100:0/830610483 conn(0x55e14c807840 msgr2=0x55e14c8097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 shutdown_connections 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 1 -- 172.21.15.100:0/830610483 wait complete. 2024-03-12T05:14:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.587+0000 7f23fd3d6740 20 client.16062 trim_cache size 0 max 16384 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 Processor -- start 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 -- start start 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c815fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c814110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c810ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c76b310 con 0x55e14c80d380 2024-03-12T05:14:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c74ef40 con 0x55e14c80dd30 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c80ccd0 con 0x55e14c8168b0 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c815fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c815fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58734/0 (socket says 172.21.15.100:58734) 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 -- 172.21.15.100:0/1988608076 learned_addr learned my addr 172.21.15.100:0/1988608076 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f37fe640 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c814110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f8da8640 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c810ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 -- 172.21.15.100:0/1988608076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 msgr2=0x55e14c810ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c810ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 -- 172.21.15.100:0/1988608076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 msgr2=0x55e14c814110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c814110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 -- 172.21.15.100:0/1988608076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e14c70fc00 con 0x55e14c80d380 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f8da8640 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c810ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:14:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.602+0000 7f23f3fff640 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c815fa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23e4005ab0 tx=0x7f23e4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff9a2e27c87383 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23d9ffb640 1 -- 172.21.15.100:0/1988608076 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e402cc20 con 0x55e14c80d380 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23d9ffb640 1 -- 172.21.15.100:0/1988608076 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23e4003950 con 0x55e14c80d380 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23d9ffb640 1 -- 172.21.15.100:0/1988608076 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4048980 con 0x55e14c80d380 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 -- 172.21.15.100:0/1988608076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 msgr2=0x55e14c815fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c815fa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23e4005ab0 tx=0x7f23e4009800 comp rx=0 tx=0).stop 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 -- 172.21.15.100:0/1988608076 shutdown_connections 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c810ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80dd30 0x55e14c814110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/1988608076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80d380 0x55e14c815fa0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 -- 172.21.15.100:0/1988608076 >> 172.21.15.100:0/1988608076 conn(0x55e14c807840 msgr2=0x55e14c812140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 -- 172.21.15.100:0/1988608076 shutdown_connections 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.603+0000 7f23fd3d6740 1 -- 172.21.15.100:0/1988608076 wait complete. 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 -- start start 2024-03-12T05:14:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 Processor -- start 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80d380 0x55e14c81c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c81cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c881830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c81d550 con 0x55e14c80dd30 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c881d70 con 0x55e14c80d380 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14c881eb0 con 0x55e14c8168b0 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23fd3d6740 10 client.0 _ll_register_callbacks cb 0x7f23ec004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c81cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c81cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58738/0 (socket says 172.21.15.100:58738) 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 -- 172.21.15.100:0/874349290 learned_addr learned my addr 172.21.15.100:0/874349290 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80d380 0x55e14c81c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f8da8640 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c881830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 -- 172.21.15.100:0/874349290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 msgr2=0x55e14c881830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c881830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 -- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80d380 msgr2=0x55e14c81c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80d380 0x55e14c81c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.604+0000 7f23f37fe640 1 -- 172.21.15.100:0/874349290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e14c773320 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23f37fe640 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c81cc80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55e14c80ce50 tx=0x7f23e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b257111c9aef4bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23d37fe640 1 -- 172.21.15.100:0/874349290 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e802f070 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23d37fe640 1 -- 172.21.15.100:0/874349290 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f23e8011df0 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e14c79a900 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23d37fe640 1 -- 172.21.15.100:0/874349290 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23e8016040 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 10 client.16068 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e14c735910 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23d37fe640 1 -- 172.21.15.100:0/874349290 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23e8041020 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23d37fe640 1 -- 172.21.15.100:0/874349290 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f23e8028050 con 0x55e14c80dd30 2024-03-12T05:14:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 10 client.16068 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 20 client.16068 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c883750 0x55e14c885b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 4 client.16068 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e14c474860 con 0x55e14c883750 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.605+0000 7f23f3fff640 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c883750 0x55e14c885b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.606+0000 7f23d37fe640 10 client.16068 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.606+0000 7f23f3fff640 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c883750 0x55e14c885b70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e424a2a0b21173 server_cookie=c52960ce2a82c046 in_seq=0 out_seq=0 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23d37fe640 1 -- 172.21.15.100:0/874349290 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e14c474860 con 0x55e14c883750 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23d37fe640 10 client.16068 handle_command_reply: tid=0 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 client.16068 shutdown 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 msgr2=0x55e14c81cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c81cc80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55e14c80ce50 tx=0x7f23e8005f90 comp rx=0 tx=0).stop 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 shutdown_connections 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e14c883750 0x55e14c885b70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e14c8168b0 0x55e14c881830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e14c80dd30 0x55e14c81cc80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 --2- 172.21.15.100:0/874349290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14c80d380 0x55e14c81c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23d37fe640 0 client.16068 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 >> 172.21.15.100:0/874349290 conn(0x55e14c807840 msgr2=0x55e14c808ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 shutdown_connections 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 1 -- 172.21.15.100:0/874349290 wait complete. 2024-03-12T05:14:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fd3d6740 20 client.16068 trim_cache size 0 max 16384 2024-03-12T05:14:39.612 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:14:39.612 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:14:39.612 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "f0475cb2-a221-4fc3-afe3-49029cf6fd14", 2024-03-12T05:14:39.612 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:14:39.612 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23dc0a1ba0 msgr2=0x7f23dc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23dc0a1ba0 0x7f23dc0a4060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f23e80058c0 tx=0x7f23e8009f90 comp rx=0 tx=0).stop 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 msgr2=0x7f23f4197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.607+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4197610 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00eb10 tx=0x7f23ec00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 shutdown_connections 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23dc0a1ba0 0x7f23dc0a4060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f4105020 0x7f23f4197610 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f4101550 0x7f23f419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 --2- 172.21.15.100:0/2283755422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f4100b90 0x7f23f419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 >> 172.21.15.100:0/2283755422 conn(0x7f23f40752e0 msgr2=0x7f23f40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 shutdown_connections 2024-03-12T05:14:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.608+0000 7f23fa80b640 1 -- 172.21.15.100:0/2283755422 wait complete. 2024-03-12T05:14:39.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:14:39.615 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:14:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 -- 172.21.15.100:0/37018833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790069ac0 msgr2=0x7f0790069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 --2- 172.21.15.100:0/37018833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790069ac0 0x7f0790069ea0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0780009930 tx=0x7f078002bc30 comp rx=0 tx=0).stop 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 -- 172.21.15.100:0/37018833 shutdown_connections 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 --2- 172.21.15.100:0/37018833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790106430 0x7f079010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 --2- 172.21.15.100:0/37018833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f079006a470 0x7f0790105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 --2- 172.21.15.100:0/37018833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790069ac0 0x7f0790069ea0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 -- 172.21.15.100:0/37018833 >> 172.21.15.100:0/37018833 conn(0x7f0790076ea0 msgr2=0x7f07900772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 -- 172.21.15.100:0/37018833 shutdown_connections 2024-03-12T05:14:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.728+0000 7f07950c3640 1 -- 172.21.15.100:0/37018833 wait complete. 2024-03-12T05:14:39.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 Processor -- start 2024-03-12T05:14:39.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 -- start start 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790069ac0 0x7f0790197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 0x7f0790197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f079019bfd0 0x7f079019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019c9c0 con 0x7f0790106430 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019cb30 con 0x7f079019bfd0 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f07950c3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019cca0 con 0x7f0790069ac0 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 0x7f0790197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 0x7f0790197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58768/0 (socket says 172.21.15.100:58768) 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 -- 172.21.15.100:0/995698757 learned_addr learned my addr 172.21.15.100:0/995698757 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078f59e640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f079019bfd0 0x7f079019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 -- 172.21.15.100:0/995698757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790069ac0 msgr2=0x7f0790197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790069ac0 0x7f0790197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 -- 172.21.15.100:0/995698757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f079019bfd0 msgr2=0x7f079019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f079019bfd0 0x7f079019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.729+0000 7f078e59c640 1 -- 172.21.15.100:0/995698757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780009560 con 0x7f0790106430 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.730+0000 7f078e59c640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 0x7f0790197c50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9c0 tx=0x7f078400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.730+0000 7f0777fff640 1 -- 172.21.15.100:0/995698757 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784009700 con 0x7f0790106430 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.730+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901a4280 con 0x7f0790106430 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.730+0000 7f0777fff640 1 -- 172.21.15.100:0/995698757 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f078400cd80 con 0x7f0790106430 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.730+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901a4730 con 0x7f0790106430 2024-03-12T05:14:39.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.730+0000 7f0777fff640 1 -- 172.21.15.100:0/995698757 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784018430 con 0x7f0790106430 2024-03-12T05:14:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0777fff640 1 -- 172.21.15.100:0/995698757 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0784005000 con 0x7f0790106430 2024-03-12T05:14:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 Processor -- start 2024-03-12T05:14:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 -- start start 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ec2b0 0x55f87e4ec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e4f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e45b280 con 0x55f87e4ecbd0 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e4587b0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.732+0000 7f0797d72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e453f90 con 0x55f87e4f57b0 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58772/0 (socket says 172.21.15.100:58772) 2024-03-12T05:14:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ec2b0 0x55f87e4ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078ed9d640 1 -- 172.21.15.100:0/3828701147 learned_addr learned my addr 172.21.15.100:0/3828701147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078e59c640 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e4f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f0777fff640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07780a1ba0 0x7f07780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 -- 172.21.15.100:0/3828701147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 msgr2=0x55f87e4f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e4f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 -- 172.21.15.100:0/3828701147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 msgr2=0x55f87e4f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 -- 172.21.15.100:0/3828701147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f87e412570 con 0x55f87e4ec2b0 2024-03-12T05:14:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078ed9d640 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07780a1ba0 0x7f07780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f0777fff640 1 -- 172.21.15.100:0/995698757 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0784014070 con 0x7f0790106430 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ec2b0 0x55f87e4ec690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0788007ec0 tx=0x7f078800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fe6b31f466513d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f07757fa640 1 -- 172.21.15.100:0/3828701147 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788020030 con 0x55f87e4ec2b0 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f078f59e640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07780a1ba0 0x7f07780a4060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0788020000 tx=0x7f078801d000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f07757fa640 1 -- 172.21.15.100:0/3828701147 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07880125f0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.733+0000 7f07757fa640 1 -- 172.21.15.100:0/3828701147 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788010630 con 0x55f87e4ec2b0 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 -- 172.21.15.100:0/3828701147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ec2b0 msgr2=0x55f87e4ec690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ec2b0 0x55f87e4ec690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0788007ec0 tx=0x7f078800f790 comp rx=0 tx=0).stop 2024-03-12T05:14:39.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 -- 172.21.15.100:0/3828701147 shutdown_connections 2024-03-12T05:14:39.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e4f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 --2- 172.21.15.100:0/3828701147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ec2b0 0x55f87e4ec690 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 -- 172.21.15.100:0/3828701147 >> 172.21.15.100:0/3828701147 conn(0x55f87e4e6990 msgr2=0x55f87e4e6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 -- 172.21.15.100:0/3828701147 shutdown_connections 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 -- 172.21.15.100:0/3828701147 wait complete. 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 -- start start 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.734+0000 7f0797d72740 1 Processor -- start 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 0x55f87e4f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e127430 con 0x55f87e4f57b0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e0e3e00 con 0x55f87e4ecbd0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e3d8fd0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 10 client.0 _ll_register_callbacks cb 0x55f87e42efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 0x55f87e4f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 0x55f87e4f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58778/0 (socket says 172.21.15.100:58778) 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 -- 172.21.15.100:0/3602925916 learned_addr learned my addr 172.21.15.100:0/3602925916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078e59c640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078ed9d640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 -- 172.21.15.100:0/3602925916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 msgr2=0x55f87e4f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 -- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 msgr2=0x55f87e4f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 -- 172.21.15.100:0/3602925916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0788007a40 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f078f59e640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 0x55f87e4f2d30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0788019e30 tx=0x7f0788019b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f1f0574dec9c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0772ff5640 1 -- 172.21.15.100:0/3602925916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078802e680 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0772ff5640 1 -- 172.21.15.100:0/3602925916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0788021070 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f87e190720 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0772ff5640 1 -- 172.21.15.100:0/3602925916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078802e7e0 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.735+0000 7f0797d72740 10 client.16080 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f87e452610 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0772ff5640 1 -- 172.21.15.100:0/3602925916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f87e190720 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0772ff5640 1 -- 172.21.15.100:0/3602925916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0788025890 con 0x55f87e4f57b0 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 10 client.16080 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 20 client.16080 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e505a10 0x55f87e507e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 4 client.16080 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f87e1df890 con 0x55f87e505a10 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f078ed9d640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e505a10 0x55f87e507e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f078ed9d640 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e505a10 0x55f87e507e30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62794e5c8a53276 server_cookie=f0a7900fcf2b9d9d in_seq=0 out_seq=0 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.736+0000 7f0772ff5640 10 client.16080 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0772ff5640 1 -- 172.21.15.100:0/3602925916 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f87e1df890 con 0x55f87e505a10 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0772ff5640 10 client.16080 handle_command_reply: tid=0 2024-03-12T05:14:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 client.16080 shutdown 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 msgr2=0x55f87e4f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 0x55f87e4f2d30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0788019e30 tx=0x7f0788019b70 comp rx=0 tx=0).stop 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 shutdown_connections 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e505a10 0x55f87e507e30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4f57b0 0x55f87e4f2d30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0772ff5640 0 client.16080 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 --2- 172.21.15.100:0/3602925916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.737+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 >> 172.21.15.100:0/3602925916 conn(0x55f87e4e6990 msgr2=0x55f87e4e8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.738+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 shutdown_connections 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.738+0000 7f0797d72740 1 -- 172.21.15.100:0/3602925916 wait complete. 2024-03-12T05:14:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.738+0000 7f0797d72740 20 client.16080 trim_cache size 0 max 16384 2024-03-12T05:14:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 Processor -- start 2024-03-12T05:14:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 -- start start 2024-03-12T05:14:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4f57b0 0x55f87e4f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e4ebf50 con 0x55f87e4ecbd0 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e4e20f0 con 0x55f87e4f57b0 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0797d72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e4e2320 con 0x55f87e4ec2b0 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58786/0 (socket says 172.21.15.100:58786) 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 -- 172.21.15.100:0/2871613542 learned_addr learned my addr 172.21.15.100:0/2871613542 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078ed9d640 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078f59e640 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4f57b0 0x55f87e4f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 -- 172.21.15.100:0/2871613542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 msgr2=0x55f87e4fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 -- 172.21.15.100:0/2871613542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4f57b0 msgr2=0x55f87e4f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4f57b0 0x55f87e4f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 -- 172.21.15.100:0/2871613542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f87e3eeac0 con 0x55f87e4ecbd0 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f078e59c640 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fcba0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f07840973f0 tx=0x7f0784096df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbdfb00045f560d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0774ff9640 1 -- 172.21.15.100:0/2871613542 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840ae070 con 0x55f87e4ecbd0 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0774ff9640 1 -- 172.21.15.100:0/2871613542 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f078409a380 con 0x55f87e4ecbd0 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.753+0000 7f0774ff9640 1 -- 172.21.15.100:0/2871613542 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784098e40 con 0x55f87e4ecbd0 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 -- 172.21.15.100:0/2871613542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 msgr2=0x55f87e4fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fcba0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f07840973f0 tx=0x7f0784096df0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 -- 172.21.15.100:0/2871613542 shutdown_connections 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4f57b0 0x55f87e4f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fcba0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 --2- 172.21.15.100:0/2871613542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 -- 172.21.15.100:0/2871613542 >> 172.21.15.100:0/2871613542 conn(0x55f87e4e6990 msgr2=0x55f87e4f0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 -- 172.21.15.100:0/2871613542 shutdown_connections 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 -- 172.21.15.100:0/2871613542 wait complete. 2024-03-12T05:14:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 -- start start 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.754+0000 7f0797d72740 1 Processor -- start 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e555320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e555860 con 0x55f87e4ec2b0 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e5559d0 con 0x55f87e4ecbd0 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e555b90 con 0x55f87e4f57b0 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 10 client.0 _ll_register_callbacks cb 0x7f0788007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58796/0 (socket says 172.21.15.100:58796) 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 -- 172.21.15.100:0/2815076271 learned_addr learned my addr 172.21.15.100:0/2815076271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078e59c640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078f59e640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e555320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 -- 172.21.15.100:0/2815076271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 msgr2=0x55f87e555320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e555320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 -- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 msgr2=0x55f87e4fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 -- 172.21.15.100:0/2815076271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901a4730 con 0x55f87e4ec2b0 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f078ed9d640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fbf90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55f87e4ebd80 tx=0x7f07800049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c359c24d667d959 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0772ff5640 1 -- 172.21.15.100:0/2815076271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780004250 con 0x55f87e4ec2b0 2024-03-12T05:14:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0772ff5640 1 -- 172.21.15.100:0/2815076271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f07800043b0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f87e1537e0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0772ff5640 1 -- 172.21.15.100:0/2815076271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078002c7e0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 10 client.16086 fetch_fsmap learned FSMap version 24 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.755+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f87e4146a0 con 0x55f87e4ec2b0 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0772ff5640 1 -- 172.21.15.100:0/2815076271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780057080 con 0x55f87e4ec2b0 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0772ff5640 1 -- 172.21.15.100:0/2815076271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f078003c040 con 0x55f87e4ec2b0 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0797d72740 10 client.16086 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0797d72740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0797d72740 20 client.16086 populate_metadata read hostname 'smithi100' 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0797d72740 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e55e700 0x55f87e560b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0797d72740 4 client.16086 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f87e1424e0 con 0x55f87e55e700 2024-03-12T05:14:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f078e59c640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e55e700 0x55f87e560b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f0772ff5640 10 client.16086 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.756+0000 7f078e59c640 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e55e700 0x55f87e560b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ce30e1669e9136 server_cookie=d80d14cd25709a02 in_seq=0 out_seq=0 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0772ff5640 1 -- 172.21.15.100:0/2815076271 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f87e1424e0 con 0x55f87e55e700 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0772ff5640 10 client.16086 handle_command_reply: tid=0 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 client.16086 shutdown 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 msgr2=0x55f87e4fbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fbf90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55f87e4ebd80 tx=0x7f07800049c0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 shutdown_connections 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f87e55e700 0x55f87e560b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87e4f57b0 0x55f87e555320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f87e4ecbd0 0x55f87e4fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0772ff5640 0 client.16086 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 --2- 172.21.15.100:0/2815076271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f87e4ec2b0 0x55f87e4fbf90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 >> 172.21.15.100:0/2815076271 conn(0x55f87e4e6990 msgr2=0x55f87e4f0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 shutdown_connections 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 1 -- 172.21.15.100:0/2815076271 wait complete. 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f0797d72740 20 client.16086 trim_cache size 0 max 16384 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout: "f0475cb2-a221-4fc3-afe3-49029cf6fd14": { 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "f0475cb2-a221-4fc3-afe3-49029cf6fd14", 2024-03-12T05:14:39.762 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:14:39.763 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:14:39.763 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:14:39.763 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:14:39.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07780a1ba0 msgr2=0x7f07780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07780a1ba0 0x7f07780a4060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0788020000 tx=0x7f078801d000 comp rx=0 tx=0).stop 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 msgr2=0x7f0790197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 0x7f0790197c50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9c0 tx=0x7f078400ee80 comp rx=0 tx=0).stop 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 shutdown_connections 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07780a1ba0 0x7f07780a4060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f079019bfd0 0x7f079019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0790106430 0x7f0790197c50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 --2- 172.21.15.100:0/995698757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790069ac0 0x7f0790197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 >> 172.21.15.100:0/995698757 conn(0x7f0790076ea0 msgr2=0x7f07900fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 shutdown_connections 2024-03-12T05:14:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:14:39.757+0000 7f07950c3640 1 -- 172.21.15.100:0/995698757 wait complete. 2024-03-12T05:14:39.765 INFO:tasks.cephfs.filesystem:scrub status for tag:f0475cb2-a221-4fc3-afe3-49029cf6fd14 - {'path': '/', 'tag': 'f0475cb2-a221-4fc3-afe3-49029cf6fd14', 'options': 'recursive,force'} 2024-03-12T05:14:39.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 213541 14.08 MB/sec warmup 83 sec latency 86.762 ms 2024-03-12T05:14:40.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:39 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:14:40.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:39 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:40.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:39 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:14:40.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:39 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:40.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:39 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:14:40.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:39 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:40.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:39 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:14:40.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:39 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:40.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:39 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:14:40.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:39 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:40.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:39 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:14:40.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:39 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:40.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 216731 14.13 MB/sec warmup 84 sec latency 11.126 ms 2024-03-12T05:14:41.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:40 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:14:41.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:40 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:41.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:40 smithi156 ceph-mon[36270]: pgmap v80: 97 pgs: 97 active+clean; 115 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-03-12T05:14:41.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:40 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:14:41.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:40 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:41.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:40 smithi161 ceph-mon[36043]: pgmap v80: 97 pgs: 97 active+clean; 115 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-03-12T05:14:41.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:40 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:14:41.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:40 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:14:41.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:40 smithi100 ceph-mon[32210]: pgmap v80: 97 pgs: 97 active+clean; 115 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-03-12T05:14:41.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 219597 14.13 MB/sec warmup 85 sec latency 20.038 ms 2024-03-12T05:14:42.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222226 14.10 MB/sec warmup 86 sec latency 13.951 ms 2024-03-12T05:14:43.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:42 smithi156 ceph-mon[36270]: pgmap v81: 97 pgs: 97 active+clean; 121 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 346 op/s 2024-03-12T05:14:43.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:42 smithi161 ceph-mon[36043]: pgmap v81: 97 pgs: 97 active+clean; 121 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 346 op/s 2024-03-12T05:14:43.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:42 smithi100 ceph-mon[32210]: pgmap v81: 97 pgs: 97 active+clean; 121 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 346 op/s 2024-03-12T05:14:43.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 224639 14.11 MB/sec warmup 87 sec latency 11.263 ms 2024-03-12T05:14:44.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 227384 14.14 MB/sec warmup 88 sec latency 107.542 ms 2024-03-12T05:14:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:44 smithi156 ceph-mon[36270]: pgmap v82: 97 pgs: 97 active+clean; 121 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-03-12T05:14:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:44 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:14:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:44 smithi161 ceph-mon[36043]: pgmap v82: 97 pgs: 97 active+clean; 121 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-03-12T05:14:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:44 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:14:45.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:44 smithi100 ceph-mon[32210]: pgmap v82: 97 pgs: 97 active+clean; 121 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-03-12T05:14:45.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:44 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:14:45.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 230222 14.13 MB/sec warmup 89 sec latency 7.385 ms 2024-03-12T05:14:46.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:45 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:14:46.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:45 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:14:46.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:45 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:14:46.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 232163 14.10 MB/sec warmup 90 sec latency 28.150 ms 2024-03-12T05:14:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:46 smithi156 ceph-mon[36270]: pgmap v83: 97 pgs: 97 active+clean; 121 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-12T05:14:47.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:46 smithi161 ceph-mon[36043]: pgmap v83: 97 pgs: 97 active+clean; 121 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-12T05:14:47.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:46 smithi100 ceph-mon[32210]: pgmap v83: 97 pgs: 97 active+clean; 121 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-12T05:14:47.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234882 14.10 MB/sec warmup 91 sec latency 9.462 ms 2024-03-12T05:14:48.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 237975 14.15 MB/sec warmup 92 sec latency 14.335 ms 2024-03-12T05:14:49.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:48 smithi156 ceph-mon[36270]: pgmap v84: 97 pgs: 97 active+clean; 127 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 356 op/s 2024-03-12T05:14:49.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:48 smithi161 ceph-mon[36043]: pgmap v84: 97 pgs: 97 active+clean; 127 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 356 op/s 2024-03-12T05:14:49.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:48 smithi100 ceph-mon[32210]: pgmap v84: 97 pgs: 97 active+clean; 127 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 356 op/s 2024-03-12T05:14:49.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 240097 14.08 MB/sec warmup 93 sec latency 74.338 ms 2024-03-12T05:14:50.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 242219 14.07 MB/sec warmup 94 sec latency 11.526 ms 2024-03-12T05:14:51.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:50 smithi156 ceph-mon[36270]: pgmap v85: 97 pgs: 97 active+clean; 127 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-03-12T05:14:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:50 smithi161 ceph-mon[36043]: pgmap v85: 97 pgs: 97 active+clean; 127 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-03-12T05:14:51.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:50 smithi100 ceph-mon[32210]: pgmap v85: 97 pgs: 97 active+clean; 127 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-03-12T05:14:51.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 245218 14.11 MB/sec warmup 95 sec latency 15.615 ms 2024-03-12T05:14:52.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 248194 14.12 MB/sec warmup 96 sec latency 45.436 ms 2024-03-12T05:14:53.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:52 smithi156 ceph-mon[36270]: pgmap v86: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-03-12T05:14:53.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:52 smithi161 ceph-mon[36043]: pgmap v86: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-03-12T05:14:53.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:52 smithi100 ceph-mon[32210]: pgmap v86: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-03-12T05:14:53.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 250736 14.09 MB/sec warmup 97 sec latency 16.370 ms 2024-03-12T05:14:54.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 252615 14.07 MB/sec warmup 98 sec latency 69.340 ms 2024-03-12T05:14:55.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:54 smithi156 ceph-mon[36270]: pgmap v87: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 248 op/s 2024-03-12T05:14:55.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:54 smithi161 ceph-mon[36043]: pgmap v87: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 248 op/s 2024-03-12T05:14:55.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:54 smithi100 ceph-mon[32210]: pgmap v87: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 248 op/s 2024-03-12T05:14:55.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 255539 14.11 MB/sec warmup 99 sec latency 38.826 ms 2024-03-12T05:14:56.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 257851 14.05 MB/sec warmup 100 sec latency 7.626 ms 2024-03-12T05:14:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:56 smithi156 ceph-mon[36270]: pgmap v88: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 235 op/s 2024-03-12T05:14:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:56 smithi161 ceph-mon[36043]: pgmap v88: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 235 op/s 2024-03-12T05:14:57.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:56 smithi100 ceph-mon[32210]: pgmap v88: 97 pgs: 97 active+clean; 132 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 235 op/s 2024-03-12T05:14:57.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 259938 14.04 MB/sec warmup 101 sec latency 19.031 ms 2024-03-12T05:14:58.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 262899 14.08 MB/sec warmup 102 sec latency 23.947 ms 2024-03-12T05:14:59.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:14:58 smithi156 ceph-mon[36270]: pgmap v89: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 358 op/s 2024-03-12T05:14:59.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:14:58 smithi161 ceph-mon[36043]: pgmap v89: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 358 op/s 2024-03-12T05:14:59.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:14:58 smithi100 ceph-mon[32210]: pgmap v89: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 358 op/s 2024-03-12T05:14:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 265289 14.05 MB/sec warmup 103 sec latency 61.707 ms 2024-03-12T05:15:00.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 267089 14.01 MB/sec warmup 104 sec latency 46.597 ms 2024-03-12T05:15:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:00 smithi156 ceph-mon[36270]: pgmap v90: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-03-12T05:15:01.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:00 smithi161 ceph-mon[36043]: pgmap v90: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-03-12T05:15:01.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:00 smithi100 ceph-mon[32210]: pgmap v90: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-03-12T05:15:01.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 269847 14.04 MB/sec warmup 105 sec latency 21.555 ms 2024-03-12T05:15:02.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 272481 14.01 MB/sec warmup 106 sec latency 7.255 ms 2024-03-12T05:15:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:02 smithi156 ceph-mon[36270]: pgmap v91: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-12T05:15:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:02 smithi161 ceph-mon[36043]: pgmap v91: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-12T05:15:03.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:02 smithi100 ceph-mon[32210]: pgmap v91: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-12T05:15:03.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 274810 14.01 MB/sec warmup 107 sec latency 14.326 ms 2024-03-12T05:15:04.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 277231 14.00 MB/sec warmup 108 sec latency 76.037 ms 2024-03-12T05:15:05.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:04 smithi156 ceph-mon[36270]: pgmap v92: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-03-12T05:15:05.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:04 smithi161 ceph-mon[36043]: pgmap v92: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-03-12T05:15:05.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:04 smithi100 ceph-mon[32210]: pgmap v92: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-03-12T05:15:05.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 280301 14.04 MB/sec warmup 109 sec latency 21.840 ms 2024-03-12T05:15:06.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 282643 13.99 MB/sec warmup 110 sec latency 24.975 ms 2024-03-12T05:15:07.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:06 smithi156 ceph-mon[36270]: pgmap v93: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-03-12T05:15:07.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:06 smithi161 ceph-mon[36043]: pgmap v93: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-03-12T05:15:07.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:06 smithi100 ceph-mon[32210]: pgmap v93: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-03-12T05:15:07.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 284734 13.98 MB/sec warmup 111 sec latency 16.777 ms 2024-03-12T05:15:08.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 287715 14.02 MB/sec warmup 112 sec latency 14.257 ms 2024-03-12T05:15:09.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:08 smithi156 ceph-mon[36270]: pgmap v94: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-12T05:15:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:08 smithi161 ceph-mon[36043]: pgmap v94: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-12T05:15:09.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:08 smithi100 ceph-mon[32210]: pgmap v94: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-12T05:15:09.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:15:09.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:15:09.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 290314 14.00 MB/sec warmup 113 sec latency 98.373 ms 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 -- 172.21.15.100:0/2063467207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc248100230 msgr2=0x7fc248105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/2063467207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc248100230 0x7fc248105dc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009930 tx=0x7fc23c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 -- 172.21.15.100:0/2063467207 shutdown_connections 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/2063467207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2481064f0 0x7fc24810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/2063467207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc248100230 0x7fc248105dc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/2063467207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2480ff880 0x7fc2480ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 -- 172.21.15.100:0/2063467207 >> 172.21.15.100:0/2063467207 conn(0x7fc248076590 msgr2=0x7fc2480fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 -- 172.21.15.100:0/2063467207 shutdown_connections 2024-03-12T05:15:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.876+0000 7fc24e9ac640 1 -- 172.21.15.100:0/2063467207 wait complete. 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 Processor -- start 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 -- start start 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 0x7fc2481007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2481064f0 0x7fc248100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248101270 0x7fc2481a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2481a6000 con 0x7fc2480ff880 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2481a6170 con 0x7fc2481064f0 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc24e9ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2481a6330 con 0x7fc248101270 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 0x7fc2481007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 0x7fc2481007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54710/0 (socket says 172.21.15.100:54710) 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 -- 172.21.15.100:0/3210236944 learned_addr learned my addr 172.21.15.100:0/3210236944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc2477fe640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2481064f0 0x7fc248100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 -- 172.21.15.100:0/3210236944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248101270 msgr2=0x7fc2481a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248101270 0x7fc2481a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 -- 172.21.15.100:0/3210236944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2481064f0 msgr2=0x7fc248100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2481064f0 0x7fc248100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.877+0000 7fc247fff640 1 -- 172.21.15.100:0/3210236944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c009560 con 0x7fc2480ff880 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.878+0000 7fc247fff640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 0x7fc2481007f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc23800d770 tx=0x7fc23800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.878+0000 7fc2457fa640 1 -- 172.21.15.100:0/3210236944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238009c30 con 0x7fc2480ff880 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.878+0000 7fc2457fa640 1 -- 172.21.15.100:0/3210236944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23800de80 con 0x7fc2480ff880 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.878+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2481a6610 con 0x7fc2480ff880 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.878+0000 7fc2457fa640 1 -- 172.21.15.100:0/3210236944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238004e80 con 0x7fc2480ff880 2024-03-12T05:15:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.878+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2481a6aa0 con 0x7fc2480ff880 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc2457fa640 1 -- 172.21.15.100:0/3210236944 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc238005020 con 0x7fc2480ff880 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 Processor -- start 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 -- start start 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 0x5560ae881690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae881bd0 0x5560ae88a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae88cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae7f0280 con 0x5560ae8812b0 2024-03-12T05:15:09.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae7ed7b0 con 0x5560ae881bd0 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc25157e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae7e8f90 con 0x5560ae88a7b0 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 0x5560ae881690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 0x5560ae881690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54718/0 (socket says 172.21.15.100:54718) 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 -- 172.21.15.100:0/395766147 learned_addr learned my addr 172.21.15.100:0/395766147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc247fff640 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae881bd0 0x5560ae88a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc2457fa640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc2300a1bd0 0x7fc2300a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc2477fe640 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae88cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc2300a1bd0 0x7fc2300a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 -- 172.21.15.100:0/395766147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 msgr2=0x5560ae88cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae88cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 -- 172.21.15.100:0/395766147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae881bd0 msgr2=0x5560ae88a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc2457fa640 1 -- 172.21.15.100:0/3210236944 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc2380cd230 con 0x7fc2480ff880 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.881+0000 7fc24cf49640 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae881bd0 0x5560ae88a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.882+0000 7fc24cf49640 1 -- 172.21.15.100:0/395766147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560ae7a7570 con 0x5560ae8812b0 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.882+0000 7fc24cf49640 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 0x5560ae881690 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc2400049d0 tx=0x7fc240012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d40157a3fb9183 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.882+0000 7fc22e7fc640 1 -- 172.21.15.100:0/395766147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240005020 con 0x5560ae8812b0 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.882+0000 7fc22e7fc640 1 -- 172.21.15.100:0/395766147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc240018070 con 0x5560ae8812b0 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.882+0000 7fc22e7fc640 1 -- 172.21.15.100:0/395766147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24001c7b0 con 0x5560ae8812b0 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 -- 172.21.15.100:0/395766147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 msgr2=0x5560ae881690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc24cf49640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc2300a1bd0 0x7fc2300a4090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc240020000 tx=0x7fc24001f000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 0x5560ae881690 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc2400049d0 tx=0x7fc240012d10 comp rx=0 tx=0).stop 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 -- 172.21.15.100:0/395766147 shutdown_connections 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae88cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae881bd0 0x5560ae88a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 --2- 172.21.15.100:0/395766147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae8812b0 0x5560ae881690 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 -- 172.21.15.100:0/395766147 >> 172.21.15.100:0/395766147 conn(0x5560ae87b990 msgr2=0x5560ae87bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 -- 172.21.15.100:0/395766147 shutdown_connections 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 -- 172.21.15.100:0/395766147 wait complete. 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 -- start start 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 Processor -- start 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.883+0000 7fc25157e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae887590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae887cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae4bc430 con 0x5560ae881bd0 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae478e00 con 0x5560ae88a7b0 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae76dfd0 con 0x5560ae8812b0 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 10 client.0 _ll_register_callbacks cb 0x5560ae7c3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc247fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae887590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc247fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41200/0 (socket says 172.21.15.100:41200) 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc247fff640 1 -- 172.21.15.100:0/418657729 learned_addr learned my addr 172.21.15.100:0/418657729 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc24cf49640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae887cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 -- 172.21.15.100:0/418657729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 msgr2=0x5560ae88a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 -- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 msgr2=0x5560ae887cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae887cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 -- 172.21.15.100:0/418657729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2400124f0 con 0x5560ae881bd0 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc247fff640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc24cf49640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae887cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2477fe640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae887590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc23c036040 tx=0x7fc23c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbbbe23ca931b36 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2277fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2277fe640 1 -- 172.21.15.100:0/418657729 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c004040 con 0x5560ae881bd0 2024-03-12T05:15:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2277fe640 1 -- 172.21.15.100:0/418657729 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc23c0041a0 con 0x5560ae881bd0 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560ae4e87e0 con 0x5560ae881bd0 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.884+0000 7fc2277fe640 1 -- 172.21.15.100:0/418657729 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc23c004360 con 0x5560ae881bd0 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 10 client.16098 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5560ae7e7610 con 0x5560ae881bd0 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc2277fe640 1 -- 172.21.15.100:0/418657729 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5560ae4e87e0 con 0x5560ae881bd0 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc2277fe640 1 -- 172.21.15.100:0/418657729 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc23c03c040 con 0x5560ae881bd0 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 10 client.16098 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 20 client.16098 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae89aa20 0x5560ae89ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 4 client.16098 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560ae525720 con 0x5560ae89aa20 2024-03-12T05:15:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.885+0000 7fc247fff640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae89aa20 0x5560ae89ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc247fff640 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae89aa20 0x5560ae89ce40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69787d6da504fe95 server_cookie=22cd6a802ce0efff in_seq=0 out_seq=0 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc2277fe640 10 client.16098 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc2277fe640 1 -- 172.21.15.100:0/418657729 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560ae525720 con 0x5560ae89aa20 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc2277fe640 10 client.16098 handle_command_reply: tid=0 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc25157e740 1 client.16098 shutdown 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 msgr2=0x5560ae887590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.886+0000 7fc25157e740 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae887590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc23c036040 tx=0x7fc23c02ca10 comp rx=0 tx=0).stop 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 shutdown_connections 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae89aa20 0x5560ae89ce40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae887cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae887590 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 --2- 172.21.15.100:0/418657729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 >> 172.21.15.100:0/418657729 conn(0x5560ae87b990 msgr2=0x5560ae87d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc2277fe640 0 client.16098 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:09.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 shutdown_connections 2024-03-12T05:15:09.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 1 -- 172.21.15.100:0/418657729 wait complete. 2024-03-12T05:15:09.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.887+0000 7fc25157e740 20 client.16098 trim_cache size 0 max 16384 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 Processor -- start 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 -- start start 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae8812b0 0x5560ae8828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae882df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae889cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae897440 con 0x5560ae881bd0 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae8772b0 con 0x5560ae8812b0 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc25157e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae875e20 con 0x5560ae88a7b0 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae882df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae882df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54740/0 (socket says 172.21.15.100:54740) 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 -- 172.21.15.100:0/3601016500 learned_addr learned my addr 172.21.15.100:0/3601016500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc24cf49640 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae889cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc247fff640 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae8812b0 0x5560ae8828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 -- 172.21.15.100:0/3601016500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 msgr2=0x5560ae889cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae889cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 -- 172.21.15.100:0/3601016500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae8812b0 msgr2=0x5560ae8828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae8812b0 0x5560ae8828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.902+0000 7fc2477fe640 1 -- 172.21.15.100:0/3601016500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560ae783ac0 con 0x5560ae881bd0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.903+0000 7fc2477fe640 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae882df0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005ab0 tx=0x7fc23c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d874907c37a9441 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.903+0000 7fc22dffb640 1 -- 172.21.15.100:0/3601016500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0035f0 con 0x5560ae881bd0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.903+0000 7fc22dffb640 1 -- 172.21.15.100:0/3601016500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c0042f0 con 0x5560ae881bd0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.903+0000 7fc22dffb640 1 -- 172.21.15.100:0/3601016500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c005050 con 0x5560ae881bd0 2024-03-12T05:15:09.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.903+0000 7fc25157e740 1 -- 172.21.15.100:0/3601016500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 msgr2=0x5560ae882df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.903+0000 7fc25157e740 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae882df0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005ab0 tx=0x7fc23c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- 172.21.15.100:0/3601016500 shutdown_connections 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae88a7b0 0x5560ae889cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae882df0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 --2- 172.21.15.100:0/3601016500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae8812b0 0x5560ae8828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- 172.21.15.100:0/3601016500 >> 172.21.15.100:0/3601016500 conn(0x5560ae87b990 msgr2=0x5560ae8a17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- 172.21.15.100:0/3601016500 shutdown_connections 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- 172.21.15.100:0/3601016500 wait complete. 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- start start 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 Processor -- start 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae88db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae88e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae8f0030 con 0x5560ae881bd0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae8f01a0 con 0x5560ae88a7b0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ae8f0360 con 0x5560ae8812b0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc25157e740 10 client.0 _ll_register_callbacks cb 0x7fc240009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc2477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae88db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc2477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae88db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54750/0 (socket says 172.21.15.100:54750) 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc2477fe640 1 -- 172.21.15.100:0/811858420 learned_addr learned my addr 172.21.15.100:0/811858420 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc247fff640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.904+0000 7fc24cf49640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae88e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2477fe640 1 -- 172.21.15.100:0/811858420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 msgr2=0x5560ae88d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2477fe640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2477fe640 1 -- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 msgr2=0x5560ae88e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2477fe640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae88e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2477fe640 1 -- 172.21.15.100:0/811858420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560ae7e7610 con 0x5560ae881bd0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2477fe640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae88db70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc23c003b50 tx=0x7fc23c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308c786c6a3b09f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2277fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2277fe640 1 -- 172.21.15.100:0/811858420 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c060070 con 0x5560ae881bd0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc240018070 con 0x5560ae881bd0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2277fe640 1 -- 172.21.15.100:0/811858420 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc23c000e40 con 0x5560ae881bd0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2277fe640 1 -- 172.21.15.100:0/811858420 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc23c004150 con 0x5560ae881bd0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 10 client.16104 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5560ae7a96a0 con 0x5560ae881bd0 2024-03-12T05:15:09.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2277fe640 1 -- 172.21.15.100:0/811858420 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc240018070 con 0x5560ae881bd0 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc2277fe640 1 -- 172.21.15.100:0/811858420 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc23c063020 con 0x5560ae881bd0 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 10 client.16104 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 20 client.16104 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae8f1620 0x5560ae8f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.905+0000 7fc25157e740 4 client.16104 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560ae574890 con 0x5560ae8f1620 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc247fff640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae8f1620 0x5560ae8f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc2277fe640 10 client.16104 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc247fff640 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae8f1620 0x5560ae8f3a40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca17b8f08e81e6ce server_cookie=e030f3e05af5d6d9 in_seq=0 out_seq=0 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc2277fe640 1 -- 172.21.15.100:0/811858420 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5560ae574890 con 0x5560ae8f1620 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc2277fe640 10 client.16104 handle_command_reply: tid=0 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 client.16104 shutdown 2024-03-12T05:15:09.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 msgr2=0x5560ae88db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae88db70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc23c003b50 tx=0x7fc23c005020 comp rx=0 tx=0).stop 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 shutdown_connections 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5560ae8f1620 0x5560ae8f3a40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560ae88a7b0 0x5560ae88e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc2277fe640 0 client.16104 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ae881bd0 0x5560ae88db70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 --2- 172.21.15.100:0/811858420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560ae8812b0 0x5560ae88d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.906+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 >> 172.21.15.100:0/811858420 conn(0x5560ae87b990 msgr2=0x5560ae8828b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 shutdown_connections 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc25157e740 1 -- 172.21.15.100:0/811858420 wait complete. 2024-03-12T05:15:09.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc25157e740 20 client.16104 trim_cache size 0 max 16384 2024-03-12T05:15:09.912 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:15:09.912 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:15:09.912 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:15:09.912 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc2300a1bd0 msgr2=0x7fc2300a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc2300a1bd0 0x7fc2300a4090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc240020000 tx=0x7fc24001f000 comp rx=0 tx=0).stop 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 msgr2=0x7fc2481007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 0x7fc2481007f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc23800d770 tx=0x7fc23800dc30 comp rx=0 tx=0).stop 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 shutdown_connections 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc2300a1bd0 0x7fc2300a4090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248101270 0x7fc2481a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2481064f0 0x7fc248100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 --2- 172.21.15.100:0/3210236944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2480ff880 0x7fc2481007f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:09.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 >> 172.21.15.100:0/3210236944 conn(0x7fc248076590 msgr2=0x7fc2481047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:09.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 shutdown_connections 2024-03-12T05:15:09.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:09.907+0000 7fc24e9ac640 1 -- 172.21.15.100:0/3210236944 wait complete. 2024-03-12T05:15:09.915 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:15:09.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.064+0000 7fcf677d0640 1 -- 172.21.15.100:0/1356621244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60100730 msgr2=0x7fcf60105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.064+0000 7fcf677d0640 1 --2- 172.21.15.100:0/1356621244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60100730 0x7fcf60105e60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009930 tx=0x7fcf5002bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- 172.21.15.100:0/1356621244 shutdown_connections 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 --2- 172.21.15.100:0/1356621244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf60106540 0x7fcf6010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 --2- 172.21.15.100:0/1356621244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60100730 0x7fcf60105e60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 --2- 172.21.15.100:0/1356621244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf600ffd80 0x7fcf60100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- 172.21.15.100:0/1356621244 >> 172.21.15.100:0/1356621244 conn(0x7fcf60075390 msgr2=0x7fcf600757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- 172.21.15.100:0/1356621244 shutdown_connections 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- 172.21.15.100:0/1356621244 wait complete. 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 Processor -- start 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- start start 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf600ffd80 0x7fcf60100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 0x7fcf60101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101770 0x7fcf601a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601a60c0 con 0x7fcf60106540 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601a6230 con 0x7fcf60101770 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf677d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601a63f0 con 0x7fcf600ffd80 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf64d6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 0x7fcf60101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf64d6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 0x7fcf60101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54780/0 (socket says 172.21.15.100:54780) 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf64d6b640 1 -- 172.21.15.100:0/3200032059 learned_addr learned my addr 172.21.15.100:0/3200032059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.065+0000 7fcf6556c640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf600ffd80 0x7fcf60100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf65d6d640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101770 0x7fcf601a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf64d6b640 1 -- 172.21.15.100:0/3200032059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf600ffd80 msgr2=0x7fcf60100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf64d6b640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf600ffd80 0x7fcf60100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf64d6b640 1 -- 172.21.15.100:0/3200032059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101770 msgr2=0x7fcf601a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf64d6b640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101770 0x7fcf601a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf64d6b640 1 -- 172.21.15.100:0/3200032059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c009490 con 0x7fcf60106540 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf64d6b640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 0x7fcf60101230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf5002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004350 con 0x7fcf60106540 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf500044b0 con 0x7fcf60106540 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50003da0 con 0x7fcf60106540 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf50009560 con 0x7fcf60106540 2024-03-12T05:15:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.066+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf601a69f0 con 0x7fcf60106540 2024-03-12T05:15:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.067+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6006aa10 con 0x7fcf60106540 2024-03-12T05:15:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.068+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fcf50004df0 con 0x7fcf60106540 2024-03-12T05:15:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.069+0000 7fcf567fc640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcf4c0a1bd0 0x7fcf4c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.069+0000 7fcf6556c640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcf4c0a1bd0 0x7fcf4c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.069+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcf500efe90 con 0x7fcf60106540 2024-03-12T05:15:10.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.069+0000 7fcf6556c640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcf4c0a1bd0 0x7fcf4c0a4090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009d60 tx=0x7fcf5c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.073+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf500b8e60 con 0x7fcf60106540 2024-03-12T05:15:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.296+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf60102220 con 0x7fcf60106540 2024-03-12T05:15:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.297+0000 7fcf567fc640 1 -- 172.21.15.100:0/3200032059 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fcf500bc520 con 0x7fcf60106540 2024-03-12T05:15:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:15:10.295 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:13:19.700942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:15:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 24 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcf4c0a1bd0 msgr2=0x7fcf4c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcf4c0a1bd0 0x7fcf4c0a4090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009d60 tx=0x7fcf5c005f90 comp rx=0 tx=0).stop 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 msgr2=0x7fcf60101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 0x7fcf60101230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf5002c680 comp rx=0 tx=0).stop 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 shutdown_connections 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcf4c0a1bd0 0x7fcf4c0a4090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101770 0x7fcf601a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf60106540 0x7fcf60101230 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 --2- 172.21.15.100:0/3200032059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf600ffd80 0x7fcf60100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 >> 172.21.15.100:0/3200032059 conn(0x7fcf60075390 msgr2=0x7fcf600fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 shutdown_connections 2024-03-12T05:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.299+0000 7fcf677d0640 1 -- 172.21.15.100:0/3200032059 wait complete. 2024-03-12T05:15:10.307 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:15:10.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.416+0000 7f57799e2640 1 -- 172.21.15.100:0/592475372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5774106430 msgr2=0x7f577410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 --2- 172.21.15.100:0/592475372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5774106430 0x7f577410e000 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f576c009cd0 tx=0x7f576c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 -- 172.21.15.100:0/592475372 shutdown_connections 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 --2- 172.21.15.100:0/592475372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5774106430 0x7f577410e000 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 --2- 172.21.15.100:0/592475372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5774068f80 0x7f5774105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 --2- 172.21.15.100:0/592475372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f57740689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 -- 172.21.15.100:0/592475372 >> 172.21.15.100:0/592475372 conn(0x7f5774077180 msgr2=0x7f5774077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 -- 172.21.15.100:0/592475372 shutdown_connections 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 -- 172.21.15.100:0/592475372 wait complete. 2024-03-12T05:15:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 Processor -- start 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 -- start start 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.417+0000 7f57799e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f577419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5774106430 0x7f577419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57741975c0 0x7f5774197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774197fb0 con 0x7f57741975c0 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774198120 con 0x7f57740685d0 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57741982e0 con 0x7f5774106430 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f577419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57741975c0 0x7f5774197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f577419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:52728/0 (socket says 172.21.15.100:52728) 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 -- 172.21.15.100:0/1282212776 learned_addr learned my addr 172.21.15.100:0/1282212776 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f576a7fc640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5774106430 0x7f577419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 -- 172.21.15.100:0/1282212776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5774106430 msgr2=0x7f577419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5774106430 0x7f577419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 -- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57741975c0 msgr2=0x7f5774197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57741975c0 0x7f5774197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 -- 172.21.15.100:0/1282212776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c009900 con 0x7f57740685d0 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57737fe640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57741975c0 0x7f5774197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f5772ffd640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f577419d410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f576400e9f0 tx=0x7f576400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57789e0640 1 -- 172.21.15.100:0/1282212776 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764009700 con 0x7f57740685d0 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57741a42d0 con 0x7f57740685d0 2024-03-12T05:15:10.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57789e0640 1 -- 172.21.15.100:0/1282212776 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f576400cd80 con 0x7f57740685d0 2024-03-12T05:15:10.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57789e0640 1 -- 172.21.15.100:0/1282212776 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764018920 con 0x7f57740685d0 2024-03-12T05:15:10.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.418+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57741a4830 con 0x7f57740685d0 2024-03-12T05:15:10.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.420+0000 7f57789e0640 1 -- 172.21.15.100:0/1282212776 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5764018a80 con 0x7f57740685d0 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 Processor -- start 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 -- start start 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bc9b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9bb70 0x56139bca4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bca4750 0x56139bca6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bc56960 con 0x56139bc9b250 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bc0a310 con 0x56139bca4750 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f577c5bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bc07840 con 0x56139bc9bb70 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f57789e0640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f57600a1ba0 0x7f57600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f57737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bc9b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f57737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bc9b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54810/0 (socket says 172.21.15.100:54810) 2024-03-12T05:15:10.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f57737fe640 1 -- 172.21.15.100:0/2340525471 learned_addr learned my addr 172.21.15.100:0/2340525471 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f576a7fc640 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bca4750 0x56139bca6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f5772ffd640 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9bb70 0x56139bca4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f57737fe640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f57600a1ba0 0x7f57600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.421+0000 7f57789e0640 1 -- 172.21.15.100:0/1282212776 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5764014070 con 0x7f57740685d0 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f57737fe640 1 -- 172.21.15.100:0/2340525471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9bb70 msgr2=0x56139bca4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f57737fe640 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9bb70 0x56139bca4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f57737fe640 1 -- 172.21.15.100:0/2340525471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bca4750 msgr2=0x56139bca6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f57737fe640 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bca4750 0x56139bca6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f57737fe640 1 -- 172.21.15.100:0/2340525471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56139bbc5060 con 0x56139bc9b250 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f576a7fc640 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bca4750 0x56139bca6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f57737fe640 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bc9b630 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f576c004af0 tx=0x7f576c0325b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae08f85ce5b7c5e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f5769ffb640 1 -- 172.21.15.100:0/2340525471 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c032b40 con 0x56139bc9b250 2024-03-12T05:15:10.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f5769ffb640 1 -- 172.21.15.100:0/2340525471 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f576c003e50 con 0x56139bc9b250 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.422+0000 7f5769ffb640 1 -- 172.21.15.100:0/2340525471 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c004790 con 0x56139bc9b250 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- 172.21.15.100:0/2340525471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 msgr2=0x56139bc9b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f57737fe640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f57600a1ba0 0x7f57600a4060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f576c04f000 tx=0x7f576c04b000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bc9b630 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f576c004af0 tx=0x7f576c0325b0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- 172.21.15.100:0/2340525471 shutdown_connections 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bca4750 0x56139bca6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9bb70 0x56139bca4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2340525471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bc9b630 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- 172.21.15.100:0/2340525471 >> 172.21.15.100:0/2340525471 conn(0x56139bc95980 msgr2=0x56139bc95d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- 172.21.15.100:0/2340525471 shutdown_connections 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- 172.21.15.100:0/2340525471 wait complete. 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- start start 2024-03-12T05:15:10.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 Processor -- start 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bca17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bca1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bca2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bc03020 con 0x56139bc9b250 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139b8d6430 con 0x56139bc9bb70 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.423+0000 7f577c5bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139b892e00 con 0x56139bca4750 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f577c5bb740 10 client.0 _ll_register_callbacks cb 0x56139bbaacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bca17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bca17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54820/0 (socket says 172.21.15.100:54820) 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 -- 172.21.15.100:0/759824671 learned_addr learned my addr 172.21.15.100:0/759824671 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f576a7fc640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bca1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f57737fe640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bca2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 -- 172.21.15.100:0/759824671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 msgr2=0x56139bca2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bca2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 -- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 msgr2=0x56139bca1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bca1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 -- 172.21.15.100:0/759824671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577410e810 con 0x56139bc9b250 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5772ffd640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bca17d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f5764014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21c3f4df091fa6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5752ffd640 1 -- 172.21.15.100:0/759824671 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57640cb750 con 0x56139bc9b250 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5752ffd640 1 -- 172.21.15.100:0/759824671 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f57640a2f50 con 0x56139bc9b250 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56139b9027e0 con 0x56139bc9b250 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5752ffd640 1 -- 172.21.15.100:0/759824671 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57640b3270 con 0x56139bc9b250 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f577c5bb740 10 client.16122 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56139bc016a0 con 0x56139bc9b250 2024-03-12T05:15:10.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.424+0000 7f5752ffd640 1 -- 172.21.15.100:0/759824671 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5764093020 con 0x56139bc9b250 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f5752ffd640 1 -- 172.21.15.100:0/759824671 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5764099cb0 con 0x56139bc9b250 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f577c5bb740 10 client.16122 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f577c5bb740 10 client.16122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f577c5bb740 20 client.16122 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f577c5bb740 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bcb4c50 0x56139bcb7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f577c5bb740 4 client.16122 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56139b93f720 con 0x56139bcb4c50 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f576a7fc640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bcb4c50 0x56139bcb7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f5752ffd640 10 client.16122 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.425+0000 7f576a7fc640 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bcb4c50 0x56139bcb7070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88e91bab5c8d1a0e server_cookie=189b63fd2115faac in_seq=0 out_seq=0 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f5752ffd640 1 -- 172.21.15.100:0/759824671 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56139b93f720 con 0x56139bcb4c50 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f5752ffd640 10 client.16122 handle_command_reply: tid=0 2024-03-12T05:15:10.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 client.16122 shutdown 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 msgr2=0x56139bca17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bca17d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f5764014600 comp rx=0 tx=0).stop 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 shutdown_connections 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bcb4c50 0x56139bcb7070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bca2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bca1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f5752ffd640 0 client.16122 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 --2- 172.21.15.100:0/759824671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9b250 0x56139bca17d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.428+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 >> 172.21.15.100:0/759824671 conn(0x56139bc95980 msgr2=0x56139bc976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.429+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 shutdown_connections 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.429+0000 7f577c5bb740 1 -- 172.21.15.100:0/759824671 wait complete. 2024-03-12T05:15:10.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.429+0000 7f577c5bb740 20 client.16122 trim_cache size 0 max 16384 2024-03-12T05:15:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 Processor -- start 2024-03-12T05:15:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 -- start start 2024-03-12T05:15:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9b250 0x56139bca3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 0x56139bca2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bc9ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bb88130 con 0x56139bc9bb70 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bcbc280 con 0x56139bc9b250 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f577c5bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bc91080 con 0x56139bca4750 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f576a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 0x56139bca2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f576a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 0x56139bca2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54836/0 (socket says 172.21.15.100:54836) 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f57737fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bc9ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f576a7fc640 1 -- 172.21.15.100:0/2262272802 learned_addr learned my addr 172.21.15.100:0/2262272802 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.441+0000 7f5772ffd640 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9b250 0x56139bca3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f576a7fc640 1 -- 172.21.15.100:0/2262272802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 msgr2=0x56139bc9ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f576a7fc640 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bc9ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f576a7fc640 1 -- 172.21.15.100:0/2262272802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9b250 msgr2=0x56139bca3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f576a7fc640 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9b250 0x56139bca3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f576a7fc640 1 -- 172.21.15.100:0/2262272802 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56139bb9dc20 con 0x56139bc9bb70 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f57737fe640 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bc9ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f576a7fc640 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 0x56139bca2070 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a036a27046a503 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f57697fa640 1 -- 172.21.15.100:0/2262272802 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c017040 con 0x56139bc9bb70 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f57697fa640 1 -- 172.21.15.100:0/2262272802 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f575c018040 con 0x56139bc9bb70 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.442+0000 7f57697fa640 1 -- 172.21.15.100:0/2262272802 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c004c40 con 0x56139bc9bb70 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 -- 172.21.15.100:0/2262272802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 msgr2=0x56139bca2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 0x56139bca2070 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c005f90 comp rx=0 tx=0).stop 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 -- 172.21.15.100:0/2262272802 shutdown_connections 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bca4750 0x56139bc9ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bc9bb70 0x56139bca2070 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 --2- 172.21.15.100:0/2262272802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9b250 0x56139bca3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 -- 172.21.15.100:0/2262272802 >> 172.21.15.100:0/2262272802 conn(0x56139bc95980 msgr2=0x56139bc9ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 -- 172.21.15.100:0/2262272802 shutdown_connections 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 -- 172.21.15.100:0/2262272802 wait complete. 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 -- start start 2024-03-12T05:15:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.443+0000 7f577c5bb740 1 Processor -- start 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9b250 0x56139bd06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bd06970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 0x56139bcaa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bcaadc0 con 0x56139bca4750 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bcaaf30 con 0x56139bc9bb70 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139bcab0f0 con 0x56139bc9b250 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 10 client.0 _ll_register_callbacks cb 0x7f5764005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 0x56139bcaa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 0x56139bcaa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54842/0 (socket says 172.21.15.100:54842) 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 -- 172.21.15.100:0/389670885 learned_addr learned my addr 172.21.15.100:0/389670885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f5772ffd640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9b250 0x56139bd06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f576a7fc640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bd06970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 -- 172.21.15.100:0/389670885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9b250 msgr2=0x56139bd06430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9b250 0x56139bd06430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 -- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 msgr2=0x56139bd06970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bd06970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 -- 172.21.15.100:0/389670885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56139bbc5060 con 0x56139bca4750 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f576a7fc640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bd06970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f57737fe640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 0x56139bcaa780 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f576c037040 tx=0x7f576c034f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeb18419f619707 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f5752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f5752ffd640 1 -- 172.21.15.100:0/389670885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c032ea0 con 0x56139bca4750 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f5752ffd640 1 -- 172.21.15.100:0/389670885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f576c05c070 con 0x56139bca4750 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.444+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f576c003e50 con 0x56139bca4750 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f5752ffd640 1 -- 172.21.15.100:0/389670885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f576c032b40 con 0x56139bca4750 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 10 client.16128 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f575c009b00 con 0x56139bca4750 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f5752ffd640 1 -- 172.21.15.100:0/389670885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f576c003e50 con 0x56139bca4750 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f5752ffd640 1 -- 172.21.15.100:0/389670885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f576c03e050 con 0x56139bca4750 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 10 client.16128 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 10 client.16128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 20 client.16128 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bd11560 0x56139bd13950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 4 client.16128 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56139b98e890 con 0x56139bd11560 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.445+0000 7f5772ffd640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bd11560 0x56139bd13950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.446+0000 7f5752ffd640 10 client.16128 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.446+0000 7f5772ffd640 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bd11560 0x56139bd13950 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb14b96a4f1a1ed server_cookie=501044ad77d9fb60 in_seq=0 out_seq=0 2024-03-12T05:15:10.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.449+0000 7f5752ffd640 1 -- 172.21.15.100:0/389670885 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56139b98e890 con 0x56139bd11560 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.449+0000 7f5752ffd640 10 client.16128 handle_command_reply: tid=0 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.449+0000 7f577c5bb740 1 client.16128 shutdown 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 msgr2=0x56139bcaa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 0x56139bcaa780 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f576c037040 tx=0x7f576c034f00 comp rx=0 tx=0).stop 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 shutdown_connections 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56139bd11560 0x56139bd13950 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56139bca4750 0x56139bcaa780 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f5752ffd640 0 client.16128 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56139bc9bb70 0x56139bd06970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 --2- 172.21.15.100:0/389670885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56139bc9b250 0x56139bd06430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 >> 172.21.15.100:0/389670885 conn(0x56139bc95980 msgr2=0x56139bc942f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 shutdown_connections 2024-03-12T05:15:10.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 1 -- 172.21.15.100:0/389670885 wait complete. 2024-03-12T05:15:10.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f577c5bb740 20 client.16128 trim_cache size 0 max 16384 2024-03-12T05:15:10.451 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f57600a1ba0 msgr2=0x7f57600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f57600a1ba0 0x7f57600a4060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f576c04f000 tx=0x7f576c04b000 comp rx=0 tx=0).stop 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 msgr2=0x7f577419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f577419d410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f576400e9f0 tx=0x7f576400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 shutdown_connections 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f57600a1ba0 0x7f57600a4060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57741975c0 0x7f5774197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5774106430 0x7f577419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 --2- 172.21.15.100:0/1282212776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57740685d0 0x7f577419d410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 >> 172.21.15.100:0/1282212776 conn(0x7f5774077180 msgr2=0x7f57740fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 shutdown_connections 2024-03-12T05:15:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.450+0000 7f57799e2640 1 -- 172.21.15.100:0/1282212776 wait complete. 2024-03-12T05:15:10.458 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:15:10.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:15:10.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 -- 172.21.15.100:0/1042792642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a80685f0 msgr2=0x7fe8a80689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 --2- 172.21.15.100:0/1042792642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a80685f0 0x7fe8a80689d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe898009cd0 tx=0x7fe89802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 -- 172.21.15.100:0/1042792642 shutdown_connections 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 --2- 172.21.15.100:0/1042792642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a81064c0 0x7fe8a810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 --2- 172.21.15.100:0/1042792642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8068fa0 0x7fe8a8105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 --2- 172.21.15.100:0/1042792642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a80685f0 0x7fe8a80689d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 -- 172.21.15.100:0/1042792642 >> 172.21.15.100:0/1042792642 conn(0x7fe8a8077180 msgr2=0x7fe8a8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 -- 172.21.15.100:0/1042792642 shutdown_connections 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 -- 172.21.15.100:0/1042792642 wait complete. 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.590+0000 7fe8addf4640 1 Processor -- start 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 -- start start 2024-03-12T05:15:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a80685f0 0x7fe8a8100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 0x7fe8a8100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a8100ac0 0x7fe8a81a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a81a5fa0 con 0x7fe8a81064c0 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a81a6110 con 0x7fe8a80685f0 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a81a62d0 con 0x7fe8a8100ac0 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 0x7fe8a8100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 0x7fe8a8100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33436/0 (socket says 172.21.15.100:33436) 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/4095122696 learned_addr learned my addr 172.21.15.100:0/4095122696 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a80685f0 0x7fe8a8100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a8100ac0 0x7fe8a81a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/4095122696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a8100ac0 msgr2=0x7fe8a81a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a8100ac0 0x7fe8a81a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/4095122696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a80685f0 msgr2=0x7fe8a8100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a80685f0 0x7fe8a8100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/4095122696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898009900 con 0x7fe8a81064c0 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 0x7fe8a8100580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe89000e9f0 tx=0x7fe89000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a4ff9640 1 -- 172.21.15.100:0/4095122696 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8900096a0 con 0x7fe8a81064c0 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a81a65b0 con 0x7fe8a81064c0 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a4ff9640 1 -- 172.21.15.100:0/4095122696 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe89000ccc0 con 0x7fe8a81064c0 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.591+0000 7fe8a4ff9640 1 -- 172.21.15.100:0/4095122696 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890018860 con 0x7fe8a81064c0 2024-03-12T05:15:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.592+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a81a6b10 con 0x7fe8a81064c0 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a4ff9640 1 -- 172.21.15.100:0/4095122696 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe890010aa0 con 0x7fe8a81064c0 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 Processor -- start 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 -- start start 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b03346630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b03346b70 0x559b0334f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0334f750 0x559b03351b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b03301960 con 0x559b03346250 2024-03-12T05:15:10.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b032b5310 con 0x559b03346b70 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b032b2840 con 0x559b0334f750 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a4ff9640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8940a1bd0 0x7fe8940a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b03346630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b03346630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33438/0 (socket says 172.21.15.100:33438) 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a7fff640 1 -- 172.21.15.100:0/3896781470 learned_addr learned my addr 172.21.15.100:0/3896781470 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b03346b70 0x559b0334f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0334f750 0x559b03351b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.594+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8940a1bd0 0x7fe8940a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3896781470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0334f750 msgr2=0x559b03351b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0334f750 0x559b03351b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3896781470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 msgr2=0x559b03346630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b03346630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3896781470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b03270060 con 0x559b03346b70 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a4ff9640 1 -- 172.21.15.100:0/4095122696 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe890014070 con 0x7fe8a81064c0 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b03346b70 0x559b0334f210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c700 tx=0x7fe898005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e9c797acb8de3b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8940a1bd0 0x7fe8940a4090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0004650 tx=0x7fe8a0004730 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe89dffb640 1 -- 172.21.15.100:0/3896781470 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898034440 con 0x559b03346b70 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe89dffb640 1 -- 172.21.15.100:0/3896781470 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8980345a0 con 0x559b03346b70 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.595+0000 7fe89dffb640 1 -- 172.21.15.100:0/3896781470 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898003bd0 con 0x559b03346b70 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3896781470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b03346b70 msgr2=0x559b0334f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b03346b70 0x559b0334f210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c700 tx=0x7fe898005f70 comp rx=0 tx=0).stop 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3896781470 shutdown_connections 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b0334f750 0x559b03351b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b03346b70 0x559b0334f210 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3896781470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b03346630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3896781470 >> 172.21.15.100:0/3896781470 conn(0x559b03340980 msgr2=0x559b03340d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3896781470 shutdown_connections 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3896781470 wait complete. 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- start start 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 Processor -- start 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b0334d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b032ae020 con 0x559b03346250 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b02f81430 con 0x559b0334f750 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b02f3de00 con 0x559b03346b70 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.596+0000 7fe8b09c8740 10 client.0 _ll_register_callbacks cb 0x559b03255cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b0334d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33450/0 (socket says 172.21.15.100:33450) 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3927544388 learned_addr learned my addr 172.21.15.100:0/3927544388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 msgr2=0x559b0334cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3927544388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 msgr2=0x559b0334d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b0334d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 -- 172.21.15.100:0/3927544388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a810e880 con 0x559b03346250 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b0334d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334c720 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c8c0 tx=0x7fe898034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ade799edc8d4b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe886ffd640 1 -- 172.21.15.100:0/3927544388 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898034c90 con 0x559b03346250 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe886ffd640 1 -- 172.21.15.100:0/3927544388 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe898034df0 con 0x559b03346250 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b02fad7e0 con 0x559b03346250 2024-03-12T05:15:10.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe886ffd640 1 -- 172.21.15.100:0/3927544388 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8980329c0 con 0x559b03346250 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8b09c8740 10 client.16140 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.597+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559b032ac6a0 con 0x559b03346250 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe886ffd640 1 -- 172.21.15.100:0/3927544388 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559b02fad7e0 con 0x559b03346250 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe886ffd640 1 -- 172.21.15.100:0/3927544388 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe89804a450 con 0x559b03346250 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8b09c8740 10 client.16140 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8b09c8740 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8b09c8740 20 client.16140 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:10.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b0335f700 0x559b03361af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8b09c8740 4 client.16140 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b02fea720 con 0x559b0335f700 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b0335f700 0x559b03361af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe886ffd640 10 client.16140 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.598+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b0335f700 0x559b03361af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86ade0ce684efce2 server_cookie=6e309280b503d3ea in_seq=0 out_seq=0 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe886ffd640 1 -- 172.21.15.100:0/3927544388 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b02fea720 con 0x559b0335f700 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe886ffd640 10 client.16140 handle_command_reply: tid=0 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 client.16140 shutdown 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 msgr2=0x559b0334c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334c720 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c8c0 tx=0x7fe898034440 comp rx=0 tx=0).stop 2024-03-12T05:15:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 shutdown_connections 2024-03-12T05:15:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b0335f700 0x559b03361af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b0334d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/3927544388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334c720 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 >> 172.21.15.100:0/3927544388 conn(0x559b03340980 msgr2=0x559b033426d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe886ffd640 0 client.16140 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 shutdown_connections 2024-03-12T05:15:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 1 -- 172.21.15.100:0/3927544388 wait complete. 2024-03-12T05:15:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.599+0000 7fe8b09c8740 20 client.16140 trim_cache size 0 max 16384 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.611+0000 7fe8b09c8740 1 Processor -- start 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.611+0000 7fe8b09c8740 1 -- start start 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b03366450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b03233130 con 0x559b03346250 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0333c080 con 0x559b0334f750 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b033404b0 con 0x559b03346b70 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33452/0 (socket says 172.21.15.100:33452) 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 -- 172.21.15.100:0/1471438663 learned_addr learned my addr 172.21.15.100:0/1471438663 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b03366450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 -- 172.21.15.100:0/1471438663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 msgr2=0x559b0334cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 -- 172.21.15.100:0/1471438663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 msgr2=0x559b03366450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b03366450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 -- 172.21.15.100:0/1471438663 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b03248c20 con 0x559b03346250 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b03366450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334ee50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c890 tx=0x7fe898005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9012cc3343efc0b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe89d7fa640 1 -- 172.21.15.100:0/1471438663 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89804a450 con 0x559b03346250 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe89d7fa640 1 -- 172.21.15.100:0/1471438663 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe89804a5b0 con 0x559b03346250 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.612+0000 7fe89d7fa640 1 -- 172.21.15.100:0/1471438663 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898003d60 con 0x559b03346250 2024-03-12T05:15:10.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1471438663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 msgr2=0x559b0334ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334ee50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c890 tx=0x7fe898005bf0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1471438663 shutdown_connections 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b03366450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346b70 0x559b0334cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1471438663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346250 0x559b0334ee50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1471438663 >> 172.21.15.100:0/1471438663 conn(0x559b03340980 msgr2=0x559b03347a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1471438663 shutdown_connections 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1471438663 wait complete. 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.613+0000 7fe8b09c8740 1 -- start start 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 Processor -- start 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 0x559b03356660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 0x559b03354ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b033550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b03355620 con 0x559b03346b70 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b03355790 con 0x559b0334f750 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b03355950 con 0x559b03346250 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8b09c8740 10 client.0 _ll_register_callbacks cb 0x7fe890005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 0x559b03354ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 0x559b03356660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 0x559b03354ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33466/0 (socket says 172.21.15.100:33466) 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/1529575480 learned_addr learned my addr 172.21.15.100:0/1529575480 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 0x559b03356660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42392/0 (socket says 172.21.15.100:42392) 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a7fff640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b033550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 msgr2=0x559b03356660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 0x559b03356660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/1529575480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 msgr2=0x559b033550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b033550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 -- 172.21.15.100:0/1529575480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b032ac6a0 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 0x559b03356660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe8a6ffd640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 0x559b03354ba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe89009a4e0 tx=0x7fe89001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3b79648e57b47 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.614+0000 7fe886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe886ffd640 1 -- 172.21.15.100:0/1529575480 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8900b8ea0 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe886ffd640 1 -- 172.21.15.100:0/1529575480 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe89009e880 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8980345a0 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe886ffd640 1 -- 172.21.15.100:0/1529575480 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8900a2ed0 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 10 client.16146 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559b0326e8c0 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe886ffd640 1 -- 172.21.15.100:0/1529575480 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe890064080 con 0x559b03346b70 2024-03-12T05:15:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe886ffd640 1 -- 172.21.15.100:0/1529575480 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe8900a2a40 con 0x559b03346b70 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 10 client.16146 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 10 client.16146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 20 client.16146 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b033b7050 0x559b033b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 4 client.16146 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe8980345a0 con 0x559b033b7050 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.615+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b033b7050 0x559b033b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe886ffd640 10 client.16146 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8a77fe640 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b033b7050 0x559b033b9470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d6c34c3204913e9 server_cookie=7f2da5e99f1a7e57 in_seq=0 out_seq=0 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe886ffd640 1 -- 172.21.15.100:0/1529575480 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe8980345a0 con 0x559b033b7050 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe886ffd640 10 client.16146 handle_command_reply: tid=0 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 client.16146 shutdown 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 msgr2=0x559b03354ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 0x559b03354ba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe89009a4e0 tx=0x7fe89001c850 comp rx=0 tx=0).stop 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 shutdown_connections 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559b033b7050 0x559b033b9470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b0334f750 0x559b033550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe886ffd640 0 client.16146 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b03346b70 0x559b03354ba0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 --2- 172.21.15.100:0/1529575480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b03346250 0x559b03356660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 >> 172.21.15.100:0/1529575480 conn(0x559b03340980 msgr2=0x559b0334abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 shutdown_connections 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 1 -- 172.21.15.100:0/1529575480 wait complete. 2024-03-12T05:15:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.616+0000 7fe8b09c8740 20 client.16146 trim_cache size 0 max 16384 2024-03-12T05:15:10.623 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:15:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8940a1bd0 msgr2=0x7fe8940a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8940a1bd0 0x7fe8940a4090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0004650 tx=0x7fe8a0004730 comp rx=0 tx=0).stop 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 msgr2=0x7fe8a8100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 0x7fe8a8100580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe89000e9f0 tx=0x7fe89000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 shutdown_connections 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8940a1bd0 0x7fe8940a4090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a8100ac0 0x7fe8a81a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a81064c0 0x7fe8a8100580 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 --2- 172.21.15.100:0/4095122696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a80685f0 0x7fe8a8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 >> 172.21.15.100:0/4095122696 conn(0x7fe8a8077180 msgr2=0x7fe8a80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 shutdown_connections 2024-03-12T05:15:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.617+0000 7fe8addf4640 1 -- 172.21.15.100:0/4095122696 wait complete. 2024-03-12T05:15:10.625 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:15:10.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:15:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 -- 172.21.15.100:0/1339953832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1064c0 msgr2=0x7f0a9c10e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/1339953832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1064c0 0x7f0a9c10e070 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009cd0 tx=0x7f0a9402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 -- 172.21.15.100:0/1339953832 shutdown_connections 2024-03-12T05:15:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/1339953832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1064c0 0x7f0a9c10e070 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/1339953832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c068fa0 0x7f0a9c105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/1339953832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 -- 172.21.15.100:0/1339953832 >> 172.21.15.100:0/1339953832 conn(0x7f0a9c077180 msgr2=0x7f0a9c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 -- 172.21.15.100:0/1339953832 shutdown_connections 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.734+0000 7f0aa44ae640 1 -- 172.21.15.100:0/1339953832 wait complete. 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 Processor -- start 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 -- start start 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9c1064c0 0x7f0a9c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c1975c0 0x7f0a9c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c197fb0 con 0x7f0a9c0685f0 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c198120 con 0x7f0a9c1064c0 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa44ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1982e0 con 0x7f0a9c1975c0 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33476/0 (socket says 172.21.15.100:33476) 2024-03-12T05:15:10.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 -- 172.21.15.100:0/904514726 learned_addr learned my addr 172.21.15.100:0/904514726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9c1064c0 0x7f0a9c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c1975c0 0x7f0a9c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 -- 172.21.15.100:0/904514726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c1975c0 msgr2=0x7f0a9c197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c1975c0 0x7f0a9c197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 -- 172.21.15.100:0/904514726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9c1064c0 msgr2=0x7f0a9c19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9c1064c0 0x7f0a9c19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.735+0000 7f0aa224a640 1 -- 172.21.15.100:0/904514726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94009900 con 0x7f0a9c0685f0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.736+0000 7f0aa224a640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c19d410 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800e9f0 tx=0x7f0a9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.736+0000 7f0a937fe640 1 -- 172.21.15.100:0/904514726 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98009700 con 0x7f0a9c0685f0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.736+0000 7f0a937fe640 1 -- 172.21.15.100:0/904514726 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a9800cd80 con 0x7f0a9c0685f0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.736+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c1a42d0 con 0x7f0a9c0685f0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.736+0000 7f0a937fe640 1 -- 172.21.15.100:0/904514726 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98018920 con 0x7f0a9c0685f0 2024-03-12T05:15:10.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.736+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1a47e0 con 0x7f0a9c0685f0 2024-03-12T05:15:10.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.738+0000 7f0a937fe640 1 -- 172.21.15.100:0/904514726 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0a98018a80 con 0x7f0a9c0685f0 2024-03-12T05:15:10.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 Processor -- start 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 -- start start 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1201630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef120a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef120cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef11bc960 con 0x556ef120a750 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef1170310 con 0x556ef1201b70 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa7080740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef116d840 con 0x556ef1201250 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0a937fe640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a880a1ba0 0x7f0a880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa1a49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef120cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a880a1ba0 0x7f0a880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa224a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef120a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa1a49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef120cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33492/0 (socket says 172.21.15.100:33492) 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa1a49640 1 -- 172.21.15.100:0/635168344 learned_addr learned my addr 172.21.15.100:0/635168344 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0a937fe640 1 -- 172.21.15.100:0/904514726 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a98014070 con 0x7f0a9c0685f0 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.739+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1201630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa1a49640 1 -- 172.21.15.100:0/635168344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 msgr2=0x556ef1201630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1201630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa1a49640 1 -- 172.21.15.100:0/635168344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 msgr2=0x556ef120a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef120a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa1a49640 1 -- 172.21.15.100:0/635168344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ef112b060 con 0x556ef120a750 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa224a640 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef120a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a880a1ba0 0x7f0a880a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a94005cc0 tx=0x7f0a94009820 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef120cb40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c004630 tx=0x7f0a8c00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d54f86b55b12e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0a90bfe640 1 -- 172.21.15.100:0/635168344 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003bd0 con 0x556ef120a750 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0a90bfe640 1 -- 172.21.15.100:0/635168344 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c003d30 con 0x556ef120a750 2024-03-12T05:15:10.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.740+0000 7f0a90bfe640 1 -- 172.21.15.100:0/635168344 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c004e90 con 0x556ef120a750 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- 172.21.15.100:0/635168344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 msgr2=0x556ef120cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef120cb40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c004630 tx=0x7f0a8c00bb20 comp rx=0 tx=0).stop 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- 172.21.15.100:0/635168344 shutdown_connections 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef120cb40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef120a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- 172.21.15.100:0/635168344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1201630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- 172.21.15.100:0/635168344 >> 172.21.15.100:0/635168344 conn(0x556ef11fb980 msgr2=0x556ef11fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- 172.21.15.100:0/635168344 shutdown_connections 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- 172.21.15.100:0/635168344 wait complete. 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- start start 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 Processor -- start 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1207700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef1207c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef1208180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef1169020 con 0x556ef120a750 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef0e3c430 con 0x556ef1201b70 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef0df8e00 con 0x556ef1201250 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.741+0000 7f0aa7080740 10 client.0 _ll_register_callbacks cb 0x556ef1110cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef1208180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef1208180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33494/0 (socket says 172.21.15.100:33494) 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 -- 172.21.15.100:0/892833078 learned_addr learned my addr 172.21.15.100:0/892833078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa224a640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1207700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef1207c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 -- 172.21.15.100:0/892833078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 msgr2=0x556ef1207700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1207700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 -- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 msgr2=0x556ef1207c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef1207c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 -- 172.21.15.100:0/892833078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c004290 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef1208180 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0a94004af0 tx=0x7f0a9402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdab6e160d21a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0a85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0a85ffb640 1 -- 172.21.15.100:0/892833078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94032ac0 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0a85ffb640 1 -- 172.21.15.100:0/892833078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a94003970 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ef0e687e0 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0a85ffb640 1 -- 172.21.15.100:0/892833078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a94003b30 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa7080740 10 client.16158 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556ef11676a0 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.742+0000 7f0a85ffb640 1 -- 172.21.15.100:0/892833078 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556ef0e687e0 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0a85ffb640 1 -- 172.21.15.100:0/892833078 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0a9404c500 con 0x556ef120a750 2024-03-12T05:15:10.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa7080740 10 client.16158 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa7080740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa7080740 20 client.16158 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa7080740 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef121aac0 0x556ef121cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa7080740 4 client.16158 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ef0ea5720 con 0x556ef121aac0 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa224a640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef121aac0 0x556ef121cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0a85ffb640 10 client.16158 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.743+0000 7f0aa224a640 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef121aac0 0x556ef121cee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9e676d68ba62fe server_cookie=f3937ebd62816743 in_seq=0 out_seq=0 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0a85ffb640 1 -- 172.21.15.100:0/892833078 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ef0ea5720 con 0x556ef121aac0 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0a85ffb640 10 client.16158 handle_command_reply: tid=0 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 client.16158 shutdown 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 msgr2=0x556ef1208180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef1208180 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0a94004af0 tx=0x7f0a9402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 shutdown_connections 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef121aac0 0x556ef121cee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef120a750 0x556ef1208180 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201b70 0x556ef1207c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0a85ffb640 0 client.16158 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 --2- 172.21.15.100:0/892833078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef1201250 0x556ef1207700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 >> 172.21.15.100:0/892833078 conn(0x556ef11fb980 msgr2=0x556ef11fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 shutdown_connections 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 1 -- 172.21.15.100:0/892833078 wait complete. 2024-03-12T05:15:10.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.744+0000 7f0aa7080740 20 client.16158 trim_cache size 0 max 16384 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 Processor -- start 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 -- start start 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1209e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1207fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1204980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef10ee130 con 0x556ef1201b70 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef0ea5720 con 0x556ef1201250 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa7080740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef11fcb40 con 0x556ef120a750 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1207fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1207fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33500/0 (socket says 172.21.15.100:33500) 2024-03-12T05:15:10.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 -- 172.21.15.100:0/3893318943 learned_addr learned my addr 172.21.15.100:0/3893318943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa224a640 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1209e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1204980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 -- 172.21.15.100:0/3893318943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 msgr2=0x556ef1204980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1204980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 -- 172.21.15.100:0/3893318943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 msgr2=0x556ef1209e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.756+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1209e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa1a49640 1 -- 172.21.15.100:0/3893318943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ef1103c20 con 0x556ef1201b70 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1207fa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005d00 tx=0x7f0a8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb3cca44bb30fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0a87fff640 1 -- 172.21.15.100:0/3893318943 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c004660 con 0x556ef1201b70 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0a87fff640 1 -- 172.21.15.100:0/3893318943 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c01f040 con 0x556ef1201b70 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0a87fff640 1 -- 172.21.15.100:0/3893318943 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0116a0 con 0x556ef1201b70 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 -- 172.21.15.100:0/3893318943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 msgr2=0x556ef1207fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1207fa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005d00 tx=0x7f0a8c004ad0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 -- 172.21.15.100:0/3893318943 shutdown_connections 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1204980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1207fa0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 --2- 172.21.15.100:0/3893318943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1209e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.757+0000 7f0aa7080740 1 -- 172.21.15.100:0/3893318943 >> 172.21.15.100:0/3893318943 conn(0x556ef11fb980 msgr2=0x556ef1221860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 -- 172.21.15.100:0/3893318943 shutdown_connections 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 -- 172.21.15.100:0/3893318943 wait complete. 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 -- start start 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 Processor -- start 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1210130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1210670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1210bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef1270ff0 con 0x556ef1201b70 2024-03-12T05:15:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef1271130 con 0x556ef1201250 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef12712f0 con 0x556ef120a750 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa7080740 10 client.0 _ll_register_callbacks cb 0x7f0a8c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1210670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1210670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33502/0 (socket says 172.21.15.100:33502) 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 -- 172.21.15.100:0/2290775005 learned_addr learned my addr 172.21.15.100:0/2290775005 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa224a640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1210130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa2a4b640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1210bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 -- 172.21.15.100:0/2290775005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 msgr2=0x556ef1210bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1210bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 -- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 msgr2=0x556ef1210130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1210130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.758+0000 7f0aa1a49640 1 -- 172.21.15.100:0/2290775005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ef112b060 con 0x556ef1201b70 2024-03-12T05:15:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa1a49640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1210670 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005880 tx=0x7f0a8c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccab7497663c9cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0a85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0a85ffb640 1 -- 172.21.15.100:0/2290775005 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c004860 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0a85ffb640 1 -- 172.21.15.100:0/2290775005 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a8c025480 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a8c003d30 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0a85ffb640 1 -- 172.21.15.100:0/2290775005 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a8c025640 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 10 client.16164 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556ef11298c0 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0a85ffb640 1 -- 172.21.15.100:0/2290775005 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c042080 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0a85ffb640 1 -- 172.21.15.100:0/2290775005 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0a8c011cf0 con 0x556ef1201b70 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 10 client.16164 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 20 client.16164 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef1272810 0x556ef1274c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 4 client.16164 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ef0ef4890 con 0x556ef1272810 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.759+0000 7f0aa224a640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef1272810 0x556ef1274c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0a85ffb640 10 client.16164 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa224a640 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef1272810 0x556ef1274c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987c2c9e1ebc6946 server_cookie=b19bbb0407b9de87 in_seq=0 out_seq=0 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0a85ffb640 1 -- 172.21.15.100:0/2290775005 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ef0ef4890 con 0x556ef1272810 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0a85ffb640 10 client.16164 handle_command_reply: tid=0 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 client.16164 shutdown 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 msgr2=0x556ef1210670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1210670 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005880 tx=0x7f0a8c00b390 comp rx=0 tx=0).stop 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 shutdown_connections 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556ef1272810 0x556ef1274c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ef120a750 0x556ef1210bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ef1201b70 0x556ef1210670 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 --2- 172.21.15.100:0/2290775005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ef1201250 0x556ef1210130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 >> 172.21.15.100:0/2290775005 conn(0x556ef11fb980 msgr2=0x556ef11fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0a85ffb640 0 client.16164 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 shutdown_connections 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 1 -- 172.21.15.100:0/2290775005 wait complete. 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.760+0000 7f0aa7080740 20 client.16164 trim_cache size 0 max 16384 2024-03-12T05:15:10.765 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:15:10.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a880a1ba0 msgr2=0x7f0a880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a880a1ba0 0x7f0a880a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a94005cc0 tx=0x7f0a94009820 comp rx=0 tx=0).stop 2024-03-12T05:15:10.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 msgr2=0x7f0a9c19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:10.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c19d410 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800e9f0 tx=0x7f0a9800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 shutdown_connections 2024-03-12T05:15:10.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a880a1ba0 0x7f0a880a4060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c1975c0 0x7f0a9c197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9c1064c0 0x7f0a9c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 --2- 172.21.15.100:0/904514726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c0685f0 0x7f0a9c19d410 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:10.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 >> 172.21.15.100:0/904514726 conn(0x7f0a9c077180 msgr2=0x7f0a9c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:10.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 shutdown_connections 2024-03-12T05:15:10.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:10.761+0000 7f0aa44ae640 1 -- 172.21.15.100:0/904514726 wait complete. 2024-03-12T05:15:10.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292394 13.98 MB/sec warmup 114 sec latency 99.412 ms 2024-03-12T05:15:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:10 smithi156 ceph-mon[36270]: pgmap v95: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-03-12T05:15:11.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:10 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3200032059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:15:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:10 smithi161 ceph-mon[36043]: pgmap v95: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-03-12T05:15:11.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:10 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3200032059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:15:11.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:10 smithi100 ceph-mon[32210]: pgmap v95: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-03-12T05:15:11.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:10 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3200032059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:15:11.769 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:15:11.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:15:11.770 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:15:11.784 INFO:tasks.workunit.client.0.smithi100.stdout: 1 295102 13.99 MB/sec warmup 115 sec latency 11.892 ms 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 -- 172.21.15.100:0/247011834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84101360 msgr2=0x7f5a84101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/247011834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84101360 0x7f5a84101740 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009930 tx=0x7f5a7402bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 -- 172.21.15.100:0/247011834 shutdown_connections 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/247011834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84106430 0x7f5a8410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/247011834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a84101d10 0x7f5a84105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/247011834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84101360 0x7f5a84101740 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 -- 172.21.15.100:0/247011834 >> 172.21.15.100:0/247011834 conn(0x7f5a840fcbe0 msgr2=0x7f5a840ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:11.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 -- 172.21.15.100:0/247011834 shutdown_connections 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.886+0000 7f5a8a73a640 1 -- 172.21.15.100:0/247011834 wait complete. 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 Processor -- start 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 -- start start 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84101360 0x7f5a8419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a84106430 0x7f5a8419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 0x7f5a84197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a84198220 con 0x7f5a84197750 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a84198390 con 0x7f5a84101360 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a8a73a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a84198550 con 0x7f5a84106430 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84101360 0x7f5a8419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.887+0000 7f5a88cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 0x7f5a84197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84101360 0x7f5a8419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34926/0 (socket says 172.21.15.100:34926) 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 0x7f5a84197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33514/0 (socket says 172.21.15.100:33514) 2024-03-12T05:15:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a83fff640 1 -- 172.21.15.100:0/1317241127 learned_addr learned my addr 172.21.15.100:0/1317241127 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a837fe640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a84106430 0x7f5a8419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 -- 172.21.15.100:0/1317241127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a84106430 msgr2=0x7f5a8419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a84106430 0x7f5a8419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 -- 172.21.15.100:0/1317241127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84101360 msgr2=0x7f5a8419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84101360 0x7f5a8419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 -- 172.21.15.100:0/1317241127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a74009560 con 0x7f5a84197750 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a88cd7640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 0x7f5a84197bd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009fd0 tx=0x7f5a7c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a817fa640 1 -- 172.21.15.100:0/1317241127 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00ce20 con 0x7f5a84197750 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a841a44a0 con 0x7f5a84197750 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a817fa640 1 -- 172.21.15.100:0/1317241127 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c005020 con 0x7f5a84197750 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a817fa640 1 -- 172.21.15.100:0/1317241127 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c010690 con 0x7f5a84197750 2024-03-12T05:15:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.888+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a841a4980 con 0x7f5a84197750 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.890+0000 7f5a817fa640 1 -- 172.21.15.100:0/1317241127 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5a7c005500 con 0x7f5a84197750 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 Processor -- start 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 -- start start 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 0x559d7ab0a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab0cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7aa6ff90 con 0x559d7ab01d30 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7aa6d4c0 con 0x559d7ab01380 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7aa68ca0 con 0x559d7ab0a8b0 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a817fa640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5a6c0a1ba0 0x7f5a6c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 0x559d7ab0a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 0x559d7ab0a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33524/0 (socket says 172.21.15.100:33524) 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 -- 172.21.15.100:0/437921444 learned_addr learned my addr 172.21.15.100:0/437921444 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a817fa640 1 -- 172.21.15.100:0/1317241127 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a7c014070 con 0x7f5a84197750 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a88cd7640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5a6c0a1ba0 0x7f5a6c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a837fe640 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab0cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 -- 172.21.15.100:0/437921444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 msgr2=0x559d7ab0cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab0cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 -- 172.21.15.100:0/437921444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 msgr2=0x559d7ab01760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab01760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a83fff640 1 -- 172.21.15.100:0/437921444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d7aa2b0b0 con 0x559d7ab01d30 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.891+0000 7f5a837fe640 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.892+0000 7f5a83fff640 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 0x559d7ab0a2e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009230 tx=0x7f5a7402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ce621689da8815 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.892+0000 7f5a88cd7640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5a6c0a1ba0 0x7f5a6c0a4060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c096f60 tx=0x7f5a7c00c890 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.892+0000 7f5a6a7fc640 1 -- 172.21.15.100:0/437921444 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74004880 con 0x559d7ab01d30 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.892+0000 7f5a6a7fc640 1 -- 172.21.15.100:0/437921444 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a740049e0 con 0x559d7ab01d30 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.892+0000 7f5a6a7fc640 1 -- 172.21.15.100:0/437921444 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a740050e0 con 0x559d7ab01d30 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/437921444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 msgr2=0x559d7ab0a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 0x559d7ab0a2e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009230 tx=0x7f5a7402c660 comp rx=0 tx=0).stop 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/437921444 shutdown_connections 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab0cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01d30 0x559d7ab0a2e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/437921444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/437921444 >> 172.21.15.100:0/437921444 conn(0x559d7aafb840 msgr2=0x559d7aafbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/437921444 shutdown_connections 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/437921444 wait complete. 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 -- start start 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.893+0000 7f5a8d3e2740 1 Processor -- start 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab07770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 0x559d7ab07cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7aa0df40 con 0x559d7ab0a8b0 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7a73c4b0 con 0x559d7ab01380 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7a9ee670 con 0x559d7ab01d30 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 10 client.0 _ll_register_callbacks cb 0x559d7aa5bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 0x559d7ab07cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33534/0 (socket says 172.21.15.100:33534) 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a83fff640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 0x559d7ab07cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42488/0 (socket says 172.21.15.100:42488) 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 -- 172.21.15.100:0/3426859049 learned_addr learned my addr 172.21.15.100:0/3426859049 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab07770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 -- 172.21.15.100:0/3426859049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 msgr2=0x559d7ab07cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 0x559d7ab07cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 -- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 msgr2=0x559d7ab07770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab07770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 -- 172.21.15.100:0/3426859049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8410e810 con 0x559d7ab0a8b0 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a83fff640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 0x559d7ab07cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a837fe640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab081f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800f3f0 tx=0x7f5a78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6581744d95f0f744 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a637fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a637fe640 1 -- 172.21.15.100:0/3426859049 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78004110 con 0x559d7ab0a8b0 2024-03-12T05:15:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a637fe640 1 -- 172.21.15.100:0/3426859049 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a78004270 con 0x559d7ab0a8b0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d7aa42f40 con 0x559d7ab0a8b0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a637fe640 1 -- 172.21.15.100:0/3426859049 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a78004430 con 0x559d7ab0a8b0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 10 client.16176 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.894+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d7aa67320 con 0x559d7ab0a8b0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a637fe640 1 -- 172.21.15.100:0/3426859049 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d7aa42f40 con 0x559d7ab0a8b0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a637fe640 1 -- 172.21.15.100:0/3426859049 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5a78025570 con 0x559d7ab0a8b0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a8d3e2740 10 client.16176 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a8d3e2740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a8d3e2740 20 client.16176 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab1adf0 0x559d7ab1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a8d3e2740 4 client.16176 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d7aa5f310 con 0x559d7ab1adf0 2024-03-12T05:15:11.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.895+0000 7f5a83fff640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab1adf0 0x559d7ab1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.896+0000 7f5a637fe640 10 client.16176 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.896+0000 7f5a83fff640 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab1adf0 0x559d7ab1d210 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f98d118f52df6f1 server_cookie=e79692837e836433 in_seq=0 out_seq=0 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.896+0000 7f5a637fe640 1 -- 172.21.15.100:0/3426859049 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d7aa5f310 con 0x559d7ab1adf0 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.896+0000 7f5a637fe640 10 client.16176 handle_command_reply: tid=0 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.896+0000 7f5a8d3e2740 1 client.16176 shutdown 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 msgr2=0x559d7ab081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab081f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800f3f0 tx=0x7f5a78004730 comp rx=0 tx=0).stop 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 shutdown_connections 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab1adf0 0x559d7ab1d210 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab081f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01d30 0x559d7ab07cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a637fe640 0 client.16176 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/3426859049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01380 0x559d7ab07770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 >> 172.21.15.100:0/3426859049 conn(0x559d7aafb840 msgr2=0x559d7aafd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 shutdown_connections 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/3426859049 wait complete. 2024-03-12T05:15:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.897+0000 7f5a8d3e2740 20 client.16176 trim_cache size 0 max 16384 2024-03-12T05:15:11.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 Processor -- start 2024-03-12T05:15:11.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 -- start start 2024-03-12T05:15:11.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 0x559d7ab09ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab06750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7aafb700 con 0x559d7ab01380 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7ab00770 con 0x559d7ab01d30 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7ab00cd0 con 0x559d7ab0a8b0 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 0x559d7ab09ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 0x559d7ab09ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33538/0 (socket says 172.21.15.100:33538) 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a83fff640 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab06750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a837fe640 1 -- 172.21.15.100:0/2798124441 learned_addr learned my addr 172.21.15.100:0/2798124441 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a837fe640 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a83fff640 1 -- 172.21.15.100:0/2798124441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 msgr2=0x559d7ab049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a83fff640 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a83fff640 1 -- 172.21.15.100:0/2798124441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 msgr2=0x559d7ab09ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.913+0000 7f5a83fff640 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 0x559d7ab09ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.914+0000 7f5a83fff640 1 -- 172.21.15.100:0/2798124441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d7aa03c00 con 0x559d7ab01d30 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.914+0000 7f5a837fe640 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 0x559d7ab09ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.914+0000 7f5a83fff640 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab06750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x559d7aafcac0 tx=0x7f5a74005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef554d5cd542bb84 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.914+0000 7f5a69ffb640 1 -- 172.21.15.100:0/2798124441 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74051b70 con 0x559d7ab01d30 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.914+0000 7f5a69ffb640 1 -- 172.21.15.100:0/2798124441 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a74051280 con 0x559d7ab01d30 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.914+0000 7f5a69ffb640 1 -- 172.21.15.100:0/2798124441 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74005070 con 0x559d7ab01d30 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/2798124441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 msgr2=0x559d7ab06750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab06750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x559d7aafcac0 tx=0x7f5a74005b50 comp rx=0 tx=0).stop 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/2798124441 shutdown_connections 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab0a8b0 0x559d7ab049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab06750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/2798124441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab01380 0x559d7ab09ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/2798124441 >> 172.21.15.100:0/2798124441 conn(0x559d7aafb840 msgr2=0x559d7ab06040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/2798124441 shutdown_connections 2024-03-12T05:15:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/2798124441 wait complete. 2024-03-12T05:15:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 -- start start 2024-03-12T05:15:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.915+0000 7f5a8d3e2740 1 Processor -- start 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01380 0x559d7ab11d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7ab10dc0 con 0x559d7ab0a8b0 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7ab10f30 con 0x559d7ab01d30 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7ab71330 con 0x559d7ab01380 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 10 client.0 _ll_register_callbacks cb 0x7f5a78005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33550/0 (socket says 172.21.15.100:33550) 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 -- 172.21.15.100:0/1538324950 learned_addr learned my addr 172.21.15.100:0/1538324950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01380 0x559d7ab11d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 -- 172.21.15.100:0/1538324950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01380 msgr2=0x559d7ab11d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01380 0x559d7ab11d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 -- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 msgr2=0x559d7ab100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 -- 172.21.15.100:0/1538324950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d7aa67320 con 0x559d7ab0a8b0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a83fff640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a837fe640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab106f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5a780097a0 tx=0x7f5a78005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700b7dd7c4b9a39d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a637fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a637fe640 1 -- 172.21.15.100:0/1538324950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78004110 con 0x559d7ab0a8b0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a637fe640 1 -- 172.21.15.100:0/1538324950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a78000e40 con 0x559d7ab0a8b0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d7aa8e900 con 0x559d7ab0a8b0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a637fe640 1 -- 172.21.15.100:0/1538324950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a78004270 con 0x559d7ab0a8b0 2024-03-12T05:15:11.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 10 client.16182 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.916+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d7aa2b0b0 con 0x559d7ab0a8b0 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a637fe640 1 -- 172.21.15.100:0/1538324950 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d7aa8e900 con 0x559d7ab0a8b0 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a637fe640 1 -- 172.21.15.100:0/1538324950 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5a7801a030 con 0x559d7ab0a8b0 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a8d3e2740 10 client.16182 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a8d3e2740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a8d3e2740 20 client.16182 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab72b60 0x559d7ab74f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a8d3e2740 4 client.16182 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559d7a768860 con 0x559d7ab72b60 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a83fff640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab72b60 0x559d7ab74f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a637fe640 10 client.16182 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.917+0000 7f5a83fff640 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab72b60 0x559d7ab74f80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f14f80d1727720 server_cookie=4802820719447b7c in_seq=0 out_seq=0 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a637fe640 1 -- 172.21.15.100:0/1538324950 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559d7a768860 con 0x559d7ab72b60 2024-03-12T05:15:11.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a637fe640 10 client.16182 handle_command_reply: tid=0 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a8d3e2740 1 client.16182 shutdown 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 msgr2=0x559d7ab106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab106f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5a780097a0 tx=0x7f5a78005850 comp rx=0 tx=0).stop 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 shutdown_connections 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559d7ab72b60 0x559d7ab74f80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.918+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d7ab0a8b0 0x559d7ab106f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d7ab01d30 0x559d7ab100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8d3e2740 1 --2- 172.21.15.100:0/1538324950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d7ab01380 0x559d7ab11d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 >> 172.21.15.100:0/1538324950 conn(0x559d7aafb840 msgr2=0x559d7ab068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a637fe640 0 client.16182 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 shutdown_connections 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8d3e2740 1 -- 172.21.15.100:0/1538324950 wait complete. 2024-03-12T05:15:11.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8d3e2740 20 client.16182 trim_cache size 0 max 16384 2024-03-12T05:15:11.922 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:15:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:15:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "ce13bfb0-4149-4b84-98c4-441249caaa14", 2024-03-12T05:15:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:15:11.922 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5a6c0a1ba0 msgr2=0x7f5a6c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5a6c0a1ba0 0x7f5a6c0a4060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c096f60 tx=0x7f5a7c00c890 comp rx=0 tx=0).stop 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 msgr2=0x7f5a84197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 0x7f5a84197bd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009fd0 tx=0x7f5a7c00ef10 comp rx=0 tx=0).stop 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 shutdown_connections 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5a6c0a1ba0 0x7f5a6c0a4060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84197750 0x7f5a84197bd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a84106430 0x7f5a8419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 --2- 172.21.15.100:0/1317241127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a84101360 0x7f5a8419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:11.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 >> 172.21.15.100:0/1317241127 conn(0x7f5a840fcbe0 msgr2=0x7f5a840fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:11.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 shutdown_connections 2024-03-12T05:15:11.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:11.919+0000 7f5a8a73a640 1 -- 172.21.15.100:0/1317241127 wait complete. 2024-03-12T05:15:11.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:15:11.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:15:12.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 -- 172.21.15.100:0/1204772915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 msgr2=0x7f535c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 --2- 172.21.15.100:0/1204772915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c100650 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5344009930 tx=0x7f534402bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:12.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 -- 172.21.15.100:0/1204772915 shutdown_connections 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 --2- 172.21.15.100:0/1204772915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f535c105340 0x7f535c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 --2- 172.21.15.100:0/1204772915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c100c20 0x7f535c104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 --2- 172.21.15.100:0/1204772915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c100650 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 -- 172.21.15.100:0/1204772915 >> 172.21.15.100:0/1204772915 conn(0x7f535c077230 msgr2=0x7f535c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 -- 172.21.15.100:0/1204772915 shutdown_connections 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.077+0000 7f53610d0640 1 -- 172.21.15.100:0/1204772915 wait complete. 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 Processor -- start 2024-03-12T05:15:12.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 -- start start 2024-03-12T05:15:12.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c105340 0x7f535c196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f535c196e70 0x7f535c1a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1a5fb0 con 0x7f535c100270 2024-03-12T05:15:12.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1a60f0 con 0x7f535c196e70 2024-03-12T05:15:12.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.078+0000 7f53610d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1a6230 con 0x7f535c105340 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33566/0 (socket says 172.21.15.100:33566) 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 -- 172.21.15.100:0/2021504868 learned_addr learned my addr 172.21.15.100:0/2021504868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535b59e640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f535c196e70 0x7f535c1a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535a59c640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c105340 0x7f535c196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 -- 172.21.15.100:0/2021504868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c105340 msgr2=0x7f535c196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c105340 0x7f535c196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 -- 172.21.15.100:0/2021504868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f535c196e70 msgr2=0x7f535c1a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f535c196e70 0x7f535c1a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 -- 172.21.15.100:0/2021504868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344009560 con 0x7f535c100270 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535a59c640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c105340 0x7f535c196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f535ad9d640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c1963f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5344009430 tx=0x7f534402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:12.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f5353fff640 1 -- 172.21.15.100:0/2021504868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344004350 con 0x7f535c100270 2024-03-12T05:15:12.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f5353fff640 1 -- 172.21.15.100:0/2021504868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53440044b0 con 0x7f535c100270 2024-03-12T05:15:12.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c1a64b0 con 0x7f535c100270 2024-03-12T05:15:12.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f5353fff640 1 -- 172.21.15.100:0/2021504868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344003da0 con 0x7f535c100270 2024-03-12T05:15:12.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.079+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c1a69c0 con 0x7f535c100270 2024-03-12T05:15:12.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5353fff640 1 -- 172.21.15.100:0/2021504868 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5344004df0 con 0x7f535c100270 2024-03-12T05:15:12.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 Processor -- start 2024-03-12T05:15:12.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 -- start start 2024-03-12T05:15:12.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146632a280 con 0x5614663c47b0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614663277b0 con 0x5614663bbbd0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5363d76740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466322f90 con 0x5614663bb2b0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f5353fff640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f53480a1ba0 0x7f53480a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f535a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f535a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33568/0 (socket says 172.21.15.100:33568) 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f535a59c640 1 -- 172.21.15.100:0/3695795941 learned_addr learned my addr 172.21.15.100:0/3695795941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.082+0000 7f535b59e640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f53480a1ba0 0x7f53480a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535b59e640 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f5353fff640 1 -- 172.21.15.100:0/2021504868 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f53440ef810 con 0x7f535c100270 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535a59c640 1 -- 172.21.15.100:0/3695795941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 msgr2=0x5614663bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535a59c640 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535a59c640 1 -- 172.21.15.100:0/3695795941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 msgr2=0x5614663c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535a59c640 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535a59c640 1 -- 172.21.15.100:0/3695795941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614662e1570 con 0x5614663c47b0 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535a59c640 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c6ba0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f534c00bce0 tx=0x7f534c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c671828b76571a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f535b59e640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f53480a1ba0 0x7f53480a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5354005fd0 tx=0x7f5354011b40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f53517fa640 1 -- 172.21.15.100:0/3695795941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c01a460 con 0x5614663c47b0 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f53517fa640 1 -- 172.21.15.100:0/3695795941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f534c01aaa0 con 0x5614663c47b0 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.083+0000 7f53517fa640 1 -- 172.21.15.100:0/3695795941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c011980 con 0x5614663c47b0 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 -- 172.21.15.100:0/3695795941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 msgr2=0x5614663c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c6ba0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f534c00bce0 tx=0x7f534c00f730 comp rx=0 tx=0).stop 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 -- 172.21.15.100:0/3695795941 shutdown_connections 2024-03-12T05:15:12.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c6ba0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 --2- 172.21.15.100:0/3695795941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 -- 172.21.15.100:0/3695795941 >> 172.21.15.100:0/3695795941 conn(0x5614663b5990 msgr2=0x5614663b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 -- 172.21.15.100:0/3695795941 shutdown_connections 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 -- 172.21.15.100:0/3695795941 wait complete. 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 -- start start 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.084+0000 7f5363d76740 1 Processor -- start 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bb2b0 0x5614663c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bbbd0 0x5614663c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561465ff6430 con 0x5614663c47b0 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561465fb2e00 con 0x5614663bb2b0 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614662a7fd0 con 0x5614663bbbd0 2024-03-12T05:15:12.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 10 client.0 _ll_register_callbacks cb 0x5614662fdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bb2b0 0x5614663c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33578/0 (socket says 172.21.15.100:33578) 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 -- 172.21.15.100:0/3600719893 learned_addr learned my addr 172.21.15.100:0/3600719893 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535a59c640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bbbd0 0x5614663c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 -- 172.21.15.100:0/3600719893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bbbd0 msgr2=0x5614663c1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bbbd0 0x5614663c1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 -- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bb2b0 msgr2=0x5614663c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bb2b0 0x5614663c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 -- 172.21.15.100:0/3600719893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c00b940 con 0x5614663c47b0 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bb2b0 0x5614663c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:12.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f535b59e640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c1cd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5354011550 tx=0x7f5354009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38040d18441bcf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f53427fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f53427fc640 1 -- 172.21.15.100:0/3600719893 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354004240 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f53427fc640 1 -- 172.21.15.100:0/3600719893 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f53540043a0 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.085+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614660227e0 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f53427fc640 1 -- 172.21.15.100:0/3600719893 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5354011020 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 10 client.16194 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561466321610 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f53427fc640 1 -- 172.21.15.100:0/3600719893 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5614660227e0 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f53427fc640 1 -- 172.21.15.100:0/3600719893 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5354010b50 con 0x5614663c47b0 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 10 client.16194 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 20 client.16194 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5614663d49d0 0x5614663d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 4 client.16194 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56146605f720 con 0x5614663d49d0 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5614663d49d0 0x5614663d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f53427fc640 10 client.16194 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.086+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5614663d49d0 0x5614663d6df0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b615bae7c24936 server_cookie=c4567bf502718db4 in_seq=0 out_seq=0 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.087+0000 7f53427fc640 1 -- 172.21.15.100:0/3600719893 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56146605f720 con 0x5614663d49d0 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.087+0000 7f53427fc640 10 client.16194 handle_command_reply: tid=0 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.087+0000 7f5363d76740 1 client.16194 shutdown 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.087+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 msgr2=0x5614663c1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.087+0000 7f5363d76740 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c1cd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5354011550 tx=0x7f5354009500 comp rx=0 tx=0).stop 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 shutdown_connections 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5614663d49d0 0x5614663d6df0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c1cd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bbbd0 0x5614663c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f53427fc640 0 client.16194 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:12.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 --2- 172.21.15.100:0/3600719893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bb2b0 0x5614663c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 >> 172.21.15.100:0/3600719893 conn(0x5614663b5990 msgr2=0x5614663b7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:12.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 shutdown_connections 2024-03-12T05:15:12.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 1 -- 172.21.15.100:0/3600719893 wait complete. 2024-03-12T05:15:12.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.088+0000 7f5363d76740 20 client.16194 trim_cache size 0 max 16384 2024-03-12T05:15:12.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 Processor -- start 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 -- start start 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466022810 con 0x5614663c47b0 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614663b1120 con 0x5614663bbbd0 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f5363d76740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614663d0570 con 0x5614663bb2b0 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f535b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663c3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42560/0 (socket says 172.21.15.100:42560) 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f535b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33582/0 (socket says 172.21.15.100:33582) 2024-03-12T05:15:12.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f535ad9d640 1 -- 172.21.15.100:0/3377287458 learned_addr learned my addr 172.21.15.100:0/3377287458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.104+0000 7f535a59c640 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535ad9d640 1 -- 172.21.15.100:0/3377287458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 msgr2=0x5614663c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535ad9d640 1 -- 172.21.15.100:0/3377287458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 msgr2=0x5614663bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535ad9d640 1 -- 172.21.15.100:0/3377287458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614662bdac0 con 0x5614663bb2b0 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535a59c640 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535b59e640 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663c3cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53440bc240 tx=0x7f53440a7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e59981f3ed4135 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f5350ff9640 1 -- 172.21.15.100:0/3377287458 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53440bc540 con 0x5614663bb2b0 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f5350ff9640 1 -- 172.21.15.100:0/3377287458 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53440c0690 con 0x5614663bb2b0 2024-03-12T05:15:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.105+0000 7f5350ff9640 1 -- 172.21.15.100:0/3377287458 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53440a70b0 con 0x5614663bb2b0 2024-03-12T05:15:12.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- 172.21.15.100:0/3377287458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 msgr2=0x5614663c3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663c3cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53440bc240 tx=0x7f53440a7040 comp rx=0 tx=0).stop 2024-03-12T05:15:12.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- 172.21.15.100:0/3377287458 shutdown_connections 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- 172.21.15.100:0/3377287458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663c3cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- 172.21.15.100:0/3377287458 >> 172.21.15.100:0/3377287458 conn(0x5614663b5990 msgr2=0x5614663bc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- 172.21.15.100:0/3377287458 shutdown_connections 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- 172.21.15.100:0/3377287458 wait complete. 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- start start 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 Processor -- start 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614663c86d0 con 0x5614663c47b0 2024-03-12T05:15:12.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614663c8840 con 0x5614663bbbd0 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.106+0000 7f5363d76740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614663c8a00 con 0x5614663bb2b0 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f5363d76740 10 client.0 _ll_register_callbacks cb 0x7f534c00b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33584/0 (socket says 172.21.15.100:33584) 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34986/0 (socket says 172.21.15.100:34986) 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42574/0 (socket says 172.21.15.100:42574) 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535a59c640 1 -- 172.21.15.100:0/3345988901 learned_addr learned my addr 172.21.15.100:0/3345988901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 -- 172.21.15.100:0/3345988901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 msgr2=0x5614663cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 -- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 msgr2=0x5614663c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:12.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 -- 172.21.15.100:0/3345988901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614662e1570 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f535b59e640 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c8190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5354005cc0 tx=0x7f535401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91823ccb34c3d47a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f53427fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f53427fc640 1 -- 172.21.15.100:0/3345988901 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354003d40 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f53427fc640 1 -- 172.21.15.100:0/3345988901 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5354009590 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f534c01aaa0 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f53427fc640 1 -- 172.21.15.100:0/3345988901 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f535401f460 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f5363d76740 10 client.16200 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.107+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561466321610 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f53427fc640 1 -- 172.21.15.100:0/3345988901 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f534c01aaa0 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f53427fc640 1 -- 172.21.15.100:0/3345988901 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5354010430 con 0x5614663c47b0 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 10 client.16200 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:12.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 20 client.16200 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146642c3a0 0x56146642e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 4 client.16200 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614660ae890 con 0x56146642c3a0 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146642c3a0 0x56146642e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f53427fc640 10 client.16200 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f535ad9d640 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146642c3a0 0x56146642e7c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63590125c352337d server_cookie=3caba8ab60ef459b in_seq=0 out_seq=0 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f53427fc640 1 -- 172.21.15.100:0/3345988901 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5614660ae890 con 0x56146642c3a0 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f53427fc640 10 client.16200 handle_command_reply: tid=0 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.108+0000 7f5363d76740 1 client.16200 shutdown 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 msgr2=0x5614663c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c8190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5354005cc0 tx=0x7f535401fac0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 shutdown_connections 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146642c3a0 0x56146642e7c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614663c47b0 0x5614663c8190 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614663bbbd0 0x5614663c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 --2- 172.21.15.100:0/3345988901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614663bb2b0 0x5614663cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53427fc640 0 client.16200 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 >> 172.21.15.100:0/3345988901 conn(0x5614663b5990 msgr2=0x5614663c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 shutdown_connections 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 1 -- 172.21.15.100:0/3345988901 wait complete. 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f5363d76740 20 client.16200 trim_cache size 0 max 16384 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:15:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "ce13bfb0-4149-4b84-98c4-441249caaa14": { 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "ce13bfb0-4149-4b84-98c4-441249caaa14", 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:15:12.113 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f53480a1ba0 msgr2=0x7f53480a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f53480a1ba0 0x7f53480a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5354005fd0 tx=0x7f5354011b40 comp rx=0 tx=0).stop 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 msgr2=0x7f535c1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c1963f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5344009430 tx=0x7f534402c6c0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 shutdown_connections 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f53480a1ba0 0x7f53480a4060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f535c196e70 0x7f535c1a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c105340 0x7f535c196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 --2- 172.21.15.100:0/2021504868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f535c100270 0x7f535c1963f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:12.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 >> 172.21.15.100:0/2021504868 conn(0x7f535c077230 msgr2=0x7f535c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:12.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 shutdown_connections 2024-03-12T05:15:12.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:12.109+0000 7f53610d0640 1 -- 172.21.15.100:0/2021504868 wait complete. 2024-03-12T05:15:12.116 INFO:tasks.cephfs.filesystem:scrub status for tag:ce13bfb0-4149-4b84-98c4-441249caaa14 - {'path': '/', 'tag': 'ce13bfb0-4149-4b84-98c4-441249caaa14', 'options': 'recursive,force'} 2024-03-12T05:15:12.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 298197 14.03 MB/sec warmup 116 sec latency 15.819 ms 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: pgmap v96: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 385 op/s 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:12 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: pgmap v96: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 385 op/s 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:12 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: pgmap v96: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 385 op/s 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:12 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:13.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 300442 13.97 MB/sec warmup 117 sec latency 35.337 ms 2024-03-12T05:15:14.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:13 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:15:14.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:13 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:15:14.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:13 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:15:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 302774 13.97 MB/sec warmup 118 sec latency 7.154 ms 2024-03-12T05:15:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:14 smithi156 ceph-mon[36270]: pgmap v97: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:15:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:15:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:15:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:14 smithi161 ceph-mon[36043]: pgmap v97: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:15:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:15:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:15:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:14 smithi100 ceph-mon[32210]: pgmap v97: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:15:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:15:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:15:15.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 305427 14.00 MB/sec warmup 119 sec latency 105.041 ms 2024-03-12T05:15:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:16 smithi156 ceph-mon[36270]: pgmap v98: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:16 smithi161 ceph-mon[36043]: pgmap v98: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:17.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:16 smithi100 ceph-mon[32210]: pgmap v98: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:17.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 310421 11.33 MB/sec execute 1 sec latency 48.449 ms 2024-03-12T05:15:18.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 312956 12.79 MB/sec execute 2 sec latency 14.230 ms 2024-03-12T05:15:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:18 smithi156 ceph-mon[36270]: pgmap v99: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 366 op/s 2024-03-12T05:15:19.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:18 smithi161 ceph-mon[36043]: pgmap v99: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 366 op/s 2024-03-12T05:15:19.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:18 smithi100 ceph-mon[32210]: pgmap v99: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 366 op/s 2024-03-12T05:15:19.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 315980 14.63 MB/sec execute 3 sec latency 17.022 ms 2024-03-12T05:15:20.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:19 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:15:20.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:19 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:15:20.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:19 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:15:20.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 318040 12.77 MB/sec execute 4 sec latency 120.463 ms 2024-03-12T05:15:21.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:20 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:15:21.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:20 smithi156 ceph-mon[36270]: pgmap v100: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-03-12T05:15:21.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:20 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:15:21.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:20 smithi161 ceph-mon[36043]: pgmap v100: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-03-12T05:15:21.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:20 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:15:21.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:20 smithi100 ceph-mon[32210]: pgmap v100: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-03-12T05:15:21.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 320173 12.82 MB/sec execute 5 sec latency 15.023 ms 2024-03-12T05:15:22.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:21 smithi161 ceph-mon[36043]: pgmap v101: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 365 op/s 2024-03-12T05:15:22.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:21 smithi100 ceph-mon[32210]: pgmap v101: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 365 op/s 2024-03-12T05:15:22.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:21 smithi156 ceph-mon[36270]: pgmap v101: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 365 op/s 2024-03-12T05:15:22.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 323161 13.74 MB/sec execute 6 sec latency 13.453 ms 2024-03-12T05:15:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 326001 13.68 MB/sec execute 7 sec latency 11.641 ms 2024-03-12T05:15:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 328233 13.37 MB/sec execute 8 sec latency 13.937 ms 2024-03-12T05:15:25.156 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:24 smithi156 ceph-mon[36270]: pgmap v102: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-12T05:15:25.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:24 smithi161 ceph-mon[36043]: pgmap v102: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-12T05:15:25.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:24 smithi100 ceph-mon[32210]: pgmap v102: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-12T05:15:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 330410 13.41 MB/sec execute 9 sec latency 75.161 ms 2024-03-12T05:15:26.785 INFO:tasks.workunit.client.0.smithi100.stdout: 1 333503 13.83 MB/sec execute 10 sec latency 7.449 ms 2024-03-12T05:15:27.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:26 smithi156 ceph-mon[36270]: pgmap v103: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-12T05:15:27.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:26 smithi161 ceph-mon[36043]: pgmap v103: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-12T05:15:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:26 smithi100 ceph-mon[32210]: pgmap v103: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-12T05:15:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 335724 13.33 MB/sec execute 11 sec latency 13.277 ms 2024-03-12T05:15:28.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 337885 13.30 MB/sec execute 12 sec latency 11.682 ms 2024-03-12T05:15:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:28 smithi156 ceph-mon[36270]: pgmap v104: 97 pgs: 97 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 365 op/s 2024-03-12T05:15:29.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:28 smithi161 ceph-mon[36043]: pgmap v104: 97 pgs: 97 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 365 op/s 2024-03-12T05:15:29.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:28 smithi100 ceph-mon[32210]: pgmap v104: 97 pgs: 97 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 365 op/s 2024-03-12T05:15:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 340770 13.67 MB/sec execute 13 sec latency 53.184 ms 2024-03-12T05:15:30.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 342920 13.22 MB/sec execute 14 sec latency 54.274 ms 2024-03-12T05:15:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:30 smithi156 ceph-mon[36270]: pgmap v105: 97 pgs: 97 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 238 op/s 2024-03-12T05:15:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:30 smithi161 ceph-mon[36043]: pgmap v105: 97 pgs: 97 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 238 op/s 2024-03-12T05:15:31.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:30 smithi100 ceph-mon[32210]: pgmap v105: 97 pgs: 97 active+clean; 178 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 238 op/s 2024-03-12T05:15:31.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345288 13.28 MB/sec execute 15 sec latency 8.578 ms 2024-03-12T05:15:32.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 348116 13.52 MB/sec execute 16 sec latency 18.329 ms 2024-03-12T05:15:33.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:32 smithi156 ceph-mon[36270]: pgmap v106: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-03-12T05:15:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:32 smithi161 ceph-mon[36043]: pgmap v106: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-03-12T05:15:33.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:32 smithi100 ceph-mon[32210]: pgmap v106: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-03-12T05:15:33.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 351272 13.77 MB/sec execute 17 sec latency 6.707 ms 2024-03-12T05:15:34.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 353626 13.50 MB/sec execute 18 sec latency 16.147 ms 2024-03-12T05:15:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:34 smithi156 ceph-mon[36270]: pgmap v107: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-03-12T05:15:35.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:34 smithi161 ceph-mon[36043]: pgmap v107: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-03-12T05:15:35.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:34 smithi100 ceph-mon[32210]: pgmap v107: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-03-12T05:15:35.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355491 13.42 MB/sec execute 19 sec latency 73.566 ms 2024-03-12T05:15:36.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 358549 13.67 MB/sec execute 20 sec latency 7.284 ms 2024-03-12T05:15:37.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:36 smithi156 ceph-mon[36270]: pgmap v108: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-03-12T05:15:37.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:36 smithi161 ceph-mon[36043]: pgmap v108: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-03-12T05:15:37.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:36 smithi100 ceph-mon[32210]: pgmap v108: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-03-12T05:15:37.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 361148 13.59 MB/sec execute 21 sec latency 15.944 ms 2024-03-12T05:15:38.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 363077 13.41 MB/sec execute 22 sec latency 7.595 ms 2024-03-12T05:15:39.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:38 smithi156 ceph-mon[36270]: pgmap v109: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-03-12T05:15:39.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:38 smithi161 ceph-mon[36043]: pgmap v109: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-03-12T05:15:39.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:38 smithi100 ceph-mon[32210]: pgmap v109: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-03-12T05:15:39.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 365918 13.56 MB/sec execute 23 sec latency 15.459 ms 2024-03-12T05:15:40.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 368550 13.55 MB/sec execute 24 sec latency 97.070 ms 2024-03-12T05:15:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:40 smithi156 ceph-mon[36270]: pgmap v110: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-03-12T05:15:41.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:40 smithi161 ceph-mon[36043]: pgmap v110: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-03-12T05:15:41.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:40 smithi100 ceph-mon[32210]: pgmap v110: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-03-12T05:15:41.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 371084 13.47 MB/sec execute 25 sec latency 7.826 ms 2024-03-12T05:15:42.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:15:42.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:15:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 -- 172.21.15.100:0/4061466049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac105340 msgr2=0x7ffaac10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 --2- 172.21.15.100:0/4061466049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac105340 0x7ffaac10cf10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009cd0 tx=0x7ffaa402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 -- 172.21.15.100:0/4061466049 shutdown_connections 2024-03-12T05:15:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 --2- 172.21.15.100:0/4061466049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac105340 0x7ffaac10cf10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 --2- 172.21.15.100:0/4061466049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100c20 0x7ffaac104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 --2- 172.21.15.100:0/4061466049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaac100270 0x7ffaac100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 -- 172.21.15.100:0/4061466049 >> 172.21.15.100:0/4061466049 conn(0x7ffaac077230 msgr2=0x7ffaac077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 -- 172.21.15.100:0/4061466049 shutdown_connections 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.229+0000 7ffab45a1640 1 -- 172.21.15.100:0/4061466049 wait complete. 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 Processor -- start 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 -- start start 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100270 0x7ffaac196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaac105340 0x7ffaac196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 0x7ffaac19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac19b8d0 con 0x7ffaac19aee0 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac19ba40 con 0x7ffaac100270 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab45a1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac19bbb0 con 0x7ffaac105340 2024-03-12T05:15:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 0x7ffaac19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab233d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100270 0x7ffaac196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 0x7ffaac19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33446/0 (socket says 172.21.15.100:33446) 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 -- 172.21.15.100:0/525644110 learned_addr learned my addr 172.21.15.100:0/525644110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaac105340 0x7ffaac196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 -- 172.21.15.100:0/525644110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaac105340 msgr2=0x7ffaac196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaac105340 0x7ffaac196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 -- 172.21.15.100:0/525644110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100270 msgr2=0x7ffaac196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100270 0x7ffaac196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab2b3e640 1 -- 172.21.15.100:0/525644110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4009900 con 0x7ffaac19aee0 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.230+0000 7ffab233d640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100270 0x7ffaac196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.231+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 0x7ffaac19b390 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005e30 tx=0x7ffaa40325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.231+0000 7ffaa37fe640 1 -- 172.21.15.100:0/525644110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4034e20 con 0x7ffaac19aee0 2024-03-12T05:15:42.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.231+0000 7ffaa37fe640 1 -- 172.21.15.100:0/525644110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4003e50 con 0x7ffaac19aee0 2024-03-12T05:15:42.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.231+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac1a3190 con 0x7ffaac19aee0 2024-03-12T05:15:42.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.231+0000 7ffaa37fe640 1 -- 172.21.15.100:0/525644110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4004750 con 0x7ffaac19aee0 2024-03-12T05:15:42.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.231+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac1a35b0 con 0x7ffaac19aee0 2024-03-12T05:15:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.233+0000 7ffaa37fe640 1 -- 172.21.15.100:0/525644110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ffaa400a040 con 0x7ffaac19aee0 2024-03-12T05:15:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.233+0000 7ffab7178740 1 Processor -- start 2024-03-12T05:15:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 -- start start 2024-03-12T05:15:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffaa37fe640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffa980a1ba0 0x7ffa980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 0x5588acc2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc2a900 0x5588acc35480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc37db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acb99280 con 0x5588acc2a900 2024-03-12T05:15:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acb967b0 con 0x5588acc29fe0 2024-03-12T05:15:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab7178740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acb91f90 con 0x5588acc359c0 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffa980a1ba0 0x7ffa980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffaa37fe640 1 -- 172.21.15.100:0/525644110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffaa40f0010 con 0x7ffaac19aee0 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 0x5588acc2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab2b3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc37db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 0x5588acc2a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:55818/0 (socket says 172.21.15.100:55818) 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 -- 172.21.15.100:0/4180845396 learned_addr learned my addr 172.21.15.100:0/4180845396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc2a900 0x5588acc35480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 -- 172.21.15.100:0/4180845396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 msgr2=0x5588acc37db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc37db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 -- 172.21.15.100:0/4180845396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc2a900 msgr2=0x5588acc35480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc2a900 0x5588acc35480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab233d640 1 -- 172.21.15.100:0/4180845396 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588acb50570 con 0x5588acc29fe0 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc37db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.234+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffa980a1ba0 0x7ffa980a4060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0fd7a0 tx=0x7ffa9c009ae0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab233d640 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 0x5588acc2a3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80115f0 tx=0x7ffaa8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714e87326fe3a891 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffaa0bfe640 1 -- 172.21.15.100:0/4180845396 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800f5c0 con 0x5588acc29fe0 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffaa0bfe640 1 -- 172.21.15.100:0/4180845396 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa800fc00 con 0x5588acc29fe0 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffaa0bfe640 1 -- 172.21.15.100:0/4180845396 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8019c30 con 0x5588acc29fe0 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 -- 172.21.15.100:0/4180845396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 msgr2=0x5588acc2a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 0x5588acc2a3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80115f0 tx=0x7ffaa8011ab0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 -- 172.21.15.100:0/4180845396 shutdown_connections 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc37db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc2a900 0x5588acc35480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 --2- 172.21.15.100:0/4180845396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc29fe0 0x5588acc2a3c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.235+0000 7ffab7178740 1 -- 172.21.15.100:0/4180845396 >> 172.21.15.100:0/4180845396 conn(0x5588acc24990 msgr2=0x5588acc24d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 -- 172.21.15.100:0/4180845396 shutdown_connections 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 -- 172.21.15.100:0/4180845396 wait complete. 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 -- start start 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 Processor -- start 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc35440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc32f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ac865430 con 0x5588acc29fe0 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ac821e00 con 0x5588acc359c0 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acb16fd0 con 0x5588acc2a900 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab7178740 10 client.0 _ll_register_callbacks cb 0x5588acb6cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab233d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc35440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab233d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc35440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33470/0 (socket says 172.21.15.100:33470) 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab233d640 1 -- 172.21.15.100:0/339563227 learned_addr learned my addr 172.21.15.100:0/339563227 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc32f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.236+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab233d640 1 -- 172.21.15.100:0/339563227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 msgr2=0x5588acc327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab233d640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab233d640 1 -- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 msgr2=0x5588acc32f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab233d640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc32f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab233d640 1 -- 172.21.15.100:0/339563227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa8009d40 con 0x5588acc29fe0 2024-03-12T05:15:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc32f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab233d640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc35440 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800ffa0 tx=0x7ffaa800ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48a024f4f161fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffa95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffa95ffb640 1 -- 172.21.15.100:0/339563227 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8004400 con 0x5588acc29fe0 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffa95ffb640 1 -- 172.21.15.100:0/339563227 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffaa800f5c0 con 0x5588acc29fe0 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588ac8917e0 con 0x5588acc29fe0 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffa95ffb640 1 -- 172.21.15.100:0/339563227 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaa800f780 con 0x5588acc29fe0 2024-03-12T05:15:42.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab7178740 10 client.16212 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5588acb90610 con 0x5588acc29fe0 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffa95ffb640 1 -- 172.21.15.100:0/339563227 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5588ac8917e0 con 0x5588acc29fe0 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffa95ffb640 1 -- 172.21.15.100:0/339563227 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ffaa8003ad0 con 0x5588acc29fe0 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab7178740 10 client.16212 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab7178740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.237+0000 7ffab7178740 20 client.16212 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.238+0000 7ffab7178740 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc45c00 0x5588acc48020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.238+0000 7ffab7178740 4 client.16212 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:42.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.238+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588ac8ce720 con 0x5588acc45c00 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.238+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc45c00 0x5588acc48020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.238+0000 7ffa95ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.238+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc45c00 0x5588acc48020 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac15337097a935d server_cookie=73fd2e0f95c2fe33 in_seq=0 out_seq=0 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffa95ffb640 1 -- 172.21.15.100:0/339563227 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588ac8ce720 con 0x5588acc45c00 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffa95ffb640 10 client.16212 handle_command_reply: tid=0 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 client.16212 shutdown 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 msgr2=0x5588acc35440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc35440 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800ffa0 tx=0x7ffaa800ffd0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 shutdown_connections 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc45c00 0x5588acc48020 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc32f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffa95ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 --2- 172.21.15.100:0/339563227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc35440 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 >> 172.21.15.100:0/339563227 conn(0x5588acc24990 msgr2=0x5588acc26730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 shutdown_connections 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 1 -- 172.21.15.100:0/339563227 wait complete. 2024-03-12T05:15:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.239+0000 7ffab7178740 20 client.16212 trim_cache size 0 max 16384 2024-03-12T05:15:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 Processor -- start 2024-03-12T05:15:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 -- start start 2024-03-12T05:15:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc32770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc2a900 0x5588acc34f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc35440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ac8917e0 con 0x5588acc29fe0 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acc200f0 con 0x5588acc2a900 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab7178740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acc1ee20 con 0x5588acc359c0 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab233d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc32770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab233d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc32770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33474/0 (socket says 172.21.15.100:33474) 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab233d640 1 -- 172.21.15.100:0/1690745844 learned_addr learned my addr 172.21.15.100:0/1690745844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc2a900 0x5588acc34f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.254+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc35440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab233d640 1 -- 172.21.15.100:0/1690745844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 msgr2=0x5588acc35440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab233d640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc35440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab233d640 1 -- 172.21.15.100:0/1690745844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc2a900 msgr2=0x5588acc34f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab233d640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc2a900 0x5588acc34f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab233d640 1 -- 172.21.15.100:0/1690745844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588acb2cac0 con 0x5588acc29fe0 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc2a900 0x5588acc34f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc35440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab233d640 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc32770 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8001eb0 tx=0x7ffaa800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c49a628fd52ba2c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffa97fff640 1 -- 172.21.15.100:0/1690745844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800f7c0 con 0x5588acc29fe0 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffa97fff640 1 -- 172.21.15.100:0/1690745844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8005550 con 0x5588acc29fe0 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffa97fff640 1 -- 172.21.15.100:0/1690745844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8019c10 con 0x5588acc29fe0 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 -- 172.21.15.100:0/1690745844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 msgr2=0x5588acc32770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc32770 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8001eb0 tx=0x7ffaa800f3c0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 -- 172.21.15.100:0/1690745844 shutdown_connections 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc359c0 0x5588acc35440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc2a900 0x5588acc34f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 --2- 172.21.15.100:0/1690745844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc32770 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.255+0000 7ffab7178740 1 -- 172.21.15.100:0/1690745844 >> 172.21.15.100:0/1690745844 conn(0x5588acc24990 msgr2=0x5588acc32060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 -- 172.21.15.100:0/1690745844 shutdown_connections 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 -- 172.21.15.100:0/1690745844 wait complete. 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 -- start start 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 Processor -- start 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc3cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc39190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acc39fa0 con 0x5588acc29fe0 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acc9c100 con 0x5588acc359c0 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588acc9c240 con 0x5588acc2a900 2024-03-12T05:15:42.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab7178740 10 client.0 _ll_register_callbacks cb 0x7ffaa8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc3cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc3cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33488/0 (socket says 172.21.15.100:33488) 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 -- 172.21.15.100:0/2743159110 learned_addr learned my addr 172.21.15.100:0/2743159110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab2b3e640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc39190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 -- 172.21.15.100:0/2743159110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 msgr2=0x5588acc39190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc39190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 -- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 msgr2=0x5588acc398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab233d640 1 -- 172.21.15.100:0/2743159110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588acb90610 con 0x5588acc29fe0 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.256+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc39190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab233d640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc3cdc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800f470 tx=0x7ffaa8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13572685afe7c894 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffa95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffa95ffb640 1 -- 172.21.15.100:0/2743159110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800f5c0 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffa95ffb640 1 -- 172.21.15.100:0/2743159110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffaa8000e40 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaa800fc00 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffa95ffb640 1 -- 172.21.15.100:0/2743159110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaa800f720 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 10 client.16218 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5588acb526a0 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffa95ffb640 1 -- 172.21.15.100:0/2743159110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa804b080 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffa95ffb640 1 -- 172.21.15.100:0/2743159110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ffaa8022ae0 con 0x5588acc29fe0 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 10 client.16218 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 20 client.16218 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc9d500 0x5588acc9f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 4 client.16218 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588ac91d890 con 0x5588acc9d500 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.257+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc9d500 0x5588acc9f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffa95ffb640 10 client.16218 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab1b3c640 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc9d500 0x5588acc9f920 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae3ce75d8827a88 server_cookie=3312786736f47c1b in_seq=0 out_seq=0 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffa95ffb640 1 -- 172.21.15.100:0/2743159110 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5588ac91d890 con 0x5588acc9d500 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffa95ffb640 10 client.16218 handle_command_reply: tid=0 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 client.16218 shutdown 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 msgr2=0x5588acc3cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc3cdc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800f470 tx=0x7ffaa8005970 comp rx=0 tx=0).stop 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 shutdown_connections 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5588acc9d500 0x5588acc9f920 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588acc359c0 0x5588acc398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffa95ffb640 0 client.16218 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588acc2a900 0x5588acc39190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 --2- 172.21.15.100:0/2743159110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588acc29fe0 0x5588acc3cdc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 >> 172.21.15.100:0/2743159110 conn(0x5588acc24990 msgr2=0x5588acc310c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 shutdown_connections 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 1 -- 172.21.15.100:0/2743159110 wait complete. 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.258+0000 7ffab7178740 20 client.16218 trim_cache size 0 max 16384 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:15:42.266 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:15:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffa980a1ba0 msgr2=0x7ffa980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffa980a1ba0 0x7ffa980a4060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0fd7a0 tx=0x7ffa9c009ae0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 msgr2=0x7ffaac19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 0x7ffaac19b390 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005e30 tx=0x7ffaa40325e0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 shutdown_connections 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ffa980a1ba0 0x7ffa980a4060 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaac19aee0 0x7ffaac19b390 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaac105340 0x7ffaac196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 --2- 172.21.15.100:0/525644110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac100270 0x7ffaac196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 >> 172.21.15.100:0/525644110 conn(0x7ffaac077230 msgr2=0x7ffaac0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 shutdown_connections 2024-03-12T05:15:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.259+0000 7ffab45a1640 1 -- 172.21.15.100:0/525644110 wait complete. 2024-03-12T05:15:42.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:15:42.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/1211584893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e4100c60 msgr2=0x7ff5e4104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/1211584893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e4100c60 0x7ff5e4104ca0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009930 tx=0x7ff5d402bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/1211584893 shutdown_connections 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/1211584893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e4105380 0x7ff5e410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/1211584893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e4100c60 0x7ff5e4104ca0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/1211584893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e41002b0 0x7ff5e4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/1211584893 >> 172.21.15.100:0/1211584893 conn(0x7ff5e4077120 msgr2=0x7ff5e4077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/1211584893 shutdown_connections 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.379+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/1211584893 wait complete. 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 Processor -- start 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 -- start start 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e41002b0 0x7ff5e419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4105380 0x7ff5e419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 0x7ff5e4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4197fc0 con 0x7ff5e41975d0 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4198130 con 0x7ff5e4105380 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ebcfb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e41982f0 con 0x7ff5e41002b0 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 0x7ff5e4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5e9296640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4105380 0x7ff5e419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 0x7ff5e4197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33510/0 (socket says 172.21.15.100:33510) 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 -- 172.21.15.100:0/3476841889 learned_addr learned my addr 172.21.15.100:0/3476841889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5e9a97640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e41002b0 0x7ff5e419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 -- 172.21.15.100:0/3476841889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e41002b0 msgr2=0x7ff5e419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e41002b0 0x7ff5e419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 -- 172.21.15.100:0/3476841889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4105380 msgr2=0x7ff5e419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4105380 0x7ff5e419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5ea298640 1 -- 172.21.15.100:0/3476841889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009490 con 0x7ff5e41975d0 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.380+0000 7ff5e9296640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4105380 0x7ff5e419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.381+0000 7ff5ea298640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 0x7ff5e4197a80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00eb10 tx=0x7ff5dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.381+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc004b20 con 0x7ff5e41975d0 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.381+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc004c80 con 0x7ff5e41975d0 2024-03-12T05:15:42.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.381+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d4009560 con 0x7ff5e41975d0 2024-03-12T05:15:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.381+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc010650 con 0x7ff5e41975d0 2024-03-12T05:15:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.381+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41a45e0 con 0x7ff5e41975d0 2024-03-12T05:15:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.382+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e40683c0 con 0x7ff5e41975d0 2024-03-12T05:15:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.383+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff5dc005020 con 0x7ff5e41975d0 2024-03-12T05:15:42.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.383+0000 7ff5daffd640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5d00a1ba0 0x7ff5d00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.384+0000 7ff5e9a97640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5d00a1ba0 0x7ff5d00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.384+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff5dc014070 con 0x7ff5e41975d0 2024-03-12T05:15:42.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.384+0000 7ff5e9a97640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5d00a1ba0 0x7ff5d00a4060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009d60 tx=0x7ff5e0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.388+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5dc096c20 con 0x7ff5e41975d0 2024-03-12T05:15:42.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.619+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5e4109620 con 0x7ff5e41975d0 2024-03-12T05:15:42.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.620+0000 7ff5daffd640 1 -- 172.21.15.100:0/3476841889 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7ff5dc09a2e0 con 0x7ff5e41975d0 2024-03-12T05:15:42.616 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:15:42.617 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:13:19.700942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:15:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 24 2024-03-12T05:15:42.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5d00a1ba0 msgr2=0x7ff5d00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5d00a1ba0 0x7ff5d00a4060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009d60 tx=0x7ff5e0005f90 comp rx=0 tx=0).stop 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 msgr2=0x7ff5e4197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 0x7ff5e4197a80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00eb10 tx=0x7ff5dc00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 shutdown_connections 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5d00a1ba0 0x7ff5d00a4060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e41975d0 0x7ff5e4197a80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4105380 0x7ff5e419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 --2- 172.21.15.100:0/3476841889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e41002b0 0x7ff5e419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 >> 172.21.15.100:0/3476841889 conn(0x7ff5e4077120 msgr2=0x7ff5e4069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 shutdown_connections 2024-03-12T05:15:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.622+0000 7ff5ebcfb640 1 -- 172.21.15.100:0/3476841889 wait complete. 2024-03-12T05:15:42.629 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:15:42.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:15:42.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 -- 172.21.15.100:0/2156975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328101380 msgr2=0x7f1328101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 --2- 172.21.15.100:0/2156975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328101380 0x7f1328101760 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1318009930 tx=0x7f131802bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 -- 172.21.15.100:0/2156975222 shutdown_connections 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 --2- 172.21.15.100:0/2156975222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1328106450 0x7f132810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 --2- 172.21.15.100:0/2156975222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1328101d30 0x7f1328105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 --2- 172.21.15.100:0/2156975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328101380 0x7f1328101760 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 -- 172.21.15.100:0/2156975222 >> 172.21.15.100:0/2156975222 conn(0x7f1328078580 msgr2=0x7f1328078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 -- 172.21.15.100:0/2156975222 shutdown_connections 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.743+0000 7f132e69a640 1 -- 172.21.15.100:0/2156975222 wait complete. 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 Processor -- start 2024-03-12T05:15:42.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 -- start start 2024-03-12T05:15:42.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1328101380 0x7f132819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 0x7f132819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1328197660 0x7f1328197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328198130 con 0x7f1328106450 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13281982a0 con 0x7f1328197660 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132e69a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328198460 con 0x7f1328101380 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 0x7f132819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 0x7f132819dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33520/0 (socket says 172.21.15.100:33520) 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 -- 172.21.15.100:0/64338831 learned_addr learned my addr 172.21.15.100:0/64338831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f1327fff640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1328101380 0x7f132819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f132cc37640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1328197660 0x7f1328197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 -- 172.21.15.100:0/64338831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1328101380 msgr2=0x7f132819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1328101380 0x7f132819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 -- 172.21.15.100:0/64338831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1328197660 msgr2=0x7f1328197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1328197660 0x7f1328197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 -- 172.21.15.100:0/64338831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318009560 con 0x7f1328106450 2024-03-12T05:15:42.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.744+0000 7f13277fe640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 0x7f132819dae0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e9f0 tx=0x7f131c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.745+0000 7f13257fa640 1 -- 172.21.15.100:0/64338831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c0096c0 con 0x7f1328106450 2024-03-12T05:15:42.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.745+0000 7f13257fa640 1 -- 172.21.15.100:0/64338831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f131c005500 con 0x7f1328106450 2024-03-12T05:15:42.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.745+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13281a4400 con 0x7f1328106450 2024-03-12T05:15:42.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.745+0000 7f13257fa640 1 -- 172.21.15.100:0/64338831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c010430 con 0x7f1328106450 2024-03-12T05:15:42.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.745+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13281a48e0 con 0x7f1328106450 2024-03-12T05:15:42.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f13257fa640 1 -- 172.21.15.100:0/64338831 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f131c005020 con 0x7f1328106450 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 Processor -- start 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 -- start start 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce543f80 0x5575ce544360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce5448a0 0x5575ce551620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 0x5575ce553f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce4ff960 con 0x5575ce551b60 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce4b3310 con 0x5575ce543f80 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f133126e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce4b0840 con 0x5575ce5448a0 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f13257fa640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13100a1c20 0x7f13100a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.747+0000 7f13277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 0x5575ce553f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f1327fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce5448a0 0x5575ce551620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 0x5575ce553f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33536/0 (socket says 172.21.15.100:33536) 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f1327fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce5448a0 0x5575ce551620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52966/0 (socket says 172.21.15.100:52966) 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 -- 172.21.15.100:0/1930344599 learned_addr learned my addr 172.21.15.100:0/1930344599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f132cc37640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce543f80 0x5575ce544360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f132cc37640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13100a1c20 0x7f13100a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13257fa640 1 -- 172.21.15.100:0/64338831 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f131c014070 con 0x7f1328106450 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 -- 172.21.15.100:0/1930344599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce5448a0 msgr2=0x5575ce551620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce5448a0 0x5575ce551620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 -- 172.21.15.100:0/1930344599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce543f80 msgr2=0x5575ce544360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce543f80 0x5575ce544360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 -- 172.21.15.100:0/1930344599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ce46e060 con 0x5575ce551b60 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f13277fe640 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 0x5575ce553f50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f131c09a440 tx=0x7f131c004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523245db7c48dedc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f132cc37640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13100a1c20 0x7f13100a40e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1320005fd0 tx=0x7f1320009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f130e7fc640 1 -- 172.21.15.100:0/1930344599 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c09a280 con 0x5575ce551b60 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f130e7fc640 1 -- 172.21.15.100:0/1930344599 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f131c09a0a0 con 0x5575ce551b60 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.748+0000 7f130e7fc640 1 -- 172.21.15.100:0/1930344599 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c0a2ee0 con 0x5575ce551b60 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 -- 172.21.15.100:0/1930344599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 msgr2=0x5575ce553f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 0x5575ce553f50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f131c09a440 tx=0x7f131c004a90 comp rx=0 tx=0).stop 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 -- 172.21.15.100:0/1930344599 shutdown_connections 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce551b60 0x5575ce553f50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce5448a0 0x5575ce551620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 --2- 172.21.15.100:0/1930344599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce543f80 0x5575ce544360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 -- 172.21.15.100:0/1930344599 >> 172.21.15.100:0/1930344599 conn(0x5575ce53e980 msgr2=0x5575ce53ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 -- 172.21.15.100:0/1930344599 shutdown_connections 2024-03-12T05:15:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 -- 172.21.15.100:0/1930344599 wait complete. 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 -- start start 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.749+0000 7f133126e740 1 Processor -- start 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce54f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce54f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce4ac020 con 0x5575ce543f80 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce17f430 con 0x5575ce5448a0 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce13be00 con 0x5575ce551b60 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 10 client.0 _ll_register_callbacks cb 0x5575ce453cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33542/0 (socket says 172.21.15.100:33542) 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 -- 172.21.15.100:0/876218960 learned_addr learned my addr 172.21.15.100:0/876218960 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f132cc37640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce54f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f13277fe640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce54f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 -- 172.21.15.100:0/876218960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 msgr2=0x5575ce54f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce54f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 -- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 msgr2=0x5575ce54f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce54f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 -- 172.21.15.100:0/876218960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13281a48e0 con 0x5575ce543f80 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f1327fff640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54eb30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f131802c240 tx=0x7f131802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41b35a43c54191c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f13077fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f13077fe640 1 -- 172.21.15.100:0/876218960 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131802c640 con 0x5575ce543f80 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f13077fe640 1 -- 172.21.15.100:0/876218960 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1318046070 con 0x5575ce543f80 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.750+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575ce1ab7e0 con 0x5575ce543f80 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f13077fe640 1 -- 172.21.15.100:0/876218960 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f131802cc00 con 0x5575ce543f80 2024-03-12T05:15:42.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 10 client.16236 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5575ce4aa6a0 con 0x5575ce543f80 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f13077fe640 1 -- 172.21.15.100:0/876218960 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575ce1ab7e0 con 0x5575ce543f80 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f13077fe640 1 -- 172.21.15.100:0/876218960 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f131803c040 con 0x5575ce543f80 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 10 client.16236 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 10 client.16236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 20 client.16236 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 4 client.16236 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575ce1e8720 con 0x5575ce561f60 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.751+0000 7f13277fe640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.752+0000 7f13277fe640 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5643a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75f59a2a46e41107 server_cookie=b79b9319af419d0b in_seq=0 out_seq=0 2024-03-12T05:15:42.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.752+0000 7f13077fe640 10 client.16236 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f13077fe640 1 -- 172.21.15.100:0/876218960 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575ce1e8720 con 0x5575ce561f60 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f13077fe640 10 client.16236 handle_command_reply: tid=0 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 client.16236 shutdown 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 msgr2=0x5575ce54eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54eb30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f131802c240 tx=0x7f131802ca10 comp rx=0 tx=0).stop 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 shutdown_connections 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5643a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce54f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce54f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f13077fe640 0 client.16236 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 --2- 172.21.15.100:0/876218960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54eb30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 >> 172.21.15.100:0/876218960 conn(0x5575ce53e980 msgr2=0x5575ce5406d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 shutdown_connections 2024-03-12T05:15:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.754+0000 7f133126e740 1 -- 172.21.15.100:0/876218960 wait complete. 2024-03-12T05:15:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.755+0000 7f133126e740 20 client.16236 trim_cache size 0 max 16384 2024-03-12T05:15:42.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 Processor -- start 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 -- start start 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce53da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce558a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce431130 con 0x5575ce543f80 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce5695f0 con 0x5575ce5448a0 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f133126e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce543bb0 con 0x5575ce551b60 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33544/0 (socket says 172.21.15.100:33544) 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 -- 172.21.15.100:0/156736868 learned_addr learned my addr 172.21.15.100:0/156736868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f13277fe640 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce53da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f132cc37640 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce558a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 -- 172.21.15.100:0/156736868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 msgr2=0x5575ce558a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce558a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 -- 172.21.15.100:0/156736868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 msgr2=0x5575ce53da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce53da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 -- 172.21.15.100:0/156736868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ce446c20 con 0x5575ce543f80 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f1327fff640 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54f3d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5e92f4ce8c44bc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f130dffb640 1 -- 172.21.15.100:0/156736868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318005d50 con 0x5575ce543f80 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f130dffb640 1 -- 172.21.15.100:0/156736868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1318044070 con 0x5575ce543f80 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.767+0000 7f130dffb640 1 -- 172.21.15.100:0/156736868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180044b0 con 0x5575ce543f80 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 -- 172.21.15.100:0/156736868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 msgr2=0x5575ce54f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54f3d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318004c00 comp rx=0 tx=0).stop 2024-03-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 -- 172.21.15.100:0/156736868 shutdown_connections 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce558a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce53da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 --2- 172.21.15.100:0/156736868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce54f3d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 -- 172.21.15.100:0/156736868 >> 172.21.15.100:0/156736868 conn(0x5575ce53e980 msgr2=0x5575ce53d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 -- 172.21.15.100:0/156736868 shutdown_connections 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 -- 172.21.15.100:0/156736868 wait complete. 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.768+0000 7f133126e740 1 -- start start 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 Processor -- start 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce557ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce558020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce5b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce558740 con 0x5575ce543f80 2024-03-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce5b8a10 con 0x5575ce5448a0 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ce5b8b80 con 0x5575ce551b60 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 10 client.0 _ll_register_callbacks cb 0x7f131c00cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce557ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce557ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33558/0 (socket says 172.21.15.100:33558) 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 -- 172.21.15.100:0/2338453299 learned_addr learned my addr 172.21.15.100:0/2338453299 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f13277fe640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce558020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f132cc37640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce5b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 -- 172.21.15.100:0/2338453299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 msgr2=0x5575ce5b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce5b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 -- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 msgr2=0x5575ce558020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce558020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 -- 172.21.15.100:0/2338453299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575ce4aa6a0 con 0x5575ce543f80 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f13277fe640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce558020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f132cc37640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce5b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f1327fff640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce557ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1318004650 tx=0x7f1318003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c49d21fb3e2bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f13077fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f13077fe640 1 -- 172.21.15.100:0/2338453299 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318005d50 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f131c09a0a0 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.769+0000 7f13077fe640 1 -- 172.21.15.100:0/2338453299 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1318046070 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f13077fe640 1 -- 172.21.15.100:0/2338453299 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f131802c5d0 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 10 client.16242 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5575ce46c8c0 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f13077fe640 1 -- 172.21.15.100:0/2338453299 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f131c09a0a0 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f13077fe640 1 -- 172.21.15.100:0/2338453299 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f131803c040 con 0x5575ce543f80 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 10 client.16242 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 10 client.16242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:15:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 20 client.16242 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 4 client.16242 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575ce237890 con 0x5575ce561f60 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f13277fe640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f13077fe640 10 client.16242 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.770+0000 7f13277fe640 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5bbed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f00ccffc72261cdb server_cookie=1ee6032122176d88 in_seq=0 out_seq=0 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f13077fe640 1 -- 172.21.15.100:0/2338453299 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575ce237890 con 0x5575ce561f60 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f13077fe640 10 client.16242 handle_command_reply: tid=0 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 client.16242 shutdown 2024-03-12T05:15:42.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 msgr2=0x5575ce557ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce557ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1318004650 tx=0x7f1318003a50 comp rx=0 tx=0).stop 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 shutdown_connections 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5575ce561f60 0x5575ce5bbed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5575ce551b60 0x5575ce5b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ce5448a0 0x5575ce558020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f13077fe640 0 client.16242 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 --2- 172.21.15.100:0/2338453299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575ce543f80 0x5575ce557ae0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 >> 172.21.15.100:0/2338453299 conn(0x5575ce53e980 msgr2=0x5575ce54cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 shutdown_connections 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 1 -- 172.21.15.100:0/2338453299 wait complete. 2024-03-12T05:15:42.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.771+0000 7f133126e740 20 client.16242 trim_cache size 0 max 16384 2024-03-12T05:15:42.777 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:15:42.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13100a1c20 msgr2=0x7f13100a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13100a1c20 0x7f13100a40e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1320005fd0 tx=0x7f1320009500 comp rx=0 tx=0).stop 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 msgr2=0x7f132819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 0x7f132819dae0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e9f0 tx=0x7f131c00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 shutdown_connections 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13100a1c20 0x7f13100a40e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1328197660 0x7f1328197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1328106450 0x7f132819dae0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 --2- 172.21.15.100:0/64338831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1328101380 0x7f132819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 >> 172.21.15.100:0/64338831 conn(0x7f1328078580 msgr2=0x7f13280ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 shutdown_connections 2024-03-12T05:15:42.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.772+0000 7f132e69a640 1 -- 172.21.15.100:0/64338831 wait complete. 2024-03-12T05:15:42.779 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:15:42.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:15:42.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 373344 13.49 MB/sec execute 26 sec latency 13.248 ms 2024-03-12T05:15:42.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 -- 172.21.15.100:0/791093908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50101064b0 msgr2=0x7f501010e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 --2- 172.21.15.100:0/791093908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50101064b0 0x7f501010e0c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5008009cd0 tx=0x7f500802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 -- 172.21.15.100:0/791093908 shutdown_connections 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 --2- 172.21.15.100:0/791093908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50101064b0 0x7f501010e0c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 --2- 172.21.15.100:0/791093908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50101006a0 0x7f5010105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 --2- 172.21.15.100:0/791093908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50100ffcf0 0x7f50101000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 -- 172.21.15.100:0/791093908 >> 172.21.15.100:0/791093908 conn(0x7f50100752e0 msgr2=0x7f50100756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 -- 172.21.15.100:0/791093908 shutdown_connections 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.889+0000 7f50170ae640 1 -- 172.21.15.100:0/791093908 wait complete. 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 Processor -- start 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 -- start start 2024-03-12T05:15:42.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 0x7f50101977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50101064b0 0x7f5010197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501019c020 0x7f501019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501019ca10 con 0x7f50100ffcf0 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501019cb80 con 0x7f50101064b0 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f50170ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501019ccf0 con 0x7f501019c020 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 0x7f50101977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 0x7f50101977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33580/0 (socket says 172.21.15.100:33580) 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 -- 172.21.15.100:0/1226394385 learned_addr learned my addr 172.21.15.100:0/1226394385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f500ffff640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50101064b0 0x7f5010197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f501564b640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501019c020 0x7f501019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 -- 172.21.15.100:0/1226394385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501019c020 msgr2=0x7f501019c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501019c020 0x7f501019c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 -- 172.21.15.100:0/1226394385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50101064b0 msgr2=0x7f5010197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50101064b0 0x7f5010197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 -- 172.21.15.100:0/1226394385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008009900 con 0x7f50100ffcf0 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f5014e4a640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 0x7f50101977b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f500400c920 tx=0x7f500400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.890+0000 7f500dffb640 1 -- 172.21.15.100:0/1226394385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004007c70 con 0x7f50100ffcf0 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.891+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50101a42d0 con 0x7f50100ffcf0 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.891+0000 7f500dffb640 1 -- 172.21.15.100:0/1226394385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5004007dd0 con 0x7f50100ffcf0 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.891+0000 7f500dffb640 1 -- 172.21.15.100:0/1226394385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004018860 con 0x7f50100ffcf0 2024-03-12T05:15:42.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.891+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50101a4780 con 0x7f50100ffcf0 2024-03-12T05:15:42.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.893+0000 7f500dffb640 1 -- 172.21.15.100:0/1226394385 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f500400fac0 con 0x7f50100ffcf0 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.893+0000 7f5019c80740 1 Processor -- start 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.893+0000 7f5019c80740 1 -- start start 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500dffb640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f50000a1ba0 0x7f50000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5019c80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e58a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5019c80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e593210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5019c80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e595b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5019c80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e545960 con 0x56086e593750 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5019c80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e4f9310 con 0x56086e58a250 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5019c80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e4f6840 con 0x56086e58ab70 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f501564b640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f50000a1ba0 0x7f50000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5014e4a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e58a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e593210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f501564b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e595b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500dffb640 1 -- 172.21.15.100:0/1226394385 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f50040cd930 con 0x7f50100ffcf0 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5014e4a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e58a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:55934/0 (socket says 172.21.15.100:55934) 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f5014e4a640 1 -- 172.21.15.100:0/1334447208 learned_addr learned my addr 172.21.15.100:0/1334447208 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 -- 172.21.15.100:0/1334447208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 msgr2=0x56086e58a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e58a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 -- 172.21.15.100:0/1334447208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 msgr2=0x56086e595b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e595b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 -- 172.21.15.100:0/1334447208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086e4b4060 con 0x56086e58ab70 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f501564b640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f50000a1ba0 0x7f50000a4060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f500802c630 tx=0x7f5008032f80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.894+0000 7f500ffff640 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e593210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc005d00 tx=0x7f4ffc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f899d32c138a5fdb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f4ffaffd640 1 -- 172.21.15.100:0/1334447208 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00be60 con 0x56086e58ab70 2024-03-12T05:15:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f4ffaffd640 1 -- 172.21.15.100:0/1334447208 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc015070 con 0x56086e58ab70 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f4ffaffd640 1 -- 172.21.15.100:0/1334447208 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc011880 con 0x56086e58ab70 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 -- 172.21.15.100:0/1334447208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 msgr2=0x56086e593210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e593210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc005d00 tx=0x7f4ffc0107a0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 -- 172.21.15.100:0/1334447208 shutdown_connections 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e595b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e593210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 --2- 172.21.15.100:0/1334447208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e58a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 -- 172.21.15.100:0/1334447208 >> 172.21.15.100:0/1334447208 conn(0x56086e584980 msgr2=0x56086e584d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 -- 172.21.15.100:0/1334447208 shutdown_connections 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.895+0000 7f5019c80740 1 -- 172.21.15.100:0/1334447208 wait complete. 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 -- start start 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 Processor -- start 2024-03-12T05:15:42.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e590740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e590c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e5911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e4f2020 con 0x56086e593750 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e1c5430 con 0x56086e58a250 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e181e00 con 0x56086e58ab70 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5019c80740 10 client.0 _ll_register_callbacks cb 0x56086e499cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e5911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e5911c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33606/0 (socket says 172.21.15.100:33606) 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f500ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e590c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 -- 172.21.15.100:0/2141147080 learned_addr learned my addr 172.21.15.100:0/2141147080 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f5014e4a640 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e590740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 -- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 msgr2=0x56086e590c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e590c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 -- 172.21.15.100:0/2141147080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 msgr2=0x56086e590740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e590740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.896+0000 7f501564b640 1 -- 172.21.15.100:0/2141147080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc00b800 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f501564b640 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e5911c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f50080098d0 tx=0x7f5008005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedf36914576851 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f4feffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f4feffff640 1 -- 172.21.15.100:0/2141147080 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008048070 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56086e1f17e0 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f4feffff640 1 -- 172.21.15.100:0/2141147080 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f50080326b0 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f4feffff640 1 -- 172.21.15.100:0/2141147080 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5008034ea0 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 10 client.16254 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56086e4f06a0 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f4feffff640 1 -- 172.21.15.100:0/2141147080 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56086e1f17e0 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f4feffff640 1 -- 172.21.15.100:0/2141147080 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5008004340 con 0x56086e593750 2024-03-12T05:15:42.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 10 client.16254 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 10 client.16254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 20 client.16254 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5a3b60 0x56086e5a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 4 client.16254 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56086e22e720 con 0x56086e5a3b60 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.897+0000 7f5014e4a640 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5a3b60 0x56086e5a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.898+0000 7f4feffff640 10 client.16254 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.898+0000 7f5014e4a640 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5a3b60 0x56086e5a5f80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=584411d9d8d11827 server_cookie=a554c6e5ac4ba59e in_seq=0 out_seq=0 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.898+0000 7f4feffff640 1 -- 172.21.15.100:0/2141147080 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56086e22e720 con 0x56086e5a3b60 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.898+0000 7f4feffff640 10 client.16254 handle_command_reply: tid=0 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.898+0000 7f5019c80740 1 client.16254 shutdown 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 msgr2=0x56086e5911c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e5911c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f50080098d0 tx=0x7f5008005d80 comp rx=0 tx=0).stop 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 shutdown_connections 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e5911c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e590c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5a3b60 0x56086e5a5f80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 --2- 172.21.15.100:0/2141147080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58a250 0x56086e590740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f4feffff640 0 client.16254 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 >> 172.21.15.100:0/2141147080 conn(0x56086e584980 msgr2=0x56086e5866d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 shutdown_connections 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 1 -- 172.21.15.100:0/2141147080 wait complete. 2024-03-12T05:15:42.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.899+0000 7f5019c80740 20 client.16254 trim_cache size 0 max 16384 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 Processor -- start 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 -- start start 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58a250 0x56086e592e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58ab70 0x56086e590fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e58d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e477130 con 0x56086e593750 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e5a0610 con 0x56086e58ab70 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5019c80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e589eb0 con 0x56086e58a250 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e58d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e58d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33614/0 (socket says 172.21.15.100:33614) 2024-03-12T05:15:42.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 -- 172.21.15.100:0/3590335122 learned_addr learned my addr 172.21.15.100:0/3590335122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f500ffff640 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58ab70 0x56086e590fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5014e4a640 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58a250 0x56086e592e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 -- 172.21.15.100:0/3590335122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58a250 msgr2=0x56086e592e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58a250 0x56086e592e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 -- 172.21.15.100:0/3590335122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58ab70 msgr2=0x56086e590fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58ab70 0x56086e590fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f501564b640 1 -- 172.21.15.100:0/3590335122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086e48cc20 con 0x56086e593750 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.911+0000 7f5014e4a640 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58a250 0x56086e592e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f501564b640 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e58d9a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f50080046f0 tx=0x7f5008009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cebdb546d6af96d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f4ffa7fc640 1 -- 172.21.15.100:0/3590335122 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008034ea0 con 0x56086e593750 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f4ffa7fc640 1 -- 172.21.15.100:0/3590335122 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5008049040 con 0x56086e593750 2024-03-12T05:15:42.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f4ffa7fc640 1 -- 172.21.15.100:0/3590335122 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008048070 con 0x56086e593750 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 -- 172.21.15.100:0/3590335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 msgr2=0x56086e58d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e58d9a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f50080046f0 tx=0x7f5008009800 comp rx=0 tx=0).stop 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 -- 172.21.15.100:0/3590335122 shutdown_connections 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e593750 0x56086e58d9a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e58ab70 0x56086e590fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 --2- 172.21.15.100:0/3590335122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58a250 0x56086e592e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 -- 172.21.15.100:0/3590335122 >> 172.21.15.100:0/3590335122 conn(0x56086e584980 msgr2=0x56086e5aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 -- 172.21.15.100:0/3590335122 shutdown_connections 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.912+0000 7f5019c80740 1 -- 172.21.15.100:0/3590335122 wait complete. 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 -- start start 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 Processor -- start 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 0x56086e59a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e598cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e593750 0x56086e599230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e599910 con 0x56086e58a250 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e599a80 con 0x56086e593750 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086e5fa070 con 0x56086e58ab70 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5019c80740 10 client.0 _ll_register_callbacks cb 0x7f4ffc00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 0x56086e59a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 0x56086e59a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33626/0 (socket says 172.21.15.100:33626) 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 -- 172.21.15.100:0/816340066 learned_addr learned my addr 172.21.15.100:0/816340066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f500ffff640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e598cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f501564b640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e593750 0x56086e599230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 -- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 msgr2=0x56086e598cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e598cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 -- 172.21.15.100:0/816340066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e593750 msgr2=0x56086e599230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e593750 0x56086e599230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.913+0000 7f5014e4a640 1 -- 172.21.15.100:0/816340066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56086e4f06a0 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5014e4a640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 0x56086e59a950 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f500409a1f0 tx=0x7f50040ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e220f38a3a78fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f4feffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f4feffff640 1 -- 172.21.15.100:0/816340066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500409e640 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f4feffff640 1 -- 172.21.15.100:0/816340066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f50040a2c90 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ffc015070 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f4feffff640 1 -- 172.21.15.100:0/816340066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50040987f0 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 10 client.16260 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56086e4b28c0 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f4feffff640 1 -- 172.21.15.100:0/816340066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc015070 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f4feffff640 1 -- 172.21.15.100:0/816340066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5004016020 con 0x56086e58a250 2024-03-12T05:15:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 10 client.16260 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 10 client.16260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 20 client.16260 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5fb6d0 0x56086e5fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 4 client.16260 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56086e27d890 con 0x56086e5fb6d0 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.914+0000 7f500ffff640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5fb6d0 0x56086e5fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f500ffff640 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5fb6d0 0x56086e5fdaf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8ad5480745bef73 server_cookie=5858d1917ceb69c7 in_seq=0 out_seq=0 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f4feffff640 10 client.16260 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f4feffff640 1 -- 172.21.15.100:0/816340066 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56086e27d890 con 0x56086e5fb6d0 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f4feffff640 10 client.16260 handle_command_reply: tid=0 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 client.16260 shutdown 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 msgr2=0x56086e59a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 0x56086e59a950 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f500409a1f0 tx=0x7f50040ba040 comp rx=0 tx=0).stop 2024-03-12T05:15:42.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 shutdown_connections 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56086e5fb6d0 0x56086e5fdaf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56086e593750 0x56086e599230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56086e58ab70 0x56086e598cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f4feffff640 0 client.16260 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 --2- 172.21.15.100:0/816340066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56086e58a250 0x56086e59a950 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 >> 172.21.15.100:0/816340066 conn(0x56086e584980 msgr2=0x56086e5832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 shutdown_connections 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 1 -- 172.21.15.100:0/816340066 wait complete. 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.915+0000 7f5019c80740 20 client.16260 trim_cache size 0 max 16384 2024-03-12T05:15:42.919 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:15:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f50000a1ba0 msgr2=0x7f50000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f50000a1ba0 0x7f50000a4060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f500802c630 tx=0x7f5008032f80 comp rx=0 tx=0).stop 2024-03-12T05:15:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 msgr2=0x7f50101977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 0x7f50101977b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f500400c920 tx=0x7f500400cde0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 shutdown_connections 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f50000a1ba0 0x7f50000a4060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501019c020 0x7f501019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50101064b0 0x7f5010197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 --2- 172.21.15.100:0/1226394385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50100ffcf0 0x7f50101977b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 >> 172.21.15.100:0/1226394385 conn(0x7f50100752e0 msgr2=0x7f50100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 shutdown_connections 2024-03-12T05:15:42.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:42.916+0000 7f50170ae640 1 -- 172.21.15.100:0/1226394385 wait complete. 2024-03-12T05:15:42.922 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:42 smithi100 ceph-mon[32210]: pgmap v111: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 354 op/s 2024-03-12T05:15:42.922 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:42 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3476841889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:15:42.923 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:15:42.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 -- 172.21.15.100:0/2401784779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 msgr2=0x7f84c010e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2401784779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c010e090 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009cd0 tx=0x7f84b802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 -- 172.21.15.100:0/2401784779 shutdown_connections 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2401784779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c010e090 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2401784779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c006a450 0x7f84c0105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2401784779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c0069aa0 0x7f84c0069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 -- 172.21.15.100:0/2401784779 >> 172.21.15.100:0/2401784779 conn(0x7f84c0077150 msgr2=0x7f84c0077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:43.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 -- 172.21.15.100:0/2401784779 shutdown_connections 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.075+0000 7f84c6db6640 1 -- 172.21.15.100:0/2401784779 wait complete. 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 Processor -- start 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 -- start start 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c0069aa0 0x7f84c0100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c01005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c0100ae0 0x7f84c01a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01a5f30 con 0x7f84c01064e0 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01a60a0 con 0x7f84c0100ae0 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c6db6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01a6260 con 0x7f84c0069aa0 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c01005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c5353640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c0100ae0 0x7f84c01a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c01005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33658/0 (socket says 172.21.15.100:33658) 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 -- 172.21.15.100:0/2579014409 learned_addr learned my addr 172.21.15.100:0/2579014409 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c4b52640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c0069aa0 0x7f84c0100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 -- 172.21.15.100:0/2579014409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c0069aa0 msgr2=0x7f84c0100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c0069aa0 0x7f84c0100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 -- 172.21.15.100:0/2579014409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c0100ae0 msgr2=0x7f84c01a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c0100ae0 0x7f84c01a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84b7fff640 1 -- 172.21.15.100:0/2579014409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8009900 con 0x7f84c01064e0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.076+0000 7f84c5353640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c0100ae0 0x7f84c01a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.077+0000 7f84b7fff640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c01005a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.077+0000 7f84b5ffb640 1 -- 172.21.15.100:0/2579014409 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b00096a0 con 0x7f84c01064e0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.077+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c01a6540 con 0x7f84c01064e0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.077+0000 7f84b5ffb640 1 -- 172.21.15.100:0/2579014409 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b000ccc0 con 0x7f84c01064e0 2024-03-12T05:15:43.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.077+0000 7f84b5ffb640 1 -- 172.21.15.100:0/2579014409 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0018860 con 0x7f84c01064e0 2024-03-12T05:15:43.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.077+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c01a6aa0 con 0x7f84c01064e0 2024-03-12T05:15:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.079+0000 7f84b5ffb640 1 -- 172.21.15.100:0/2579014409 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f84b0010aa0 con 0x7f84c01064e0 2024-03-12T05:15:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 Processor -- start 2024-03-12T05:15:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 -- start start 2024-03-12T05:15:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84b5ffb640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84a80a1ba0 0x7f84a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5ddb70 0x5616bf5e6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf598960 con 0x5616bf5e6750 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf54c310 con 0x5616bf5ddb70 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c9984740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf549840 con 0x5616bf5dd250 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84b5ffb640 1 -- 172.21.15.100:0/2579014409 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84b0014070 con 0x7f84c01064e0 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c5353640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84a80a1ba0 0x7f84a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5dd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:53046/0 (socket says 172.21.15.100:53046) 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 -- 172.21.15.100:0/1548071430 learned_addr learned my addr 172.21.15.100:0/1548071430 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c5353640 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84b7fff640 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5ddb70 0x5616bf5e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 -- 172.21.15.100:0/1548071430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5ddb70 msgr2=0x5616bf5e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5ddb70 0x5616bf5e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 -- 172.21.15.100:0/1548071430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 msgr2=0x5616bf5e8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.080+0000 7f84c4b52640 1 -- 172.21.15.100:0/1548071430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616bf507060 con 0x5616bf5dd250 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c5353640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84a80a1ba0 0x7f84a80a4060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005cc0 tx=0x7f84b8005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c4b52640 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5dd630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009d20 tx=0x7f84bc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f1cf936921451 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84a6ffd640 1 -- 172.21.15.100:0/1548071430 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc019630 con 0x5616bf5dd250 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84a6ffd640 1 -- 172.21.15.100:0/1548071430 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84bc019c70 con 0x5616bf5dd250 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84a6ffd640 1 -- 172.21.15.100:0/1548071430 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00ebd0 con 0x5616bf5dd250 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 -- 172.21.15.100:0/1548071430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 msgr2=0x5616bf5dd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5dd630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009d20 tx=0x7f84bc011810 comp rx=0 tx=0).stop 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 -- 172.21.15.100:0/1548071430 shutdown_connections 2024-03-12T05:15:43.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5ddb70 0x5616bf5e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 --2- 172.21.15.100:0/1548071430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5dd630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 -- 172.21.15.100:0/1548071430 >> 172.21.15.100:0/1548071430 conn(0x5616bf5d7980 msgr2=0x5616bf5d7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.081+0000 7f84c9984740 1 -- 172.21.15.100:0/1548071430 shutdown_connections 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 -- 172.21.15.100:0/1548071430 wait complete. 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 -- start start 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 Processor -- start 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 0x5616bf5e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5e6750 0x5616bf5e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf545020 con 0x5616bf5ddb70 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf218430 con 0x5616bf5e6750 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf1d4e00 con 0x5616bf5dd250 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c9984740 10 client.0 _ll_register_callbacks cb 0x5616bf4eccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 0x5616bf5e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c4b52640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 0x5616bf5e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33690/0 (socket says 172.21.15.100:33690) 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84b7fff640 1 -- 172.21.15.100:0/241471424 learned_addr learned my addr 172.21.15.100:0/241471424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.082+0000 7f84c5353640 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5e6750 0x5616bf5e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84b7fff640 1 -- 172.21.15.100:0/241471424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 msgr2=0x5616bf5e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84b7fff640 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84b7fff640 1 -- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5e6750 msgr2=0x5616bf5e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84b7fff640 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5e6750 0x5616bf5e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84b7fff640 1 -- 172.21.15.100:0/241471424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc009950 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84b7fff640 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 0x5616bf5e3cc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f84b00971a0 tx=0x7f84b009a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166a1086411df306 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f849ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f849ffff640 1 -- 172.21.15.100:0/241471424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b00ba2b0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f849ffff640 1 -- 172.21.15.100:0/241471424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84b00a2ec0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616bf2447e0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f849ffff640 1 -- 172.21.15.100:0/241471424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b00cb6d0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 10 client.16272 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616bf5436a0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f849ffff640 1 -- 172.21.15.100:0/241471424 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616bf2447e0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f849ffff640 1 -- 172.21.15.100:0/241471424 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f84b004f0a0 con 0x5616bf5ddb70 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 10 client.16272 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 20 client.16272 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:43.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf5f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 4 client.16272 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.083+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616bf281720 con 0x5616bf5f6b90 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.084+0000 7f84c4b52640 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf5f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.084+0000 7f849ffff640 10 client.16272 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.084+0000 7f84c4b52640 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf5f8fb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13784f07b138044a server_cookie=8ca9fecea689af05 in_seq=0 out_seq=0 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f849ffff640 1 -- 172.21.15.100:0/241471424 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616bf281720 con 0x5616bf5f6b90 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f849ffff640 10 client.16272 handle_command_reply: tid=0 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 client.16272 shutdown 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 msgr2=0x5616bf5e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 0x5616bf5e3cc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f84b00971a0 tx=0x7f84b009a400 comp rx=0 tx=0).stop 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 shutdown_connections 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf5f8fb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5e6750 0x5616bf5e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f849ffff640 0 client.16272 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5ddb70 0x5616bf5e3cc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 --2- 172.21.15.100:0/241471424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5dd250 0x5616bf5e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 >> 172.21.15.100:0/241471424 conn(0x5616bf5d7980 msgr2=0x5616bf5d96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 shutdown_connections 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 1 -- 172.21.15.100:0/241471424 wait complete. 2024-03-12T05:15:43.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.085+0000 7f84c9984740 20 client.16272 trim_cache size 0 max 16384 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 Processor -- start 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 -- start start 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf4ca130 con 0x5616bf5e6750 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf5d8980 con 0x5616bf5dd250 2024-03-12T05:15:43.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.097+0000 7f84c9984740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf5d8550 con 0x5616bf5ddb70 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c4b52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33698/0 (socket says 172.21.15.100:33698) 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 -- 172.21.15.100:0/809189361 learned_addr learned my addr 172.21.15.100:0/809189361 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 -- 172.21.15.100:0/809189361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 msgr2=0x5616bf5e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5e4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 -- 172.21.15.100:0/809189361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 msgr2=0x5616bf5e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 -- 172.21.15.100:0/809189361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616bf4dfc20 con 0x5616bf5e6750 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84c5353640 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e09e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009cd0 tx=0x7f84b804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4f600df71a440 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:43.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84a67fc640 1 -- 172.21.15.100:0/809189361 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8048070 con 0x5616bf5e6750 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84a67fc640 1 -- 172.21.15.100:0/809189361 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b8004c00 con 0x5616bf5e6750 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.098+0000 7f84a67fc640 1 -- 172.21.15.100:0/809189361 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80042a0 con 0x5616bf5e6750 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- 172.21.15.100:0/809189361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 msgr2=0x5616bf5e09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e09e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009cd0 tx=0x7f84b804b000 comp rx=0 tx=0).stop 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- 172.21.15.100:0/809189361 shutdown_connections 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5e09e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- 172.21.15.100:0/809189361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- 172.21.15.100:0/809189361 >> 172.21.15.100:0/809189361 conn(0x5616bf5d7980 msgr2=0x5616bf5dea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- 172.21.15.100:0/809189361 shutdown_connections 2024-03-12T05:15:43.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- 172.21.15.100:0/809189361 wait complete. 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- start start 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 Processor -- start 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5ebff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf64d0a0 con 0x5616bf5e6750 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf64d1e0 con 0x5616bf5dd250 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616bf64d320 con 0x5616bf5ddb70 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.099+0000 7f84c9984740 10 client.0 _ll_register_callbacks cb 0x7f84bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c4b52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5ebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5eca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33706/0 (socket says 172.21.15.100:33706) 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 -- 172.21.15.100:0/2795091968 learned_addr learned my addr 172.21.15.100:0/2795091968 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 -- 172.21.15.100:0/2795091968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 msgr2=0x5616bf5ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 -- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 msgr2=0x5616bf5ebff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5ebff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 -- 172.21.15.100:0/2795091968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c010e8a0 con 0x5616bf5e6750 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c5353640 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5eca70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f84b8037040 tx=0x7f84b8034f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e2d58cff14d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f849ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f849ffff640 1 -- 172.21.15.100:0/2795091968 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8048040 con 0x5616bf5e6750 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f849ffff640 1 -- 172.21.15.100:0/2795091968 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84b8003ea0 con 0x5616bf5e6750 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84bc019c70 con 0x5616bf5e6750 2024-03-12T05:15:43.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f849ffff640 1 -- 172.21.15.100:0/2795091968 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b80040a0 con 0x5616bf5e6750 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c9984740 10 client.16278 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616bf5058c0 con 0x5616bf5e6750 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.100+0000 7f849ffff640 1 -- 172.21.15.100:0/2795091968 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b8066020 con 0x5616bf5e6750 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f849ffff640 1 -- 172.21.15.100:0/2795091968 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f84b8062840 con 0x5616bf5e6750 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 10 client.16278 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 20 client.16278 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf650780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 4 client.16278 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5616bf2d0890 con 0x5616bf5f6b90 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c4b52640 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf650780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f849ffff640 10 client.16278 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c4b52640 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf650780 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8832addb4dd6cc0 server_cookie=85b57018fed186ec in_seq=0 out_seq=0 2024-03-12T05:15:43.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f849ffff640 1 -- 172.21.15.100:0/2795091968 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616bf2d0890 con 0x5616bf5f6b90 2024-03-12T05:15:43.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f849ffff640 10 client.16278 handle_command_reply: tid=0 2024-03-12T05:15:43.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 1 client.16278 shutdown 2024-03-12T05:15:43.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 msgr2=0x5616bf5eca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.101+0000 7f84c9984740 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5eca70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f84b8037040 tx=0x7f84b8034f50 comp rx=0 tx=0).stop 2024-03-12T05:15:43.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 shutdown_connections 2024-03-12T05:15:43.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5616bf5f6b90 0x5616bf650780 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616bf5e6750 0x5616bf5eca70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616bf5ddb70 0x5616bf5ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 --2- 172.21.15.100:0/2795091968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616bf5dd250 0x5616bf5ebff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 >> 172.21.15.100:0/2795091968 conn(0x5616bf5d7980 msgr2=0x5616bf5d62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:43.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f849ffff640 0 client.16278 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:43.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 shutdown_connections 2024-03-12T05:15:43.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 1 -- 172.21.15.100:0/2795091968 wait complete. 2024-03-12T05:15:43.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c9984740 20 client.16278 trim_cache size 0 max 16384 2024-03-12T05:15:43.108 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:15:43.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84a80a1ba0 msgr2=0x7f84a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84a80a1ba0 0x7f84a80a4060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005cc0 tx=0x7f84b8005fd0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 msgr2=0x7f84c01005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c01005a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 shutdown_connections 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84a80a1ba0 0x7f84a80a4060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c0100ae0 0x7f84c01a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c01064e0 0x7f84c01005a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 --2- 172.21.15.100:0/2579014409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c0069aa0 0x7f84c0100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 >> 172.21.15.100:0/2579014409 conn(0x7f84c0077150 msgr2=0x7f84c00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 shutdown_connections 2024-03-12T05:15:43.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:43.102+0000 7f84c6db6640 1 -- 172.21.15.100:0/2579014409 wait complete. 2024-03-12T05:15:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:42 smithi156 ceph-mon[36270]: pgmap v111: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 354 op/s 2024-03-12T05:15:43.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:42 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3476841889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:15:43.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:42 smithi161 ceph-mon[36043]: pgmap v111: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 354 op/s 2024-03-12T05:15:43.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:42 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3476841889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:15:43.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 376346 13.67 MB/sec execute 27 sec latency 6.688 ms 2024-03-12T05:15:44.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:15:44.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:15:44.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:15:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.225+0000 7f45a49c9640 1 -- 172.21.15.100:0/3479360383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a01002a0 msgr2=0x7f45a0100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.225+0000 7f45a49c9640 1 --2- 172.21.15.100:0/3479360383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a01002a0 0x7f45a0100680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4590009930 tx=0x7f459002bc30 comp rx=0 tx=0).stop 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 -- 172.21.15.100:0/3479360383 shutdown_connections 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 --2- 172.21.15.100:0/3479360383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a0105370 0x7f45a010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 --2- 172.21.15.100:0/3479360383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45a0100c50 0x7f45a0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 --2- 172.21.15.100:0/3479360383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a01002a0 0x7f45a0100680 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 -- 172.21.15.100:0/3479360383 >> 172.21.15.100:0/3479360383 conn(0x7f45a0076400 msgr2=0x7f45a0076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 -- 172.21.15.100:0/3479360383 shutdown_connections 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 -- 172.21.15.100:0/3479360383 wait complete. 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 Processor -- start 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 -- start start 2024-03-12T05:15:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.226+0000 7f45a49c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45a01002a0 0x7f45a019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 0x7f45a019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a0197650 0x7f45a0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0198120 con 0x7f45a0105370 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0198290 con 0x7f45a01002a0 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0198450 con 0x7f45a0197650 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 0x7f45a019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 0x7f45a019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33734/0 (socket says 172.21.15.100:33734) 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 -- 172.21.15.100:0/1970725704 learned_addr learned my addr 172.21.15.100:0/1970725704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459ed9d640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a0197650 0x7f45a0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459e59c640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45a01002a0 0x7f45a019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 -- 172.21.15.100:0/1970725704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a0197650 msgr2=0x7f45a0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a0197650 0x7f45a0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 -- 172.21.15.100:0/1970725704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45a01002a0 msgr2=0x7f45a019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45a01002a0 0x7f45a019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 -- 172.21.15.100:0/1970725704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590009560 con 0x7f45a0105370 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 0x7f45a019dad0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f459400e9c0 tx=0x7f459400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f458f7fe640 1 -- 172.21.15.100:0/1970725704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594009700 con 0x7f45a0105370 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f458f7fe640 1 -- 172.21.15.100:0/1970725704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f459400cd80 con 0x7f45a0105370 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a01a43f0 con 0x7f45a0105370 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f458f7fe640 1 -- 172.21.15.100:0/1970725704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594010430 con 0x7f45a0105370 2024-03-12T05:15:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.227+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01a48d0 con 0x7f45a0105370 2024-03-12T05:15:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f458f7fe640 1 -- 172.21.15.100:0/1970725704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4594005000 con 0x7f45a0105370 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 Processor -- start 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 -- start start 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 0x55f729a5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5dab0 0x55f729a6a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729a6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7299cbf90 con 0x55f729a5d100 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7299c94c0 con 0x55f729a6acc0 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f45a759f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7299c4ca0 con 0x55f729a5dab0 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f458f7fe640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45880a1ba0 0x7f45880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f459ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 0x55f729a5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729a6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f459ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 0x55f729a5d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33750/0 (socket says 172.21.15.100:33750) 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f459ed9d640 1 -- 172.21.15.100:0/2660361750 learned_addr learned my addr 172.21.15.100:0/2660361750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.230+0000 7f459ed9d640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45880a1ba0 0x7f45880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459e59c640 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5dab0 0x55f729a6a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f458f7fe640 1 -- 172.21.15.100:0/1970725704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4594014070 con 0x7f45a0105370 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 -- 172.21.15.100:0/2660361750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5dab0 msgr2=0x55f729a6a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5dab0 0x55f729a6a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 -- 172.21.15.100:0/2660361750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 msgr2=0x55f729a6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729a6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 -- 172.21.15.100:0/2660361750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7299870b0 con 0x55f729a5d100 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459dd9b640 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729a6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 0x55f729a5d4e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4598005cc0 tx=0x7f45980117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ace172b9587636 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f459ed9d640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45880a1ba0 0x7f45880a4060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4598011cb0 tx=0x7f4598016000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f458cff9640 1 -- 172.21.15.100:0/2660361750 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800f450 con 0x55f729a5d100 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f458cff9640 1 -- 172.21.15.100:0/2660361750 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f459801a040 con 0x55f729a5d100 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.231+0000 7f458cff9640 1 -- 172.21.15.100:0/2660361750 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598020550 con 0x55f729a5d100 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 -- 172.21.15.100:0/2660361750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 msgr2=0x55f729a5d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 0x55f729a5d4e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4598005cc0 tx=0x7f45980117b0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 -- 172.21.15.100:0/2660361750 shutdown_connections 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729a6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5dab0 0x55f729a6a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 --2- 172.21.15.100:0/2660361750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5d100 0x55f729a5d4e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 -- 172.21.15.100:0/2660361750 >> 172.21.15.100:0/2660361750 conn(0x55f729a57840 msgr2=0x55f729a57c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 -- 172.21.15.100:0/2660361750 shutdown_connections 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 -- 172.21.15.100:0/2660361750 wait complete. 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 -- start start 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.232+0000 7f45a759f740 1 Processor -- start 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a67c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5dab0 0x55f729a68170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 0x55f729a686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729969f40 con 0x55f729a6acc0 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7296984b0 con 0x55f729a5dab0 2024-03-12T05:15:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72994a670 con 0x55f729a5d100 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 10 client.0 _ll_register_callbacks cb 0x55f7299b7f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 0x55f729a686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 0x55f729a686b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33752/0 (socket says 172.21.15.100:33752) 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 -- 172.21.15.100:0/2045336214 learned_addr learned my addr 172.21.15.100:0/2045336214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459dd9b640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5dab0 0x55f729a68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459e59c640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a67c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 -- 172.21.15.100:0/2045336214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 msgr2=0x55f729a67c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a67c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 -- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5dab0 msgr2=0x55f729a68170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5dab0 0x55f729a68170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 -- 172.21.15.100:0/2045336214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4598014a90 con 0x55f729a6acc0 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f459ed9d640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 0x55f729a686b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f459800fec0 tx=0x7f4598004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab731b71768c6cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f4585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f4585ffb640 1 -- 172.21.15.100:0/2045336214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598004af0 con 0x55f729a6acc0 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f4585ffb640 1 -- 172.21.15.100:0/2045336214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4598004c50 con 0x55f729a6acc0 2024-03-12T05:15:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f72999ef40 con 0x55f729a6acc0 2024-03-12T05:15:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f4585ffb640 1 -- 172.21.15.100:0/2045336214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4598004e10 con 0x55f729a6acc0 2024-03-12T05:15:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 10 client.16290 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.233+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f7299c3320 con 0x55f729a6acc0 2024-03-12T05:15:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f4585ffb640 1 -- 172.21.15.100:0/2045336214 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f72999ef40 con 0x55f729a6acc0 2024-03-12T05:15:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f4585ffb640 1 -- 172.21.15.100:0/2045336214 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f459801d070 con 0x55f729a6acc0 2024-03-12T05:15:44.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f45a759f740 10 client.16290 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:44.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f45a759f740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:44.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f45a759f740 20 client.16290 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:44.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f45a759f740 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729a7b1c0 0x55f729a7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f45a759f740 4 client.16290 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:44.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7299bb310 con 0x55f729a7b1c0 2024-03-12T05:15:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f459e59c640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729a7b1c0 0x55f729a7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f459e59c640 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729a7b1c0 0x55f729a7d5e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a63bae869837e9 server_cookie=3786a031f0e4a0b5 in_seq=0 out_seq=0 2024-03-12T05:15:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.234+0000 7f4585ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f4585ffb640 1 -- 172.21.15.100:0/2045336214 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7299bb310 con 0x55f729a7b1c0 2024-03-12T05:15:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f4585ffb640 10 client.16290 handle_command_reply: tid=0 2024-03-12T05:15:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 client.16290 shutdown 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 msgr2=0x55f729a686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 0x55f729a686b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f459800fec0 tx=0x7f4598004a80 comp rx=0 tx=0).stop 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 shutdown_connections 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729a7b1c0 0x55f729a7d5e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a6acc0 0x55f729a686b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5dab0 0x55f729a68170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f4585ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 --2- 172.21.15.100:0/2045336214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a67c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 >> 172.21.15.100:0/2045336214 conn(0x55f729a57840 msgr2=0x55f729a597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 shutdown_connections 2024-03-12T05:15:44.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 1 -- 172.21.15.100:0/2045336214 wait complete. 2024-03-12T05:15:44.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.235+0000 7f45a759f740 20 client.16290 trim_cache size 0 max 16384 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.250+0000 7f45a759f740 1 Processor -- start 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 -- start start 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5d100 0x55f729a6a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a6acc0 0x55f729a64e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729a58a50 con 0x55f729a5dab0 2024-03-12T05:15:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729a6da40 con 0x55f729a5d100 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f45a759f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729a63820 con 0x55f729a6acc0 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33762/0 (socket says 172.21.15.100:33762) 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 -- 172.21.15.100:0/1524334296 learned_addr learned my addr 172.21.15.100:0/1524334296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459e59c640 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5d100 0x55f729a6a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459ed9d640 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a6acc0 0x55f729a64e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 -- 172.21.15.100:0/1524334296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a6acc0 msgr2=0x55f729a64e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a6acc0 0x55f729a64e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 -- 172.21.15.100:0/1524334296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5d100 msgr2=0x55f729a6a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5d100 0x55f729a6a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 -- 172.21.15.100:0/1524334296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72995fc00 con 0x55f729a5dab0 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a684d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f459400c6a0 tx=0x7f459406dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7471ff0c7e78decd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f4587fff640 1 -- 172.21.15.100:0/1524334296 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594098fe0 con 0x55f729a5dab0 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f4587fff640 1 -- 172.21.15.100:0/1524334296 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45940a2930 con 0x55f729a5dab0 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.251+0000 7f4587fff640 1 -- 172.21.15.100:0/1524334296 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940a2fd0 con 0x55f729a5dab0 2024-03-12T05:15:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 -- 172.21.15.100:0/1524334296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 msgr2=0x55f729a684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a684d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f459400c6a0 tx=0x7f459406dae0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 -- 172.21.15.100:0/1524334296 shutdown_connections 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a6acc0 0x55f729a64e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a684d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 --2- 172.21.15.100:0/1524334296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a5d100 0x55f729a6a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 -- 172.21.15.100:0/1524334296 >> 172.21.15.100:0/1524334296 conn(0x55f729a57840 msgr2=0x55f729a81f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 -- 172.21.15.100:0/1524334296 shutdown_connections 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 -- 172.21.15.100:0/1524334296 wait complete. 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 -- start start 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.252+0000 7f45a759f740 1 Processor -- start 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a70f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a71470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729acb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729acbbe0 con 0x55f729a5dab0 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729acbd50 con 0x55f729a6acc0 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f729acbf10 con 0x55f729a5d100 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 10 client.0 _ll_register_callbacks cb 0x7f459800aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a71470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a71470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33776/0 (socket says 172.21.15.100:33776) 2024-03-12T05:15:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 -- 172.21.15.100:0/1133741059 learned_addr learned my addr 172.21.15.100:0/1133741059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459e59c640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a70f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459ed9d640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729acb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 -- 172.21.15.100:0/1133741059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 msgr2=0x55f729a70f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a70f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 -- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 msgr2=0x55f729acb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729acb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 -- 172.21.15.100:0/1133741059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01a43f0 con 0x55f729a5dab0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f459dd9b640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a71470 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4594005740 tx=0x7f45940a2200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41b35dd1c710287 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f4585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f4585ffb640 1 -- 172.21.15.100:0/1133741059 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459409af30 con 0x55f729a5dab0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f4585ffb640 1 -- 172.21.15.100:0/1133741059 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f45940a2fd0 con 0x55f729a5dab0 2024-03-12T05:15:44.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7299ea900 con 0x55f729a5dab0 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f4585ffb640 1 -- 172.21.15.100:0/1133741059 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45940a1ec0 con 0x55f729a5dab0 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 10 client.16296 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.253+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f729985910 con 0x55f729a5dab0 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f4585ffb640 1 -- 172.21.15.100:0/1133741059 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4594080020 con 0x55f729a5dab0 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f4585ffb640 1 -- 172.21.15.100:0/1133741059 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f459401f060 con 0x55f729a5dab0 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f45a759f740 10 client.16296 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f45a759f740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f45a759f740 20 client.16296 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f45a759f740 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729ad3520 0x55f729ad5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f45a759f740 4 client.16296 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f7296c4860 con 0x55f729ad3520 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f459e59c640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729ad3520 0x55f729ad5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f459e59c640 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729ad3520 0x55f729ad5940 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfc5bbd03e6a834 server_cookie=d329b4cee19799ed in_seq=0 out_seq=0 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.254+0000 7f4585ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.255+0000 7f4585ffb640 1 -- 172.21.15.100:0/1133741059 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f7296c4860 con 0x55f729ad3520 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.255+0000 7f4585ffb640 10 client.16296 handle_command_reply: tid=0 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.255+0000 7f45a759f740 1 client.16296 shutdown 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.255+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 msgr2=0x55f729a71470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.255+0000 7f45a759f740 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a71470 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4594005740 tx=0x7f45940a2200 comp rx=0 tx=0).stop 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 shutdown_connections 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f729ad3520 0x55f729ad5940 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f729a6acc0 0x55f729acb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f729a5dab0 0x55f729a71470 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 --2- 172.21.15.100:0/1133741059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f729a5d100 0x55f729a70f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 >> 172.21.15.100:0/1133741059 conn(0x55f729a57840 msgr2=0x55f729a56050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f4585ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 shutdown_connections 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 1 -- 172.21.15.100:0/1133741059 wait complete. 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a759f740 20 client.16296 trim_cache size 0 max 16384 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:15:44.265 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:15:44.266 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "99b72903-b5fb-4c33-a5d4-ceee984ac622", 2024-03-12T05:15:44.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:15:44.266 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:15:44.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45880a1ba0 msgr2=0x7f45880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45880a1ba0 0x7f45880a4060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4598011cb0 tx=0x7f4598016000 comp rx=0 tx=0).stop 2024-03-12T05:15:44.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 msgr2=0x7f45a019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 0x7f45a019dad0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f459400e9c0 tx=0x7f459400ee80 comp rx=0 tx=0).stop 2024-03-12T05:15:44.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 shutdown_connections 2024-03-12T05:15:44.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45880a1ba0 0x7f45880a4060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45a0197650 0x7f45a0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0105370 0x7f45a019dad0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 --2- 172.21.15.100:0/1970725704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45a01002a0 0x7f45a019d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 >> 172.21.15.100:0/1970725704 conn(0x7f45a0076400 msgr2=0x7f45a00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 shutdown_connections 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.256+0000 7f45a49c9640 1 -- 172.21.15.100:0/1970725704 wait complete. 2024-03-12T05:15:44.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:15:44.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:15:44.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 -- 172.21.15.100:0/2212911474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 msgr2=0x7f59a010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 --2- 172.21.15.100:0/2212911474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a010db80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5998009ca0 tx=0x7f599802c180 comp rx=0 tx=0).stop 2024-03-12T05:15:44.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 -- 172.21.15.100:0/2212911474 shutdown_connections 2024-03-12T05:15:44.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 --2- 172.21.15.100:0/2212911474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a010db80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 --2- 172.21.15.100:0/2212911474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59a0102ce0 0x7f59a010b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 --2- 172.21.15.100:0/2212911474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59a01023c0 0x7f59a01027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.378+0000 7f59a773a640 1 -- 172.21.15.100:0/2212911474 >> 172.21.15.100:0/2212911474 conn(0x7f59a0077230 msgr2=0x7f59a0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 -- 172.21.15.100:0/2212911474 shutdown_connections 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 -- 172.21.15.100:0/2212911474 wait complete. 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 Processor -- start 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 -- start start 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59a01023c0 0x7f59a0100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59a0102ce0 0x7f59a0100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a0100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a5a90 con 0x7f59a010b700 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a5c00 con 0x7f59a0102ce0 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a773a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a5dc0 con 0x7f59a01023c0 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a0100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a0100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33802/0 (socket says 172.21.15.100:33802) 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2220583842 learned_addr learned my addr 172.21.15.100:0/2220583842 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a54d6640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59a01023c0 0x7f59a0100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.379+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59a0102ce0 0x7f59a0100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2220583842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59a01023c0 msgr2=0x7f59a0100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59a01023c0 0x7f59a0100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2220583842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59a0102ce0 msgr2=0x7f59a0100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59a0102ce0 0x7f59a0100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2220583842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998009900 con 0x7f59a010b700 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a0100a90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f599802c790 tx=0x7f5998034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59967fc640 1 -- 172.21.15.100:0/2220583842 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998003d80 con 0x7f59a010b700 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59967fc640 1 -- 172.21.15.100:0/2220583842 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5998032c70 con 0x7f59a010b700 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a01a6040 con 0x7f59a010b700 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59967fc640 1 -- 172.21.15.100:0/2220583842 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980047f0 con 0x7f59a010b700 2024-03-12T05:15:44.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.380+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a01a6540 con 0x7f59a010b700 2024-03-12T05:15:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.382+0000 7f59967fc640 1 -- 172.21.15.100:0/2220583842 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f599800a040 con 0x7f59a010b700 2024-03-12T05:15:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 Processor -- start 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 -- start start 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59967fc640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f598c0a1c20 0x7f598c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 0x5579aa500ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa464280 con 0x5579aa4f5bd0 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa4617b0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa45cf90 con 0x5579aa4f52b0 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a54d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f598c0a1c20 0x7f598c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 0x5579aa500ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59967fc640 1 -- 172.21.15.100:0/2220583842 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59980f0240 con 0x7f59a010b700 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a54d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33810/0 (socket says 172.21.15.100:33810) 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 0x5579aa500ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:56076/0 (socket says 172.21.15.100:56076) 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a54d6640 1 -- 172.21.15.100:0/1177536315 learned_addr learned my addr 172.21.15.100:0/1177536315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 -- 172.21.15.100:0/1177536315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 msgr2=0x5579aa4f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 -- 172.21.15.100:0/1177536315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f5bd0 msgr2=0x5579aa4fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 -- 172.21.15.100:0/1177536315 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579aa41b570 con 0x5579aa4fe7b0 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f598c0a1c20 0x7f598c0a40e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f599000a500 tx=0x7f599000aa00 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.383+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 0x5579aa500ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59980eda90 tx=0x7f599804b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19bc9868f2822567 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f598b7fe640 1 -- 172.21.15.100:0/1177536315 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980bc8d0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f598b7fe640 1 -- 172.21.15.100:0/1177536315 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59980cf9a0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f598b7fe640 1 -- 172.21.15.100:0/1177536315 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980c10c0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 -- 172.21.15.100:0/1177536315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 msgr2=0x5579aa500ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 0x5579aa500ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59980eda90 tx=0x7f599804b000 comp rx=0 tx=0).stop 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 -- 172.21.15.100:0/1177536315 shutdown_connections 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4fe7b0 0x5579aa500ba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/1177536315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.384+0000 7f59aa3e5740 1 -- 172.21.15.100:0/1177536315 >> 172.21.15.100:0/1177536315 conn(0x5579aa4ef990 msgr2=0x5579aa4efd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 -- 172.21.15.100:0/1177536315 shutdown_connections 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 -- 172.21.15.100:0/1177536315 wait complete. 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 -- start start 2024-03-12T05:15:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 Processor -- start 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa4fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa130430 con 0x5579aa4fe7b0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa0ece00 con 0x5579aa4f5bd0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa3e1fd0 con 0x5579aa4f52b0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59aa3e5740 10 client.0 _ll_register_callbacks cb 0x5579aa437fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa4fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59a54d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa4fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33822/0 (socket says 172.21.15.100:33822) 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2964104240 learned_addr learned my addr 172.21.15.100:0/2964104240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.385+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2964104240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 msgr2=0x5579aa4fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 msgr2=0x5579aa4fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59a5cd7640 1 -- 172.21.15.100:0/2964104240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a01a6540 con 0x5579aa4fe7b0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa4fba50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5998004af0 tx=0x7f59980bc9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35331fe88a1f421 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f5988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f5988ff9640 1 -- 172.21.15.100:0/2964104240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980bd670 con 0x5579aa4fe7b0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f5988ff9640 1 -- 172.21.15.100:0/2964104240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59980c5710 con 0x5579aa4fe7b0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579aa15c7e0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f5988ff9640 1 -- 172.21.15.100:0/2964104240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59980c10c0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 10 client.16308 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5579aa45b610 con 0x5579aa4fe7b0 2024-03-12T05:15:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f5988ff9640 1 -- 172.21.15.100:0/2964104240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59980c9020 con 0x5579aa4fe7b0 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f5988ff9640 1 -- 172.21.15.100:0/2964104240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f59980bd1d0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 10 client.16308 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 20 client.16308 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa50e1e0 0x5579aa510600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 4 client.16308 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.386+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579aa199720 con 0x5579aa50e1e0 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.387+0000 7f59a54d6640 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa50e1e0 0x5579aa510600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.387+0000 7f5988ff9640 10 client.16308 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.387+0000 7f59a54d6640 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa50e1e0 0x5579aa510600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f1bce3178c7e91 server_cookie=26f302199c52531f in_seq=0 out_seq=0 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f5988ff9640 1 -- 172.21.15.100:0/2964104240 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579aa199720 con 0x5579aa50e1e0 2024-03-12T05:15:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f5988ff9640 10 client.16308 handle_command_reply: tid=0 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 client.16308 shutdown 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 msgr2=0x5579aa4fba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa4fba50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5998004af0 tx=0x7f59980bc9c0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 shutdown_connections 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa50e1e0 0x5579aa510600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa4fba50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f5988ff9640 0 client.16308 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2964104240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 >> 172.21.15.100:0/2964104240 conn(0x5579aa4ef990 msgr2=0x5579aa4f1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 shutdown_connections 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2964104240 wait complete. 2024-03-12T05:15:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.388+0000 7f59aa3e5740 20 client.16308 trim_cache size 0 max 16384 2024-03-12T05:15:44.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.403+0000 7f59aa3e5740 1 Processor -- start 2024-03-12T05:15:44.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.403+0000 7f59aa3e5740 1 -- start start 2024-03-12T05:15:44.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa505410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa515840 con 0x5579aa4fe7b0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa4e9e20 con 0x5579aa4f5bd0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa4f9a20 con 0x5579aa4f52b0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa505410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa505410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33830/0 (socket says 172.21.15.100:33830) 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 -- 172.21.15.100:0/4285576021 learned_addr learned my addr 172.21.15.100:0/4285576021 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a54d6640 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 -- 172.21.15.100:0/4285576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 msgr2=0x5579aa4fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 -- 172.21.15.100:0/4285576021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 msgr2=0x5579aa4fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 -- 172.21.15.100:0/4285576021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579aa3f7ac0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa505410 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f59980edf80 tx=0x7f59980bb480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaa6aac11dc9041 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f598affd640 1 -- 172.21.15.100:0/4285576021 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980bd670 con 0x5579aa4fe7b0 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f598affd640 1 -- 172.21.15.100:0/4285576021 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59980c9070 con 0x5579aa4fe7b0 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.404+0000 7f598affd640 1 -- 172.21.15.100:0/4285576021 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980bd3a0 con 0x5579aa4fe7b0 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 -- 172.21.15.100:0/4285576021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 msgr2=0x5579aa505410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa505410 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f59980edf80 tx=0x7f59980bb480 comp rx=0 tx=0).stop 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 -- 172.21.15.100:0/4285576021 shutdown_connections 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4fe7b0 0x5579aa505410 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa4fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/4285576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4f52b0 0x5579aa4fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 -- 172.21.15.100:0/4285576021 >> 172.21.15.100:0/4285576021 conn(0x5579aa4ef990 msgr2=0x5579aa4f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 -- 172.21.15.100:0/4285576021 shutdown_connections 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 -- 172.21.15.100:0/4285576021 wait complete. 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 -- start start 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.405+0000 7f59aa3e5740 1 Processor -- start 2024-03-12T05:15:44.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 0x5579aa5058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa501c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4fe7b0 0x5579aa502260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa502960 con 0x5579aa4f52b0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa502ad0 con 0x5579aa4f5bd0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa564710 con 0x5579aa4fe7b0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 10 client.0 _ll_register_callbacks cb 0x7f5998004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 0x5579aa5058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 0x5579aa5058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33836/0 (socket says 172.21.15.100:33836) 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 -- 172.21.15.100:0/2253415432 learned_addr learned my addr 172.21.15.100:0/2253415432 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a5cd7640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4fe7b0 0x5579aa502260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa501c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 -- 172.21.15.100:0/2253415432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4fe7b0 msgr2=0x5579aa502260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4fe7b0 0x5579aa502260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 -- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 msgr2=0x5579aa501c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa501c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 -- 172.21.15.100:0/2253415432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579aa45b610 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59a54d6640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 0x5579aa5058d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f599c009d70 tx=0x7f599c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907c1c58c6641c7a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f5988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f5988ff9640 1 -- 172.21.15.100:0/2253415432 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c017040 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f5988ff9640 1 -- 172.21.15.100:0/2253415432 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f599c003a00 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.406+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59980cf9a0 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f5988ff9640 1 -- 172.21.15.100:0/2253415432 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599c003bc0 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 10 client.16314 fetch_fsmap learned FSMap version 24 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5579aa41d6a0 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f5988ff9640 1 -- 172.21.15.100:0/2253415432 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59980cf9a0 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f5988ff9640 1 -- 172.21.15.100:0/2253415432 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f599c028420 con 0x5579aa4f52b0 2024-03-12T05:15:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 10 client.16314 fetch_fsmap finished waiting for FSMap version 24 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 20 client.16314 populate_metadata read hostname 'smithi100' 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa565f70 0x5579aa568390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 4 client.16314 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5579aa1e8890 con 0x5579aa565f70 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa565f70 0x5579aa568390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f5988ff9640 10 client.16314 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.407+0000 7f59a4cd5640 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa565f70 0x5579aa568390 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879b13470db33e2a server_cookie=d7c63c095009a511 in_seq=0 out_seq=0 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f5988ff9640 1 -- 172.21.15.100:0/2253415432 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5579aa1e8890 con 0x5579aa565f70 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f5988ff9640 10 client.16314 handle_command_reply: tid=0 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 client.16314 shutdown 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 msgr2=0x5579aa5058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 0x5579aa5058d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f599c009d70 tx=0x7f599c0044a0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 shutdown_connections 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5579aa565f70 0x5579aa568390 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5579aa4fe7b0 0x5579aa502260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f5988ff9640 0 client.16314 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579aa4f5bd0 0x5579aa501c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 --2- 172.21.15.100:0/2253415432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579aa4f52b0 0x5579aa5058d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 >> 172.21.15.100:0/2253415432 conn(0x5579aa4ef990 msgr2=0x5579aa4ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 shutdown_connections 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 1 -- 172.21.15.100:0/2253415432 wait complete. 2024-03-12T05:15:44.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59aa3e5740 20 client.16314 trim_cache size 0 max 16384 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "99b72903-b5fb-4c33-a5d4-ceee984ac622": { 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "99b72903-b5fb-4c33-a5d4-ceee984ac622", 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:15:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:15:44.414 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f598c0a1c20 msgr2=0x7f598c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f598c0a1c20 0x7f598c0a40e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f599000a500 tx=0x7f599000aa00 comp rx=0 tx=0).stop 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 msgr2=0x7f59a0100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a0100a90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f599802c790 tx=0x7f5998034760 comp rx=0 tx=0).stop 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 shutdown_connections 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f598c0a1c20 0x7f598c0a40e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a010b700 0x7f59a0100a90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59a0102ce0 0x7f59a0100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 --2- 172.21.15.100:0/2220583842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59a01023c0 0x7f59a0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:15:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.408+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 >> 172.21.15.100:0/2220583842 conn(0x7f59a0077230 msgr2=0x7f59a00fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:15:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.409+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 shutdown_connections 2024-03-12T05:15:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:15:44.409+0000 7f59a773a640 1 -- 172.21.15.100:0/2220583842 wait complete. 2024-03-12T05:15:44.416 INFO:tasks.cephfs.filesystem:scrub status for tag:99b72903-b5fb-4c33-a5d4-ceee984ac622 - {'path': '/', 'tag': '99b72903-b5fb-4c33-a5d4-ceee984ac622', 'options': 'recursive,force'} 2024-03-12T05:15:44.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 378933 13.64 MB/sec execute 28 sec latency 31.003 ms 2024-03-12T05:15:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: pgmap v112: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-12T05:15:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:15:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:15:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:15:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:44 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: pgmap v112: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:44 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: pgmap v112: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-12T05:15:45.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:15:45.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:15:45.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:15:45.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:44 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:15:45.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 380667 13.46 MB/sec execute 29 sec latency 120.754 ms 2024-03-12T05:15:46.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 383554 13.58 MB/sec execute 30 sec latency 22.068 ms 2024-03-12T05:15:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:46 smithi156 ceph-mon[36270]: pgmap v113: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-12T05:15:47.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:46 smithi161 ceph-mon[36043]: pgmap v113: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-12T05:15:47.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:46 smithi100 ceph-mon[32210]: pgmap v113: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-12T05:15:47.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 386627 13.71 MB/sec execute 31 sec latency 16.249 ms 2024-03-12T05:15:48.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 389124 13.56 MB/sec execute 32 sec latency 7.678 ms 2024-03-12T05:15:49.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:48 smithi156 ceph-mon[36270]: pgmap v114: 97 pgs: 97 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-03-12T05:15:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:48 smithi161 ceph-mon[36043]: pgmap v114: 97 pgs: 97 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-03-12T05:15:49.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:48 smithi100 ceph-mon[32210]: pgmap v114: 97 pgs: 97 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-03-12T05:15:49.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 391355 13.57 MB/sec execute 33 sec latency 12.989 ms 2024-03-12T05:15:50.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:49 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:15:50.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:49 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:15:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:49 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:15:50.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 394091 13.66 MB/sec execute 34 sec latency 52.699 ms 2024-03-12T05:15:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:50 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:15:51.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:50 smithi156 ceph-mon[36270]: pgmap v115: 97 pgs: 97 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:51.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:50 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:15:51.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:50 smithi161 ceph-mon[36043]: pgmap v115: 97 pgs: 97 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:51.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:50 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:15:51.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:50 smithi100 ceph-mon[32210]: pgmap v115: 97 pgs: 97 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:15:51.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 396977 13.66 MB/sec execute 35 sec latency 17.473 ms 2024-03-12T05:15:52.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 399607 13.63 MB/sec execute 36 sec latency 14.348 ms 2024-03-12T05:15:53.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:52 smithi156 ceph-mon[36270]: pgmap v116: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-03-12T05:15:53.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:52 smithi161 ceph-mon[36043]: pgmap v116: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-03-12T05:15:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:52 smithi100 ceph-mon[32210]: pgmap v116: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-03-12T05:15:53.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 401838 13.62 MB/sec execute 37 sec latency 7.656 ms 2024-03-12T05:15:54.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 404795 13.74 MB/sec execute 38 sec latency 14.474 ms 2024-03-12T05:15:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:54 smithi156 ceph-mon[36270]: pgmap v117: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-03-12T05:15:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:54 smithi161 ceph-mon[36043]: pgmap v117: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-03-12T05:15:55.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:54 smithi100 ceph-mon[32210]: pgmap v117: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-03-12T05:15:55.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 407607 13.73 MB/sec execute 39 sec latency 34.709 ms 2024-03-12T05:15:56.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 409384 13.66 MB/sec execute 40 sec latency 109.755 ms 2024-03-12T05:15:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:56 smithi156 ceph-mon[36270]: pgmap v118: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-03-12T05:15:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:56 smithi161 ceph-mon[36043]: pgmap v118: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-03-12T05:15:57.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:56 smithi100 ceph-mon[32210]: pgmap v118: 97 pgs: 97 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-03-12T05:15:57.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 412173 13.68 MB/sec execute 41 sec latency 15.640 ms 2024-03-12T05:15:58.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 415313 13.79 MB/sec execute 42 sec latency 33.680 ms 2024-03-12T05:15:59.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:15:58 smithi156 ceph-mon[36270]: pgmap v119: 97 pgs: 97 active+clean; 218 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-03-12T05:15:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:15:58 smithi161 ceph-mon[36043]: pgmap v119: 97 pgs: 97 active+clean; 218 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-03-12T05:15:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:15:58 smithi100 ceph-mon[32210]: pgmap v119: 97 pgs: 97 active+clean; 218 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-03-12T05:15:59.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 417972 13.77 MB/sec execute 43 sec latency 41.666 ms 2024-03-12T05:16:00.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 419614 13.64 MB/sec execute 44 sec latency 29.921 ms 2024-03-12T05:16:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:00 smithi156 ceph-mon[36270]: pgmap v120: 97 pgs: 97 active+clean; 218 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:16:01.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:00 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:16:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:00 smithi161 ceph-mon[36043]: pgmap v120: 97 pgs: 97 active+clean; 218 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:16:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:00 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:16:01.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:00 smithi100 ceph-mon[32210]: pgmap v120: 97 pgs: 97 active+clean; 218 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-03-12T05:16:01.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:00 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:16:01.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 422201 13.70 MB/sec execute 45 sec latency 53.900 ms 2024-03-12T05:16:02.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 424517 13.58 MB/sec execute 46 sec latency 18.588 ms 2024-03-12T05:16:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:02 smithi156 ceph-mon[36270]: pgmap v121: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 388 op/s 2024-03-12T05:16:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:02 smithi161 ceph-mon[36043]: pgmap v121: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 388 op/s 2024-03-12T05:16:03.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:02 smithi100 ceph-mon[32210]: pgmap v121: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 388 op/s 2024-03-12T05:16:03.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 426746 13.59 MB/sec execute 47 sec latency 6.830 ms 2024-03-12T05:16:04.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 429654 13.67 MB/sec execute 48 sec latency 15.085 ms 2024-03-12T05:16:05.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:04 smithi156 ceph-mon[36270]: pgmap v122: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-03-12T05:16:05.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:04 smithi161 ceph-mon[36043]: pgmap v122: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-03-12T05:16:05.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:04 smithi100 ceph-mon[32210]: pgmap v122: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-03-12T05:16:05.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 432598 13.69 MB/sec execute 49 sec latency 7.808 ms 2024-03-12T05:16:06.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 434113 13.58 MB/sec execute 50 sec latency 48.413 ms 2024-03-12T05:16:07.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:06 smithi156 ceph-mon[36270]: pgmap v123: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-03-12T05:16:07.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:06 smithi161 ceph-mon[36043]: pgmap v123: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-03-12T05:16:07.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:06 smithi100 ceph-mon[32210]: pgmap v123: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 270 op/s 2024-03-12T05:16:07.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 436904 13.61 MB/sec execute 51 sec latency 6.742 ms 2024-03-12T05:16:08.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 440067 13.71 MB/sec execute 52 sec latency 17.340 ms 2024-03-12T05:16:09.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:08 smithi156 ceph-mon[36270]: pgmap v124: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 387 op/s 2024-03-12T05:16:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:08 smithi161 ceph-mon[36043]: pgmap v124: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 387 op/s 2024-03-12T05:16:09.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:08 smithi100 ceph-mon[32210]: pgmap v124: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 387 op/s 2024-03-12T05:16:09.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 442451 13.61 MB/sec execute 53 sec latency 15.503 ms 2024-03-12T05:16:10.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 444804 13.66 MB/sec execute 54 sec latency 7.540 ms 2024-03-12T05:16:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:10 smithi156 ceph-mon[36270]: pgmap v125: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-03-12T05:16:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:10 smithi161 ceph-mon[36043]: pgmap v125: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-03-12T05:16:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:10 smithi100 ceph-mon[32210]: pgmap v125: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-03-12T05:16:11.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 447316 13.67 MB/sec execute 55 sec latency 74.641 ms 2024-03-12T05:16:12.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 450283 13.68 MB/sec execute 56 sec latency 7.014 ms 2024-03-12T05:16:13.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:12 smithi156 ceph-mon[36270]: pgmap v126: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-03-12T05:16:13.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:12 smithi161 ceph-mon[36043]: pgmap v126: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-03-12T05:16:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:12 smithi100 ceph-mon[32210]: pgmap v126: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-03-12T05:16:13.787 INFO:tasks.workunit.client.0.smithi100.stdout: 1 452489 13.62 MB/sec execute 57 sec latency 26.486 ms 2024-03-12T05:16:14.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:16:14.417 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:16:14.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 -- 172.21.15.100:0/3113059218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c105380 msgr2=0x7fc46c10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 --2- 172.21.15.100:0/3113059218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c105380 0x7fc46c10cf50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc464009c10 tx=0x7fc46402bef0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 -- 172.21.15.100:0/3113059218 shutdown_connections 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 --2- 172.21.15.100:0/3113059218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c105380 0x7fc46c10cf50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 --2- 172.21.15.100:0/3113059218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100c60 0x7fc46c104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 --2- 172.21.15.100:0/3113059218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c1002b0 0x7fc46c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 -- 172.21.15.100:0/3113059218 >> 172.21.15.100:0/3113059218 conn(0x7fc46c077190 msgr2=0x7fc46c0775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 -- 172.21.15.100:0/3113059218 shutdown_connections 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.535+0000 7fc471c96640 1 -- 172.21.15.100:0/3113059218 wait complete. 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 Processor -- start 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 -- start start 2024-03-12T05:16:14.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c1002b0 0x7fc46c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c105380 0x7fc46c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 0x7fc46c1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a5fd0 con 0x7fc46c1027c0 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a6140 con 0x7fc46c1002b0 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc471c96640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a6300 con 0x7fc46c105380 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 0x7fc46c1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 0x7fc46c1a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51838/0 (socket says 172.21.15.100:51838) 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 -- 172.21.15.100:0/1923020607 learned_addr learned my addr 172.21.15.100:0/1923020607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c1002b0 0x7fc46c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46affd640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c105380 0x7fc46c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 -- 172.21.15.100:0/1923020607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c105380 msgr2=0x7fc46c102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c105380 0x7fc46c102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 -- 172.21.15.100:0/1923020607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c1002b0 msgr2=0x7fc46c101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c1002b0 0x7fc46c101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.536+0000 7fc46bfff640 1 -- 172.21.15.100:0/1923020607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464009840 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.537+0000 7fc46bfff640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 0x7fc46c1a5a00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc46402c430 tx=0x7fc464033610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.537+0000 7fc468ff9640 1 -- 172.21.15.100:0/1923020607 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464003b70 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.537+0000 7fc468ff9640 1 -- 172.21.15.100:0/1923020607 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc46402ce40 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.537+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c1a6580 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.537+0000 7fc468ff9640 1 -- 172.21.15.100:0/1923020607 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46403d720 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.537+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c1a6a30 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.539+0000 7fc468ff9640 1 -- 172.21.15.100:0/1923020607 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc464004040 con 0x7fc46c1027c0 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 Processor -- start 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 -- start start 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc468ff9640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4540a1bd0 0x7fc4540a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 0x5594ffeae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffeb7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc46affd640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4540a1bd0 0x7fc4540a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeb77b0 0x5594ffeb9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffe1d280 con 0x5594ffeb77b0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffe1a7b0 con 0x5594ffeaebd0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc474862740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffe15f90 con 0x5594ffeae2b0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.540+0000 7fc468ff9640 1 -- 172.21.15.100:0/1923020607 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc4640efbd0 con 0x7fc46c1027c0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 0x5594ffeae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 0x5594ffeae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:36832/0 (socket says 172.21.15.100:36832) 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2150019445 learned_addr learned my addr 172.21.15.100:0/2150019445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46bfff640 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeb77b0 0x5594ffeb9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46affd640 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffeb7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2150019445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 msgr2=0x5594ffeb7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffeb7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2150019445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeb77b0 msgr2=0x5594ffeb9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeb77b0 0x5594ffeb9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2150019445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594ffdd4570 con 0x5594ffeae2b0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46bfff640 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeb77b0 0x5594ffeb9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46affd640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4540a1bd0 0x7fc4540a4090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc46c1037d0 tx=0x7fc460011810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 0x5594ffeae690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009cd0 tx=0x7fc45c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95d04832e3295cf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc451ffb640 1 -- 172.21.15.100:0/2150019445 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c009e80 con 0x5594ffeae2b0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc451ffb640 1 -- 172.21.15.100:0/2150019445 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0198f0 con 0x5594ffeae2b0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.541+0000 7fc451ffb640 1 -- 172.21.15.100:0/2150019445 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c018900 con 0x5594ffeae2b0 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 -- 172.21.15.100:0/2150019445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 msgr2=0x5594ffeae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 0x5594ffeae690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009cd0 tx=0x7fc45c011810 comp rx=0 tx=0).stop 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 -- 172.21.15.100:0/2150019445 shutdown_connections 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeb77b0 0x5594ffeb9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffeb7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 --2- 172.21.15.100:0/2150019445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeae2b0 0x5594ffeae690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 -- 172.21.15.100:0/2150019445 >> 172.21.15.100:0/2150019445 conn(0x5594ffea8990 msgr2=0x5594ffea8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 -- 172.21.15.100:0/2150019445 shutdown_connections 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 -- 172.21.15.100:0/2150019445 wait complete. 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.542+0000 7fc474862740 1 -- start start 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 Processor -- start 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeae2b0 0x5594ffeb71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 0x5594ffeb4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffeb4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffae9430 con 0x5594ffeaebd0 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffaa5e00 con 0x5594ffeae2b0 2024-03-12T05:16:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffd9afd0 con 0x5594ffeb77b0 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc474862740 10 client.0 _ll_register_callbacks cb 0x5594ffdf0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 0x5594ffeb4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeae2b0 0x5594ffeb71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 0x5594ffeb4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51856/0 (socket says 172.21.15.100:51856) 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 -- 172.21.15.100:0/1929496628 learned_addr learned my addr 172.21.15.100:0/1929496628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46bfff640 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffeb4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 -- 172.21.15.100:0/1929496628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 msgr2=0x5594ffeb4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffeb4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 -- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeae2b0 msgr2=0x5594ffeb71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeae2b0 0x5594ffeb71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 -- 172.21.15.100:0/1929496628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c009930 con 0x5594ffeaebd0 2024-03-12T05:16:14.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc46affd640 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 0x5594ffeb4550 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc460009d60 tx=0x7fc460004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb8377231a21b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.543+0000 7fc44affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc44affd640 1 -- 172.21.15.100:0/1929496628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460020d20 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc44affd640 1 -- 172.21.15.100:0/1929496628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc460028400 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594ffb157e0 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc44affd640 1 -- 172.21.15.100:0/1929496628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc460020e80 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 10 client.16326 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5594ffe14610 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc44affd640 1 -- 172.21.15.100:0/1929496628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5594ffb157e0 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc44affd640 1 -- 172.21.15.100:0/1929496628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc460026030 con 0x5594ffeaebd0 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 10 client.16326 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 20 client.16326 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:14.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594ffec7a00 0x5594ffec9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 4 client.16326 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:14.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594ffb52720 con 0x5594ffec7a00 2024-03-12T05:16:14.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.544+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594ffec7a00 0x5594ffec9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc44affd640 10 client.16326 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:14.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594ffec7a00 0x5594ffec9e20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704b69310ca23698 server_cookie=18ca741fc3f24c55 in_seq=0 out_seq=0 2024-03-12T05:16:14.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc44affd640 1 -- 172.21.15.100:0/1929496628 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594ffb52720 con 0x5594ffec7a00 2024-03-12T05:16:14.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc44affd640 10 client.16326 handle_command_reply: tid=0 2024-03-12T05:16:14.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc474862740 1 client.16326 shutdown 2024-03-12T05:16:14.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 msgr2=0x5594ffeb4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.545+0000 7fc474862740 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 0x5594ffeb4550 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc460009d60 tx=0x7fc460004730 comp rx=0 tx=0).stop 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 shutdown_connections 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594ffec7a00 0x5594ffec9e20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffeb4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeaebd0 0x5594ffeb4550 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc44affd640 0 client.16326 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 --2- 172.21.15.100:0/1929496628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeae2b0 0x5594ffeb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 >> 172.21.15.100:0/1929496628 conn(0x5594ffea8990 msgr2=0x5594ffeaa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 shutdown_connections 2024-03-12T05:16:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 1 -- 172.21.15.100:0/1929496628 wait complete. 2024-03-12T05:16:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.546+0000 7fc474862740 20 client.16326 trim_cache size 0 max 16384 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.562+0000 7fc474862740 1 Processor -- start 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.562+0000 7fc474862740 1 -- start start 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc474862740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffeb6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc474862740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeaebd0 0x5594ffeb71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc474862740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeb77b0 0x5594ffeb2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc474862740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffead680 con 0x5594ffeae2b0 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc474862740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffecf030 con 0x5594ffeb77b0 2024-03-12T05:16:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc474862740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffea2e20 con 0x5594ffeaebd0 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffeb6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffeb6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51866/0 (socket says 172.21.15.100:51866) 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 -- 172.21.15.100:0/3610561365 learned_addr learned my addr 172.21.15.100:0/3610561365 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46affd640 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeaebd0 0x5594ffeb71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46bfff640 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeb77b0 0x5594ffeb2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 -- 172.21.15.100:0/3610561365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeaebd0 msgr2=0x5594ffeb71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeaebd0 0x5594ffeb71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 -- 172.21.15.100:0/3610561365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeb77b0 msgr2=0x5594ffeb2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeb77b0 0x5594ffeb2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 -- 172.21.15.100:0/3610561365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594ffdb0ac0 con 0x5594ffeae2b0 2024-03-12T05:16:14.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46affd640 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeaebd0 0x5594ffeb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffeb6c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d24af24b25d4c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc4517fa640 1 -- 172.21.15.100:0/3610561365 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c028ea0 con 0x5594ffeae2b0 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc4517fa640 1 -- 172.21.15.100:0/3610561365 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45c028070 con 0x5594ffeae2b0 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.563+0000 7fc4517fa640 1 -- 172.21.15.100:0/3610561365 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c005600 con 0x5594ffeae2b0 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 -- 172.21.15.100:0/3610561365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 msgr2=0x5594ffeb6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffeb6c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c009460 comp rx=0 tx=0).stop 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 -- 172.21.15.100:0/3610561365 shutdown_connections 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeb77b0 0x5594ffeb2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeaebd0 0x5594ffeb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 --2- 172.21.15.100:0/3610561365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffeb6c80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 -- 172.21.15.100:0/3610561365 >> 172.21.15.100:0/3610561365 conn(0x5594ffea8990 msgr2=0x5594ffeaf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 -- 172.21.15.100:0/3610561365 shutdown_connections 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 -- 172.21.15.100:0/3610561365 wait complete. 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.564+0000 7fc474862740 1 -- start start 2024-03-12T05:16:14.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 Processor -- start 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffebea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffebae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffebb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffebba80 con 0x5594ffeae2b0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594ffebbbf0 con 0x5594ffeaebd0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594fff1df20 con 0x5594ffeb77b0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc474862740 10 client.0 _ll_register_callbacks cb 0x7fc45c0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffebea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffebea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51868/0 (socket says 172.21.15.100:51868) 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2660620572 learned_addr learned my addr 172.21.15.100:0/2660620572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46affd640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffebae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46bfff640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffebb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2660620572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 msgr2=0x5594ffebb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffebb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 msgr2=0x5594ffebae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffebae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46b7fe640 1 -- 172.21.15.100:0/2660620572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594ffe14610 con 0x5594ffeae2b0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.565+0000 7fc46affd640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffebae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc46b7fe640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffebea80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc45c01c040 tx=0x7fc45c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992dc9c3b57e0753 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc44affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc44affd640 1 -- 172.21.15.100:0/2660620572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c028ea0 con 0x5594ffeae2b0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc44affd640 1 -- 172.21.15.100:0/2660620572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc45c005280 con 0x5594ffeae2b0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc45c0198f0 con 0x5594ffeae2b0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc44affd640 1 -- 172.21.15.100:0/2660620572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc45c005c50 con 0x5594ffeae2b0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 10 client.16332 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5594ffdd4570 con 0x5594ffeae2b0 2024-03-12T05:16:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc44affd640 1 -- 172.21.15.100:0/2660620572 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0198f0 con 0x5594ffeae2b0 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc44affd640 1 -- 172.21.15.100:0/2660620572 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc45c0285b0 con 0x5594ffeae2b0 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 10 client.16332 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 20 client.16332 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594fff1f330 0x5594fff21750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 4 client.16332 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5594ffba1890 con 0x5594fff1f330 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.566+0000 7fc46affd640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594fff1f330 0x5594fff21750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc44affd640 10 client.16332 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc46affd640 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594fff1f330 0x5594fff21750 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab000daa75161de server_cookie=4f992f8e659f02f0 in_seq=0 out_seq=0 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc44affd640 1 -- 172.21.15.100:0/2660620572 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5594ffba1890 con 0x5594fff1f330 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc44affd640 10 client.16332 handle_command_reply: tid=0 2024-03-12T05:16:14.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 client.16332 shutdown 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 msgr2=0x5594ffebea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffebea80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc45c01c040 tx=0x7fc45c005970 comp rx=0 tx=0).stop 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 shutdown_connections 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5594fff1f330 0x5594fff21750 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc44affd640 0 client.16332 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ffeb77b0 0x5594ffebb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594ffeaebd0 0x5594ffebae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 --2- 172.21.15.100:0/2660620572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594ffeae2b0 0x5594ffebea80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 >> 172.21.15.100:0/2660620572 conn(0x5594ffea8990 msgr2=0x5594ffeb2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 shutdown_connections 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.567+0000 7fc474862740 1 -- 172.21.15.100:0/2660620572 wait complete. 2024-03-12T05:16:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc474862740 20 client.16332 trim_cache size 0 max 16384 2024-03-12T05:16:14.572 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:16:14.572 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:16:14.572 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:16:14.572 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4540a1bd0 msgr2=0x7fc4540a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4540a1bd0 0x7fc4540a4090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc46c1037d0 tx=0x7fc460011810 comp rx=0 tx=0).stop 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 msgr2=0x7fc46c1a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 0x7fc46c1a5a00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc46402c430 tx=0x7fc464033610 comp rx=0 tx=0).stop 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 shutdown_connections 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4540a1bd0 0x7fc4540a4090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1027c0 0x7fc46c1a5a00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c105380 0x7fc46c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 --2- 172.21.15.100:0/1923020607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c1002b0 0x7fc46c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 >> 172.21.15.100:0/1923020607 conn(0x7fc46c077190 msgr2=0x7fc46c0697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 shutdown_connections 2024-03-12T05:16:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.568+0000 7fc471c96640 1 -- 172.21.15.100:0/1923020607 wait complete. 2024-03-12T05:16:14.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:16:14.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:16:14.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 -- 172.21.15.100:0/640753669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98100360 msgr2=0x7fba981007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 --2- 172.21.15.100:0/640753669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98100360 0x7fba981007e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fba88009900 tx=0x7fba8802bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 -- 172.21.15.100:0/640753669 shutdown_connections 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 --2- 172.21.15.100:0/640753669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100d20 0x7fba9810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 --2- 172.21.15.100:0/640753669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98100360 0x7fba981007e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 --2- 172.21.15.100:0/640753669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98106130 0x7fba98106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 -- 172.21.15.100:0/640753669 >> 172.21.15.100:0/640753669 conn(0x7fba98077060 msgr2=0x7fba98077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 -- 172.21.15.100:0/640753669 shutdown_connections 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.693+0000 7fbaa000b640 1 -- 172.21.15.100:0/640753669 wait complete. 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 Processor -- start 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 -- start start 2024-03-12T05:16:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 0x7fba98196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98100d20 0x7fba98196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 0x7fba9819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819b510 con 0x7fba98106130 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819b680 con 0x7fba98100360 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fbaa000b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819b840 con 0x7fba98100d20 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fba9e5a8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 0x7fba9819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fba9dda7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 0x7fba98196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fba9e5a8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 0x7fba9819af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51894/0 (socket says 172.21.15.100:51894) 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fba9e5a8640 1 -- 172.21.15.100:0/4153061941 learned_addr learned my addr 172.21.15.100:0/4153061941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fba9dda7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 0x7fba98196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60470/0 (socket says 172.21.15.100:60470) 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.694+0000 7fba9d5a6640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98100d20 0x7fba98196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9e5a8640 1 -- 172.21.15.100:0/4153061941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98100d20 msgr2=0x7fba98196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9e5a8640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98100d20 0x7fba98196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9e5a8640 1 -- 172.21.15.100:0/4153061941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 msgr2=0x7fba98196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9e5a8640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 0x7fba98196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9e5a8640 1 -- 172.21.15.100:0/4153061941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009840 con 0x7fba98106130 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9d5a6640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98100d20 0x7fba98196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9dda7640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 0x7fba98196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba9e5a8640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 0x7fba9819af10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fba9000efd0 tx=0x7fba9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90009b40 con 0x7fba98106130 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba90003b70 con 0x7fba98106130 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900108f0 con 0x7fba98106130 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba88009560 con 0x7fba98106130 2024-03-12T05:16:14.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.695+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981a31c0 con 0x7fba98106130 2024-03-12T05:16:14.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.696+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba98101780 con 0x7fba98106130 2024-03-12T05:16:14.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.697+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fba90003ce0 con 0x7fba98106130 2024-03-12T05:16:14.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.698+0000 7fba8effd640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba840a1ba0 0x7fba840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:14.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.698+0000 7fba9dda7640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba840a1ba0 0x7fba840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:14.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.698+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fba90014070 con 0x7fba98106130 2024-03-12T05:16:14.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.699+0000 7fba9dda7640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba840a1ba0 0x7fba840a4060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba94009fc0 tx=0x7fba94005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:14.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.702+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba90097020 con 0x7fba98106130 2024-03-12T05:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 454828 13.63 MB/sec execute 58 sec latency 14.480 ms 2024-03-12T05:16:14.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.950+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba981979a0 con 0x7fba98106130 2024-03-12T05:16:14.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.951+0000 7fba8effd640 1 -- 172.21.15.100:0/4153061941 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fba9009a6e0 con 0x7fba98106130 2024-03-12T05:16:14.947 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:16:14.947 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:16:14.948 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:16:14.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba840a1ba0 msgr2=0x7fba840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba840a1ba0 0x7fba840a4060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba94009fc0 tx=0x7fba94005f90 comp rx=0 tx=0).stop 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 msgr2=0x7fba9819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 0x7fba9819af10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fba9000efd0 tx=0x7fba9000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 shutdown_connections 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba840a1ba0 0x7fba840a4060 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba98106130 0x7fba9819af10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba98100d20 0x7fba98196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 --2- 172.21.15.100:0/4153061941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba98100360 0x7fba98196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 >> 172.21.15.100:0/4153061941 conn(0x7fba98077060 msgr2=0x7fba980fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 shutdown_connections 2024-03-12T05:16:14.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:14.953+0000 7fbaa000b640 1 -- 172.21.15.100:0/4153061941 wait complete. 2024-03-12T05:16:14.961 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:16:14.961 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:16:15.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 -- 172.21.15.100:0/613942910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68105330 msgr2=0x7fca6810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/613942910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68105330 0x7fca6810cf00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca60009be0 tx=0x7fca6002bef0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 -- 172.21.15.100:0/613942910 shutdown_connections 2024-03-12T05:16:15.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/613942910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68105330 0x7fca6810cf00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/613942910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100c10 0x7fca68104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/613942910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca68100260 0x7fca68100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 -- 172.21.15.100:0/613942910 >> 172.21.15.100:0/613942910 conn(0x7fca68077180 msgr2=0x7fca68077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 -- 172.21.15.100:0/613942910 shutdown_connections 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.078+0000 7fca6f6d4640 1 -- 172.21.15.100:0/613942910 wait complete. 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 Processor -- start 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 -- start start 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 0x7fca68195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca68105330 0x7fca68195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68199e50 0x7fca6819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6819a840 con 0x7fca68100260 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6819a9b0 con 0x7fca68105330 2024-03-12T05:16:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6f6d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6819ab20 con 0x7fca68199e50 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 0x7fca68195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 0x7fca68195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51910/0 (socket says 172.21.15.100:51910) 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6cc6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca68105330 0x7fca68195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 -- 172.21.15.100:0/3311161404 learned_addr learned my addr 172.21.15.100:0/3311161404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6dc71640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68199e50 0x7fca6819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 -- 172.21.15.100:0/3311161404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68199e50 msgr2=0x7fca6819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68199e50 0x7fca6819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 -- 172.21.15.100:0/3311161404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca68105330 msgr2=0x7fca68195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca68105330 0x7fca68195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 -- 172.21.15.100:0/3311161404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5800d570 con 0x7fca68100260 2024-03-12T05:16:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.079+0000 7fca6d470640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 0x7fca68195590 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fca6400e9f0 tx=0x7fca6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.080+0000 7fca5e7fc640 1 -- 172.21.15.100:0/3311161404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64009720 con 0x7fca68100260 2024-03-12T05:16:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.080+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca60009840 con 0x7fca68100260 2024-03-12T05:16:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.080+0000 7fca5e7fc640 1 -- 172.21.15.100:0/3311161404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca6400cd80 con 0x7fca68100260 2024-03-12T05:16:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.080+0000 7fca5e7fc640 1 -- 172.21.15.100:0/3311161404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64010690 con 0x7fca68100260 2024-03-12T05:16:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.080+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca681a2420 con 0x7fca68100260 2024-03-12T05:16:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.082+0000 7fca5e7fc640 1 -- 172.21.15.100:0/3311161404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fca640046c0 con 0x7fca68100260 2024-03-12T05:16:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.082+0000 7fca7237f740 1 Processor -- start 2024-03-12T05:16:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.082+0000 7fca7237f740 1 -- start start 2024-03-12T05:16:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca7237f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca7237f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca7237f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca7237f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df76185960 con 0x55df761d3750 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca7237f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df76139310 con 0x55df761ca250 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca7237f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df76136840 con 0x55df761cab70 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6dc71640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51922/0 (socket says 172.21.15.100:51922) 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 -- 172.21.15.100:0/3745797077 learned_addr learned my addr 172.21.15.100:0/3745797077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6d470640 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca5e7fc640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fca540a1ba0 0x7fca540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 -- 172.21.15.100:0/3745797077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 msgr2=0x55df761d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 -- 172.21.15.100:0/3745797077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 msgr2=0x55df761ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 -- 172.21.15.100:0/3745797077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df760f4060 con 0x55df761d3750 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6dc71640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fca540a1ba0 0x7fca540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca5e7fc640 1 -- 172.21.15.100:0/3311161404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fca64014070 con 0x7fca68100260 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.083+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d5b40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d870 tx=0x7fca58009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cb5a503c015fe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca537fe640 1 -- 172.21.15.100:0/3745797077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58012040 con 0x55df761d3750 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca537fe640 1 -- 172.21.15.100:0/3745797077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca58009c40 con 0x55df761d3750 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca537fe640 1 -- 172.21.15.100:0/3745797077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58003dd0 con 0x55df761d3750 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 -- 172.21.15.100:0/3745797077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 msgr2=0x55df761d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca6dc71640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fca540a1ba0 0x7fca540a4060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca60005f00 tx=0x7fca60040040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d5b40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d870 tx=0x7fca58009260 comp rx=0 tx=0).stop 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 -- 172.21.15.100:0/3745797077 shutdown_connections 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d5b40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 --2- 172.21.15.100:0/3745797077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.084+0000 7fca7237f740 1 -- 172.21.15.100:0/3745797077 >> 172.21.15.100:0/3745797077 conn(0x55df761c4980 msgr2=0x55df761c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 -- 172.21.15.100:0/3745797077 shutdown_connections 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 -- 172.21.15.100:0/3745797077 wait complete. 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 -- start start 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 Processor -- start 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df76132020 con 0x55df761ca250 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df75e05430 con 0x55df761cab70 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df75dc1e00 con 0x55df761d3750 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca7237f740 10 client.0 _ll_register_callbacks cb 0x55df760d9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca6d470640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca6d470640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51924/0 (socket says 172.21.15.100:51924) 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.085+0000 7fca6d470640 1 -- 172.21.15.100:0/46314356 learned_addr learned my addr 172.21.15.100:0/46314356 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6dc71640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6d470640 1 -- 172.21.15.100:0/46314356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 msgr2=0x55df761d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6d470640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6d470640 1 -- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 msgr2=0x55df761d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6d470640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6d470640 1 -- 172.21.15.100:0/46314356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5800d270 con 0x55df761ca250 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca6d470640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761d0670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fca6409a4b0 tx=0x7fca6409a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3902f134fc499cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca50ff9640 1 -- 172.21.15.100:0/46314356 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6409e900 con 0x55df761ca250 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca50ff9640 1 -- 172.21.15.100:0/46314356 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fca640cde40 con 0x55df761ca250 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df75e317e0 con 0x55df761ca250 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca50ff9640 1 -- 172.21.15.100:0/46314356 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca640a2f50 con 0x55df761ca250 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca7237f740 10 client.16350 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55df761306a0 con 0x55df761ca250 2024-03-12T05:16:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca50ff9640 1 -- 172.21.15.100:0/46314356 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55df75e317e0 con 0x55df761ca250 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.086+0000 7fca50ff9640 1 -- 172.21.15.100:0/46314356 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca6401f060 con 0x55df761ca250 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca7237f740 10 client.16350 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca7237f740 10 client.16350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca7237f740 20 client.16350 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca7237f740 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df761e3a50 0x55df761e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca7237f740 4 client.16350 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df75e6e720 con 0x55df761e3a50 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df761e3a50 0x55df761e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca50ff9640 10 client.16350 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.087+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df761e3a50 0x55df761e5e60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f0a5233a10d750f server_cookie=4f031744ebce6097 in_seq=0 out_seq=0 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca50ff9640 1 -- 172.21.15.100:0/46314356 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df75e6e720 con 0x55df761e3a50 2024-03-12T05:16:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca50ff9640 10 client.16350 handle_command_reply: tid=0 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 client.16350 shutdown 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 msgr2=0x55df761d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761d0670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fca6409a4b0 tx=0x7fca6409a3b0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 shutdown_connections 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df761e3a50 0x55df761e5e60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca50ff9640 0 client.16350 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 --2- 172.21.15.100:0/46314356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761d0670 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 >> 172.21.15.100:0/46314356 conn(0x55df761c4980 msgr2=0x55df761c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 shutdown_connections 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 1 -- 172.21.15.100:0/46314356 wait complete. 2024-03-12T05:16:15.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.088+0000 7fca7237f740 20 client.16350 trim_cache size 0 max 16384 2024-03-12T05:16:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 Processor -- start 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 -- start start 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df760b7130 con 0x55df761ca250 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df761c9eb0 con 0x55df761cab70 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca7237f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df761c05f0 con 0x55df761d3750 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51936/0 (socket says 172.21.15.100:51936) 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 -- 172.21.15.100:0/187233895 learned_addr learned my addr 172.21.15.100:0/187233895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6dc71640 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 -- 172.21.15.100:0/187233895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 msgr2=0x55df761c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 -- 172.21.15.100:0/187233895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 msgr2=0x55df761d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 -- 172.21.15.100:0/187233895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df760ccc20 con 0x55df761ca250 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca6d470640 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761cf7d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fca6409a6f0 tx=0x7fca64005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df81d5f00c17109 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.101+0000 7fca52ffd640 1 -- 172.21.15.100:0/187233895 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64096a50 con 0x55df761ca250 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca52ffd640 1 -- 172.21.15.100:0/187233895 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca6409e900 con 0x55df761ca250 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca52ffd640 1 -- 172.21.15.100:0/187233895 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640b32d0 con 0x55df761ca250 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 -- 172.21.15.100:0/187233895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 msgr2=0x55df761cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761cf7d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fca6409a6f0 tx=0x7fca64005720 comp rx=0 tx=0).stop 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 -- 172.21.15.100:0/187233895 shutdown_connections 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761d3750 0x55df761c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761cab70 0x55df761d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 --2- 172.21.15.100:0/187233895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761ca250 0x55df761cf7d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 -- 172.21.15.100:0/187233895 >> 172.21.15.100:0/187233895 conn(0x55df761c4980 msgr2=0x55df761c32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 -- 172.21.15.100:0/187233895 shutdown_connections 2024-03-12T05:16:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.102+0000 7fca7237f740 1 -- 172.21.15.100:0/187233895 wait complete. 2024-03-12T05:16:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 -- start start 2024-03-12T05:16:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 Processor -- start 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761d9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7623a110 con 0x55df761d3750 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7623a280 con 0x55df761ca250 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7623a440 con 0x55df761cab70 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca7237f740 10 client.0 _ll_register_callbacks cb 0x7fca6400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6dc71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6dc71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51950/0 (socket says 172.21.15.100:51950) 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6d470640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761d9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6dc71640 1 -- 172.21.15.100:0/1577191088 learned_addr learned my addr 172.21.15.100:0/1577191088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6d470640 1 -- 172.21.15.100:0/1577191088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 msgr2=0x55df761d97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6d470640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6d470640 1 -- 172.21.15.100:0/1577191088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 msgr2=0x55df761d9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6d470640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6d470640 1 -- 172.21.15.100:0/1577191088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df761306a0 con 0x55df761ca250 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.103+0000 7fca6dc71640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca6d470640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761d9290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca64097180 tx=0x7fca6409aff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1034011a4b99336f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca50ff9640 1 -- 172.21.15.100:0/1577191088 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64096a50 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca50ff9640 1 -- 172.21.15.100:0/1577191088 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fca64099c00 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca58009c40 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca50ff9640 1 -- 172.21.15.100:0/1577191088 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca640ba3e0 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 10 client.25736 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55df760f28c0 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca50ff9640 1 -- 172.21.15.100:0/1577191088 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca58009c40 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca50ff9640 1 -- 172.21.15.100:0/1577191088 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca6409bce0 con 0x55df761ca250 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 10 client.25736 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 10 client.25736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 20 client.25736 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df7623b730 0x55df7623db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 4 client.25736 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df75ebd890 con 0x55df7623b730 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.104+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df7623b730 0x55df7623db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca6cc6f640 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df7623b730 0x55df7623db20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c44f08353011ec0 server_cookie=cb721c163361521 in_seq=0 out_seq=0 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca50ff9640 10 client.25736 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca50ff9640 1 -- 172.21.15.100:0/1577191088 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df75ebd890 con 0x55df7623b730 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca50ff9640 10 client.25736 handle_command_reply: tid=0 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 client.25736 shutdown 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 msgr2=0x55df761d9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761d9290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca64097180 tx=0x7fca6409aff0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 shutdown_connections 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55df7623b730 0x55df7623db20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df761d3750 0x55df761d9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df761cab70 0x55df761d97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca50ff9640 0 client.25736 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 --2- 172.21.15.100:0/1577191088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df761ca250 0x55df761d9290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 >> 172.21.15.100:0/1577191088 conn(0x55df761c4980 msgr2=0x55df761c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 shutdown_connections 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 1 -- 172.21.15.100:0/1577191088 wait complete. 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.105+0000 7fca7237f740 20 client.25736 trim_cache size 0 max 16384 2024-03-12T05:16:15.109 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fca540a1ba0 msgr2=0x7fca540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fca540a1ba0 0x7fca540a4060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca60005f00 tx=0x7fca60040040 comp rx=0 tx=0).stop 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 msgr2=0x7fca68195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 0x7fca68195590 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fca6400e9f0 tx=0x7fca6400edf0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 shutdown_connections 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fca540a1ba0 0x7fca540a4060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca68199e50 0x7fca6819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca68105330 0x7fca68195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 --2- 172.21.15.100:0/3311161404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68100260 0x7fca68195590 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 >> 172.21.15.100:0/3311161404 conn(0x7fca68077180 msgr2=0x7fca680fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 shutdown_connections 2024-03-12T05:16:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.106+0000 7fca6f6d4640 1 -- 172.21.15.100:0/3311161404 wait complete. 2024-03-12T05:16:15.113 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:16:15.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:16:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:14 smithi156 ceph-mon[36270]: pgmap v127: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-03-12T05:16:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:16:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:16:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:16:15.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:14 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:16:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:14 smithi161 ceph-mon[36043]: pgmap v127: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-03-12T05:16:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:16:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:16:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:16:15.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:14 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 -- 172.21.15.100:0/418939312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 msgr2=0x7f529c106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/418939312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c106620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f528c009900 tx=0x7f528c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 -- 172.21.15.100:0/418939312 shutdown_connections 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/418939312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100d20 0x7f529c10ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/418939312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100360 0x7f529c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/418939312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c106620 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 -- 172.21.15.100:0/418939312 >> 172.21.15.100:0/418939312 conn(0x7f529c077060 msgr2=0x7f529c077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 -- 172.21.15.100:0/418939312 shutdown_connections 2024-03-12T05:16:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 -- 172.21.15.100:0/418939312 wait complete. 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.231+0000 7f52a1b4b640 1 Processor -- start 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 -- start start 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100360 0x7f529c101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100d20 0x7f529c1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a5b60 con 0x7f529c106240 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a5cd0 con 0x7f529c100d20 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f52a1b4b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a5e90 con 0x7f529c100360 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100360 0x7f529c101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100d20 0x7f529c1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c1027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51970/0 (socket says 172.21.15.100:51970) 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 -- 172.21.15.100:0/671550155 learned_addr learned my addr 172.21.15.100:0/671550155 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529b7fe640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100360 0x7f529c101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:36956/0 (socket says 172.21.15.100:36956) 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 -- 172.21.15.100:0/671550155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100360 msgr2=0x7f529c101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100360 0x7f529c101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 -- 172.21.15.100:0/671550155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100d20 msgr2=0x7f529c1022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100d20 0x7f529c1022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 -- 172.21.15.100:0/671550155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c009560 con 0x7f529c106240 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529affd640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100d20 0x7f529c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.232+0000 7f529bfff640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c1027e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5294009fd0 tx=0x7f529400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.233+0000 7f5298ff9640 1 -- 172.21.15.100:0/671550155 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400bea0 con 0x7f529c106240 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.233+0000 7f5298ff9640 1 -- 172.21.15.100:0/671550155 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5294009940 con 0x7f529c106240 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.233+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c1a6170 con 0x7f529c106240 2024-03-12T05:16:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.233+0000 7f5298ff9640 1 -- 172.21.15.100:0/671550155 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940106c0 con 0x7f529c106240 2024-03-12T05:16:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.233+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1a6630 con 0x7f529c106240 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f5298ff9640 1 -- 172.21.15.100:0/671550155 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5294004240 con 0x7f529c106240 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 Processor -- start 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 -- start start 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472846630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472851b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563472801960 con 0x56347284f750 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634727b5310 con 0x563472846250 2024-03-12T05:16:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.235+0000 7f52a47f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634727b2840 con 0x563472846b70 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472851b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:36958/0 (socket says 172.21.15.100:36958) 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 -- 172.21.15.100:0/718748428 learned_addr learned my addr 172.21.15.100:0/718748428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529affd640 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472846630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f5298ff9640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52840a1ba0 0x7f52840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 -- 172.21.15.100:0/718748428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 msgr2=0x563472846630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472846630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 -- 172.21.15.100:0/718748428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 msgr2=0x563472851b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472851b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 -- 172.21.15.100:0/718748428 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563472770060 con 0x563472846b70 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529affd640 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472846630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f5298ff9640 1 -- 172.21.15.100:0/671550155 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f52940cd430 con 0x7f529c106240 2024-03-12T05:16:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529bfff640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52840a1ba0 0x7f52840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f529b7fe640 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284f210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f528c009900 tx=0x7f528c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41cf776bb015d1ad server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f5281ffb640 1 -- 172.21.15.100:0/718748428 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0048b0 con 0x563472846b70 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f5281ffb640 1 -- 172.21.15.100:0/718748428 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f528c004a10 con 0x563472846b70 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.236+0000 7f5281ffb640 1 -- 172.21.15.100:0/718748428 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0051f0 con 0x563472846b70 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 -- 172.21.15.100:0/718748428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 msgr2=0x56347284f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284f210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f528c009900 tx=0x7f528c0046c0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f529bfff640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52840a1ba0 0x7f52840a4060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5294096ad0 tx=0x7f5294004540 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 -- 172.21.15.100:0/718748428 shutdown_connections 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472851b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284f210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 --2- 172.21.15.100:0/718748428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472846630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 -- 172.21.15.100:0/718748428 >> 172.21.15.100:0/718748428 conn(0x563472840980 msgr2=0x563472840d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 -- 172.21.15.100:0/718748428 shutdown_connections 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 -- 172.21.15.100:0/718748428 wait complete. 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.237+0000 7f52a47f4740 1 -- start start 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 Processor -- start 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x56347284c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x56347284d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634727ae020 con 0x56347284f750 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563472481430 con 0x563472846250 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56347243de00 con 0x563472846b70 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f52a47f4740 10 client.0 _ll_register_callbacks cb 0x563472755cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x56347284d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x56347284d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51990/0 (socket says 172.21.15.100:51990) 2024-03-12T05:16:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529affd640 1 -- 172.21.15.100:0/2061714065 learned_addr learned my addr 172.21.15.100:0/2061714065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529affd640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x56347284c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529b7fe640 1 -- 172.21.15.100:0/2061714065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 msgr2=0x56347284c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529b7fe640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x56347284c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529b7fe640 1 -- 172.21.15.100:0/2061714065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 msgr2=0x56347284d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529b7fe640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x56347284d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529b7fe640 1 -- 172.21.15.100:0/2061714065 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c10d7a0 con 0x563472846b70 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.238+0000 7f529affd640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x56347284d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f529b7fe640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284cc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f528c004650 tx=0x7f528c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac274de5e055992 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f527affd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f527affd640 1 -- 172.21.15.100:0/2061714065 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0048b0 con 0x563472846b70 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f527affd640 1 -- 172.21.15.100:0/2061714065 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f528c005180 con 0x563472846b70 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634724ad7e0 con 0x563472846b70 2024-03-12T05:16:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f527affd640 1 -- 172.21.15.100:0/2061714065 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528c005340 con 0x563472846b70 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 10 client.25816 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634727ac6a0 con 0x563472846b70 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f527affd640 1 -- 172.21.15.100:0/2061714065 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5634724ad7e0 con 0x563472846b70 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f527affd640 1 -- 172.21.15.100:0/2061714065 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f528c03c050 con 0x563472846b70 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 10 client.25816 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 10 client.25816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 20 client.25816 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.239+0000 7f52a47f4740 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56347285f6c0 0x563472861ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.240+0000 7f52a47f4740 4 client.25816 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.240+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634724ea720 con 0x56347285f6c0 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.240+0000 7f529affd640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56347285f6c0 0x563472861ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.240+0000 7f527affd640 10 client.25816 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.240+0000 7f529affd640 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56347285f6c0 0x563472861ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1759663eff2a24d0 server_cookie=334aa37e053d7455 in_seq=0 out_seq=0 2024-03-12T05:16:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f527affd640 1 -- 172.21.15.100:0/2061714065 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634724ea720 con 0x56347285f6c0 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f527affd640 10 client.25816 handle_command_reply: tid=0 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 client.25816 shutdown 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 msgr2=0x56347284cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284cc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f528c004650 tx=0x7f528c000bc0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 shutdown_connections 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x56347284d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846b70 0x56347284cc30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56347285f6c0 0x563472861ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 --2- 172.21.15.100:0/2061714065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x56347284c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 >> 172.21.15.100:0/2061714065 conn(0x563472840980 msgr2=0x5634728426d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f527affd640 0 client.25816 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 shutdown_connections 2024-03-12T05:16:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 1 -- 172.21.15.100:0/2061714065 wait complete. 2024-03-12T05:16:15.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.241+0000 7f52a47f4740 20 client.25816 trim_cache size 0 max 16384 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.253+0000 7f52a47f4740 1 Processor -- start 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.253+0000 7f52a47f4740 1 -- start start 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 0x56347284ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846b70 0x56347284cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472849970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52a47f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563472733130 con 0x56347284f750 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52a47f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56347285c560 con 0x563472846b70 2024-03-12T05:16:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52a47f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56347283c5f0 con 0x563472846250 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472849970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 0x56347284ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 0x56347284ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:36968/0 (socket says 172.21.15.100:36968) 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 -- 172.21.15.100:0/1674784211 learned_addr learned my addr 172.21.15.100:0/1674784211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529b7fe640 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846b70 0x56347284cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 -- 172.21.15.100:0/1674784211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846b70 msgr2=0x56347284cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846b70 0x56347284cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 -- 172.21.15.100:0/1674784211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 msgr2=0x563472849970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472849970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 -- 172.21.15.100:0/1674784211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563472748c20 con 0x563472846250 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f529affd640 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 0x56347284ee20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5290012010 tx=0x7f5290004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94e5820a7c22e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52817fa640 1 -- 172.21.15.100:0/1674784211 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290026ea0 con 0x563472846250 2024-03-12T05:16:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52817fa640 1 -- 172.21.15.100:0/1674784211 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5290016040 con 0x563472846250 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.254+0000 7f52817fa640 1 -- 172.21.15.100:0/1674784211 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290004ea0 con 0x563472846250 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 -- 172.21.15.100:0/1674784211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 msgr2=0x56347284ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 0x56347284ee20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5290012010 tx=0x7f5290004730 comp rx=0 tx=0).stop 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 -- 172.21.15.100:0/1674784211 shutdown_connections 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56347284f750 0x563472849970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846b70 0x56347284cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1674784211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563472846250 0x56347284ee20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 -- 172.21.15.100:0/1674784211 >> 172.21.15.100:0/1674784211 conn(0x563472840980 msgr2=0x563472847990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 -- 172.21.15.100:0/1674784211 shutdown_connections 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 -- 172.21.15.100:0/1674784211 wait complete. 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.255+0000 7f52a47f4740 1 -- start start 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 Processor -- start 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472855110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 0x563472855650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56347284f750 0x563472855b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634728b5c50 con 0x563472846b70 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634728b5dc0 con 0x563472846250 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634728b5f80 con 0x56347284f750 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 10 client.0 _ll_register_callbacks cb 0x7f528c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 0x563472855650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 0x563472855650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52004/0 (socket says 172.21.15.100:52004) 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 -- 172.21.15.100:0/1064185308 learned_addr learned my addr 172.21.15.100:0/1064185308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529affd640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472855110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529affd640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56347284f750 0x563472855b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 -- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56347284f750 msgr2=0x563472855b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56347284f750 0x563472855b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 -- 172.21.15.100:0/1064185308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 msgr2=0x563472855110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472855110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 -- 172.21.15.100:0/1064185308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529002e040 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529affd640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472855110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f529b7fe640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 0x563472855650 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x56347284bf50 tx=0x7f528c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47260f830b2fe36 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f527affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f527affd640 1 -- 172.21.15.100:0/1064185308 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0048b0 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f527affd640 1 -- 172.21.15.100:0/1064185308 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f528c000e40 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.256+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f528c004a10 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f527affd640 1 -- 172.21.15.100:0/1064185308 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528c005d50 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 10 client.16374 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56347276e8c0 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f527affd640 1 -- 172.21.15.100:0/1064185308 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f528c004a10 con 0x563472846b70 2024-03-12T05:16:15.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f527affd640 1 -- 172.21.15.100:0/1064185308 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f528c03c040 con 0x563472846b70 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 10 client.16374 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 10 client.16374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 20 client.16374 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5634728b73d0 0x5634728b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 4 client.16374 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563472539890 con 0x5634728b73d0 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.257+0000 7f529affd640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5634728b73d0 0x5634728b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f527affd640 10 client.16374 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f529affd640 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5634728b73d0 0x5634728b97c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e688976388f76bde server_cookie=46a87f3d25d24db5 in_seq=0 out_seq=0 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f527affd640 1 -- 172.21.15.100:0/1064185308 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563472539890 con 0x5634728b73d0 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f527affd640 10 client.16374 handle_command_reply: tid=0 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 client.16374 shutdown 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 msgr2=0x563472855650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 0x563472855650 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x56347284bf50 tx=0x7f528c004b80 comp rx=0 tx=0).stop 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 shutdown_connections 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56347284f750 0x563472855b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563472846b70 0x563472855650 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5634728b73d0 0x5634728b97c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 --2- 172.21.15.100:0/1064185308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563472846250 0x563472855110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f527affd640 0 client.16374 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 >> 172.21.15.100:0/1064185308 conn(0x563472840980 msgr2=0x56347283f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 shutdown_connections 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 1 -- 172.21.15.100:0/1064185308 wait complete. 2024-03-12T05:16:15.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.258+0000 7f52a47f4740 20 client.16374 trim_cache size 0 max 16384 2024-03-12T05:16:15.262 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:16:15.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52840a1ba0 msgr2=0x7f52840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52840a1ba0 0x7f52840a4060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5294096ad0 tx=0x7f5294004540 comp rx=0 tx=0).stop 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 msgr2=0x7f529c1027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c1027e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5294009fd0 tx=0x7f529400b260 comp rx=0 tx=0).stop 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 shutdown_connections 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52840a1ba0 0x7f52840a4060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f529c106240 0x7f529c1027e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f529c100d20 0x7f529c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 --2- 172.21.15.100:0/671550155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529c100360 0x7f529c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 >> 172.21.15.100:0/671550155 conn(0x7f529c077060 msgr2=0x7f529c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 shutdown_connections 2024-03-12T05:16:15.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.259+0000 7f52a1b4b640 1 -- 172.21.15.100:0/671550155 wait complete. 2024-03-12T05:16:15.265 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:14 smithi100 ceph-mon[32210]: pgmap v127: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-03-12T05:16:15.265 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:16:15.265 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:16:15.265 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:16:15.265 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:14 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:16:15.266 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:16:15.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 -- 172.21.15.100:0/337772895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c105330 msgr2=0x7fa16c10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 --2- 172.21.15.100:0/337772895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c105330 0x7fa16c10cf00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa164009cd0 tx=0x7fa16402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 -- 172.21.15.100:0/337772895 shutdown_connections 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 --2- 172.21.15.100:0/337772895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c105330 0x7fa16c10cf00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 --2- 172.21.15.100:0/337772895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa16c100c10 0x7fa16c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 --2- 172.21.15.100:0/337772895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa16c100260 0x7fa16c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 -- 172.21.15.100:0/337772895 >> 172.21.15.100:0/337772895 conn(0x7fa16c077180 msgr2=0x7fa16c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 -- 172.21.15.100:0/337772895 shutdown_connections 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.380+0000 7fa1722f4640 1 -- 172.21.15.100:0/337772895 wait complete. 2024-03-12T05:16:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 Processor -- start 2024-03-12T05:16:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 -- start start 2024-03-12T05:16:15.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 0x7fa16c10d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa16c105330 0x7fa16c10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa16c10e260 0x7fa16c1a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1a5f30 con 0x7fa16c100260 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1a60a0 con 0x7fa16c105330 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa1722f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1a6260 con 0x7fa16c10e260 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.381+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 0x7fa16c10d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 0x7fa16c10d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52012/0 (socket says 172.21.15.100:52012) 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 -- 172.21.15.100:0/1859692662 learned_addr learned my addr 172.21.15.100:0/1859692662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa170891640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa16c10e260 0x7fa16c1a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa16c105330 0x7fa16c10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 -- 172.21.15.100:0/1859692662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa16c10e260 msgr2=0x7fa16c1a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa16c10e260 0x7fa16c1a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 -- 172.21.15.100:0/1859692662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa16c105330 msgr2=0x7fa16c10dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa16c105330 0x7fa16c10dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 -- 172.21.15.100:0/1859692662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164009900 con 0x7fa16c100260 2024-03-12T05:16:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa16bfff640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 0x7fa16c10d7e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c970 tx=0x7fa15c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa1697fa640 1 -- 172.21.15.100:0/1859692662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c007bb0 con 0x7fa16c100260 2024-03-12T05:16:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c1a6540 con 0x7fa16c100260 2024-03-12T05:16:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa1697fa640 1 -- 172.21.15.100:0/1859692662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c007d10 con 0x7fa16c100260 2024-03-12T05:16:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa1697fa640 1 -- 172.21.15.100:0/1859692662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00f450 con 0x7fa16c100260 2024-03-12T05:16:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.382+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c1a6a00 con 0x7fa16c100260 2024-03-12T05:16:15.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.384+0000 7fa1697fa640 1 -- 172.21.15.100:0/1859692662 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fa15c004f70 con 0x7fa16c100260 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 Processor -- start 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 -- start start 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f43630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f4c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f4c750 0x563742f4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742efe960 con 0x563742f43250 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742eb2310 con 0x563742f43b70 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa174fa3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742eaf840 con 0x563742f4c750 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa1697fa640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa1540a1ba0 0x7fa1540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f43630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f43630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52018/0 (socket says 172.21.15.100:52018) 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 -- 172.21.15.100:0/894413077 learned_addr learned my addr 172.21.15.100:0/894413077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa16bfff640 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f4c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f4c750 0x563742f4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa1540a1ba0 0x7fa1540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 -- 172.21.15.100:0/894413077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f4c750 msgr2=0x563742f4eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f4c750 0x563742f4eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 -- 172.21.15.100:0/894413077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 msgr2=0x563742f4c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f4c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa1697fa640 1 -- 172.21.15.100:0/1859692662 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa15c0cd4b0 con 0x7fa16c100260 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa170891640 1 -- 172.21.15.100:0/894413077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563742e6d060 con 0x563742f43250 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.385+0000 7fa16bfff640 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f4c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.386+0000 7fa170891640 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f43630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa16402c700 tx=0x7fa164034540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1785d6cf499ca043 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.386+0000 7fa1527fc640 1 -- 172.21.15.100:0/894413077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640328b0 con 0x563742f43250 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.386+0000 7fa1527fc640 1 -- 172.21.15.100:0/894413077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa164034e60 con 0x563742f43250 2024-03-12T05:16:15.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.386+0000 7fa170891640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa1540a1ba0 0x7fa1540a4060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa16404f000 tx=0x7fa16404b000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.386+0000 7fa1527fc640 1 -- 172.21.15.100:0/894413077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164004440 con 0x563742f43250 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 -- 172.21.15.100:0/894413077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 msgr2=0x563742f43630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f43630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa16402c700 tx=0x7fa164034540 comp rx=0 tx=0).stop 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 -- 172.21.15.100:0/894413077 shutdown_connections 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f4c750 0x563742f4eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f4c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 --2- 172.21.15.100:0/894413077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f43630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 -- 172.21.15.100:0/894413077 >> 172.21.15.100:0/894413077 conn(0x563742f3d980 msgr2=0x563742f3dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 -- 172.21.15.100:0/894413077 shutdown_connections 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 -- 172.21.15.100:0/894413077 wait complete. 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 -- start start 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.387+0000 7fa174fa3740 1 Processor -- start 2024-03-12T05:16:15.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f49740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43b70 0x563742f49c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f4c750 0x563742f4a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742eab020 con 0x563742f43250 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742b7e430 con 0x563742f4c750 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742b3ae00 con 0x563742f43b70 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f49740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f49740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52026/0 (socket says 172.21.15.100:52026) 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 10 client.0 _ll_register_callbacks cb 0x563742e52cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 -- 172.21.15.100:0/3052837347 learned_addr learned my addr 172.21.15.100:0/3052837347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43b70 0x563742f49c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa170891640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f4c750 0x563742f4a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 -- 172.21.15.100:0/3052837347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43b70 msgr2=0x563742f49c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43b70 0x563742f49c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 -- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f4c750 msgr2=0x563742f4a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f4c750 0x563742f4a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 -- 172.21.15.100:0/3052837347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c0682e0 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43b70 0x563742f49c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa16bfff640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f49740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0cac50 tx=0x7fa15c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a93556bd34ebb32 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa14b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa14b7fe640 1 -- 172.21.15.100:0/3052837347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c09e220 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa14b7fe640 1 -- 172.21.15.100:0/3052837347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa15c0a2870 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.388+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563742baa7e0 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa14b7fe640 1 -- 172.21.15.100:0/3052837347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15c005220 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 10 client.16386 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563742ea96a0 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa14b7fe640 1 -- 172.21.15.100:0/3052837347 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563742baa7e0 con 0x563742f43250 2024-03-12T05:16:15.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa14b7fe640 1 -- 172.21.15.100:0/3052837347 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa15c016030 con 0x563742f43250 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 10 client.16386 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 20 client.16386 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742f5cb50 0x563742f5ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 4 client.16386 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563742be7720 con 0x563742f5cb50 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.389+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742f5cb50 0x563742f5ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa14b7fe640 10 client.16386 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742f5cb50 0x563742f5ef40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330f6b2a264c2fd0 server_cookie=13db3ee76e90fb70 in_seq=0 out_seq=0 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa14b7fe640 1 -- 172.21.15.100:0/3052837347 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563742be7720 con 0x563742f5cb50 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa14b7fe640 10 client.16386 handle_command_reply: tid=0 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 client.16386 shutdown 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 msgr2=0x563742f49740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f49740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0cac50 tx=0x7fa15c00a570 comp rx=0 tx=0).stop 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 shutdown_connections 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742f5cb50 0x563742f5ef40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f4c750 0x563742f4a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa14b7fe640 0 client.16386 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43b70 0x563742f49c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 --2- 172.21.15.100:0/3052837347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f43250 0x563742f49740 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 >> 172.21.15.100:0/3052837347 conn(0x563742f3d980 msgr2=0x563742f3f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 shutdown_connections 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.390+0000 7fa174fa3740 1 -- 172.21.15.100:0/3052837347 wait complete. 2024-03-12T05:16:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.391+0000 7fa174fa3740 20 client.16386 trim_cache size 0 max 16384 2024-03-12T05:16:15.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 Processor -- start 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 -- start start 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f49fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742f469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742e30130 con 0x563742f4c750 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742f45340 con 0x563742f43b70 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.403+0000 7fa174fa3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742f39080 con 0x563742f43250 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742f469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742f469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52040/0 (socket says 172.21.15.100:52040) 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 -- 172.21.15.100:0/1311019331 learned_addr learned my addr 172.21.15.100:0/1311019331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f49fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 -- 172.21.15.100:0/1311019331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 msgr2=0x563742f4be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f4be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 -- 172.21.15.100:0/1311019331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 msgr2=0x563742f49fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f49fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 -- 172.21.15.100:0/1311019331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563742e45c20 con 0x563742f4c750 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa16bfff640 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f4be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa170891640 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742f469a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa164009cd0 tx=0x7fa164046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e20584542ac6fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa151ffb640 1 -- 172.21.15.100:0/1311019331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640323a0 con 0x563742f4c750 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa151ffb640 1 -- 172.21.15.100:0/1311019331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa164004660 con 0x563742f4c750 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.404+0000 7fa151ffb640 1 -- 172.21.15.100:0/1311019331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164003bd0 con 0x563742f4c750 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 -- 172.21.15.100:0/1311019331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 msgr2=0x563742f469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742f469a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa164009cd0 tx=0x7fa164046040 comp rx=0 tx=0).stop 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 -- 172.21.15.100:0/1311019331 shutdown_connections 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742f469a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f49fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 --2- 172.21.15.100:0/1311019331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f4be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 -- 172.21.15.100:0/1311019331 >> 172.21.15.100:0/1311019331 conn(0x563742f3d980 msgr2=0x563742f638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 -- 172.21.15.100:0/1311019331 shutdown_connections 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 -- 172.21.15.100:0/1311019331 wait complete. 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.405+0000 7fa174fa3740 1 -- start start 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 Processor -- start 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f52930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f52e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742fad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742fad5f0 con 0x563742f4c750 2024-03-12T05:16:15.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742fad730 con 0x563742f43b70 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742fad8f0 con 0x563742f43250 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa174fa3740 10 client.0 _ll_register_callbacks cb 0x7fa160005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f52930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa170891640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742fad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f52930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37040/0 (socket says 172.21.15.100:37040) 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 -- 172.21.15.100:0/914178212 learned_addr learned my addr 172.21.15.100:0/914178212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f52e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 -- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 msgr2=0x563742f52e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f52e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 -- 172.21.15.100:0/914178212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 msgr2=0x563742fad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742fad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa16bfff640 1 -- 172.21.15.100:0/914178212 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563742ea96a0 con 0x563742f43250 2024-03-12T05:16:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.406+0000 7fa170891640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742fad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa16bfff640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f52930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa15c09a110 tx=0x7fa15c099890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6603c1bf6cde07 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa14b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa14b7fe640 1 -- 172.21.15.100:0/914178212 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0a2870 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa14b7fe640 1 -- 172.21.15.100:0/914178212 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa15c09e220 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa164034e60 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa14b7fe640 1 -- 172.21.15.100:0/914178212 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15c099040 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 10 client.25837 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563742e6d060 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa14b7fe640 1 -- 172.21.15.100:0/914178212 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c0a6020 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa14b7fe640 1 -- 172.21.15.100:0/914178212 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa15c016040 con 0x563742f43250 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 10 client.25837 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 20 client.25837 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742fb4d90 0x563742fb7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 4 client.25837 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563742c36890 con 0x563742fb4d90 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.407+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742fb4d90 0x563742fb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa14b7fe640 10 client.25837 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa16b7fe640 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742fb4d90 0x563742fb7180 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e90486d6ac527ea8 server_cookie=8449f99c981f8ffa in_seq=0 out_seq=0 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa14b7fe640 1 -- 172.21.15.100:0/914178212 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563742c36890 con 0x563742fb4d90 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa14b7fe640 10 client.25837 handle_command_reply: tid=0 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 client.25837 shutdown 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 msgr2=0x563742f52930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f52930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa15c09a110 tx=0x7fa15c099890 comp rx=0 tx=0).stop 2024-03-12T05:16:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 shutdown_connections 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563742fb4d90 0x563742fb7180 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742f4c750 0x563742fad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa14b7fe640 0 client.25837 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563742f43b70 0x563742f52e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 --2- 172.21.15.100:0/914178212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563742f43250 0x563742f52930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 >> 172.21.15.100:0/914178212 conn(0x563742f3d980 msgr2=0x563742f3c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 shutdown_connections 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 1 -- 172.21.15.100:0/914178212 wait complete. 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.408+0000 7fa174fa3740 20 client.25837 trim_cache size 0 max 16384 2024-03-12T05:16:15.412 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:16:15.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa1540a1ba0 msgr2=0x7fa1540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa1540a1ba0 0x7fa1540a4060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa16404f000 tx=0x7fa16404b000 comp rx=0 tx=0).stop 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 msgr2=0x7fa16c10d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 0x7fa16c10d7e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c970 tx=0x7fa15c00ce30 comp rx=0 tx=0).stop 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 shutdown_connections 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa1540a1ba0 0x7fa1540a4060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa16c10e260 0x7fa16c1a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa16c105330 0x7fa16c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 --2- 172.21.15.100:0/1859692662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16c100260 0x7fa16c10d7e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 >> 172.21.15.100:0/1859692662 conn(0x7fa16c077180 msgr2=0x7fa16c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 shutdown_connections 2024-03-12T05:16:15.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:15.409+0000 7fa1722f4640 1 -- 172.21.15.100:0/1859692662 wait complete. 2024-03-12T05:16:15.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 457871 13.71 MB/sec execute 59 sec latency 13.965 ms 2024-03-12T05:16:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:15 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4153061941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:16:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:15 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4153061941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:16:16.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:15 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4153061941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:16:16.416 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:16:16.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:16:16.417 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 -- 172.21.15.100:0/2368259865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 msgr2=0x7f92fc1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 --2- 172.21.15.100:0/2368259865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc1007e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009900 tx=0x7f92f002bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 -- 172.21.15.100:0/2368259865 shutdown_connections 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 --2- 172.21.15.100:0/2368259865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92fc100d20 0x7f92fc10ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 --2- 172.21.15.100:0/2368259865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc1007e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 --2- 172.21.15.100:0/2368259865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc106240 0x7f92fc106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 -- 172.21.15.100:0/2368259865 >> 172.21.15.100:0/2368259865 conn(0x7f92fc077060 msgr2=0x7f92fc077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 -- 172.21.15.100:0/2368259865 shutdown_connections 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.534+0000 7f9301855640 1 -- 172.21.15.100:0/2368259865 wait complete. 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 Processor -- start 2024-03-12T05:16:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 -- start start 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc100d20 0x7f92fc196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92fc106240 0x7f92fc19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc19b520 con 0x7f92fc100360 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc19b690 con 0x7f92fc100d20 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f9301855640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc19b850 con 0x7f92fc106240 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f92faffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.535+0000 7f92faffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52060/0 (socket says 172.21.15.100:52060) 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 -- 172.21.15.100:0/2197374753 learned_addr learned my addr 172.21.15.100:0/2197374753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc100d20 0x7f92fc196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92fc106240 0x7f92fc19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 -- 172.21.15.100:0/2197374753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92fc106240 msgr2=0x7f92fc19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92fc106240 0x7f92fc19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 -- 172.21.15.100:0/2197374753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc100d20 msgr2=0x7f92fc196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc100d20 0x7f92fc196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 -- 172.21.15.100:0/2197374753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f0009560 con 0x7f92fc100360 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc100d20 0x7f92fc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f92faffd640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc196660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00e9f0 tx=0x7f92ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f9300853640 1 -- 172.21.15.100:0/2197374753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec009700 con 0x7f92fc100360 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f9300853640 1 -- 172.21.15.100:0/2197374753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92ec00cd80 con 0x7f92fc100360 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc19bb30 con 0x7f92fc100360 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f9300853640 1 -- 172.21.15.100:0/2197374753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec018920 con 0x7f92fc100360 2024-03-12T05:16:16.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.536+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc1a33b0 con 0x7f92fc100360 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f9300853640 1 -- 172.21.15.100:0/2197374753 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f92ec018a80 con 0x7f92fc100360 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 Processor -- start 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 -- start start 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf2863b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf28642aa0 0x55bf28642f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf285a9f90 con 0x55bf2863bba0 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf285a74c0 con 0x55bf28642aa0 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f930442e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf285a2ca0 con 0x55bf2863b1f0 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f9300853640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92e40a1ba0 0x7f92e40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f92faffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f92faffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52074/0 (socket says 172.21.15.100:52074) 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f92faffd640 1 -- 172.21.15.100:0/4227794621 learned_addr learned my addr 172.21.15.100:0/4227794621 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf28642aa0 0x55bf28642f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92e40a1ba0 0x7f92e40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf2863b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.539+0000 7f9300853640 1 -- 172.21.15.100:0/2197374753 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f92ec014070 con 0x7f92fc100360 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fa7fc640 1 -- 172.21.15.100:0/4227794621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 msgr2=0x55bf2863b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf2863b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fa7fc640 1 -- 172.21.15.100:0/4227794621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 msgr2=0x55bf286424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fa7fc640 1 -- 172.21.15.100:0/4227794621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf285650b0 con 0x55bf28642aa0 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92faffd640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf2863b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92e40a1ba0 0x7f92e40a4060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009270 tx=0x7f92f4009ba0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf28642aa0 0x55bf28642f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f002c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec5eb9d4cc96de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92e1ffb640 1 -- 172.21.15.100:0/4227794621 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f002c7e0 con 0x55bf28642aa0 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92e1ffb640 1 -- 172.21.15.100:0/4227794621 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f0004510 con 0x55bf28642aa0 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.540+0000 7f92e1ffb640 1 -- 172.21.15.100:0/4227794621 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f00054a0 con 0x55bf28642aa0 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 -- 172.21.15.100:0/4227794621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf28642aa0 msgr2=0x55bf28642f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf28642aa0 0x55bf28642f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f002c680 comp rx=0 tx=0).stop 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 -- 172.21.15.100:0/4227794621 shutdown_connections 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf28642aa0 0x55bf28642f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 --2- 172.21.15.100:0/4227794621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf2863b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 -- 172.21.15.100:0/4227794621 >> 172.21.15.100:0/4227794621 conn(0x55bf28635840 msgr2=0x55bf28635c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 -- 172.21.15.100:0/4227794621 shutdown_connections 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 -- 172.21.15.100:0/4227794621 wait complete. 2024-03-12T05:16:16.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 -- start start 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.541+0000 7f930442e740 1 Processor -- start 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf286464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863bba0 0x55bf28646a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf28646f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf28547f40 con 0x55bf28642aa0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf282764b0 con 0x55bf2863bba0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf28528670 con 0x55bf2863b1f0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 10 client.0 _ll_register_callbacks cb 0x55bf28595f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf28646f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf28646f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52082/0 (socket says 172.21.15.100:52082) 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 -- 172.21.15.100:0/3239087014 learned_addr learned my addr 172.21.15.100:0/3239087014 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863bba0 0x55bf28646a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92faffd640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf286464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 -- 172.21.15.100:0/3239087014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 msgr2=0x55bf286464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf286464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 -- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863bba0 msgr2=0x55bf28646a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863bba0 0x55bf28646a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 -- 172.21.15.100:0/3239087014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc10d7a0 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf28646f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f92f4010b70 tx=0x7f92f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430905fbfeacfa34 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92daffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92daffd640 1 -- 172.21.15.100:0/3239087014 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40045c0 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f92daffd640 1 -- 172.21.15.100:0/3239087014 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f92f4004720 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.542+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf2857cf40 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f92daffd640 1 -- 172.21.15.100:0/3239087014 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f4012020 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 10 client.16404 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bf285a1320 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f92daffd640 1 -- 172.21.15.100:0/3239087014 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bf2857cf40 con 0x55bf28642aa0 2024-03-12T05:16:16.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f92daffd640 1 -- 172.21.15.100:0/3239087014 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f92f4011440 con 0x55bf28642aa0 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 10 client.16404 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 20 client.16404 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf28659440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 4 client.16404 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf28599310 con 0x55bf28657020 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.543+0000 7f92faffd640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf28659440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f92daffd640 10 client.16404 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f92faffd640 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf28659440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e489eaf6a33148a server_cookie=2bc1683e1e408c1a in_seq=0 out_seq=0 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f92daffd640 1 -- 172.21.15.100:0/3239087014 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf28599310 con 0x55bf28657020 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f92daffd640 10 client.16404 handle_command_reply: tid=0 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 client.16404 shutdown 2024-03-12T05:16:16.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 msgr2=0x55bf28646f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf28646f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f92f4010b70 tx=0x7f92f4004ab0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 shutdown_connections 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf28659440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf28646f70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863bba0 0x55bf28646a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 --2- 172.21.15.100:0/3239087014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863b1f0 0x55bf286464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f92daffd640 0 client.16404 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.544+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 >> 172.21.15.100:0/3239087014 conn(0x55bf28635840 msgr2=0x55bf286377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.545+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 shutdown_connections 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.545+0000 7f930442e740 1 -- 172.21.15.100:0/3239087014 wait complete. 2024-03-12T05:16:16.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.545+0000 7f930442e740 20 client.16404 trim_cache size 0 max 16384 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 Processor -- start 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 -- start start 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf28648c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf28642aa0 0x55bf28643e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf2863c0f0 con 0x55bf2863bba0 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf2862fda0 con 0x55bf2863b1f0 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f930442e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf28655830 con 0x55bf28642aa0 2024-03-12T05:16:16.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f92fa7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f92fa7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52094/0 (socket says 172.21.15.100:52094) 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f92fa7fc640 1 -- 172.21.15.100:0/1115745654 learned_addr learned my addr 172.21.15.100:0/1115745654 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f92faffd640 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf28648c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.560+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf28642aa0 0x55bf28643e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92fa7fc640 1 -- 172.21.15.100:0/1115745654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf28642aa0 msgr2=0x55bf28643e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf28642aa0 0x55bf28643e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92fa7fc640 1 -- 172.21.15.100:0/1115745654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 msgr2=0x55bf28648c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf28648c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92fa7fc640 1 -- 172.21.15.100:0/1115745654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf2853dc00 con 0x55bf2863bba0 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286423a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f92f0004c50 tx=0x7f92f0005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82e79cda5c6bbba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92e17fa640 1 -- 172.21.15.100:0/1115745654 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f004f070 con 0x55bf2863bba0 2024-03-12T05:16:16.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92e17fa640 1 -- 172.21.15.100:0/1115745654 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f0004e70 con 0x55bf2863bba0 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.561+0000 7f92e17fa640 1 -- 172.21.15.100:0/1115745654 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0005510 con 0x55bf2863bba0 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- 172.21.15.100:0/1115745654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 msgr2=0x55bf286423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286423a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f92f0004c50 tx=0x7f92f0005080 comp rx=0 tx=0).stop 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- 172.21.15.100:0/1115745654 shutdown_connections 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf28642aa0 0x55bf28643e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf2863bba0 0x55bf286423a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- 172.21.15.100:0/1115745654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf28648c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- 172.21.15.100:0/1115745654 >> 172.21.15.100:0/1115745654 conn(0x55bf28635840 msgr2=0x55bf28641c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- 172.21.15.100:0/1115745654 shutdown_connections 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- 172.21.15.100:0/1115745654 wait complete. 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- start start 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 Processor -- start 2024-03-12T05:16:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf2863ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863bba0 0x55bf2863f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf286ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf2863fdb0 con 0x55bf28642aa0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf286ada80 con 0x55bf2863b1f0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf286adbc0 con 0x55bf2863bba0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.562+0000 7f930442e740 10 client.0 _ll_register_callbacks cb 0x7f92f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf286ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf2863ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92fa7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863bba0 0x55bf2863f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf2863ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60690/0 (socket says 172.21.15.100:60690) 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf286ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52098/0 (socket says 172.21.15.100:52098) 2024-03-12T05:16:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 -- 172.21.15.100:0/1955764613 learned_addr learned my addr 172.21.15.100:0/1955764613 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92fa7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863bba0 0x55bf2863f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37098/0 (socket says 172.21.15.100:37098) 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 -- 172.21.15.100:0/1955764613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863bba0 msgr2=0x55bf2863f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863bba0 0x55bf2863f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 -- 172.21.15.100:0/1955764613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 msgr2=0x55bf286ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf286ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 -- 172.21.15.100:0/1955764613 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf285650b0 con 0x55bf2863b1f0 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92fb7fe640 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf286ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92faffd640 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf2863ef70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec09a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ac8a3aee9f814f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92daffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92daffd640 1 -- 172.21.15.100:0/1955764613 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0a2fe0 con 0x55bf2863b1f0 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92daffd640 1 -- 172.21.15.100:0/1955764613 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f92ec0cb5e0 con 0x55bf2863b1f0 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf285c8900 con 0x55bf2863b1f0 2024-03-12T05:16:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f92daffd640 1 -- 172.21.15.100:0/1955764613 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92ec0cb7a0 con 0x55bf2863b1f0 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f930442e740 10 client.25790 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.563+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bf28563910 con 0x55bf2863b1f0 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f92daffd640 1 -- 172.21.15.100:0/1955764613 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bf285c8900 con 0x55bf2863b1f0 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f92daffd640 1 -- 172.21.15.100:0/1955764613 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f92ec0ba0f0 con 0x55bf2863b1f0 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f930442e740 10 client.25790 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f930442e740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f930442e740 20 client.25790 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f930442e740 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf286b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f930442e740 4 client.25790 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bf282a2860 con 0x55bf28657020 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf286b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f92daffd640 10 client.25790 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.564+0000 7f92fa7fc640 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf286b12d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0506f7d609552 server_cookie=8f14892ade2305bd in_seq=0 out_seq=0 2024-03-12T05:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.565+0000 7f92daffd640 1 -- 172.21.15.100:0/1955764613 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bf282a2860 con 0x55bf28657020 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.565+0000 7f92daffd640 10 client.25790 handle_command_reply: tid=0 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.565+0000 7f930442e740 1 client.25790 shutdown 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.565+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 msgr2=0x55bf2863ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.565+0000 7f930442e740 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf2863ef70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec09a740 comp rx=0 tx=0).stop 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 shutdown_connections 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf28657020 0x55bf286b12d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf28642aa0 0x55bf286ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf2863bba0 0x55bf2863f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 --2- 172.21.15.100:0/1955764613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf2863b1f0 0x55bf2863ef70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f92daffd640 0 client.25790 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 >> 172.21.15.100:0/1955764613 conn(0x55bf28635840 msgr2=0x55bf28634050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 shutdown_connections 2024-03-12T05:16:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 1 -- 172.21.15.100:0/1955764613 wait complete. 2024-03-12T05:16:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f930442e740 20 client.25790 trim_cache size 0 max 16384 2024-03-12T05:16:16.569 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:16:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:16:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "5b37910e-bdff-4a5d-8b26-fe5da8bdaba0", 2024-03-12T05:16:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:16:16.569 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:16:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92e40a1ba0 msgr2=0x7f92e40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92e40a1ba0 0x7f92e40a4060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009270 tx=0x7f92f4009ba0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 msgr2=0x7f92fc196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc196660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00e9f0 tx=0x7f92ec00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 shutdown_connections 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92e40a1ba0 0x7f92e40a4060 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92fc106240 0x7f92fc19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc100d20 0x7f92fc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 --2- 172.21.15.100:0/2197374753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92fc100360 0x7f92fc196660 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 >> 172.21.15.100:0/2197374753 conn(0x7f92fc077060 msgr2=0x7f92fc0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 shutdown_connections 2024-03-12T05:16:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.566+0000 7f9301855640 1 -- 172.21.15.100:0/2197374753 wait complete. 2024-03-12T05:16:16.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:16:16.573 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:16:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 -- 172.21.15.100:0/3622174424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50105340 msgr2=0x7f2f5010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/3622174424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50105340 0x7f2f5010cf10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009cd0 tx=0x7f2f48034900 comp rx=0 tx=0).stop 2024-03-12T05:16:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 -- 172.21.15.100:0/3622174424 shutdown_connections 2024-03-12T05:16:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/3622174424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50105340 0x7f2f5010cf10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/3622174424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100c20 0x7f2f50104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/3622174424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f50100270 0x7f2f50100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 -- 172.21.15.100:0/3622174424 >> 172.21.15.100:0/3622174424 conn(0x7f2f50077230 msgr2=0x7f2f50077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 -- 172.21.15.100:0/3622174424 shutdown_connections 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.687+0000 7f2f55cbb640 1 -- 172.21.15.100:0/3622174424 wait complete. 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 Processor -- start 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 -- start start 2024-03-12T05:16:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100270 0x7f2f50195520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f50105340 0x7f2f50195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 0x7f2f5019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019a7d0 con 0x7f2f50199de0 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019a940 con 0x7f2f50100270 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.688+0000 7f2f55cbb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019aab0 con 0x7f2f50105340 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 0x7f2f5019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f50105340 0x7f2f50195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 0x7f2f5019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52116/0 (socket says 172.21.15.100:52116) 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 -- 172.21.15.100:0/390666479 learned_addr learned my addr 172.21.15.100:0/390666479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100270 0x7f2f50195520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 -- 172.21.15.100:0/390666479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f50105340 msgr2=0x7f2f50195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f50105340 0x7f2f50195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 -- 172.21.15.100:0/390666479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100270 msgr2=0x7f2f50195520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100270 0x7f2f50195520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 -- 172.21.15.100:0/390666479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f48009900 con 0x7f2f50199de0 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100270 0x7f2f50195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4ffff640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 0x7f2f5019a290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009df0 tx=0x7f2f48009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4d7fa640 1 -- 172.21.15.100:0/390666479 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48039d60 con 0x7f2f50199de0 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f501a2090 con 0x7f2f50199de0 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4d7fa640 1 -- 172.21.15.100:0/390666479 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f4803ab40 con 0x7f2f50199de0 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f4d7fa640 1 -- 172.21.15.100:0/390666479 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f480046e0 con 0x7f2f50199de0 2024-03-12T05:16:16.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.689+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f501a2510 con 0x7f2f50199de0 2024-03-12T05:16:16.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f4d7fa640 1 -- 172.21.15.100:0/390666479 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2f4800a040 con 0x7f2f50199de0 2024-03-12T05:16:16.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 Processor -- start 2024-03-12T05:16:16.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 -- start start 2024-03-12T05:16:16.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a34690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c49a3280 con 0x5598c4a34bd0 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c49a07b0 con 0x5598c4a3d7b0 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f58885740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c499bf90 con 0x5598c4a342b0 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a34690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52132/0 (socket says 172.21.15.100:52132) 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/520396287 learned_addr learned my addr 172.21.15.100:0/520396287 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.692+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f4d7fa640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f3c0a1ba0 0x7f2f3c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f46ffd640 1 -- 172.21.15.100:0/520396287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a3d7b0 msgr2=0x5598c4a3fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f46ffd640 1 -- 172.21.15.100:0/520396287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 msgr2=0x5598c4a3d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f46ffd640 1 -- 172.21.15.100:0/520396287 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c495a570 con 0x5598c4a342b0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f4ffff640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f3c0a1ba0 0x7f2f3c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f4d7fa640 1 -- 172.21.15.100:0/390666479 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2f480f8c30 con 0x7f2f50199de0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a34690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f500ffaa0 tx=0x7f2f38005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c73b6c4effc0ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f45ffb640 1 -- 172.21.15.100:0/520396287 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38014070 con 0x5598c4a342b0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f45ffb640 1 -- 172.21.15.100:0/520396287 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f38015040 con 0x5598c4a342b0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f4ffff640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f3c0a1ba0 0x7f2f3c0a4060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2f480f6420 tx=0x7f2f48091030 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.693+0000 7f2f45ffb640 1 -- 172.21.15.100:0/520396287 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38004d00 con 0x5598c4a342b0 2024-03-12T05:16:16.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 -- 172.21.15.100:0/520396287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 msgr2=0x5598c4a34690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a34690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f500ffaa0 tx=0x7f2f38005f90 comp rx=0 tx=0).stop 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 -- 172.21.15.100:0/520396287 shutdown_connections 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 --2- 172.21.15.100:0/520396287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a34690 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 -- 172.21.15.100:0/520396287 >> 172.21.15.100:0/520396287 conn(0x5598c4a2e990 msgr2=0x5598c4a2ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 -- 172.21.15.100:0/520396287 shutdown_connections 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 -- 172.21.15.100:0/520396287 wait complete. 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 -- start start 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.694+0000 7f2f58885740 1 Processor -- start 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 0x5598c4a3a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c466f430 con 0x5598c4a3d7b0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c462be00 con 0x5598c4a34bd0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4920fd0 con 0x5598c4a342b0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 10 client.0 _ll_register_callbacks cb 0x5598c4976fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 0x5598c4a3a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 0x5598c4a3a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60736/0 (socket says 172.21.15.100:60736) 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52134/0 (socket says 172.21.15.100:52134) 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/3380358414 learned_addr learned my addr 172.21.15.100:0/3380358414 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/3380358414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 msgr2=0x5598c4a3d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a3d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/3380358414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a3d7b0 msgr2=0x5598c4a3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/3380358414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f380093f0 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a3d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 0x5598c4a3a5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f40015d80 tx=0x7f2f40013660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac2912c4c8a26a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f2effd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f2effd640 1 -- 172.21.15.100:0/3380358414 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40010040 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598c469b7e0 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.695+0000 7f2f2effd640 1 -- 172.21.15.100:0/3380358414 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2f4001b070 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f2effd640 1 -- 172.21.15.100:0/3380358414 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f4001f420 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 10 client.25802 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5598c499a610 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f2effd640 1 -- 172.21.15.100:0/3380358414 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f4002b080 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f2effd640 1 -- 172.21.15.100:0/3380358414 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f40027770 con 0x5598c4a34bd0 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 10 client.25802 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 20 client.25802 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4a4da30 0x5598c4a4fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 4 client.25802 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598c46d8720 con 0x5598c4a4da30 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4a4da30 0x5598c4a4fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f2effd640 10 client.25802 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.696+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4a4da30 0x5598c4a4fe50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca432a2cd637eee server_cookie=edbde06ce2c55a in_seq=0 out_seq=0 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.697+0000 7f2f2effd640 1 -- 172.21.15.100:0/3380358414 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598c46d8720 con 0x5598c4a4da30 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.697+0000 7f2f2effd640 10 client.25802 handle_command_reply: tid=0 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.697+0000 7f2f58885740 1 client.25802 shutdown 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 msgr2=0x5598c4a3a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 0x5598c4a3a5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f40015d80 tx=0x7f2f40013660 comp rx=0 tx=0).stop 2024-03-12T05:16:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 shutdown_connections 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4a4da30 0x5598c4a4fe50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a3d7b0 0x5598c4a3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a34bd0 0x5598c4a3a5b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 --2- 172.21.15.100:0/3380358414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a342b0 0x5598c4a3d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f2effd640 0 client.25802 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 >> 172.21.15.100:0/3380358414 conn(0x5598c4a2e990 msgr2=0x5598c4a30730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 shutdown_connections 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 1 -- 172.21.15.100:0/3380358414 wait complete. 2024-03-12T05:16:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.698+0000 7f2f58885740 20 client.25802 trim_cache size 0 max 16384 2024-03-12T05:16:16.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.713+0000 7f2f58885740 1 Processor -- start 2024-03-12T05:16:16.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.713+0000 7f2f58885740 1 -- start start 2024-03-12T05:16:16.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f58885740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f58885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f58885740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4a39e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f58885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4a49510 con 0x5598c4a34bd0 2024-03-12T05:16:16.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f58885740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4a55040 con 0x5598c4a342b0 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f58885740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4a38b00 con 0x5598c4a3d7b0 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4a39e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4a39e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:37136/0 (socket says 172.21.15.100:37136) 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 -- 172.21.15.100:0/2697076831 learned_addr learned my addr 172.21.15.100:0/2697076831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 -- 172.21.15.100:0/2697076831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 msgr2=0x5598c4a3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a3cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 -- 172.21.15.100:0/2697076831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 msgr2=0x5598c4a3d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f46ffd640 1 -- 172.21.15.100:0/2697076831 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c4936ac0 con 0x5598c4a3d7b0 2024-03-12T05:16:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.714+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:16.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.715+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4a39e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800ffd0 tx=0x7f2f380048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a648494d0e8999 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.715+0000 7f2f457fa640 1 -- 172.21.15.100:0/2697076831 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38014040 con 0x5598c4a3d7b0 2024-03-12T05:16:16.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.716+0000 7f2f457fa640 1 -- 172.21.15.100:0/2697076831 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f38034070 con 0x5598c4a3d7b0 2024-03-12T05:16:16.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.716+0000 7f2f457fa640 1 -- 172.21.15.100:0/2697076831 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3802e700 con 0x5598c4a3d7b0 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.716+0000 7f2f58885740 1 -- 172.21.15.100:0/2697076831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 msgr2=0x5598c4a39e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.716+0000 7f2f58885740 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4a39e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800ffd0 tx=0x7f2f380048e0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 -- 172.21.15.100:0/2697076831 shutdown_connections 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4a39e40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a3d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 --2- 172.21.15.100:0/2697076831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 -- 172.21.15.100:0/2697076831 >> 172.21.15.100:0/2697076831 conn(0x5598c4a2e990 msgr2=0x5598c4a35810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 -- 172.21.15.100:0/2697076831 shutdown_connections 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 -- 172.21.15.100:0/2697076831 wait complete. 2024-03-12T05:16:16.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 -- start start 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.717+0000 7f2f58885740 1 Processor -- start 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a41320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4aa3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4a41fd0 con 0x5598c4a34bd0 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4a42140 con 0x5598c4a342b0 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c4aa4470 con 0x5598c4a3d7b0 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f58885740 10 client.0 _ll_register_callbacks cb 0x7f2f3800f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a41860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52148/0 (socket says 172.21.15.100:52148) 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/631445388 learned_addr learned my addr 172.21.15.100:0/631445388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a41320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4aa3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/631445388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 msgr2=0x5598c4aa3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4aa3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 msgr2=0x5598c4a41320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a41320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f4f7fe640 1 -- 172.21.15.100:0/631445388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598c499a610 con 0x5598c4a34bd0 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.718+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a41320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f4f7fe640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a41860 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2f40015730 tx=0x7f2f40013fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa66e45708a24c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f2effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f2effd640 1 -- 172.21.15.100:0/631445388 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40005450 con 0x5598c4a34bd0 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f38015040 con 0x5598c4a34bd0 2024-03-12T05:16:16.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f2effd640 1 -- 172.21.15.100:0/631445388 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2f4001b040 con 0x5598c4a34bd0 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f2effd640 1 -- 172.21.15.100:0/631445388 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f40013b60 con 0x5598c4a34bd0 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f58885740 10 client.16428 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.719+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f2f3802ee70 con 0x5598c4a34bd0 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f2effd640 1 -- 172.21.15.100:0/631445388 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5598c469b7e0 con 0x5598c4a34bd0 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f2effd640 1 -- 172.21.15.100:0/631445388 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f4001f420 con 0x5598c4a34bd0 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f58885740 10 client.16428 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f58885740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f58885740 20 client.16428 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f58885740 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4aa5cc0 0x5598c4aa80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f58885740 4 client.16428 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f2f38015040 con 0x5598c4aa5cc0 2024-03-12T05:16:16.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.720+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4aa5cc0 0x5598c4aa80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.721+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4aa5cc0 0x5598c4aa80e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d6d88ecaee8f13 server_cookie=e5ed10b0c614e280 in_seq=0 out_seq=0 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.721+0000 7f2f2effd640 10 client.16428 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.721+0000 7f2f2effd640 1 -- 172.21.15.100:0/631445388 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f2f38015040 con 0x5598c4aa5cc0 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.721+0000 7f2f2effd640 10 client.16428 handle_command_reply: tid=0 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.721+0000 7f2f58885740 1 client.16428 shutdown 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 msgr2=0x5598c4a41860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a41860 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2f40015730 tx=0x7f2f40013fa0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 shutdown_connections 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5598c4aa5cc0 0x5598c4aa80e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598c4a3d7b0 0x5598c4aa3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598c4a34bd0 0x5598c4a41860 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 --2- 172.21.15.100:0/631445388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598c4a342b0 0x5598c4a41320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 >> 172.21.15.100:0/631445388 conn(0x5598c4a2e990 msgr2=0x5598c4a39e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f2effd640 0 client.16428 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 shutdown_connections 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 1 -- 172.21.15.100:0/631445388 wait complete. 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.722+0000 7f2f58885740 20 client.16428 trim_cache size 0 max 16384 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "5b37910e-bdff-4a5d-8b26-fe5da8bdaba0": { 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "5b37910e-bdff-4a5d-8b26-fe5da8bdaba0", 2024-03-12T05:16:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:16:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:16:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:16:16.723 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.723+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f3c0a1ba0 msgr2=0x7f2f3c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.723+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f3c0a1ba0 0x7f2f3c0a4060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2f480f6420 tx=0x7f2f48091030 comp rx=0 tx=0).stop 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.723+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 msgr2=0x7f2f5019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.723+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 0x7f2f5019a290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009df0 tx=0x7f2f48009890 comp rx=0 tx=0).stop 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.723+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 shutdown_connections 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f3c0a1ba0 0x7f2f3c0a4060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f50199de0 0x7f2f5019a290 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f50105340 0x7f2f50195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 --2- 172.21.15.100:0/390666479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f50100270 0x7f2f50195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 >> 172.21.15.100:0/390666479 conn(0x7f2f50077230 msgr2=0x7f2f500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:16.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 shutdown_connections 2024-03-12T05:16:16.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:16.724+0000 7f2f55cbb640 1 -- 172.21.15.100:0/390666479 wait complete. 2024-03-12T05:16:16.736 INFO:tasks.cephfs.filesystem:scrub status for tag:5b37910e-bdff-4a5d-8b26-fe5da8bdaba0 - {'path': '/', 'tag': '5b37910e-bdff-4a5d-8b26-fe5da8bdaba0', 'options': 'recursive,force'} 2024-03-12T05:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 458950 13.84 MB/sec execute 60 sec latency 47.423 ms 2024-03-12T05:16:17.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:16 smithi100 ceph-mon[32210]: pgmap v128: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-03-12T05:16:17.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:16 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:16:17.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:16 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:17.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:16 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:16:17.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:16 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:16 smithi156 ceph-mon[36270]: pgmap v128: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-03-12T05:16:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:16 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:16:17.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:16 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:17.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:16 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:16:17.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:16 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:16 smithi161 ceph-mon[36043]: pgmap v128: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-03-12T05:16:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:16 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:16:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:16 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:17.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:16 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:16:17.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:16 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:17.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 462139 13.92 MB/sec execute 61 sec latency 47.078 ms 2024-03-12T05:16:18.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:17 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:16:18.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:17 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:18.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:17 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:16:18.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:17 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:18.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:17 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:16:18.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:17 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:18.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 464899 13.90 MB/sec execute 62 sec latency 7.074 ms 2024-03-12T05:16:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:18 smithi156 ceph-mon[36270]: pgmap v129: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 348 op/s 2024-03-12T05:16:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:18 smithi161 ceph-mon[36043]: pgmap v129: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 348 op/s 2024-03-12T05:16:19.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:18 smithi100 ceph-mon[32210]: pgmap v129: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 348 op/s 2024-03-12T05:16:19.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 467114 13.87 MB/sec execute 63 sec latency 26.583 ms 2024-03-12T05:16:20.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 469866 13.88 MB/sec execute 64 sec latency 7.405 ms 2024-03-12T05:16:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:20 smithi156 ceph-mon[36270]: pgmap v130: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-03-12T05:16:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:20 smithi161 ceph-mon[36043]: pgmap v130: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-03-12T05:16:21.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:20 smithi100 ceph-mon[32210]: pgmap v130: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-03-12T05:16:21.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 472613 13.93 MB/sec execute 65 sec latency 71.736 ms 2024-03-12T05:16:22.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 474956 13.85 MB/sec execute 66 sec latency 8.143 ms 2024-03-12T05:16:23.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:22 smithi156 ceph-mon[36270]: pgmap v131: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 345 op/s 2024-03-12T05:16:23.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:22 smithi161 ceph-mon[36043]: pgmap v131: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 345 op/s 2024-03-12T05:16:23.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:22 smithi100 ceph-mon[32210]: pgmap v131: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 345 op/s 2024-03-12T05:16:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 477147 13.85 MB/sec execute 67 sec latency 7.979 ms 2024-03-12T05:16:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 479977 13.89 MB/sec execute 68 sec latency 114.692 ms 2024-03-12T05:16:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:24 smithi156 ceph-mon[36270]: pgmap v132: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:16:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:24 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:16:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:24 smithi161 ceph-mon[36043]: pgmap v132: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:16:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:24 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:16:25.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:24 smithi100 ceph-mon[32210]: pgmap v132: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:16:25.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:24 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:16:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 482864 13.89 MB/sec execute 69 sec latency 7.051 ms 2024-03-12T05:16:26.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:25 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:16:26.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:25 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:16:26.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:25 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:16:26.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 484671 13.85 MB/sec execute 70 sec latency 65.719 ms 2024-03-12T05:16:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:26 smithi156 ceph-mon[36270]: pgmap v133: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:16:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:26 smithi161 ceph-mon[36043]: pgmap v133: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:16:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:26 smithi100 ceph-mon[32210]: pgmap v133: 97 pgs: 97 active+clean; 261 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:16:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 487360 13.85 MB/sec execute 71 sec latency 20.968 ms 2024-03-12T05:16:28.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 490550 13.92 MB/sec execute 72 sec latency 8.165 ms 2024-03-12T05:16:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:28 smithi156 ceph-mon[36270]: pgmap v134: 97 pgs: 97 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-03-12T05:16:29.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:28 smithi161 ceph-mon[36043]: pgmap v134: 97 pgs: 97 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-03-12T05:16:29.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:28 smithi100 ceph-mon[32210]: pgmap v134: 97 pgs: 97 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-03-12T05:16:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 493224 13.90 MB/sec execute 73 sec latency 7.642 ms 2024-03-12T05:16:30.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 495411 13.87 MB/sec execute 74 sec latency 19.929 ms 2024-03-12T05:16:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:30 smithi156 ceph-mon[36270]: pgmap v135: 97 pgs: 97 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-03-12T05:16:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:30 smithi161 ceph-mon[36043]: pgmap v135: 97 pgs: 97 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-03-12T05:16:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:30 smithi100 ceph-mon[32210]: pgmap v135: 97 pgs: 97 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-03-12T05:16:31.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 497838 13.87 MB/sec execute 75 sec latency 57.081 ms 2024-03-12T05:16:32.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 500889 13.92 MB/sec execute 76 sec latency 10.626 ms 2024-03-12T05:16:33.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:32 smithi156 ceph-mon[36270]: pgmap v136: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 331 op/s 2024-03-12T05:16:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:32 smithi161 ceph-mon[36043]: pgmap v136: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 331 op/s 2024-03-12T05:16:33.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:32 smithi100 ceph-mon[32210]: pgmap v136: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 331 op/s 2024-03-12T05:16:33.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 503503 13.87 MB/sec execute 77 sec latency 7.329 ms 2024-03-12T05:16:34.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 505901 13.90 MB/sec execute 78 sec latency 37.587 ms 2024-03-12T05:16:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:34 smithi156 ceph-mon[36270]: pgmap v137: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 217 op/s 2024-03-12T05:16:35.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:34 smithi161 ceph-mon[36043]: pgmap v137: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 217 op/s 2024-03-12T05:16:35.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:34 smithi100 ceph-mon[32210]: pgmap v137: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 217 op/s 2024-03-12T05:16:35.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 508635 13.90 MB/sec execute 79 sec latency 15.019 ms 2024-03-12T05:16:36.788 INFO:tasks.workunit.client.0.smithi100.stdout: 1 511807 13.96 MB/sec execute 80 sec latency 32.019 ms 2024-03-12T05:16:37.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:36 smithi156 ceph-mon[36270]: pgmap v138: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 217 op/s 2024-03-12T05:16:37.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:36 smithi161 ceph-mon[36043]: pgmap v138: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 217 op/s 2024-03-12T05:16:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:36 smithi100 ceph-mon[32210]: pgmap v138: 97 pgs: 97 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 217 op/s 2024-03-12T05:16:37.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 514035 13.88 MB/sec execute 81 sec latency 124.668 ms 2024-03-12T05:16:38.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 516332 13.88 MB/sec execute 82 sec latency 10.573 ms 2024-03-12T05:16:39.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:38 smithi156 ceph-mon[36270]: pgmap v139: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 335 op/s 2024-03-12T05:16:39.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:38 smithi161 ceph-mon[36043]: pgmap v139: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 335 op/s 2024-03-12T05:16:39.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:38 smithi100 ceph-mon[32210]: pgmap v139: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 335 op/s 2024-03-12T05:16:39.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 519233 13.92 MB/sec execute 83 sec latency 12.039 ms 2024-03-12T05:16:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:39 smithi100 ceph-mon[32210]: pgmap v140: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-03-12T05:16:40.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:39 smithi156 ceph-mon[36270]: pgmap v140: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-03-12T05:16:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:39 smithi161 ceph-mon[36043]: pgmap v140: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-03-12T05:16:40.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 522333 13.97 MB/sec execute 84 sec latency 21.585 ms 2024-03-12T05:16:41.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 524785 13.91 MB/sec execute 85 sec latency 31.577 ms 2024-03-12T05:16:42.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 526743 13.89 MB/sec execute 86 sec latency 73.436 ms 2024-03-12T05:16:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:42 smithi156 ceph-mon[36270]: pgmap v141: 97 pgs: 97 active+clean; 285 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2024-03-12T05:16:43.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:42 smithi161 ceph-mon[36043]: pgmap v141: 97 pgs: 97 active+clean; 285 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2024-03-12T05:16:43.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:42 smithi100 ceph-mon[32210]: pgmap v141: 97 pgs: 97 active+clean; 285 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2024-03-12T05:16:43.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 529650 13.94 MB/sec execute 87 sec latency 16.095 ms 2024-03-12T05:16:44.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 532634 13.95 MB/sec execute 88 sec latency 14.635 ms 2024-03-12T05:16:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:44 smithi156 ceph-mon[36270]: pgmap v142: 97 pgs: 97 active+clean; 286 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-03-12T05:16:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:44 smithi161 ceph-mon[36043]: pgmap v142: 97 pgs: 97 active+clean; 286 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-03-12T05:16:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:44 smithi100 ceph-mon[32210]: pgmap v142: 97 pgs: 97 active+clean; 286 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-03-12T05:16:45.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 535256 13.92 MB/sec execute 89 sec latency 7.832 ms 2024-03-12T05:16:46.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:16:46.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:16:46.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 537411 13.91 MB/sec execute 90 sec latency 8.395 ms 2024-03-12T05:16:46.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 -- 172.21.15.100:0/11477047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac101360 msgr2=0x7f4bac101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 --2- 172.21.15.100:0/11477047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac101360 0x7f4bac101740 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009930 tx=0x7f4b9c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 -- 172.21.15.100:0/11477047 shutdown_connections 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 --2- 172.21.15.100:0/11477047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac106430 0x7f4bac10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 --2- 172.21.15.100:0/11477047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bac101d10 0x7f4bac105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 --2- 172.21.15.100:0/11477047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac101360 0x7f4bac101740 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 -- 172.21.15.100:0/11477047 >> 172.21.15.100:0/11477047 conn(0x7f4bac0fcc20 msgr2=0x7f4bac0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 -- 172.21.15.100:0/11477047 shutdown_connections 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.866+0000 7f4bb230d640 1 -- 172.21.15.100:0/11477047 wait complete. 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 Processor -- start 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 -- start start 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bac101360 0x7f4bac19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 0x7f4bac19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac197650 0x7f4bac197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac198040 con 0x7f4bac106430 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac1981b0 con 0x7f4bac101360 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb230d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac198370 con 0x7f4bac197650 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 0x7f4bac19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 0x7f4bac19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44876/0 (socket says 172.21.15.100:44876) 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 -- 172.21.15.100:0/2711521532 learned_addr learned my addr 172.21.15.100:0/2711521532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4babfff640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bac101360 0x7f4bac19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac197650 0x7f4bac197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 -- 172.21.15.100:0/2711521532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac197650 msgr2=0x7f4bac197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac197650 0x7f4bac197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 -- 172.21.15.100:0/2711521532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bac101360 msgr2=0x7f4bac19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bac101360 0x7f4bac19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.867+0000 7f4bab7fe640 1 -- 172.21.15.100:0/2711521532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c009560 con 0x7f4bac106430 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.868+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 0x7f4bac19d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000e9c0 tx=0x7f4ba000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.868+0000 7f4ba97fa640 1 -- 172.21.15.100:0/2711521532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0009700 con 0x7f4bac106430 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.868+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac1a4310 con 0x7f4bac106430 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.868+0000 7f4ba97fa640 1 -- 172.21.15.100:0/2711521532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba000cd80 con 0x7f4bac106430 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.868+0000 7f4ba97fa640 1 -- 172.21.15.100:0/2711521532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0010430 con 0x7f4bac106430 2024-03-12T05:16:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.868+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac1a4820 con 0x7f4bac106430 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.870+0000 7f4ba97fa640 1 -- 172.21.15.100:0/2711521532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4ba0005000 con 0x7f4bac106430 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 Processor -- start 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 -- start start 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4ba97fa640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4b940a1ba0 0x7f4b940a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104e0a20 0x55fa104e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa10442280 con 0x55fa104cde00 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1043f7b0 con 0x55fa104e0a20 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1043af90 con 0x55fa104cd440 2024-03-12T05:16:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4b940a1ba0 0x7f4b940a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4ba97fa640 1 -- 172.21.15.100:0/2711521532 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4ba0014070 con 0x7f4bac106430 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bb08aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52200/0 (socket says 172.21.15.100:52200) 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 -- 172.21.15.100:0/1630376623 learned_addr learned my addr 172.21.15.100:0/1630376623 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4babfff640 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104e0a20 0x55fa104e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 -- 172.21.15.100:0/1630376623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104e0a20 msgr2=0x55fa104e0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104e0a20 0x55fa104e0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 -- 172.21.15.100:0/1630376623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 msgr2=0x55fa104e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104e7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.871+0000 7f4bab7fe640 1 -- 172.21.15.100:0/1630376623 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa103f9570 con 0x55fa104cd440 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.872+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104cd8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00cb390 tx=0x7f4ba009a350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8719bd33b9446182 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.872+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4b940a1ba0 0x7f4b940a4060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55fa104d1070 tx=0x7f4ba4004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:46.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.872+0000 7f4b927fc640 1 -- 172.21.15.100:0/1630376623 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba009a810 con 0x55fa104cd440 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.872+0000 7f4b927fc640 1 -- 172.21.15.100:0/1630376623 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba009e960 con 0x55fa104cd440 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.872+0000 7f4b927fc640 1 -- 172.21.15.100:0/1630376623 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0096e80 con 0x55fa104cd440 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.872+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/1630376623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 msgr2=0x55fa104cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104cd8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00cb390 tx=0x7f4ba009a350 comp rx=0 tx=0).stop 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/1630376623 shutdown_connections 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104cd8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/1630376623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104e0a20 0x55fa104e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/1630376623 >> 172.21.15.100:0/1630376623 conn(0x55fa104cffb0 msgr2=0x55fa104d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/1630376623 shutdown_connections 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/1630376623 wait complete. 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- start start 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 Processor -- start 2024-03-12T05:16:46.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa104e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1010e430 con 0x55fa104e0a20 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa100cae00 con 0x55fa104cde00 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.873+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa103bffd0 con 0x55fa104cd440 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb4fb9740 10 client.0 _ll_register_callbacks cb 0x55fa10415fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa104e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa104e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44894/0 (socket says 172.21.15.100:44894) 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 -- 172.21.15.100:0/4220045885 learned_addr learned my addr 172.21.15.100:0/4220045885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4babfff640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 -- 172.21.15.100:0/4220045885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 msgr2=0x55fa104e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 -- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 msgr2=0x55fa104e4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104e4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 -- 172.21.15.100:0/4220045885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac1a4820 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4babfff640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa104e4fc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4010460 tx=0x7f4ba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7449fac878b8d6fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4b8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/4220045885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba402b620 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/4220045885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4ba402b780 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa1013a7e0 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/4220045885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba402b940 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb4fb9740 10 client.16440 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.874+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fa10439610 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/4220045885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fa1013a7e0 con 0x55fa104e0a20 2024-03-12T05:16:46.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/4220045885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ba402d9d0 con 0x55fa104e0a20 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4bb4fb9740 10 client.16440 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4bb4fb9740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4bb4fb9740 20 client.16440 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa104f5070 0x55fa104f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4bb4fb9740 4 client.16440 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa10177720 con 0x55fa104f5070 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4babfff640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa104f5070 0x55fa104f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4b8b7fe640 10 client.16440 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.875+0000 7f4babfff640 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa104f5070 0x55fa104f7490 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37433033db174704 server_cookie=8d032e3563931827 in_seq=0 out_seq=0 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.876+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/4220045885 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa10177720 con 0x55fa104f5070 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.876+0000 7f4b8b7fe640 10 client.16440 handle_command_reply: tid=0 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.876+0000 7f4bb4fb9740 1 client.16440 shutdown 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.876+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 msgr2=0x55fa104e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.876+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa104e4fc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4010460 tx=0x7f4ba4004a40 comp rx=0 tx=0).stop 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 shutdown_connections 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa104f5070 0x55fa104f7490 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa104e4fc0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104e4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4b8b7fe640 0 client.16440 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/4220045885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 >> 172.21.15.100:0/4220045885 conn(0x55fa104cffb0 msgr2=0x55fa104cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 shutdown_connections 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/4220045885 wait complete. 2024-03-12T05:16:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.877+0000 7f4bb4fb9740 20 client.16440 trim_cache size 0 max 16384 2024-03-12T05:16:46.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 Processor -- start 2024-03-12T05:16:46.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 -- start start 2024-03-12T05:16:46.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cd440 0x55fa104e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104e0a20 0x55fa104e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa1013a7e0 con 0x55fa104cde00 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa104ce4e0 con 0x55fa104cd440 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa104fc6d0 con 0x55fa104e0a20 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44896/0 (socket says 172.21.15.100:44896) 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 -- 172.21.15.100:0/3807432284 learned_addr learned my addr 172.21.15.100:0/3807432284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4babfff640 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cd440 0x55fa104e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104e0a20 0x55fa104e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 -- 172.21.15.100:0/3807432284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104e0a20 msgr2=0x55fa104e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104e0a20 0x55fa104e7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 -- 172.21.15.100:0/3807432284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cd440 msgr2=0x55fa104e26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cd440 0x55fa104e26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.892+0000 7f4bab7fe640 1 -- 172.21.15.100:0/3807432284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa103d5ac0 con 0x55fa104cde00 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104cc8d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4ba009a7a0 tx=0x7f4ba009a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6749645869948dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4b91ffb640 1 -- 172.21.15.100:0/3807432284 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00a2fb0 con 0x55fa104cde00 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4b91ffb640 1 -- 172.21.15.100:0/3807432284 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba0098b10 con 0x55fa104cde00 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4b91ffb640 1 -- 172.21.15.100:0/3807432284 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00052b0 con 0x55fa104cde00 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/3807432284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 msgr2=0x55fa104cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104cc8d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4ba009a7a0 tx=0x7f4ba009a5e0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/3807432284 shutdown_connections 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104e0a20 0x55fa104e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104cde00 0x55fa104cc8d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/3807432284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cd440 0x55fa104e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/3807432284 >> 172.21.15.100:0/3807432284 conn(0x55fa104cffb0 msgr2=0x55fa104cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/3807432284 shutdown_connections 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.893+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/3807432284 wait complete. 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 -- start start 2024-03-12T05:16:46.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 Processor -- start 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa10544620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa10544b60 con 0x55fa104e0a20 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa10544cd0 con 0x55fa104cde00 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa10544e90 con 0x55fa104cd440 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb4fb9740 10 client.0 _ll_register_callbacks cb 0x7f4ba0005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb08aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa10544620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4babfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4babfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104eb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52232/0 (socket says 172.21.15.100:52232) 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bb08aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa10544620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44908/0 (socket says 172.21.15.100:44908) 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4babfff640 1 -- 172.21.15.100:0/2526258128 learned_addr learned my addr 172.21.15.100:0/2526258128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.894+0000 7f4bab7fe640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb08aa640 1 -- 172.21.15.100:0/2526258128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 msgr2=0x55fa104eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb08aa640 1 -- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 msgr2=0x55fa104eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104eb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb08aa640 1 -- 172.21.15.100:0/2526258128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa103f9570 con 0x55fa104e0a20 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4babfff640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb08aa640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa10544620 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006010 tx=0x7f4ba4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54178c015b7499ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4b8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/2526258128 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400ec40 con 0x55fa104e0a20 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/2526258128 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4ba4004b20 con 0x55fa104e0a20 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ba009e960 con 0x55fa104e0a20 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/2526258128 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba4004ce0 con 0x55fa104e0a20 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 10 client.16446 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fa103fb6a0 con 0x55fa104e0a20 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/2526258128 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba009e960 con 0x55fa104e0a20 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/2526258128 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ba402b420 con 0x55fa104e0a20 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 10 client.16446 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 20 client.16446 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa1054d860 0x55fa1054fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 4 client.16446 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.895+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fa101c6890 con 0x55fa1054d860 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4babfff640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa1054d860 0x55fa1054fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4b8b7fe640 10 client.16446 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4babfff640 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa1054d860 0x55fa1054fc80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2487b37c0014889 server_cookie=a9b1addcf9571423 in_seq=0 out_seq=0 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4b8b7fe640 1 -- 172.21.15.100:0/2526258128 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fa101c6890 con 0x55fa1054d860 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4b8b7fe640 10 client.16446 handle_command_reply: tid=0 2024-03-12T05:16:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 client.16446 shutdown 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 msgr2=0x55fa10544620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa10544620 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006010 tx=0x7f4ba4005e30 comp rx=0 tx=0).stop 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 shutdown_connections 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fa1054d860 0x55fa1054fc80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa104e0a20 0x55fa10544620 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4b8b7fe640 0 client.16446 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa104cde00 0x55fa104eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 --2- 172.21.15.100:0/2526258128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa104cd440 0x55fa104eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 >> 172.21.15.100:0/2526258128 conn(0x55fa104cffb0 msgr2=0x55fa104cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 shutdown_connections 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 1 -- 172.21.15.100:0/2526258128 wait complete. 2024-03-12T05:16:46.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.896+0000 7f4bb4fb9740 20 client.16446 trim_cache size 0 max 16384 2024-03-12T05:16:46.900 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:16:46.900 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:16:46.900 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:16:46.900 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4b940a1ba0 msgr2=0x7f4b940a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4b940a1ba0 0x7f4b940a4060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55fa104d1070 tx=0x7f4ba4004ab0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 msgr2=0x7f4bac19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 0x7f4bac19d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000e9c0 tx=0x7f4ba000ee80 comp rx=0 tx=0).stop 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 shutdown_connections 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4b940a1ba0 0x7f4b940a4060 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac197650 0x7f4bac197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac106430 0x7f4bac19d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 --2- 172.21.15.100:0/2711521532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bac101360 0x7f4bac19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 >> 172.21.15.100:0/2711521532 conn(0x7f4bac0fcc20 msgr2=0x7f4bac0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 shutdown_connections 2024-03-12T05:16:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:46.897+0000 7f4bb230d640 1 -- 172.21.15.100:0/2711521532 wait complete. 2024-03-12T05:16:46.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:16:46.904 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:16:47.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 -- 172.21.15.100:0/2967115157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 msgr2=0x7f89fc10dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 --2- 172.21.15.100:0/2967115157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc10dc60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009ca0 tx=0x7f89f402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 -- 172.21.15.100:0/2967115157 shutdown_connections 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 --2- 172.21.15.100:0/2967115157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc10dc60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 --2- 172.21.15.100:0/2967115157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc102d10 0x7f89fc10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 --2- 172.21.15.100:0/2967115157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89fc1023f0 0x7f89fc1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 -- 172.21.15.100:0/2967115157 >> 172.21.15.100:0/2967115157 conn(0x7f89fc077180 msgr2=0x7f89fc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 -- 172.21.15.100:0/2967115157 shutdown_connections 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 -- 172.21.15.100:0/2967115157 wait complete. 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.040+0000 7f8a018f1640 1 Processor -- start 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 -- start start 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 0x7f89fc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89fc102d10 0x7f89fc19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc197bf0 con 0x7f89fc10b7c0 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc197d60 con 0x7f89fc1023f0 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc197f20 con 0x7f89fc102d10 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89faffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 0x7f89fc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44926/0 (socket says 172.21.15.100:44926) 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89faffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 0x7f89fc19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:46892/0 (socket says 172.21.15.100:46892) 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 -- 172.21.15.100:0/4140210249 learned_addr learned my addr 172.21.15.100:0/4140210249 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89f3fff640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89fc102d10 0x7f89fc19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 -- 172.21.15.100:0/4140210249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89fc102d10 msgr2=0x7f89fc19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89fc102d10 0x7f89fc19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 -- 172.21.15.100:0/4140210249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 msgr2=0x7f89fc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 0x7f89fc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 -- 172.21.15.100:0/4140210249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4009900 con 0x7f89fc10b7c0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89faffd640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 0x7f89fc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f89fb7fe640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc1975a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f89f402c700 tx=0x7f89f4034b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40328b0 con 0x7f89fc10b7c0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89f4003e50 con 0x7f89fc10b7c0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc1981a0 con 0x7f89fc10b7c0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.041+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40048a0 con 0x7f89fc10b7c0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.042+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc1a4490 con 0x7f89fc10b7c0 2024-03-12T05:16:47.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.042+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc067f70 con 0x7f89fc10b7c0 2024-03-12T05:16:47.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.043+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f89f400a040 con 0x7f89fc10b7c0 2024-03-12T05:16:47.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.044+0000 7f8a008ef640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89e80a1c20 0x7f89e80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.044+0000 7f89faffd640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89e80a1c20 0x7f89e80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.044+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89f40f0850 con 0x7f89fc10b7c0 2024-03-12T05:16:47.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.045+0000 7f89faffd640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89e80a1c20 0x7f89e80a40e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89ec005d00 tx=0x7f89ec00b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.048+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89f40b9820 con 0x7f89fc10b7c0 2024-03-12T05:16:47.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:46 smithi100 ceph-mon[32210]: pgmap v143: 97 pgs: 97 active+clean; 286 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-03-12T05:16:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:46 smithi156 ceph-mon[36270]: pgmap v143: 97 pgs: 97 active+clean; 286 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-03-12T05:16:47.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:46 smithi161 ceph-mon[36043]: pgmap v143: 97 pgs: 97 active+clean; 286 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-03-12T05:16:47.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.268+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89fc111300 con 0x7f89fc10b7c0 2024-03-12T05:16:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.269+0000 7f8a008ef640 1 -- 172.21.15.100:0/4140210249 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f89f40bcee0 con 0x7f89fc10b7c0 2024-03-12T05:16:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:16:47.265 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:16:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89e80a1c20 msgr2=0x7f89e80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89e80a1c20 0x7f89e80a40e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89ec005d00 tx=0x7f89ec00b410 comp rx=0 tx=0).stop 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 msgr2=0x7f89fc1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc1975a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f89f402c700 tx=0x7f89f4034b10 comp rx=0 tx=0).stop 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 shutdown_connections 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89e80a1c20 0x7f89e80a40e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89fc10b7c0 0x7f89fc1975a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89fc102d10 0x7f89fc19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.271+0000 7f8a018f1640 1 --2- 172.21.15.100:0/4140210249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89fc1023f0 0x7f89fc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.272+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 >> 172.21.15.100:0/4140210249 conn(0x7f89fc077180 msgr2=0x7f89fc0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.272+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 shutdown_connections 2024-03-12T05:16:47.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.272+0000 7f8a018f1640 1 -- 172.21.15.100:0/4140210249 wait complete. 2024-03-12T05:16:47.278 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:16:47.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:16:47.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 -- 172.21.15.100:0/2117442000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84280ffcf0 msgr2=0x7f84281000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 --2- 172.21.15.100:0/2117442000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84280ffcf0 0x7f84281000d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8418009930 tx=0x7f841802bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 -- 172.21.15.100:0/2117442000 shutdown_connections 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 --2- 172.21.15.100:0/2117442000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84281064b0 0x7f842810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 --2- 172.21.15.100:0/2117442000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84281006a0 0x7f8428105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 --2- 172.21.15.100:0/2117442000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84280ffcf0 0x7f84281000d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 -- 172.21.15.100:0/2117442000 >> 172.21.15.100:0/2117442000 conn(0x7f84280752e0 msgr2=0x7f84280756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 -- 172.21.15.100:0/2117442000 shutdown_connections 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.390+0000 7f842e5d0640 1 -- 172.21.15.100:0/2117442000 wait complete. 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 Processor -- start 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 -- start start 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84280ffcf0 0x7f842819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84281064b0 0x7f842819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 0x7f8428197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428198140 con 0x7f8428197670 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281982b0 con 0x7f84280ffcf0 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842e5d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428198470 con 0x7f84281064b0 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842cb6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 0x7f8428197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842cb6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 0x7f8428197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44956/0 (socket says 172.21.15.100:44956) 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f842cb6d640 1 -- 172.21.15.100:0/799893813 learned_addr learned my addr 172.21.15.100:0/799893813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f8427fff640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84280ffcf0 0x7f842819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.391+0000 7f84277fe640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84281064b0 0x7f842819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842cb6d640 1 -- 172.21.15.100:0/799893813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84281064b0 msgr2=0x7f842819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842cb6d640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84281064b0 0x7f842819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842cb6d640 1 -- 172.21.15.100:0/799893813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84280ffcf0 msgr2=0x7f842819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842cb6d640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84280ffcf0 0x7f842819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842cb6d640 1 -- 172.21.15.100:0/799893813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418009560 con 0x7f8428197670 2024-03-12T05:16:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842cb6d640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 0x7f8428197af0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f842000eb10 tx=0x7f842000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f84257fa640 1 -- 172.21.15.100:0/799893813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420004b20 con 0x7f8428197670 2024-03-12T05:16:47.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281a4410 con 0x7f8428197670 2024-03-12T05:16:47.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f84257fa640 1 -- 172.21.15.100:0/799893813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8420004c80 con 0x7f8428197670 2024-03-12T05:16:47.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f84257fa640 1 -- 172.21.15.100:0/799893813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420010650 con 0x7f8428197670 2024-03-12T05:16:47.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.392+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84281a48f0 con 0x7f8428197670 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84257fa640 1 -- 172.21.15.100:0/799893813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8420005020 con 0x7f8428197670 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 Processor -- start 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 -- start start 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c780630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780b70 0x56431c789210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c78bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c73b960 con 0x56431c780250 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c6ef310 con 0x56431c789750 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.394+0000 7f84311a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c6ec840 con 0x56431c780b70 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c780630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84257fa640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84100a1ba0 0x7f84100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c780630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44962/0 (socket says 172.21.15.100:44962) 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 -- 172.21.15.100:0/2329741866 learned_addr learned my addr 172.21.15.100:0/2329741866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f8427fff640 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780b70 0x56431c789210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c78bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f842cb6d640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84100a1ba0 0x7f84100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84257fa640 1 -- 172.21.15.100:0/799893813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8420014070 con 0x7f8428197670 2024-03-12T05:16:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 -- 172.21.15.100:0/2329741866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780b70 msgr2=0x56431c789210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780b70 0x56431c789210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 -- 172.21.15.100:0/2329741866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 msgr2=0x56431c78bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c78bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 -- 172.21.15.100:0/2329741866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431c6aa060 con 0x56431c780250 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f8427fff640 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780b70 0x56431c789210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f84277fe640 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c780630 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f841c0136f0 tx=0x7f841c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca1fefd8c50c2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f842cb6d640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84100a1ba0 0x7f84100a4060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f84200cb5b0 tx=0x7f84200900b0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f840e7fc640 1 -- 172.21.15.100:0/2329741866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c011080 con 0x56431c780250 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f840e7fc640 1 -- 172.21.15.100:0/2329741866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f841c004510 con 0x56431c780250 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.395+0000 7f840e7fc640 1 -- 172.21.15.100:0/2329741866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c011d80 con 0x56431c780250 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 -- 172.21.15.100:0/2329741866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 msgr2=0x56431c780630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c780630 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f841c0136f0 tx=0x7f841c0096c0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 -- 172.21.15.100:0/2329741866 shutdown_connections 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c78bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780b70 0x56431c789210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 --2- 172.21.15.100:0/2329741866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c780630 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 -- 172.21.15.100:0/2329741866 >> 172.21.15.100:0/2329741866 conn(0x56431c77a980 msgr2=0x56431c77ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 -- 172.21.15.100:0/2329741866 shutdown_connections 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 -- 172.21.15.100:0/2329741866 wait complete. 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.396+0000 7f84311a3740 1 -- start start 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 Processor -- start 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780250 0x56431c786770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c786cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c7871f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c6e8020 con 0x56431c780b70 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c3bb430 con 0x56431c780250 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c377e00 con 0x56431c789750 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84311a3740 10 client.0 _ll_register_callbacks cb 0x56431c68fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c786cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780250 0x56431c786770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780250 0x56431c786770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:46940/0 (socket says 172.21.15.100:46940) 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84277fe640 1 -- 172.21.15.100:0/50190907 learned_addr learned my addr 172.21.15.100:0/50190907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c786cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44964/0 (socket says 172.21.15.100:44964) 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84277fe640 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c7871f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 -- 172.21.15.100:0/50190907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 msgr2=0x56431c7871f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c7871f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 -- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780250 msgr2=0x56431c786770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780250 0x56431c786770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 -- 172.21.15.100:0/50190907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c01b040 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f8427fff640 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c786cb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8418009230 tx=0x7f841802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ff1dd0be040ccc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.397+0000 7f84077fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84077fe640 1 -- 172.21.15.100:0/50190907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418004040 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84077fe640 1 -- 172.21.15.100:0/50190907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84180041a0 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56431c3e77e0 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84077fe640 1 -- 172.21.15.100:0/50190907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8418004360 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 10 client.16464 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56431c6e66a0 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84077fe640 1 -- 172.21.15.100:0/50190907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56431c3e77e0 con 0x56431c780b70 2024-03-12T05:16:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84077fe640 1 -- 172.21.15.100:0/50190907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f841803c040 con 0x56431c780b70 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 10 client.16464 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 10 client.16464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 20 client.16464 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c799a60 0x56431c79be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 4 client.16464 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56431c424720 con 0x56431c799a60 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.398+0000 7f84277fe640 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c799a60 0x56431c79be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.399+0000 7f84077fe640 10 client.16464 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.399+0000 7f84277fe640 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c799a60 0x56431c79be80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1cb948371daaa04 server_cookie=7d338dbd9270fb98 in_seq=0 out_seq=0 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84077fe640 1 -- 172.21.15.100:0/50190907 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56431c424720 con 0x56431c799a60 2024-03-12T05:16:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84077fe640 10 client.16464 handle_command_reply: tid=0 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 client.16464 shutdown 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 msgr2=0x56431c786cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c786cb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8418009230 tx=0x7f841802ca10 comp rx=0 tx=0).stop 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 shutdown_connections 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c799a60 0x56431c79be80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c7871f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84077fe640 0 client.16464 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c786cb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 --2- 172.21.15.100:0/50190907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780250 0x56431c786770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 >> 172.21.15.100:0/50190907 conn(0x56431c77a980 msgr2=0x56431c77c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 shutdown_connections 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 1 -- 172.21.15.100:0/50190907 wait complete. 2024-03-12T05:16:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.400+0000 7f84311a3740 20 client.16464 trim_cache size 0 max 16384 2024-03-12T05:16:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 Processor -- start 2024-03-12T05:16:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 -- start start 2024-03-12T05:16:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780250 0x56431c788e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c787010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c781a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c66d130 con 0x56431c780b70 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c76ca40 con 0x56431c789750 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84311a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c7a10d0 con 0x56431c780250 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c787010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c787010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44966/0 (socket says 172.21.15.100:44966) 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 -- 172.21.15.100:0/3601264237 learned_addr learned my addr 172.21.15.100:0/3601264237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f84277fe640 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c781a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 -- 172.21.15.100:0/3601264237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780250 msgr2=0x56431c788e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780250 0x56431c788e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 -- 172.21.15.100:0/3601264237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 msgr2=0x56431c781a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c781a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.412+0000 7f8427fff640 1 -- 172.21.15.100:0/3601264237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431c682c20 con 0x56431c780b70 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f8427fff640 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c787010 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8418004a60 tx=0x7f8418000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3df9d0d6c051ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f840dffb640 1 -- 172.21.15.100:0/3601264237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418003eb0 con 0x56431c780b70 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f840dffb640 1 -- 172.21.15.100:0/3601264237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84180044f0 con 0x56431c780b70 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f840dffb640 1 -- 172.21.15.100:0/3601264237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418053780 con 0x56431c780b70 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 -- 172.21.15.100:0/3601264237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 msgr2=0x56431c787010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c787010 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8418004a60 tx=0x7f8418000bc0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 -- 172.21.15.100:0/3601264237 shutdown_connections 2024-03-12T05:16:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c789750 0x56431c781a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780b70 0x56431c787010 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 --2- 172.21.15.100:0/3601264237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c780250 0x56431c788e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 -- 172.21.15.100:0/3601264237 >> 172.21.15.100:0/3601264237 conn(0x56431c77a980 msgr2=0x56431c775a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 -- 172.21.15.100:0/3601264237 shutdown_connections 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.413+0000 7f84311a3740 1 -- 172.21.15.100:0/3601264237 wait complete. 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 -- start start 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 Processor -- start 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c78efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780b70 0x56431c78f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c78fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c7effb0 con 0x56431c780250 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c7f00f0 con 0x56431c780b70 2024-03-12T05:16:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431c7f0230 con 0x56431c789750 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84311a3740 10 client.0 _ll_register_callbacks cb 0x7f841c0099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c78efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c78efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44978/0 (socket says 172.21.15.100:44978) 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 -- 172.21.15.100:0/1160204415 learned_addr learned my addr 172.21.15.100:0/1160204415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f8427fff640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780b70 0x56431c78f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c78fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 -- 172.21.15.100:0/1160204415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 msgr2=0x56431c78fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c78fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 -- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780b70 msgr2=0x56431c78f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.414+0000 7f84277fe640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780b70 0x56431c78f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84277fe640 1 -- 172.21.15.100:0/1160204415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56431c6e66a0 con 0x56431c780250 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84277fe640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c78efd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f841c0119d0 tx=0x7f841c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c99d4563c216c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84077fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84077fe640 1 -- 172.21.15.100:0/1160204415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c011c30 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84077fe640 1 -- 172.21.15.100:0/1160204415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f841c011d90 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f841c004510 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84077fe640 1 -- 172.21.15.100:0/1160204415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f841c02f340 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 10 client.16470 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56431c6a88c0 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84077fe640 1 -- 172.21.15.100:0/1160204415 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f841c004510 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84077fe640 1 -- 172.21.15.100:0/1160204415 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f841c004680 con 0x56431c780250 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 10 client.16470 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 10 client.16470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 20 client.16470 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c7f19c0 0x56431c7f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 4 client.16470 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.415+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56431c473890 con 0x56431c7f19c0 2024-03-12T05:16:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.416+0000 7f8427fff640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c7f19c0 0x56431c7f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.416+0000 7f84077fe640 10 client.16470 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.416+0000 7f8427fff640 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c7f19c0 0x56431c7f3de0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a4cde829016ce81 server_cookie=a726513e6ca93036 in_seq=0 out_seq=0 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.417+0000 7f84077fe640 1 -- 172.21.15.100:0/1160204415 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56431c473890 con 0x56431c7f19c0 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.417+0000 7f84077fe640 10 client.16470 handle_command_reply: tid=0 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.417+0000 7f84311a3740 1 client.16470 shutdown 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 msgr2=0x56431c78efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c78efd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f841c0119d0 tx=0x7f841c009b20 comp rx=0 tx=0).stop 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 shutdown_connections 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56431c7f19c0 0x56431c7f3de0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56431c789750 0x56431c78fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431c780b70 0x56431c78f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84077fe640 0 client.16470 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 --2- 172.21.15.100:0/1160204415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431c780250 0x56431c78efd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 >> 172.21.15.100:0/1160204415 conn(0x56431c77a980 msgr2=0x56431c781a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 shutdown_connections 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 1 -- 172.21.15.100:0/1160204415 wait complete. 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f84311a3740 20 client.16470 trim_cache size 0 max 16384 2024-03-12T05:16:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84100a1ba0 msgr2=0x7f84100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84100a1ba0 0x7f84100a4060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f84200cb5b0 tx=0x7f84200900b0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 msgr2=0x7f8428197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 0x7f8428197af0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f842000eb10 tx=0x7f842000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 shutdown_connections 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84100a1ba0 0x7f84100a4060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428197670 0x7f8428197af0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84281064b0 0x7f842819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 --2- 172.21.15.100:0/799893813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84280ffcf0 0x7f842819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 >> 172.21.15.100:0/799893813 conn(0x7f84280752e0 msgr2=0x7f84280fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 shutdown_connections 2024-03-12T05:16:47.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.418+0000 7f842e5d0640 1 -- 172.21.15.100:0/799893813 wait complete. 2024-03-12T05:16:47.425 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:16:47.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:16:47.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- 172.21.15.100:0/4207840739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81981002a0 msgr2=0x7f8198105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- 172.21.15.100:0/4207840739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81981002a0 0x7f8198105dd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8188009930 tx=0x7f818802bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- 172.21.15.100:0/4207840739 shutdown_connections 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- 172.21.15.100:0/4207840739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8198106500 0x7f819810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- 172.21.15.100:0/4207840739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81981002a0 0x7f8198105dd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- 172.21.15.100:0/4207840739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980ff8f0 0x7f81980ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- 172.21.15.100:0/4207840739 >> 172.21.15.100:0/4207840739 conn(0x7f8198076640 msgr2=0x7f81980fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- 172.21.15.100:0/4207840739 shutdown_connections 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- 172.21.15.100:0/4207840739 wait complete. 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 Processor -- start 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- start start 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 0x7f8198100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198106500 0x7f8198100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81981012e0 0x7f81981a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981a5fd0 con 0x7f81980ff8f0 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981a6140 con 0x7f81981012e0 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.536+0000 7f819f0a3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981a6300 con 0x7f8198106500 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 0x7f8198100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 0x7f8198100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45000/0 (socket says 172.21.15.100:45000) 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819d640640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81981012e0 0x7f81981a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f8197fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198106500 0x7f8198100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 -- 172.21.15.100:0/2775921419 learned_addr learned my addr 172.21.15.100:0/2775921419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 -- 172.21.15.100:0/2775921419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198106500 msgr2=0x7f8198100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198106500 0x7f8198100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 -- 172.21.15.100:0/2775921419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81981012e0 msgr2=0x7f81981a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81981012e0 0x7f81981a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 -- 172.21.15.100:0/2775921419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188009560 con 0x7f81980ff8f0 2024-03-12T05:16:47.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819d640640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81981012e0 0x7f81981a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819ce3f640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 0x7f8198100860 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f819000ebc0 tx=0x7f819000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f8195ffb640 1 -- 172.21.15.100:0/2775921419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900099b0 con 0x7f81980ff8f0 2024-03-12T05:16:47.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81981a65e0 con 0x7f81980ff8f0 2024-03-12T05:16:47.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f8195ffb640 1 -- 172.21.15.100:0/2775921419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f819000eec0 con 0x7f81980ff8f0 2024-03-12T05:16:47.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f8195ffb640 1 -- 172.21.15.100:0/2775921419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190018b60 con 0x7f81980ff8f0 2024-03-12T05:16:47.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.537+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981a6ac0 con 0x7f81980ff8f0 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f8195ffb640 1 -- 172.21.15.100:0/2775921419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8190018cc0 con 0x7f81980ff8f0 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f8195ffb640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f81840a1bd0 0x7f81840a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 Processor -- start 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 -- start start 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f8197fff640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f81840a1bd0 0x7f81840a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f8195ffb640 1 -- 172.21.15.100:0/2775921419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8190014070 con 0x7f81980ff8f0 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98250 0x55ba66c98630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66c98b70 0x55ba66ca1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66ca1750 0x55ba66ca3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66c53960 con 0x55ba66c98b70 2024-03-12T05:16:47.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66c07310 con 0x55ba66ca1750 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f81a1c70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66c04840 con 0x55ba66c98250 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f8197fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66c98b70 0x55ba66ca1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.540+0000 7f819ce3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98250 0x55ba66c98630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f8197fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66c98b70 0x55ba66ca1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45006/0 (socket says 172.21.15.100:45006) 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f8197fff640 1 -- 172.21.15.100:0/172855225 learned_addr learned my addr 172.21.15.100:0/172855225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819d640640 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66ca1750 0x55ba66ca3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819ce3f640 1 -- 172.21.15.100:0/172855225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66ca1750 msgr2=0x55ba66ca3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819ce3f640 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66ca1750 0x55ba66ca3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819ce3f640 1 -- 172.21.15.100:0/172855225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66c98b70 msgr2=0x55ba66ca1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819ce3f640 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66c98b70 0x55ba66ca1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819ce3f640 1 -- 172.21.15.100:0/172855225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba66bc2060 con 0x55ba66c98250 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f8197fff640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f81840a1bd0 0x7f81840a4090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8188005ab0 tx=0x7f81880046c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f819ce3f640 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98250 0x55ba66c98630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81900cb570 tx=0x7f819009a700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b1d9a3bdca9bf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f818effd640 1 -- 172.21.15.100:0/172855225 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819009eba0 con 0x55ba66c98250 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f818effd640 1 -- 172.21.15.100:0/172855225 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81900a31f0 con 0x55ba66c98250 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.541+0000 7f818effd640 1 -- 172.21.15.100:0/172855225 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819009a1b0 con 0x55ba66c98250 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 -- 172.21.15.100:0/172855225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98250 msgr2=0x55ba66c98630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98250 0x55ba66c98630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81900cb570 tx=0x7f819009a700 comp rx=0 tx=0).stop 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 -- 172.21.15.100:0/172855225 shutdown_connections 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66ca1750 0x55ba66ca3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66c98b70 0x55ba66ca1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 --2- 172.21.15.100:0/172855225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98250 0x55ba66c98630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 -- 172.21.15.100:0/172855225 >> 172.21.15.100:0/172855225 conn(0x55ba66c92980 msgr2=0x55ba66c92d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 -- 172.21.15.100:0/172855225 shutdown_connections 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 -- 172.21.15.100:0/172855225 wait complete. 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 -- start start 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.542+0000 7f81a1c70740 1 Processor -- start 2024-03-12T05:16:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c9eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66c9f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66c00020 con 0x55ba66ca1750 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba668d3430 con 0x55ba66c98250 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba6688fe00 con 0x55ba66c98b70 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 10 client.0 _ll_register_callbacks cb 0x55ba66ba7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c9eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f819d640640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66c9f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f819ce3f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c9eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52354/0 (socket says 172.21.15.100:52354) 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 -- 172.21.15.100:0/2531393742 learned_addr learned my addr 172.21.15.100:0/2531393742 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 -- 172.21.15.100:0/2531393742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 msgr2=0x55ba66c9e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 -- 172.21.15.100:0/2531393742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 msgr2=0x55ba66c9f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66c9f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 -- 172.21.15.100:0/2531393742 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981a6ac0 con 0x55ba66c98b70 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f819d640640 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66c9f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8197fff640 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c9eca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8188009930 tx=0x7f818802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c4be554a44b38a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8177fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8177fff640 1 -- 172.21.15.100:0/2531393742 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188048070 con 0x55ba66c98b70 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba668ff7e0 con 0x55ba66c98b70 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.543+0000 7f8177fff640 1 -- 172.21.15.100:0/2531393742 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8188005000 con 0x55ba66c98b70 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f8177fff640 1 -- 172.21.15.100:0/2531393742 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8188004510 con 0x55ba66c98b70 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 10 client.25921 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba66bfe6a0 con 0x55ba66c98b70 2024-03-12T05:16:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f8177fff640 1 -- 172.21.15.100:0/2531393742 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f818805b020 con 0x55ba66c98b70 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f8177fff640 1 -- 172.21.15.100:0/2531393742 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f818803c070 con 0x55ba66c98b70 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 10 client.25921 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 10 client.25921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 20 client.25921 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66cb1bc0 0x55ba66cb3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 4 client.25921 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba6693c720 con 0x55ba66cb1bc0 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.544+0000 7f819ce3f640 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66cb1bc0 0x55ba66cb3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f8177fff640 10 client.25921 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f819ce3f640 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66cb1bc0 0x55ba66cb3fe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fcbb008abf748ef server_cookie=25bef425a0226d5b in_seq=0 out_seq=0 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f8177fff640 1 -- 172.21.15.100:0/2531393742 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba6693c720 con 0x55ba66cb1bc0 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f8177fff640 10 client.25921 handle_command_reply: tid=0 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 client.25921 shutdown 2024-03-12T05:16:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 msgr2=0x55ba66c9eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c9eca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8188009930 tx=0x7f818802bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 shutdown_connections 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66c9f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c9eca0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66cb1bc0 0x55ba66cb3fe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 --2- 172.21.15.100:0/2531393742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f8177fff640 0 client.25921 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 >> 172.21.15.100:0/2531393742 conn(0x55ba66c92980 msgr2=0x55ba66c946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 shutdown_connections 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.545+0000 7f81a1c70740 1 -- 172.21.15.100:0/2531393742 wait complete. 2024-03-12T05:16:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.546+0000 7f81a1c70740 20 client.25921 trim_cache size 0 max 16384 2024-03-12T05:16:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 Processor -- start 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 -- start start 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c91a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66b85130 con 0x55ba66ca1750 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66c97eb0 con 0x55ba66c98250 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f81a1c70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66cb91f0 con 0x55ba66c98b70 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45026/0 (socket says 172.21.15.100:45026) 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 -- 172.21.15.100:0/577785033 learned_addr learned my addr 172.21.15.100:0/577785033 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f8197fff640 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c91a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819ce3f640 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 -- 172.21.15.100:0/577785033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 msgr2=0x55ba66c91a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c91a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 -- 172.21.15.100:0/577785033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 msgr2=0x55ba66c9f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 -- 172.21.15.100:0/577785033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba66b9ac20 con 0x55ba66ca1750 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f819d640640 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca85a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8180012ab0 tx=0x7f8180005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a81e796232b873 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.558+0000 7f818e7fc640 1 -- 172.21.15.100:0/577785033 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180020070 con 0x55ba66ca1750 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f818e7fc640 1 -- 172.21.15.100:0/577785033 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8180018040 con 0x55ba66ca1750 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f818e7fc640 1 -- 172.21.15.100:0/577785033 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180017050 con 0x55ba66ca1750 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 -- 172.21.15.100:0/577785033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 msgr2=0x55ba66ca85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca85a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8180012ab0 tx=0x7f8180005f90 comp rx=0 tx=0).stop 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 -- 172.21.15.100:0/577785033 shutdown_connections 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca85a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66c91a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 --2- 172.21.15.100:0/577785033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66c9f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 -- 172.21.15.100:0/577785033 >> 172.21.15.100:0/577785033 conn(0x55ba66c92980 msgr2=0x55ba66c912f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 -- 172.21.15.100:0/577785033 shutdown_connections 2024-03-12T05:16:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.559+0000 7f81a1c70740 1 -- 172.21.15.100:0/577785033 wait complete. 2024-03-12T05:16:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 -- start start 2024-03-12T05:16:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 Processor -- start 2024-03-12T05:16:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66d02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66d02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66ca82e0 con 0x55ba66ca1750 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66ca8450 con 0x55ba66c98250 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba66ca8610 con 0x55ba66c98b70 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f81a1c70740 10 client.0 _ll_register_callbacks cb 0x7f81900184c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f819d640640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f819ce3f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66d02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f819d640640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45030/0 (socket says 172.21.15.100:45030) 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f819ce3f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66d02a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47020/0 (socket says 172.21.15.100:47020) 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f819d640640 1 -- 172.21.15.100:0/946950947 learned_addr learned my addr 172.21.15.100:0/946950947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.560+0000 7f8197fff640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66d02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819d640640 1 -- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 msgr2=0x55ba66d02f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819d640640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66d02f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819d640640 1 -- 172.21.15.100:0/946950947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 msgr2=0x55ba66d02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819d640640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66d02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819d640640 1 -- 172.21.15.100:0/946950947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8180012760 con 0x55ba66ca1750 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8197fff640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66d02f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819ce3f640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66d02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f819d640640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca7d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8180012a80 tx=0x7f8180005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51845aa04ad8c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8177fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8177fff640 1 -- 172.21.15.100:0/946950947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180020040 con 0x55ba66ca1750 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8177fff640 1 -- 172.21.15.100:0/946950947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f818000fdb0 con 0x55ba66ca1750 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81900a31f0 con 0x55ba66ca1750 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8177fff640 1 -- 172.21.15.100:0/946950947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8180017020 con 0x55ba66ca1750 2024-03-12T05:16:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f81a1c70740 10 client.16488 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba66bc08c0 con 0x55ba66ca1750 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8177fff640 1 -- 172.21.15.100:0/946950947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81900a31f0 con 0x55ba66ca1750 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f8177fff640 1 -- 172.21.15.100:0/946950947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8180010700 con 0x55ba66ca1750 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.561+0000 7f81a1c70740 10 client.16488 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 10 client.16488 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 20 client.16488 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66d0a780 0x55ba66d0cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 4 client.16488 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ba6698b890 con 0x55ba66d0a780 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f819ce3f640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66d0a780 0x55ba66d0cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f8177fff640 10 client.16488 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f819ce3f640 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66d0a780 0x55ba66d0cb70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a65b55a5d33d2a2b server_cookie=5ef1b3b620d5be3e in_seq=0 out_seq=0 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f8177fff640 1 -- 172.21.15.100:0/946950947 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ba6698b890 con 0x55ba66d0a780 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f8177fff640 10 client.16488 handle_command_reply: tid=0 2024-03-12T05:16:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 client.16488 shutdown 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 msgr2=0x55ba66ca7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca7d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8180012a80 tx=0x7f8180005850 comp rx=0 tx=0).stop 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 shutdown_connections 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba66ca1750 0x55ba66ca7d20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba66c98b70 0x55ba66d02f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ba66d0a780 0x55ba66d0cb70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 --2- 172.21.15.100:0/946950947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba66c98250 0x55ba66d02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f8177fff640 0 client.16488 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.562+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 >> 172.21.15.100:0/946950947 conn(0x55ba66c92980 msgr2=0x55ba66c944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 shutdown_connections 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f81a1c70740 1 -- 172.21.15.100:0/946950947 wait complete. 2024-03-12T05:16:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f81a1c70740 20 client.16488 trim_cache size 0 max 16384 2024-03-12T05:16:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:16:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f81840a1bd0 msgr2=0x7f81840a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f81840a1bd0 0x7f81840a4090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8188005ab0 tx=0x7f81880046c0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 msgr2=0x7f8198100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 0x7f8198100860 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f819000ebc0 tx=0x7f819000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 shutdown_connections 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f81840a1bd0 0x7f81840a4090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81981012e0 0x7f81981a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198106500 0x7f8198100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 --2- 172.21.15.100:0/2775921419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980ff8f0 0x7f8198100860 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 >> 172.21.15.100:0/2775921419 conn(0x7f8198076640 msgr2=0x7f8198104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 shutdown_connections 2024-03-12T05:16:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.563+0000 7f819f0a3640 1 -- 172.21.15.100:0/2775921419 wait complete. 2024-03-12T05:16:47.570 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:16:47.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:16:47.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 -- 172.21.15.100:0/1653417224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 msgr2=0x7fc5941004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 --2- 172.21.15.100:0/1653417224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc5941004a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009900 tx=0x7fc57c02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 -- 172.21.15.100:0/1653417224 shutdown_connections 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 --2- 172.21.15.100:0/1653417224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5941009e0 0x7fc59410cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 --2- 172.21.15.100:0/1653417224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc5941004a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 --2- 172.21.15.100:0/1653417224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594106100 0x7fc5941064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 -- 172.21.15.100:0/1653417224 >> 172.21.15.100:0/1653417224 conn(0x7fc594077110 msgr2=0x7fc594077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 -- 172.21.15.100:0/1653417224 shutdown_connections 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.680+0000 7fc598c23640 1 -- 172.21.15.100:0/1653417224 wait complete. 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 Processor -- start 2024-03-12T05:16:47.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 -- start start 2024-03-12T05:16:47.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc594198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5941009e0 0x7fc594198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc594106100 0x7fc5941a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941993e0 con 0x7fc594100040 2024-03-12T05:16:47.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941a3be0 con 0x7fc594106100 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc598c23640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941a3d20 con 0x7fc5941009e0 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc594198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc594198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45044/0 (socket says 172.21.15.100:45044) 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 -- 172.21.15.100:0/1615179964 learned_addr learned my addr 172.21.15.100:0/1615179964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc592d9d640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc594106100 0x7fc5941a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc58bfff640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5941009e0 0x7fc594198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 -- 172.21.15.100:0/1615179964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5941009e0 msgr2=0x7fc594198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5941009e0 0x7fc594198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 -- 172.21.15.100:0/1615179964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc594106100 msgr2=0x7fc5941a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc594106100 0x7fc5941a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc59259c640 1 -- 172.21.15.100:0/1615179964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c009560 con 0x7fc594100040 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.681+0000 7fc58bfff640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5941009e0 0x7fc594198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.682+0000 7fc59259c640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc594198620 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ea20 tx=0x7fc58400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.682+0000 7fc58b7fe640 1 -- 172.21.15.100:0/1615179964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584009720 con 0x7fc594100040 2024-03-12T05:16:47.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.682+0000 7fc58b7fe640 1 -- 172.21.15.100:0/1615179964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc584005500 con 0x7fc594100040 2024-03-12T05:16:47.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.682+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5941a3f80 con 0x7fc594100040 2024-03-12T05:16:47.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.682+0000 7fc58b7fe640 1 -- 172.21.15.100:0/1615179964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584010430 con 0x7fc594100040 2024-03-12T05:16:47.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.682+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5941a43b0 con 0x7fc594100040 2024-03-12T05:16:47.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc58b7fe640 1 -- 172.21.15.100:0/1615179964 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc584005020 con 0x7fc594100040 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 Processor -- start 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 -- start start 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d2b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2bb70 0x5585a8d34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d34750 0x5585a8d36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8ce6960 con 0x5585a8d2b250 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c9a310 con 0x5585a8d34750 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c97840 con 0x5585a8d2bb70 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc58b7fe640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5800a1ba0 0x7fc5800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc592d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d2b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc592d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d2b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45054/0 (socket says 172.21.15.100:45054) 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc592d9d640 1 -- 172.21.15.100:0/2652891973 learned_addr learned my addr 172.21.15.100:0/2652891973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc58bfff640 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d34750 0x5585a8d36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc59259c640 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2bb70 0x5585a8d34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.685+0000 7fc592d9d640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5800a1ba0 0x7fc5800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58bfff640 1 -- 172.21.15.100:0/2652891973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2bb70 msgr2=0x5585a8d34210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58bfff640 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2bb70 0x5585a8d34210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58bfff640 1 -- 172.21.15.100:0/2652891973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 msgr2=0x5585a8d2b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58bfff640 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d2b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58b7fe640 1 -- 172.21.15.100:0/1615179964 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc584014070 con 0x7fc594100040 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58bfff640 1 -- 172.21.15.100:0/2652891973 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585a8c55060 con 0x5585a8d34750 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc592d9d640 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d2b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc58bfff640 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d34750 0x5585a8d36b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009230 tx=0x7fc57c02c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1e0cc78b3a8a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc592d9d640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5800a1ba0 0x7fc5800a4060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009fd0 tx=0x7fc58c004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc588ff9640 1 -- 172.21.15.100:0/2652891973 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c004a40 con 0x5585a8d34750 2024-03-12T05:16:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc588ff9640 1 -- 172.21.15.100:0/2652891973 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c004040 con 0x5585a8d34750 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.686+0000 7fc588ff9640 1 -- 172.21.15.100:0/2652891973 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0051d0 con 0x5585a8d34750 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2652891973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d34750 msgr2=0x5585a8d36b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d34750 0x5585a8d36b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009230 tx=0x7fc57c02c770 comp rx=0 tx=0).stop 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2652891973 shutdown_connections 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d34750 0x5585a8d36b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2bb70 0x5585a8d34210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2652891973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d2b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2652891973 >> 172.21.15.100:0/2652891973 conn(0x5585a8d25980 msgr2=0x5585a8d25d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2652891973 shutdown_connections 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2652891973 wait complete. 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 -- start start 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.687+0000 7fc59b7f7740 1 Processor -- start 2024-03-12T05:16:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d32130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c93020 con 0x5585a8d2b250 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8966430 con 0x5585a8d2bb70 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8922e00 con 0x5585a8d34750 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 10 client.0 _ll_register_callbacks cb 0x5585a8c3acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d316b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45058/0 (socket says 172.21.15.100:45058) 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 -- 172.21.15.100:0/3105278958 learned_addr learned my addr 172.21.15.100:0/3105278958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc58bfff640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc592d9d640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d32130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 -- 172.21.15.100:0/3105278958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 msgr2=0x5585a8d32130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d32130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 -- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 msgr2=0x5585a8d31bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d31bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 -- 172.21.15.100:0/3105278958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59410d7a0 con 0x5585a8d2b250 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59259c640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d316b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc58400e9f0 tx=0x7fc58401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4a4bcc6847000a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc579ffb640 1 -- 172.21.15.100:0/3105278958 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5840b1150 con 0x5585a8d2b250 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc579ffb640 1 -- 172.21.15.100:0/3105278958 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc584005670 con 0x5585a8d2b250 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585a89927e0 con 0x5585a8d2b250 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc579ffb640 1 -- 172.21.15.100:0/3105278958 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58409a7c0 con 0x5585a8d2b250 2024-03-12T05:16:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 10 client.16500 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.688+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5585a8c916a0 con 0x5585a8d2b250 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc579ffb640 1 -- 172.21.15.100:0/3105278958 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5585a89927e0 con 0x5585a8d2b250 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc579ffb640 1 -- 172.21.15.100:0/3105278958 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc58401f060 con 0x5585a8d2b250 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc59b7f7740 10 client.16500 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc59b7f7740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc59b7f7740 20 client.16500 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d44800 0x5585a8d46c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc59b7f7740 4 client.16500 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585a89cf720 con 0x5585a8d44800 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc58bfff640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d44800 0x5585a8d46c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc579ffb640 10 client.16500 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.689+0000 7fc58bfff640 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d44800 0x5585a8d46c10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e71924c6ab2d716 server_cookie=247f534eb46fe8ed in_seq=0 out_seq=0 2024-03-12T05:16:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc579ffb640 1 -- 172.21.15.100:0/3105278958 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585a89cf720 con 0x5585a8d44800 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc579ffb640 10 client.16500 handle_command_reply: tid=0 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 client.16500 shutdown 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 msgr2=0x5585a8d316b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d316b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc58400e9f0 tx=0x7fc58401d000 comp rx=0 tx=0).stop 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 shutdown_connections 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d44800 0x5585a8d46c10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d32130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d31bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc579ffb640 0 client.16500 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3105278958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2b250 0x5585a8d316b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 >> 172.21.15.100:0/3105278958 conn(0x5585a8d25980 msgr2=0x5585a8d276d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 shutdown_connections 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3105278958 wait complete. 2024-03-12T05:16:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.690+0000 7fc59b7f7740 20 client.16500 trim_cache size 0 max 16384 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 Processor -- start 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 -- start start 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2b250 0x5585a8d24a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d3b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 0x5585a8d3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c18130 con 0x5585a8d34750 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8d269b0 con 0x5585a8d2bb70 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8d2ac90 con 0x5585a8d2b250 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc592d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 0x5585a8d3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc592d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 0x5585a8d3b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45072/0 (socket says 172.21.15.100:45072) 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc592d9d640 1 -- 172.21.15.100:0/2691631328 learned_addr learned my addr 172.21.15.100:0/2691631328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc59259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2b250 0x5585a8d24a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.702+0000 7fc58bfff640 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d3b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc592d9d640 1 -- 172.21.15.100:0/2691631328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2b250 msgr2=0x5585a8d24a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc592d9d640 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2b250 0x5585a8d24a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc592d9d640 1 -- 172.21.15.100:0/2691631328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 msgr2=0x5585a8d3b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc592d9d640 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d3b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc592d9d640 1 -- 172.21.15.100:0/2691631328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585a8c2dc20 con 0x5585a8d34750 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc592d9d640 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 0x5585a8d3b9c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc58c013880 tx=0x7fc58c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28342dcbd2850bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc57bfff640 1 -- 172.21.15.100:0/2691631328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c013170 con 0x5585a8d34750 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc57bfff640 1 -- 172.21.15.100:0/2691631328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc58c010e10 con 0x5585a8d34750 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc57bfff640 1 -- 172.21.15.100:0/2691631328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c02ad10 con 0x5585a8d34750 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2691631328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 msgr2=0x5585a8d3b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.703+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 0x5585a8d3b9c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc58c013880 tx=0x7fc58c009be0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2691631328 shutdown_connections 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d34750 0x5585a8d3b9c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2bb70 0x5585a8d3b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/2691631328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d2b250 0x5585a8d24a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2691631328 >> 172.21.15.100:0/2691631328 conn(0x5585a8d25980 msgr2=0x5585a8d242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2691631328 shutdown_connections 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- 172.21.15.100:0/2691631328 wait complete. 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- start start 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 Processor -- start 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 0x5585a8d3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 0x5585a8d3a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d9ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8d3af10 con 0x5585a8d2bb70 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8d3b080 con 0x5585a8d2b250 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8d9b280 con 0x5585a8d34750 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59b7f7740 10 client.0 _ll_register_callbacks cb 0x7fc58c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 0x5585a8d3a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 0x5585a8d3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 0x5585a8d3a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47066/0 (socket says 172.21.15.100:47066) 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 0x5585a8d3a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45084/0 (socket says 172.21.15.100:45084) 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 -- 172.21.15.100:0/3343351149 learned_addr learned my addr 172.21.15.100:0/3343351149 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc592d9d640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d9ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 -- 172.21.15.100:0/3343351149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 msgr2=0x5585a8d9ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d9ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 -- 172.21.15.100:0/3343351149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 msgr2=0x5585a8d3a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 0x5585a8d3a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.704+0000 7fc59259c640 1 -- 172.21.15.100:0/3343351149 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c0135c0 con 0x5585a8d2b250 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc58bfff640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 0x5585a8d3a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59259c640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 0x5585a8d3a210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc584098a70 tx=0x7fc584004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82bb8f9844b49b94 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc579ffb640 1 -- 172.21.15.100:0/3343351149 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400cd20 con 0x5585a8d2b250 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc579ffb640 1 -- 172.21.15.100:0/3343351149 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc584005670 con 0x5585a8d2b250 2024-03-12T05:16:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc57c004040 con 0x5585a8d2b250 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc579ffb640 1 -- 172.21.15.100:0/3343351149 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58409a120 con 0x5585a8d2b250 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 10 client.25886 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5585a8c538c0 con 0x5585a8d2b250 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc579ffb640 1 -- 172.21.15.100:0/3343351149 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c004040 con 0x5585a8d2b250 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc579ffb640 1 -- 172.21.15.100:0/3343351149 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc58401f040 con 0x5585a8d2b250 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 10 client.25886 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 20 client.25886 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d9c680 0x5585a8d9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 4 client.25886 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.705+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585a8a1e890 con 0x5585a8d9c680 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc58bfff640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d9c680 0x5585a8d9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc579ffb640 10 client.25886 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc58bfff640 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d9c680 0x5585a8d9eaa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d140f0cd3801e963 server_cookie=754f662e0a3ed3df in_seq=0 out_seq=0 2024-03-12T05:16:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc579ffb640 1 -- 172.21.15.100:0/3343351149 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585a8a1e890 con 0x5585a8d9c680 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc579ffb640 10 client.25886 handle_command_reply: tid=0 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 client.25886 shutdown 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 msgr2=0x5585a8d3a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 0x5585a8d3a210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc584098a70 tx=0x7fc584004920 comp rx=0 tx=0).stop 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 shutdown_connections 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585a8d9c680 0x5585a8d9eaa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585a8d34750 0x5585a8d9ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc579ffb640 0 client.25886 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585a8d2bb70 0x5585a8d3a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 --2- 172.21.15.100:0/3343351149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585a8d2b250 0x5585a8d3a210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 >> 172.21.15.100:0/3343351149 conn(0x5585a8d25980 msgr2=0x5585a8d24b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 shutdown_connections 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 1 -- 172.21.15.100:0/3343351149 wait complete. 2024-03-12T05:16:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.706+0000 7fc59b7f7740 20 client.25886 trim_cache size 0 max 16384 2024-03-12T05:16:47.710 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:16:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5800a1ba0 msgr2=0x7fc5800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5800a1ba0 0x7fc5800a4060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009fd0 tx=0x7fc58c004ab0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 msgr2=0x7fc594198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc594198620 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ea20 tx=0x7fc58400ee20 comp rx=0 tx=0).stop 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 shutdown_connections 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5800a1ba0 0x7fc5800a4060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc594106100 0x7fc5941a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5941009e0 0x7fc594198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 --2- 172.21.15.100:0/1615179964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc594100040 0x7fc594198620 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 >> 172.21.15.100:0/1615179964 conn(0x7fc594077110 msgr2=0x7fc5940fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 shutdown_connections 2024-03-12T05:16:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:47.707+0000 7fc598c23640 1 -- 172.21.15.100:0/1615179964 wait complete. 2024-03-12T05:16:47.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 540186 13.95 MB/sec execute 91 sec latency 68.763 ms 2024-03-12T05:16:48.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:47 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4140210249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:16:48.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:47 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4140210249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:16:48.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:47 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4140210249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:16:48.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:16:48.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:16:48.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:16:48.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 542434 13.88 MB/sec execute 92 sec latency 22.401 ms 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.837+0000 7f07881ac640 1 -- 172.21.15.100:0/583727253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0780101dc0 msgr2=0x7f0780105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- 172.21.15.100:0/583727253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0780101dc0 0x7f0780105e00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0770009900 tx=0x7f077002bc30 comp rx=0 tx=0).stop 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 -- 172.21.15.100:0/583727253 shutdown_connections 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- 172.21.15.100:0/583727253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801064e0 0x7f078010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- 172.21.15.100:0/583727253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0780101dc0 0x7f0780105e00 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- 172.21.15.100:0/583727253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f07801017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 -- 172.21.15.100:0/583727253 >> 172.21.15.100:0/583727253 conn(0x7f07800fcc70 msgr2=0x7f07800ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 -- 172.21.15.100:0/583727253 shutdown_connections 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 -- 172.21.15.100:0/583727253 wait complete. 2024-03-12T05:16:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 Processor -- start 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 -- start start 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f078019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 0x7f078019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801998f0 0x7f0780199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.838+0000 7f07881ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019a2e0 con 0x7f07801064e0 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f07881ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019a450 con 0x7f07801998f0 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f07881ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019a610 con 0x7f0780101410 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785747640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 0x7f078019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f078019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f078019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52456/0 (socket says 172.21.15.100:52456) 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 -- 172.21.15.100:0/642226600 learned_addr learned my addr 172.21.15.100:0/642226600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785747640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 0x7f078019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45088/0 (socket says 172.21.15.100:45088) 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0786749640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801998f0 0x7f0780199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 -- 172.21.15.100:0/642226600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801998f0 msgr2=0x7f0780199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801998f0 0x7f0780199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 -- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 msgr2=0x7f078019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 0x7f078019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 -- 172.21.15.100:0/642226600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009560 con 0x7f0780101410 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785747640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 0x7f078019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0786749640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801998f0 0x7f0780199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0785f48640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f078019f740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e9f0 tx=0x7f077c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0776ffd640 1 -- 172.21.15.100:0/642226600 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c009720 con 0x7f0780101410 2024-03-12T05:16:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07801a6600 con 0x7f0780101410 2024-03-12T05:16:48.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0776ffd640 1 -- 172.21.15.100:0/642226600 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c00cd80 con 0x7f0780101410 2024-03-12T05:16:48.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f0776ffd640 1 -- 172.21.15.100:0/642226600 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c010690 con 0x7f0780101410 2024-03-12T05:16:48.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.839+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07801a6ae0 con 0x7f0780101410 2024-03-12T05:16:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.841+0000 7f0776ffd640 1 -- 172.21.15.100:0/642226600 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f077c0046c0 con 0x7f0780101410 2024-03-12T05:16:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 Processor -- start 2024-03-12T05:16:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 -- start start 2024-03-12T05:16:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 0x56348abd02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abd08b0 0x56348abd2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348ab35f90 con 0x56348abd08b0 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348ab334c0 con 0x56348abc7380 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f078ad80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348ab2eca0 con 0x56348abc7d30 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0785f48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 0x56348abd02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0786749640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0776ffd640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f076c0a1ba0 0x7f076c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0786749640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abc7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47102/0 (socket says 172.21.15.100:47102) 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0786749640 1 -- 172.21.15.100:0/876665663 learned_addr learned my addr 172.21.15.100:0/876665663 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0785f48640 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 0x56348abd02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52470/0 (socket says 172.21.15.100:52470) 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.842+0000 7f0785747640 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abd08b0 0x56348abd2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0786749640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f076c0a1ba0 0x7f076c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0776ffd640 1 -- 172.21.15.100:0/642226600 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f077c014070 con 0x7f0780101410 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0785f48640 1 -- 172.21.15.100:0/876665663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 msgr2=0x56348abc7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0785f48640 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abc7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0785f48640 1 -- 172.21.15.100:0/876665663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abd08b0 msgr2=0x56348abd2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0785f48640 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abd08b0 0x56348abd2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0785f48640 1 -- 172.21.15.100:0/876665663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56348aaf10b0 con 0x56348abc7d30 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0785f48640 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 0x56348abd02e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cb2d0 tx=0x7f077c004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8702360929ff3a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f0786749640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f076c0a1ba0 0x7f076c0a4060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0778005c40 tx=0x7f0778004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f076bfff640 1 -- 172.21.15.100:0/876665663 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c09a7b0 con 0x56348abc7d30 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f076bfff640 1 -- 172.21.15.100:0/876665663 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c09aeb0 con 0x56348abc7d30 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.843+0000 7f076bfff640 1 -- 172.21.15.100:0/876665663 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c0a2d20 con 0x56348abc7d30 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 -- 172.21.15.100:0/876665663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 msgr2=0x56348abd02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 0x56348abd02e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cb2d0 tx=0x7f077c004b30 comp rx=0 tx=0).stop 2024-03-12T05:16:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 -- 172.21.15.100:0/876665663 shutdown_connections 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abd08b0 0x56348abd2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7d30 0x56348abd02e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 --2- 172.21.15.100:0/876665663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abc7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 -- 172.21.15.100:0/876665663 >> 172.21.15.100:0/876665663 conn(0x56348abc1840 msgr2=0x56348abc1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 -- 172.21.15.100:0/876665663 shutdown_connections 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.844+0000 7f078ad80740 1 -- 172.21.15.100:0/876665663 wait complete. 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 -- start start 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 Processor -- start 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abcd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abcdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abd08b0 0x56348abce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348aad3f40 con 0x56348abc7d30 2024-03-12T05:16:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348a8024b0 con 0x56348abc7380 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348aab4670 con 0x56348abd08b0 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f078ad80740 10 client.0 _ll_register_callbacks cb 0x56348ab21f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785f48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abcd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785747640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abcdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785f48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abcd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47104/0 (socket says 172.21.15.100:47104) 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785747640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abcdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45104/0 (socket says 172.21.15.100:45104) 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785f48640 1 -- 172.21.15.100:0/461291275 learned_addr learned my addr 172.21.15.100:0/461291275 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0786749640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abd08b0 0x56348abce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785747640 1 -- 172.21.15.100:0/461291275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abd08b0 msgr2=0x56348abce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.845+0000 7f0785747640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abd08b0 0x56348abce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f0785747640 1 -- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 msgr2=0x56348abcd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f0785747640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abcd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f0785747640 1 -- 172.21.15.100:0/461291275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07801a6ae0 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f0785f48640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abcd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f0785747640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abcdda0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f077002c170 tx=0x7f0770004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8266a2a63724076 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f07697fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f07697fa640 1 -- 172.21.15.100:0/461291275 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770004480 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56348ab08f40 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f07697fa640 1 -- 172.21.15.100:0/461291275 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0770004df0 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f07697fa640 1 -- 172.21.15.100:0/461291275 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0770004fb0 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 10 client.16518 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56348ab2d320 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f07697fa640 1 -- 172.21.15.100:0/461291275 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56348ab08f40 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f07697fa640 1 -- 172.21.15.100:0/461291275 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f077003c040 con 0x56348abc7d30 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 10 client.16518 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 20 client.16518 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abe0dc0 0x56348abe31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 4 client.16518 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.846+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56348ab25310 con 0x56348abe0dc0 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.847+0000 7f0785f48640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abe0dc0 0x56348abe31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.847+0000 7f07697fa640 10 client.16518 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.847+0000 7f0785f48640 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abe0dc0 0x56348abe31e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e44cdbfd547b54 server_cookie=de4597a85cbc84b1 in_seq=0 out_seq=0 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.847+0000 7f07697fa640 1 -- 172.21.15.100:0/461291275 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56348ab25310 con 0x56348abe0dc0 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.847+0000 7f07697fa640 10 client.16518 handle_command_reply: tid=0 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 client.16518 shutdown 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 msgr2=0x56348abcdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abcdda0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f077002c170 tx=0x7f0770004040 comp rx=0 tx=0).stop 2024-03-12T05:16:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 shutdown_connections 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abe0dc0 0x56348abe31e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abd08b0 0x56348abce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abcdda0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f07697fa640 0 client.16518 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 --2- 172.21.15.100:0/461291275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7380 0x56348abcd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 >> 172.21.15.100:0/461291275 conn(0x56348abc1840 msgr2=0x56348abc37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 shutdown_connections 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 1 -- 172.21.15.100:0/461291275 wait complete. 2024-03-12T05:16:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.848+0000 7f078ad80740 20 client.16518 trim_cache size 0 max 16384 2024-03-12T05:16:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 Processor -- start 2024-03-12T05:16:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 -- start start 2024-03-12T05:16:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc1840 0x56348abce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abc3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abd7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348abbd180 con 0x56348abc7d30 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348abdd900 con 0x56348abc1840 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f078ad80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348abc1090 con 0x56348abc7380 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abd7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abd7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45118/0 (socket says 172.21.15.100:45118) 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 -- 172.21.15.100:0/4162898649 learned_addr learned my addr 172.21.15.100:0/4162898649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0785f48640 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc1840 0x56348abce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0785747640 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abc3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 -- 172.21.15.100:0/4162898649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 msgr2=0x56348abc3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abc3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 -- 172.21.15.100:0/4162898649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc1840 msgr2=0x56348abce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc1840 0x56348abce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.865+0000 7f0786749640 1 -- 172.21.15.100:0/4162898649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56348aac9c00 con 0x56348abc7d30 2024-03-12T05:16:48.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.866+0000 7f0786749640 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abd7880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f077800fe00 tx=0x7f0778005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8873c688b2d86e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.866+0000 7f076b7fe640 1 -- 172.21.15.100:0/4162898649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778025070 con 0x56348abc7d30 2024-03-12T05:16:48.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.866+0000 7f076b7fe640 1 -- 172.21.15.100:0/4162898649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0778010dc0 con 0x56348abc7d30 2024-03-12T05:16:48.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.866+0000 7f076b7fe640 1 -- 172.21.15.100:0/4162898649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778005790 con 0x56348abc7d30 2024-03-12T05:16:48.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 -- 172.21.15.100:0/4162898649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 msgr2=0x56348abd7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abd7880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f077800fe00 tx=0x7f0778005bd0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 -- 172.21.15.100:0/4162898649 shutdown_connections 2024-03-12T05:16:48.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc7d30 0x56348abd7880 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abc3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 --2- 172.21.15.100:0/4162898649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc1840 0x56348abce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 -- 172.21.15.100:0/4162898649 >> 172.21.15.100:0/4162898649 conn(0x56348abc8480 msgr2=0x56348abc2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 -- 172.21.15.100:0/4162898649 shutdown_connections 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 -- 172.21.15.100:0/4162898649 wait complete. 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 -- start start 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.867+0000 7f078ad80740 1 Processor -- start 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 0x56348abd6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abd74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7d30 0x56348ac30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348ac30aa0 con 0x56348abc1840 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348ac30c10 con 0x56348abc7d30 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348ac30dd0 con 0x56348abc7380 2024-03-12T05:16:48.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 0x56348abd6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 0x56348abd6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45120/0 (socket says 172.21.15.100:45120) 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 -- 172.21.15.100:0/1976742059 learned_addr learned my addr 172.21.15.100:0/1976742059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785747640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abd74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0786749640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7d30 0x56348ac30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f078ad80740 10 client.0 _ll_register_callbacks cb 0x7f077c09a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 -- 172.21.15.100:0/1976742059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 msgr2=0x56348abd74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abd74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 -- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7d30 msgr2=0x56348ac30560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7d30 0x56348ac30560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785f48640 1 -- 172.21.15.100:0/1976742059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0778013e40 con 0x56348abc1840 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0785747640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abd74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.868+0000 7f0786749640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7d30 0x56348ac30560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:48.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.869+0000 7f0785f48640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 0x56348abd6fb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cb2f0 tx=0x7f077c085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bec9ad2259fdd3d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.869+0000 7f07697fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f07697fa640 1 -- 172.21.15.100:0/1976742059 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c0b3320 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f07697fa640 1 -- 172.21.15.100:0/1976742059 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f077c09aeb0 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56348ab54900 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f07697fa640 1 -- 172.21.15.100:0/1976742059 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077c096a50 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 10 client.16524 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56348aaef910 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f07697fa640 1 -- 172.21.15.100:0/1976742059 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56348ab54900 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f07697fa640 1 -- 172.21.15.100:0/1976742059 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f077c01f060 con 0x56348abc1840 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 10 client.16524 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 20 client.16524 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:48.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abd08b0 0x56348ac3b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 4 client.16524 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:16:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.870+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56348a82e860 con 0x56348abd08b0 2024-03-12T05:16:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.871+0000 7f0785747640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abd08b0 0x56348ac3b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.871+0000 7f07697fa640 10 client.16524 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.871+0000 7f0785747640 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abd08b0 0x56348ac3b4f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e60396a25328982 server_cookie=fdfc9799ac9bb8e in_seq=0 out_seq=0 2024-03-12T05:16:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f07697fa640 1 -- 172.21.15.100:0/1976742059 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56348a82e860 con 0x56348abd08b0 2024-03-12T05:16:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f07697fa640 10 client.16524 handle_command_reply: tid=0 2024-03-12T05:16:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 client.16524 shutdown 2024-03-12T05:16:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 msgr2=0x56348abd6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 0x56348abd6fb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cb2f0 tx=0x7f077c085040 comp rx=0 tx=0).stop 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 shutdown_connections 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56348abd08b0 0x56348ac3b4f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56348abc7d30 0x56348ac30560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56348abc7380 0x56348abd74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f07697fa640 0 client.16524 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 --2- 172.21.15.100:0/1976742059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56348abc1840 0x56348abd6fb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.872+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 >> 172.21.15.100:0/1976742059 conn(0x56348abc8480 msgr2=0x56348abc3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 shutdown_connections 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f078ad80740 1 -- 172.21.15.100:0/1976742059 wait complete. 2024-03-12T05:16:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f078ad80740 20 client.16524 trim_cache size 0 max 16384 2024-03-12T05:16:48.874 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:16:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:16:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "95223c2f-58eb-4b2e-b3ac-e75734a31062", 2024-03-12T05:16:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:16:48.874 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f076c0a1ba0 msgr2=0x7f076c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f076c0a1ba0 0x7f076c0a4060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0778005c40 tx=0x7f0778004ab0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 msgr2=0x7f078019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f078019f740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e9f0 tx=0x7f077c00edf0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 shutdown_connections 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f076c0a1ba0 0x7f076c0a4060 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07801998f0 0x7f0780199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07801064e0 0x7f078019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 --2- 172.21.15.100:0/642226600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0780101410 0x7f078019f740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 >> 172.21.15.100:0/642226600 conn(0x7f07800fcc70 msgr2=0x7f07800fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 shutdown_connections 2024-03-12T05:16:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.873+0000 7f07881ac640 1 -- 172.21.15.100:0/642226600 wait complete. 2024-03-12T05:16:48.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:16:48.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:16:48.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 -- 172.21.15.100:0/992278241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674100270 msgr2=0x7f0674100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 --2- 172.21.15.100:0/992278241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674100270 0x7f0674100650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0670009cd0 tx=0x7f067002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 -- 172.21.15.100:0/992278241 shutdown_connections 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 --2- 172.21.15.100:0/992278241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674105340 0x7f067410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 --2- 172.21.15.100:0/992278241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0674100c20 0x7f0674104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 --2- 172.21.15.100:0/992278241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674100270 0x7f0674100650 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 -- 172.21.15.100:0/992278241 >> 172.21.15.100:0/992278241 conn(0x7f0674077230 msgr2=0x7f0674077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 -- 172.21.15.100:0/992278241 shutdown_connections 2024-03-12T05:16:48.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.997+0000 7f067ace1640 1 -- 172.21.15.100:0/992278241 wait complete. 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 Processor -- start 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 -- start start 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0674100270 0x7f0674198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 0x7f0674198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06741990d0 0x7f06741a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06741a3d20 con 0x7f0674105340 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06741a3e90 con 0x7f06741990d0 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067ace1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06741a4050 con 0x7f0674100270 2024-03-12T05:16:48.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 0x7f0674198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 0x7f0674198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45130/0 (socket says 172.21.15.100:45130) 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 -- 172.21.15.100:0/2619713638 learned_addr learned my addr 172.21.15.100:0/2619713638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f0678a7d640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0674100270 0x7f0674198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f067927e640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06741990d0 0x7f06741a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 -- 172.21.15.100:0/2619713638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0674100270 msgr2=0x7f0674198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0674100270 0x7f0674198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 -- 172.21.15.100:0/2619713638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06741990d0 msgr2=0x7f06741a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06741990d0 0x7f06741a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.998+0000 7f066bfff640 1 -- 172.21.15.100:0/2619713638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0670009900 con 0x7f0674105340 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.999+0000 7f066bfff640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 0x7f0674198b40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f066400d770 tx=0x7f066400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.999+0000 7f0669ffb640 1 -- 172.21.15.100:0/2619713638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664009c30 con 0x7f0674105340 2024-03-12T05:16:48.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.999+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06741a4330 con 0x7f0674105340 2024-03-12T05:16:48.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.999+0000 7f0669ffb640 1 -- 172.21.15.100:0/2619713638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f066400de80 con 0x7f0674105340 2024-03-12T05:16:48.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.999+0000 7f0669ffb640 1 -- 172.21.15.100:0/2619713638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400f6b0 con 0x7f0674105340 2024-03-12T05:16:48.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:48.999+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06741a4890 con 0x7f0674105340 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.001+0000 7f0669ffb640 1 -- 172.21.15.100:0/2619713638 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0664010460 con 0x7f0674105340 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.001+0000 7f067d98a740 1 Processor -- start 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.001+0000 7f067d98a740 1 -- start start 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067d98a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067d98a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fd0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067d98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fd2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067d98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f36280 con 0x560015fd07b0 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067d98a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f337b0 con 0x560015fc72b0 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f0669ffb640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f065c0a1ba0 0x7f065c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067d98a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f2ef90 con 0x560015fc7bd0 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f066bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fd2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47160/0 (socket says 172.21.15.100:47160) 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f0678a7d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fd0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 -- 172.21.15.100:0/979218972 learned_addr learned my addr 172.21.15.100:0/979218972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f065c0a1ba0 0x7f065c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f0669ffb640 1 -- 172.21.15.100:0/2619713638 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f06640cd520 con 0x7f0674105340 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 -- 172.21.15.100:0/979218972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 msgr2=0x560015fd0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fd0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 -- 172.21.15.100:0/979218972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 msgr2=0x560015fd2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fd2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 -- 172.21.15.100:0/979218972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560015eed570 con 0x560015fc72b0 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f0678a7d640 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fd0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:49.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f065c0a1ba0 0x7f065c0a4060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f066c00f850 tx=0x7f066c018000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.002+0000 7f067927e640 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc7690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f066c011a30 tx=0x7f066c011ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129260ae45ca160 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f065affd640 1 -- 172.21.15.100:0/979218972 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c004030 con 0x560015fc72b0 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f065affd640 1 -- 172.21.15.100:0/979218972 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f066c009a20 con 0x560015fc72b0 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f065affd640 1 -- 172.21.15.100:0/979218972 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c003a50 con 0x560015fc72b0 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 -- 172.21.15.100:0/979218972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 msgr2=0x560015fc7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc7690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f066c011a30 tx=0x7f066c011ef0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 -- 172.21.15.100:0/979218972 shutdown_connections 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fd2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fd0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 --2- 172.21.15.100:0/979218972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc7690 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 -- 172.21.15.100:0/979218972 >> 172.21.15.100:0/979218972 conn(0x560015fc1990 msgr2=0x560015fc1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 -- 172.21.15.100:0/979218972 shutdown_connections 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.003+0000 7f067d98a740 1 -- 172.21.15.100:0/979218972 wait complete. 2024-03-12T05:16:49.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 -- start start 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 Processor -- start 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 0x560015fd01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc7bd0 0x560015fcd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fd07b0 0x560015fcdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015c02430 con 0x560015fc72b0 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015bbee00 con 0x560015fc7bd0 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015eb3fd0 con 0x560015fd07b0 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067d98a740 10 client.0 _ll_register_callbacks cb 0x560015f09fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f0678a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 0x560015fd01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f0678a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 0x560015fd01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45148/0 (socket says 172.21.15.100:45148) 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f0678a7d640 1 -- 172.21.15.100:0/604508055 learned_addr learned my addr 172.21.15.100:0/604508055 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f066bfff640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc7bd0 0x560015fcd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.004+0000 7f067927e640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fd07b0 0x560015fcdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0678a7d640 1 -- 172.21.15.100:0/604508055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fd07b0 msgr2=0x560015fcdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0678a7d640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fd07b0 0x560015fcdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0678a7d640 1 -- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc7bd0 msgr2=0x560015fcd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0678a7d640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc7bd0 0x560015fcd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0678a7d640 1 -- 172.21.15.100:0/604508055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c019040 con 0x560015fc72b0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f066bfff640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc7bd0 0x560015fcd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0678a7d640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 0x560015fd01e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0670037040 tx=0x7f0670034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91ebb0cbc9d8a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0653fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0653fff640 1 -- 172.21.15.100:0/604508055 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06700328b0 con 0x560015fc72b0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0653fff640 1 -- 172.21.15.100:0/604508055 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0670046070 con 0x560015fc72b0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560015c2e7e0 con 0x560015fc72b0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0653fff640 1 -- 172.21.15.100:0/604508055 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0670032eb0 con 0x560015fc72b0 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f067d98a740 10 client.16536 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560015f2d610 con 0x560015fc72b0 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.005+0000 7f0653fff640 1 -- 172.21.15.100:0/604508055 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560015c2e7e0 con 0x560015fc72b0 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f0653fff640 1 -- 172.21.15.100:0/604508055 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0670004570 con 0x560015fc72b0 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f067d98a740 10 client.16536 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f067d98a740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f067d98a740 20 client.16536 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f067d98a740 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560015fe0970 0x560015fe2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f067d98a740 4 client.16536 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560015c6b720 con 0x560015fe0970 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f066bfff640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560015fe0970 0x560015fe2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f0653fff640 10 client.16536 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.006+0000 7f066bfff640 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560015fe0970 0x560015fe2d60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e004f9b74d447ef server_cookie=12b4d8d5f33c84b3 in_seq=0 out_seq=0 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f0653fff640 1 -- 172.21.15.100:0/604508055 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560015c6b720 con 0x560015fe0970 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f0653fff640 10 client.16536 handle_command_reply: tid=0 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 client.16536 shutdown 2024-03-12T05:16:49.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 msgr2=0x560015fd01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 0x560015fd01e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0670037040 tx=0x7f0670034670 comp rx=0 tx=0).stop 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 shutdown_connections 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560015fe0970 0x560015fe2d60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fd07b0 0x560015fcdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc7bd0 0x560015fcd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f0653fff640 0 client.16536 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 --2- 172.21.15.100:0/604508055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc72b0 0x560015fd01e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 >> 172.21.15.100:0/604508055 conn(0x560015fc1990 msgr2=0x560015fc3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.007+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 shutdown_connections 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.008+0000 7f067d98a740 1 -- 172.21.15.100:0/604508055 wait complete. 2024-03-12T05:16:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.008+0000 7f067d98a740 20 client.16536 trim_cache size 0 max 16384 2024-03-12T05:16:49.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 Processor -- start 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 -- start start 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc72b0 0x560015fcfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 0x560015fd01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fd07b0 0x560015fcbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015fc6f10 con 0x560015fc7bd0 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015fc2790 con 0x560015fd07b0 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067d98a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015fe7f90 con 0x560015fc72b0 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 0x560015fd01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 0x560015fd01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45152/0 (socket says 172.21.15.100:45152) 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 -- 172.21.15.100:0/2483687732 learned_addr learned my addr 172.21.15.100:0/2483687732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f067927e640 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fd07b0 0x560015fcbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f0678a7d640 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc72b0 0x560015fcfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 -- 172.21.15.100:0/2483687732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc72b0 msgr2=0x560015fcfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc72b0 0x560015fcfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 -- 172.21.15.100:0/2483687732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fd07b0 msgr2=0x560015fcbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fd07b0 0x560015fcbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.024+0000 7f066bfff640 1 -- 172.21.15.100:0/2483687732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560015ec9ac0 con 0x560015fc7bd0 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.025+0000 7f066bfff640 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 0x560015fd01e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f06640cad10 tx=0x7f0664065530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8217a003b1b85c7f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.025+0000 7f065a7fc640 1 -- 172.21.15.100:0/2483687732 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066409e290 con 0x560015fc7bd0 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.025+0000 7f065a7fc640 1 -- 172.21.15.100:0/2483687732 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06640a28e0 con 0x560015fc7bd0 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.025+0000 7f065a7fc640 1 -- 172.21.15.100:0/2483687732 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664096860 con 0x560015fc7bd0 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- 172.21.15.100:0/2483687732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 msgr2=0x560015fd01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 0x560015fd01e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f06640cad10 tx=0x7f0664065530 comp rx=0 tx=0).stop 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- 172.21.15.100:0/2483687732 shutdown_connections 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fd07b0 0x560015fcbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fc7bd0 0x560015fd01e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- 172.21.15.100:0/2483687732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc72b0 0x560015fcfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- 172.21.15.100:0/2483687732 >> 172.21.15.100:0/2483687732 conn(0x560015fc1990 msgr2=0x560015fc8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- 172.21.15.100:0/2483687732 shutdown_connections 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- 172.21.15.100:0/2483687732 wait complete. 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- start start 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 Processor -- start 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fc9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015fcb6c0 con 0x560015fd07b0 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001603aeb0 con 0x560015fc72b0 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.026+0000 7f067d98a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001603b020 con 0x560015fc7bd0 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f067d98a740 10 client.0 _ll_register_callbacks cb 0x7f066c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:16:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f067927e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f0678a7d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fc9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f067927e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45166/0 (socket says 172.21.15.100:45166) 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f0678a7d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47188/0 (socket says 172.21.15.100:47188) 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f067927e640 1 -- 172.21.15.100:0/524981912 learned_addr learned my addr 172.21.15.100:0/524981912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 -- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 msgr2=0x560015fc9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 -- 172.21.15.100:0/524981912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 msgr2=0x560015fca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 -- 172.21.15.100:0/524981912 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06741a4890 con 0x560015fc7bd0 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f067927e640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f0678a7d640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:16:49.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f066bfff640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fc9d60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x560015fc7050 tx=0x7f06640cac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a4addd7495aef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f0653fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f0653fff640 1 -- 172.21.15.100:0/524981912 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640caee0 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f0653fff640 1 -- 172.21.15.100:0/524981912 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f06640af020 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.027+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f066c009a20 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f0653fff640 1 -- 172.21.15.100:0/524981912 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06640cb040 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 10 client.25981 fetch_fsmap learned FSMap version 25 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560015eef6a0 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f0653fff640 1 -- 172.21.15.100:0/524981912 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f066c009a20 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f0653fff640 1 -- 172.21.15.100:0/524981912 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0664099640 con 0x560015fc7bd0 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 10 client.25981 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 20 client.25981 populate_metadata read hostname 'smithi100' 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56001603c3f0 0x56001603e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:16:49.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 4 client.25981 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560015cba890 con 0x56001603c3f0 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f0678a7d640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56001603c3f0 0x56001603e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.028+0000 7f0653fff640 10 client.25981 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f0678a7d640 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56001603c3f0 0x56001603e810 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f2ccdd0104fa22 server_cookie=1c73e643885578d9 in_seq=0 out_seq=0 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f0653fff640 1 -- 172.21.15.100:0/524981912 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560015cba890 con 0x56001603c3f0 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f0653fff640 10 client.25981 handle_command_reply: tid=0 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 client.25981 shutdown 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 msgr2=0x560015fc9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fc9d60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x560015fc7050 tx=0x7f06640cac40 comp rx=0 tx=0).stop 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 shutdown_connections 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56001603c3f0 0x56001603e810 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560015fd07b0 0x560015fca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f0653fff640 0 client.25981 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:16:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560015fc7bd0 0x560015fc9d60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 --2- 172.21.15.100:0/524981912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560015fc72b0 0x560015fc9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 >> 172.21.15.100:0/524981912 conn(0x560015fc1990 msgr2=0x560015fcbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 shutdown_connections 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 1 -- 172.21.15.100:0/524981912 wait complete. 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.029+0000 7f067d98a740 20 client.25981 trim_cache size 0 max 16384 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:16:49.033 INFO:teuthology.orchestra.run.smithi100.stdout: "95223c2f-58eb-4b2e-b3ac-e75734a31062": { 2024-03-12T05:16:49.034 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:16:49.034 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "95223c2f-58eb-4b2e-b3ac-e75734a31062", 2024-03-12T05:16:49.034 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:16:49.034 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:16:49.034 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:16:49.034 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f065c0a1ba0 msgr2=0x7f065c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f065c0a1ba0 0x7f065c0a4060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f066c00f850 tx=0x7f066c018000 comp rx=0 tx=0).stop 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 msgr2=0x7f0674198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 0x7f0674198b40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f066400d770 tx=0x7f066400dc30 comp rx=0 tx=0).stop 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 shutdown_connections 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f065c0a1ba0 0x7f065c0a4060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06741990d0 0x7f06741a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674105340 0x7f0674198b40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 --2- 172.21.15.100:0/2619713638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0674100270 0x7f0674198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:16:49.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 >> 172.21.15.100:0/2619713638 conn(0x7f0674077230 msgr2=0x7f06740fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:16:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 shutdown_connections 2024-03-12T05:16:49.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:16:49.030+0000 7f067ace1640 1 -- 172.21.15.100:0/2619713638 wait complete. 2024-03-12T05:16:49.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:48 smithi100 ceph-mon[32210]: pgmap v144: 97 pgs: 97 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 332 op/s 2024-03-12T05:16:49.037 INFO:tasks.cephfs.filesystem:scrub status for tag:95223c2f-58eb-4b2e-b3ac-e75734a31062 - {'path': '/', 'tag': '95223c2f-58eb-4b2e-b3ac-e75734a31062', 'options': 'recursive,force'} 2024-03-12T05:16:49.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:48 smithi156 ceph-mon[36270]: pgmap v144: 97 pgs: 97 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 332 op/s 2024-03-12T05:16:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:48 smithi161 ceph-mon[36043]: pgmap v144: 97 pgs: 97 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 332 op/s 2024-03-12T05:16:49.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 544340 13.87 MB/sec execute 93 sec latency 43.349 ms 2024-03-12T05:16:50.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:49 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:16:50.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:49 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:49 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:16:50.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:49 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:16:50.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:49 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:49 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:49 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:16:50.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:49 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:49 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:16:50.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:49 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:16:50.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:49 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:49 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:49 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:16:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:49 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:49 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:16:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:49 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:16:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:49 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:49 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:16:50.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 547334 13.92 MB/sec execute 94 sec latency 12.445 ms 2024-03-12T05:16:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:50 smithi156 ceph-mon[36270]: pgmap v145: 97 pgs: 97 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-12T05:16:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:50 smithi161 ceph-mon[36043]: pgmap v145: 97 pgs: 97 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-12T05:16:51.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:50 smithi100 ceph-mon[32210]: pgmap v145: 97 pgs: 97 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-12T05:16:51.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 549788 13.89 MB/sec execute 95 sec latency 13.791 ms 2024-03-12T05:16:52.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 551673 13.85 MB/sec execute 96 sec latency 121.725 ms 2024-03-12T05:16:53.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:52 smithi156 ceph-mon[36270]: pgmap v146: 97 pgs: 97 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 328 op/s 2024-03-12T05:16:53.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:52 smithi161 ceph-mon[36043]: pgmap v146: 97 pgs: 97 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 328 op/s 2024-03-12T05:16:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:52 smithi100 ceph-mon[32210]: pgmap v146: 97 pgs: 97 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 328 op/s 2024-03-12T05:16:53.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 554533 13.88 MB/sec execute 97 sec latency 15.737 ms 2024-03-12T05:16:54.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 557183 13.87 MB/sec execute 98 sec latency 16.305 ms 2024-03-12T05:16:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:54 smithi156 ceph-mon[36270]: pgmap v147: 97 pgs: 97 active+clean; 321 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-03-12T05:16:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:54 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:16:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:54 smithi161 ceph-mon[36043]: pgmap v147: 97 pgs: 97 active+clean; 321 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-03-12T05:16:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:54 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:16:55.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:54 smithi100 ceph-mon[32210]: pgmap v147: 97 pgs: 97 active+clean; 321 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-03-12T05:16:55.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:54 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:16:55.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 559345 13.85 MB/sec execute 99 sec latency 8.389 ms 2024-03-12T05:16:56.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:55 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:16:56.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:55 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:16:56.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:55 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:16:56.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 561970 13.85 MB/sec execute 100 sec latency 11.906 ms 2024-03-12T05:16:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:56 smithi156 ceph-mon[36270]: pgmap v148: 97 pgs: 97 active+clean; 321 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 229 op/s 2024-03-12T05:16:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:56 smithi161 ceph-mon[36043]: pgmap v148: 97 pgs: 97 active+clean; 321 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 229 op/s 2024-03-12T05:16:57.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:56 smithi100 ceph-mon[32210]: pgmap v148: 97 pgs: 97 active+clean; 321 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 229 op/s 2024-03-12T05:16:57.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 564640 13.86 MB/sec execute 101 sec latency 147.459 ms 2024-03-12T05:16:58.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 566489 13.82 MB/sec execute 102 sec latency 19.336 ms 2024-03-12T05:16:59.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:16:58 smithi156 ceph-mon[36270]: pgmap v149: 97 pgs: 97 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 343 op/s 2024-03-12T05:16:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:16:58 smithi161 ceph-mon[36043]: pgmap v149: 97 pgs: 97 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 343 op/s 2024-03-12T05:16:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:16:58 smithi100 ceph-mon[32210]: pgmap v149: 97 pgs: 97 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 343 op/s 2024-03-12T05:16:59.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 569086 13.82 MB/sec execute 103 sec latency 11.596 ms 2024-03-12T05:17:00.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 571601 13.82 MB/sec execute 104 sec latency 20.222 ms 2024-03-12T05:17:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:00 smithi156 ceph-mon[36270]: pgmap v150: 97 pgs: 97 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-03-12T05:17:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:00 smithi161 ceph-mon[36043]: pgmap v150: 97 pgs: 97 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-03-12T05:17:01.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:00 smithi100 ceph-mon[32210]: pgmap v150: 97 pgs: 97 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-03-12T05:17:01.789 INFO:tasks.workunit.client.0.smithi100.stdout: 1 573579 13.79 MB/sec execute 105 sec latency 19.725 ms 2024-03-12T05:17:02.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 575936 13.79 MB/sec execute 106 sec latency 70.806 ms 2024-03-12T05:17:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:02 smithi156 ceph-mon[36270]: pgmap v151: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 379 op/s 2024-03-12T05:17:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:02 smithi161 ceph-mon[36043]: pgmap v151: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 379 op/s 2024-03-12T05:17:03.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:02 smithi100 ceph-mon[32210]: pgmap v151: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 379 op/s 2024-03-12T05:17:03.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 578976 13.82 MB/sec execute 107 sec latency 14.497 ms 2024-03-12T05:17:04.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 581431 13.77 MB/sec execute 108 sec latency 11.485 ms 2024-03-12T05:17:05.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:04 smithi156 ceph-mon[36270]: pgmap v152: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 265 op/s 2024-03-12T05:17:05.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:04 smithi161 ceph-mon[36043]: pgmap v152: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 265 op/s 2024-03-12T05:17:05.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:04 smithi100 ceph-mon[32210]: pgmap v152: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 265 op/s 2024-03-12T05:17:05.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 583710 13.78 MB/sec execute 109 sec latency 9.940 ms 2024-03-12T05:17:06.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586632 13.81 MB/sec execute 110 sec latency 12.311 ms 2024-03-12T05:17:07.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:06 smithi156 ceph-mon[36270]: pgmap v153: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 264 op/s 2024-03-12T05:17:07.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:06 smithi161 ceph-mon[36043]: pgmap v153: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 264 op/s 2024-03-12T05:17:07.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:06 smithi100 ceph-mon[32210]: pgmap v153: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 264 op/s 2024-03-12T05:17:07.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 13.68 MB/sec execute 111 sec latency 962.516 ms 2024-03-12T05:17:08.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 13.56 MB/sec execute 112 sec latency 1962.593 ms 2024-03-12T05:17:09.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:08 smithi156 ceph-mon[36270]: pgmap v154: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 385 op/s 2024-03-12T05:17:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:08 smithi161 ceph-mon[36043]: pgmap v154: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 385 op/s 2024-03-12T05:17:09.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:08 smithi100 ceph-mon[32210]: pgmap v154: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 385 op/s 2024-03-12T05:17:09.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 13.44 MB/sec execute 113 sec latency 2962.717 ms 2024-03-12T05:17:10.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 13.32 MB/sec execute 114 sec latency 3962.804 ms 2024-03-12T05:17:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:10 smithi156 ceph-mon[36270]: pgmap v155: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-03-12T05:17:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:10 smithi161 ceph-mon[36043]: pgmap v155: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-03-12T05:17:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:10 smithi100 ceph-mon[32210]: pgmap v155: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-03-12T05:17:11.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 13.21 MB/sec execute 115 sec latency 4962.914 ms 2024-03-12T05:17:12.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 13.09 MB/sec execute 116 sec latency 5963.032 ms 2024-03-12T05:17:13.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:12 smithi156 ceph-mon[36270]: pgmap v156: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 311 op/s 2024-03-12T05:17:13.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:12 smithi161 ceph-mon[36043]: pgmap v156: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 311 op/s 2024-03-12T05:17:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:12 smithi100 ceph-mon[32210]: pgmap v156: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 311 op/s 2024-03-12T05:17:13.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 12.98 MB/sec execute 117 sec latency 6963.153 ms 2024-03-12T05:17:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 12.87 MB/sec execute 118 sec latency 7963.230 ms 2024-03-12T05:17:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:14 smithi156 ceph-mon[36270]: pgmap v157: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-03-12T05:17:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:14 smithi161 ceph-mon[36043]: pgmap v157: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-03-12T05:17:15.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:14 smithi100 ceph-mon[32210]: pgmap v157: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-03-12T05:17:15.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 12.76 MB/sec execute 119 sec latency 8963.260 ms 2024-03-12T05:17:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:15 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:17:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:15 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:17:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:15 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:17:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:15 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:17:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:15 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:17:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:15 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:17:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:15 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:17:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:15 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:17:16.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:15 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:17:16.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:15 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:17:16.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:15 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:17:16.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:15 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:17:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 12.66 MB/sec execute 120 sec latency 9963.367 ms 2024-03-12T05:17:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:16 smithi156 ceph-mon[36270]: pgmap v158: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-03-12T05:17:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:16 smithi161 ceph-mon[36043]: pgmap v158: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-03-12T05:17:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:16 smithi100 ceph-mon[32210]: pgmap v158: 97 pgs: 97 active+clean; 342 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-03-12T05:17:17.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 586664 12.55 MB/sec execute 121 sec latency 10963.432 ms 2024-03-12T05:17:18.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 587213 12.47 MB/sec execute 122 sec latency 11707.258 ms 2024-03-12T05:17:19.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:17:19.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 -- 172.21.15.100:0/1337341618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750100c20 msgr2=0x7f4750104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 --2- 172.21.15.100:0/1337341618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750100c20 0x7f4750104c60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f473c009930 tx=0x7f473c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 -- 172.21.15.100:0/1337341618 shutdown_connections 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 --2- 172.21.15.100:0/1337341618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4750105340 0x7f475010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 --2- 172.21.15.100:0/1337341618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750100c20 0x7f4750104c60 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 --2- 172.21.15.100:0/1337341618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4750100270 0x7f4750100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 -- 172.21.15.100:0/1337341618 >> 172.21.15.100:0/1337341618 conn(0x7f4750077230 msgr2=0x7f4750077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 -- 172.21.15.100:0/1337341618 shutdown_connections 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.152+0000 7f4756e76640 1 -- 172.21.15.100:0/1337341618 wait complete. 2024-03-12T05:17:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 Processor -- start 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 -- start start 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4750100270 0x7f47501985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 0x7f4750198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4750199090 0x7f47501a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501a3ce0 con 0x7f4750105340 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501a3e50 con 0x7f4750199090 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4756e76640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501a4010 con 0x7f4750100270 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f474ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 0x7f4750198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f474ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 0x7f4750198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51074/0 (socket says 172.21.15.100:51074) 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f474ffff640 1 -- 172.21.15.100:0/1283473302 learned_addr learned my addr 172.21.15.100:0/1283473302 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4755413640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4750199090 0x7f47501a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.153+0000 7f4754c12640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4750100270 0x7f47501985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474ffff640 1 -- 172.21.15.100:0/1283473302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4750100270 msgr2=0x7f47501985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474ffff640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4750100270 0x7f47501985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474ffff640 1 -- 172.21.15.100:0/1283473302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4750199090 msgr2=0x7f47501a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474ffff640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4750199090 0x7f47501a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474ffff640 1 -- 172.21.15.100:0/1283473302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c009560 con 0x7f4750105340 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474ffff640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 0x7f4750198b00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f473c009230 tx=0x7f473c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474dffb640 1 -- 172.21.15.100:0/1283473302 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c004350 con 0x7f4750105340 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474dffb640 1 -- 172.21.15.100:0/1283473302 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f473c0044b0 con 0x7f4750105340 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47501a4290 con 0x7f4750105340 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f474dffb640 1 -- 172.21.15.100:0/1283473302 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c003da0 con 0x7f4750105340 2024-03-12T05:17:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.154+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47501a47a0 con 0x7f4750105340 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.156+0000 7f474dffb640 1 -- 172.21.15.100:0/1283473302 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f473c004df0 con 0x7f4750105340 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 Processor -- start 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 -- start start 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 0x55da4300a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f474dffb640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47400a1ba0 0x7f47400a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da43013270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da430137b0 0x55da43015ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42f79280 con 0x55da4300a2b0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42f767b0 con 0x55da4300abd0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4759a47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42f71f90 con 0x55da430137b0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 0x55da4300a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4754c12640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47400a1ba0 0x7f47400a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 0x55da4300a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51092/0 (socket says 172.21.15.100:51092) 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 -- 172.21.15.100:0/305442418 learned_addr learned my addr 172.21.15.100:0/305442418 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f474ffff640 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da43013270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da430137b0 0x55da43015ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f474dffb640 1 -- 172.21.15.100:0/1283473302 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f473c0ef7f0 con 0x7f4750105340 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 -- 172.21.15.100:0/305442418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da430137b0 msgr2=0x55da43015ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da430137b0 0x55da43015ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 -- 172.21.15.100:0/305442418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 msgr2=0x55da43013270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da43013270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f4755413640 1 -- 172.21.15.100:0/305442418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da42f30570 con 0x55da4300a2b0 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.157+0000 7f474ffff640 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da43013270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4754c12640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47400a1ba0 0x7f47400a4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4744009bd0 tx=0x7f4744005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4755413640 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 0x55da4300a690 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4748013ad0 tx=0x7f4748005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa80d954589287e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f473affd640 1 -- 172.21.15.100:0/305442418 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748013130 con 0x55da4300a2b0 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f473affd640 1 -- 172.21.15.100:0/305442418 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4748009d30 con 0x55da4300a2b0 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f473affd640 1 -- 172.21.15.100:0/305442418 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800fc30 con 0x55da4300a2b0 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 -- 172.21.15.100:0/305442418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 msgr2=0x55da4300a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 0x55da4300a690 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4748013ad0 tx=0x7f4748005bd0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 -- 172.21.15.100:0/305442418 shutdown_connections 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da430137b0 0x55da43015ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da43013270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 --2- 172.21.15.100:0/305442418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300a2b0 0x55da4300a690 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 -- 172.21.15.100:0/305442418 >> 172.21.15.100:0/305442418 conn(0x55da43004990 msgr2=0x55da43004d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 -- 172.21.15.100:0/305442418 shutdown_connections 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.158+0000 7f4759a47740 1 -- 172.21.15.100:0/305442418 wait complete. 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 -- start start 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 Processor -- start 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43010560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43010b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42c45430 con 0x55da4300abd0 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42c01e00 con 0x55da430137b0 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42ef6fd0 con 0x55da4300a2b0 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4759a47740 10 client.0 _ll_register_callbacks cb 0x55da42f4cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f474ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43010560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4754c12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f474ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43010560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51106/0 (socket says 172.21.15.100:51106) 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f474ffff640 1 -- 172.21.15.100:0/233789662 learned_addr learned my addr 172.21.15.100:0/233789662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.159+0000 7f4755413640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43010b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f474ffff640 1 -- 172.21.15.100:0/233789662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 msgr2=0x55da430131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f474ffff640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f474ffff640 1 -- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 msgr2=0x55da43010b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f474ffff640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43010b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f474ffff640 1 -- 172.21.15.100:0/233789662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474801b040 con 0x55da4300abd0 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4754c12640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f474ffff640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43010560 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f473c0ed0a0 tx=0x7f473c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e10c6b1c687ec8a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f472ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f472ffff640 1 -- 172.21.15.100:0/233789662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0c1590 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f472ffff640 1 -- 172.21.15.100:0/233789662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f473c0bcc20 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da42cae720 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f472ffff640 1 -- 172.21.15.100:0/233789662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473c0bd650 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 10 client.16554 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55da42f70610 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f472ffff640 1 -- 172.21.15.100:0/233789662 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da42cae720 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f472ffff640 1 -- 172.21.15.100:0/233789662 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f473c03c070 con 0x55da4300abd0 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 10 client.16554 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 20 client.16554 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da43025a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 4 client.16554 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.160+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da42cfd890 con 0x55da43023630 2024-03-12T05:17:19.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.161+0000 7f4754c12640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da43025a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.161+0000 7f472ffff640 10 client.16554 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.161+0000 7f4754c12640 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da43025a20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f64b8a0fda1900 server_cookie=2d0fa07505a0f257 in_seq=0 out_seq=0 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.161+0000 7f472ffff640 1 -- 172.21.15.100:0/233789662 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da42cfd890 con 0x55da43023630 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.161+0000 7f472ffff640 10 client.16554 handle_command_reply: tid=0 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 client.16554 shutdown 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 msgr2=0x55da43010560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43010560 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f473c0ed0a0 tx=0x7f473c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 shutdown_connections 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da43025a20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f472ffff640 0 client.16554 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43010b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43010560 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 --2- 172.21.15.100:0/233789662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 >> 172.21.15.100:0/233789662 conn(0x55da43004990 msgr2=0x55da43006730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 shutdown_connections 2024-03-12T05:17:19.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 1 -- 172.21.15.100:0/233789662 wait complete. 2024-03-12T05:17:19.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.162+0000 7f4759a47740 20 client.16554 trim_cache size 0 max 16384 2024-03-12T05:17:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 Processor -- start 2024-03-12T05:17:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 -- start start 2024-03-12T05:17:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da43012c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da430131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 0x55da4300ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43000320 con 0x55da430137b0 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43009cf0 con 0x55da4300abd0 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.178+0000 7f4759a47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da430057c0 con 0x55da4300a2b0 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f4754c12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da43012c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f4755413640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 0x55da4300ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da430131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da430131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:35844/0 (socket says 172.21.15.100:35844) 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f4755413640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 0x55da4300ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51112/0 (socket says 172.21.15.100:51112) 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 -- 172.21.15.100:0/270094599 learned_addr learned my addr 172.21.15.100:0/270094599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f4754c12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da43012c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40238/0 (socket says 172.21.15.100:40238) 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 -- 172.21.15.100:0/270094599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 msgr2=0x55da43012c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da43012c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 -- 172.21.15.100:0/270094599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 msgr2=0x55da4300ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 0x55da4300ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 -- 172.21.15.100:0/270094599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da42f0cac0 con 0x55da4300abd0 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f4755413640 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 0x55da4300ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f474ffff640 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da430131d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f473c005ec0 tx=0x7f473c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732139a4c81bd709 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f473a7fc640 1 -- 172.21.15.100:0/270094599 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0d5090 con 0x55da4300abd0 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f473a7fc640 1 -- 172.21.15.100:0/270094599 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f473c0b87c0 con 0x55da4300abd0 2024-03-12T05:17:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.179+0000 7f473a7fc640 1 -- 172.21.15.100:0/270094599 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0c4cc0 con 0x55da4300abd0 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 -- 172.21.15.100:0/270094599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 msgr2=0x55da430131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da430131d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f473c005ec0 tx=0x7f473c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 -- 172.21.15.100:0/270094599 shutdown_connections 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da430137b0 0x55da4300ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4300abd0 0x55da430131d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 --2- 172.21.15.100:0/270094599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da43012c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 -- 172.21.15.100:0/270094599 >> 172.21.15.100:0/270094599 conn(0x55da43004990 msgr2=0x55da4302a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 -- 172.21.15.100:0/270094599 shutdown_connections 2024-03-12T05:17:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 -- 172.21.15.100:0/270094599 wait complete. 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 -- start start 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.180+0000 7f4759a47740 1 Processor -- start 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43018e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43073040 con 0x55da4300abd0 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da430731b0 con 0x55da430137b0 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43073370 con 0x55da4300a2b0 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4759a47740 10 client.0 _ll_register_callbacks cb 0x7f4748009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4754c12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43018e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4754c12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40246/0 (socket says 172.21.15.100:40246) 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43018e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51116/0 (socket says 172.21.15.100:51116) 2024-03-12T05:17:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4754c12640 1 -- 172.21.15.100:0/46182219 learned_addr learned my addr 172.21.15.100:0/46182219 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f4755413640 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 -- 172.21.15.100:0/46182219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 msgr2=0x55da430188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 -- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 msgr2=0x55da43072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 -- 172.21.15.100:0/46182219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da42f30570 con 0x55da4300abd0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f474ffff640 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43018e20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55da43006e00 tx=0x7f473c0c0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfbecd7610717ed server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.181+0000 7f472ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f472ffff640 1 -- 172.21.15.100:0/46182219 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0bc2f0 con 0x55da4300abd0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f472ffff640 1 -- 172.21.15.100:0/46182219 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f473c0bcc20 con 0x55da4300abd0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da42c604e0 con 0x55da4300abd0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f472ffff640 1 -- 172.21.15.100:0/46182219 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473c0bd610 con 0x55da4300abd0 2024-03-12T05:17:19.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 10 client.16560 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55da42f70610 con 0x55da4300abd0 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f472ffff640 1 -- 172.21.15.100:0/46182219 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da42c604e0 con 0x55da4300abd0 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f472ffff640 1 -- 172.21.15.100:0/46182219 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f473c03c070 con 0x55da4300abd0 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 10 client.16560 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 20 client.16560 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da4307c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 4 client.16560 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da42c1d700 con 0x55da43023630 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.182+0000 7f4754c12640 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da4307c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4754c12640 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da4307c670 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ad7d0e81275836 server_cookie=b43ba3bb1e08cb1e in_seq=0 out_seq=0 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f472ffff640 10 client.16560 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f472ffff640 1 -- 172.21.15.100:0/46182219 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da42c1d700 con 0x55da43023630 2024-03-12T05:17:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f472ffff640 10 client.16560 handle_command_reply: tid=0 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 client.16560 shutdown 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 msgr2=0x55da43018e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43018e20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55da43006e00 tx=0x7f473c0c0040 comp rx=0 tx=0).stop 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 shutdown_connections 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da43023630 0x55da4307c670 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da430137b0 0x55da43072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4300abd0 0x55da43018e20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 --2- 172.21.15.100:0/46182219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da4300a2b0 0x55da430188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 >> 172.21.15.100:0/46182219 conn(0x55da43004990 msgr2=0x55da4300edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f472ffff640 0 client.16560 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 shutdown_connections 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 1 -- 172.21.15.100:0/46182219 wait complete. 2024-03-12T05:17:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.183+0000 7f4759a47740 20 client.16560 trim_cache size 0 max 16384 2024-03-12T05:17:19.187 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:17:19.187 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:17:19.187 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:17:19.187 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47400a1ba0 msgr2=0x7f47400a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47400a1ba0 0x7f47400a4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4744009bd0 tx=0x7f4744005f90 comp rx=0 tx=0).stop 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 msgr2=0x7f4750198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 0x7f4750198b00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f473c009230 tx=0x7f473c02c680 comp rx=0 tx=0).stop 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 shutdown_connections 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47400a1ba0 0x7f47400a4060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4750199090 0x7f47501a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4750105340 0x7f4750198b00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 --2- 172.21.15.100:0/1283473302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4750100270 0x7f47501985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 >> 172.21.15.100:0/1283473302 conn(0x7f4750077230 msgr2=0x7f47500fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 shutdown_connections 2024-03-12T05:17:19.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.184+0000 7f4756e76640 1 -- 172.21.15.100:0/1283473302 wait complete. 2024-03-12T05:17:19.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:18 smithi100 ceph-mon[32210]: pgmap v159: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:17:19.190 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:17:19.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:17:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:18 smithi156 ceph-mon[36270]: pgmap v159: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:17:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:18 smithi161 ceph-mon[36043]: pgmap v159: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:17:19.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 -- 172.21.15.100:0/3847858051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041053c0 msgr2=0x7fcd0410cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3847858051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041053c0 0x7fcd0410cf90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009cd0 tx=0x7fccfc02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 -- 172.21.15.100:0/3847858051 shutdown_connections 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3847858051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041053c0 0x7fcd0410cf90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3847858051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd04100ca0 0x7fcd04104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3847858051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd041002f0 0x7fcd041006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 -- 172.21.15.100:0/3847858051 >> 172.21.15.100:0/3847858051 conn(0x7fcd04077180 msgr2=0x7fcd04077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 -- 172.21.15.100:0/3847858051 shutdown_connections 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.302+0000 7fcd099b5640 1 -- 172.21.15.100:0/3847858051 wait complete. 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 Processor -- start 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 -- start start 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 0x7fcd041966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd041053c0 0x7fcd04196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd0419afb0 0x7fcd0419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419b9a0 con 0x7fcd041002f0 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419bb10 con 0x7fcd041053c0 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd099b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419bc80 con 0x7fcd0419afb0 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd02ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 0x7fcd041966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd02ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 0x7fcd041966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51146/0 (socket says 172.21.15.100:51146) 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd02ffd640 1 -- 172.21.15.100:0/3552742817 learned_addr learned my addr 172.21.15.100:0/3552742817 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd027fc640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd041053c0 0x7fcd04196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.303+0000 7fcd037fe640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd0419afb0 0x7fcd0419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd02ffd640 1 -- 172.21.15.100:0/3552742817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd0419afb0 msgr2=0x7fcd0419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd02ffd640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd0419afb0 0x7fcd0419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd02ffd640 1 -- 172.21.15.100:0/3552742817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd041053c0 msgr2=0x7fcd04196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd02ffd640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd041053c0 0x7fcd04196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd02ffd640 1 -- 172.21.15.100:0/3552742817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc009900 con 0x7fcd041002f0 2024-03-12T05:17:19.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd02ffd640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 0x7fcd041966f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9f0 tx=0x7fccf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4009700 con 0x7fcd041002f0 2024-03-12T05:17:19.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccf400cd80 con 0x7fcd041002f0 2024-03-12T05:17:19.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041a3260 con 0x7fcd041002f0 2024-03-12T05:17:19.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4018920 con 0x7fcd041002f0 2024-03-12T05:17:19.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.304+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041a3710 con 0x7fcd041002f0 2024-03-12T05:17:19.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.305+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd04067f70 con 0x7fcd041002f0 2024-03-12T05:17:19.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.306+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fccf4018a80 con 0x7fcd041002f0 2024-03-12T05:17:19.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.307+0000 7fcd089b3640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fccec0a1ba0 0x7fccec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.307+0000 7fcd027fc640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fccec0a1ba0 0x7fccec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.307+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fccf4014070 con 0x7fcd041002f0 2024-03-12T05:17:19.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.307+0000 7fcd027fc640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fccec0a1ba0 0x7fccec0a4060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ffba0 tx=0x7fccf800b390 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.311+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fccf4096fa0 con 0x7fcd041002f0 2024-03-12T05:17:19.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.543+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd041085f0 con 0x7fcd041002f0 2024-03-12T05:17:19.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.544+0000 7fcd089b3640 1 -- 172.21.15.100:0/3552742817 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fccf409a660 con 0x7fcd041002f0 2024-03-12T05:17:19.540 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:17:19.540 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:17:19.541 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:17:19.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fccec0a1ba0 msgr2=0x7fccec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fccec0a1ba0 0x7fccec0a4060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ffba0 tx=0x7fccf800b390 comp rx=0 tx=0).stop 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 msgr2=0x7fcd041966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 0x7fcd041966f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9f0 tx=0x7fccf400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 shutdown_connections 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fccec0a1ba0 0x7fccec0a4060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd0419afb0 0x7fcd0419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd041053c0 0x7fcd04196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 --2- 172.21.15.100:0/3552742817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd041002f0 0x7fcd041966f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 >> 172.21.15.100:0/3552742817 conn(0x7fcd04077180 msgr2=0x7fcd040fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 shutdown_connections 2024-03-12T05:17:19.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.546+0000 7fcd099b5640 1 -- 172.21.15.100:0/3552742817 wait complete. 2024-03-12T05:17:19.553 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:17:19.554 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 -- 172.21.15.100:0/812071036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184106450 msgr2=0x7f918410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 --2- 172.21.15.100:0/812071036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184106450 0x7f918410e020 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f917c009cd0 tx=0x7f917c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 -- 172.21.15.100:0/812071036 shutdown_connections 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 --2- 172.21.15.100:0/812071036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184106450 0x7f918410e020 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 --2- 172.21.15.100:0/812071036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184101d30 0x7f9184105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 --2- 172.21.15.100:0/812071036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9184101380 0x7f9184101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 -- 172.21.15.100:0/812071036 >> 172.21.15.100:0/812071036 conn(0x7f9184078580 msgr2=0x7f9184078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 -- 172.21.15.100:0/812071036 shutdown_connections 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.669+0000 7f9188a9d640 1 -- 172.21.15.100:0/812071036 wait complete. 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 Processor -- start 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 -- start start 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 0x7f918419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184106450 0x7f918419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91841975a0 0x7f9184197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184197f90 con 0x7f9184101380 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184198100 con 0x7f9184106450 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9188a9d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841982c0 con 0x7f91841975a0 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 0x7f918419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 0x7f918419d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51162/0 (socket says 172.21.15.100:51162) 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 -- 172.21.15.100:0/2247189404 learned_addr learned my addr 172.21.15.100:0/2247189404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184106450 0x7f918419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9182d9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91841975a0 0x7f9184197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 -- 172.21.15.100:0/2247189404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91841975a0 msgr2=0x7f9184197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91841975a0 0x7f9184197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 -- 172.21.15.100:0/2247189404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184106450 msgr2=0x7f918419d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184106450 0x7f918419d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f918259c640 1 -- 172.21.15.100:0/2247189404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009900 con 0x7f9184101380 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.670+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184106450 0x7f918419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.671+0000 7f918259c640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 0x7f918419d3f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f917400c970 tx=0x7f917400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.671+0000 7f917bfff640 1 -- 172.21.15.100:0/2247189404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174007bb0 con 0x7f9184101380 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.671+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91841a42b0 con 0x7f9184101380 2024-03-12T05:17:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.671+0000 7f917bfff640 1 -- 172.21.15.100:0/2247189404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9174007d10 con 0x7f9184101380 2024-03-12T05:17:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.671+0000 7f917bfff640 1 -- 172.21.15.100:0/2247189404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400f450 con 0x7f9184101380 2024-03-12T05:17:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.671+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841a4770 con 0x7f9184101380 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.673+0000 7f917bfff640 1 -- 172.21.15.100:0/2247189404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9174004f70 con 0x7f9184101380 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 Processor -- start 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 -- start start 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b995360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b950960 con 0x55fb2b994f80 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b904310 con 0x55fb2b9a2b60 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918b677740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b901840 con 0x55fb2b9958a0 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f917bfff640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91700a1ba0 0x7f91700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f918259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f9179d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f9182d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b995360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f9179d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:35894/0 (socket says 172.21.15.100:35894) 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.674+0000 7f9179d9b640 1 -- 172.21.15.100:0/2118838493 learned_addr learned my addr 172.21.15.100:0/2118838493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9182d9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91700a1ba0 0x7f91700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f917bfff640 1 -- 172.21.15.100:0/2247189404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91740cd4b0 con 0x7f9184101380 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9179d9b640 1 -- 172.21.15.100:0/2118838493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 msgr2=0x55fb2b9a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9179d9b640 1 -- 172.21.15.100:0/2118838493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 msgr2=0x55fb2b995360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b995360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9179d9b640 1 -- 172.21.15.100:0/2118838493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b8bf060 con 0x55fb2b9a2b60 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a4f50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916c009fd0 tx=0x7f916c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ee49027a412243 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9182d9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91700a1ba0 0x7f91700a4060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f917c005cc0 tx=0x7f917c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9178d99640 1 -- 172.21.15.100:0/2118838493 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c004030 con 0x55fb2b9a2b60 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9178d99640 1 -- 172.21.15.100:0/2118838493 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f916c0035f0 con 0x55fb2b9a2b60 2024-03-12T05:17:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.675+0000 7f9178d99640 1 -- 172.21.15.100:0/2118838493 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c00f810 con 0x55fb2b9a2b60 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 -- 172.21.15.100:0/2118838493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 msgr2=0x55fb2b9a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a4f50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916c009fd0 tx=0x7f916c0118d0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 -- 172.21.15.100:0/2118838493 shutdown_connections 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a4f50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 --2- 172.21.15.100:0/2118838493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b995360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 -- 172.21.15.100:0/2118838493 >> 172.21.15.100:0/2118838493 conn(0x55fb2b98f980 msgr2=0x55fb2b98fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 -- 172.21.15.100:0/2118838493 shutdown_connections 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 -- 172.21.15.100:0/2118838493 wait complete. 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 -- start start 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.676+0000 7f918b677740 1 Processor -- start 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b994f80 0x55fb2b99fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b8fd020 con 0x55fb2b9a2b60 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b5d0430 con 0x55fb2b9958a0 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b58ce00 con 0x55fb2b994f80 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 10 client.0 _ll_register_callbacks cb 0x55fb2b8a4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51170/0 (socket says 172.21.15.100:51170) 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 -- 172.21.15.100:0/1393441426 learned_addr learned my addr 172.21.15.100:0/1393441426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9179d9b640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918259c640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b994f80 0x55fb2b99fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 -- 172.21.15.100:0/1393441426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b994f80 msgr2=0x55fb2b99fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b994f80 0x55fb2b99fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 -- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 msgr2=0x55fb2b9a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 -- 172.21.15.100:0/1393441426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c009c00 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918259c640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b994f80 0x55fb2b99fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9182d9d640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a05f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f917c009cd0 tx=0x7f917c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32ade843a45eb5e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9169ffb640 1 -- 172.21.15.100:0/1393441426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c032b40 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9169ffb640 1 -- 172.21.15.100:0/1393441426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f917c003970 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb2b5fc7e0 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f9169ffb640 1 -- 172.21.15.100:0/1393441426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f917c003b30 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 10 client.16578 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.677+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fb2b8fb6a0 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f9169ffb640 1 -- 172.21.15.100:0/1393441426 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f917c05d020 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f9169ffb640 1 -- 172.21.15.100:0/1393441426 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f917c050460 con 0x55fb2b9a2b60 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918b677740 10 client.16578 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918b677740 10 client.16578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918b677740 20 client.16578 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918b677740 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2b9b2fa0 0x55fb2b9b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918b677740 4 client.16578 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb2b639720 con 0x55fb2b9b2fa0 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918259c640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2b9b2fa0 0x55fb2b9b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f918259c640 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2b9b2fa0 0x55fb2b9b53c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d798c73380420f6 server_cookie=165394bdb946917a in_seq=0 out_seq=0 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.678+0000 7f9169ffb640 10 client.16578 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f9169ffb640 1 -- 172.21.15.100:0/1393441426 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb2b639720 con 0x55fb2b9b2fa0 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f9169ffb640 10 client.16578 handle_command_reply: tid=0 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 client.16578 shutdown 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 msgr2=0x55fb2b9a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a05f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f917c009cd0 tx=0x7f917c0095c0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 shutdown_connections 2024-03-12T05:17:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2b9b2fa0 0x55fb2b9b53c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a05f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 --2- 172.21.15.100:0/1393441426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b994f80 0x55fb2b99fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 >> 172.21.15.100:0/1393441426 conn(0x55fb2b98f980 msgr2=0x55fb2b9916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f9169ffb640 0 client.16578 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 shutdown_connections 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.679+0000 7f918b677740 1 -- 172.21.15.100:0/1393441426 wait complete. 2024-03-12T05:17:19.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.680+0000 7f918b677740 20 client.16578 trim_cache size 0 max 16384 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 Processor -- start 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 -- start start 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9a2b60 0x55fb2b99aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b882130 con 0x55fb2b994f80 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b98b2b0 con 0x55fb2b9958a0 2024-03-12T05:17:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918b677740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b995fa0 con 0x55fb2b9a2b60 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51186/0 (socket says 172.21.15.100:51186) 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 -- 172.21.15.100:0/2306745421 learned_addr learned my addr 172.21.15.100:0/2306745421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f9182d9d640 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9a2b60 0x55fb2b99aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 -- 172.21.15.100:0/2306745421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9a2b60 msgr2=0x55fb2b99aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9a2b60 0x55fb2b99aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 -- 172.21.15.100:0/2306745421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 msgr2=0x55fb2b9a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f918259c640 1 -- 172.21.15.100:0/2306745421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b897c20 con 0x55fb2b994f80 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.692+0000 7f9179d9b640 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918259c640 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a22a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f917409a110 tx=0x7f9174096420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe3ff2e437eee15 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f916bfff640 1 -- 172.21.15.100:0/2306745421 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917409a180 con 0x55fb2b994f80 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f916bfff640 1 -- 172.21.15.100:0/2306745421 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f917409e220 con 0x55fb2b994f80 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f916bfff640 1 -- 172.21.15.100:0/2306745421 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174098560 con 0x55fb2b994f80 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 -- 172.21.15.100:0/2306745421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 msgr2=0x55fb2b9a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a22a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f917409a110 tx=0x7f9174096420 comp rx=0 tx=0).stop 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 -- 172.21.15.100:0/2306745421 shutdown_connections 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9a2b60 0x55fb2b99aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 --2- 172.21.15.100:0/2306745421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a22a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 -- 172.21.15.100:0/2306745421 >> 172.21.15.100:0/2306745421 conn(0x55fb2b98f980 msgr2=0x55fb2b9b9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 -- 172.21.15.100:0/2306745421 shutdown_connections 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.693+0000 7f918b677740 1 -- 172.21.15.100:0/2306745421 wait complete. 2024-03-12T05:17:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 -- start start 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 Processor -- start 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2ba094d0 con 0x55fb2b994f80 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2ba09610 con 0x55fb2b9a2b60 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2ba09780 con 0x55fb2b9958a0 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918b677740 10 client.0 _ll_register_callbacks cb 0x7f916c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51194/0 (socket says 172.21.15.100:51194) 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f918259c640 1 -- 172.21.15.100:0/1647947411 learned_addr learned my addr 172.21.15.100:0/1647947411 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f9179d9b640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.694+0000 7f9182d9d640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918259c640 1 -- 172.21.15.100:0/1647947411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 msgr2=0x55fb2b9a8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918259c640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918259c640 1 -- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 msgr2=0x55fb2b9a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918259c640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918259c640 1 -- 172.21.15.100:0/1647947411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb2b8bf060 con 0x55fb2b994f80 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918259c640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a8500 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f91740cb1f0 tx=0x7f9174007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d989e30e712963d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f9169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f9169ffb640 1 -- 172.21.15.100:0/1647947411 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174098220 con 0x55fb2b994f80 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f9169ffb640 1 -- 172.21.15.100:0/1647947411 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f917409a180 con 0x55fb2b994f80 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f916c0035f0 con 0x55fb2b994f80 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f9169ffb640 1 -- 172.21.15.100:0/1647947411 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91740967f0 con 0x55fb2b994f80 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 10 client.16584 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f91841a4770 con 0x55fb2b994f80 2024-03-12T05:17:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f9169ffb640 1 -- 172.21.15.100:0/1647947411 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f916c0035f0 con 0x55fb2b994f80 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f9169ffb640 1 -- 172.21.15.100:0/1647947411 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9174016030 con 0x55fb2b994f80 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 10 client.16584 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 10 client.16584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 20 client.16584 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2ba0ac00 0x55fb2ba0cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 4 client.16584 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.695+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fb2b688890 con 0x55fb2ba0ac00 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.696+0000 7f9179d9b640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2ba0ac00 0x55fb2ba0cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.696+0000 7f9169ffb640 10 client.16584 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.696+0000 7f9179d9b640 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2ba0ac00 0x55fb2ba0cff0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d03b02ca637fd41e server_cookie=f421493c1a1cabe5 in_seq=0 out_seq=0 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.696+0000 7f9169ffb640 1 -- 172.21.15.100:0/1647947411 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fb2b688890 con 0x55fb2ba0ac00 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.696+0000 7f9169ffb640 10 client.16584 handle_command_reply: tid=0 2024-03-12T05:17:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.696+0000 7f918b677740 1 client.16584 shutdown 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 msgr2=0x55fb2b9a8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a8500 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f91740cb1f0 tx=0x7f9174007700 comp rx=0 tx=0).stop 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 shutdown_connections 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb2ba0ac00 0x55fb2ba0cff0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb2b9a2b60 0x55fb2b9a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2b9958a0 0x55fb2b9a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 --2- 172.21.15.100:0/1647947411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb2b994f80 0x55fb2b9a8500 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9169ffb640 0 client.16584 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 >> 172.21.15.100:0/1647947411 conn(0x55fb2b98f980 msgr2=0x55fb2b99aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 shutdown_connections 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 1 -- 172.21.15.100:0/1647947411 wait complete. 2024-03-12T05:17:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f918b677740 20 client.16584 trim_cache size 0 max 16384 2024-03-12T05:17:19.701 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:17:19.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91700a1ba0 msgr2=0x7f91700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91700a1ba0 0x7f91700a4060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f917c005cc0 tx=0x7f917c005c50 comp rx=0 tx=0).stop 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 msgr2=0x7f918419d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 0x7f918419d3f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f917400c970 tx=0x7f917400ce30 comp rx=0 tx=0).stop 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 shutdown_connections 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f91700a1ba0 0x7f91700a4060 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91841975a0 0x7f9184197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9184106450 0x7f918419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 --2- 172.21.15.100:0/2247189404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184101380 0x7f918419d3f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 >> 172.21.15.100:0/2247189404 conn(0x7f9184078580 msgr2=0x7f91840ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 shutdown_connections 2024-03-12T05:17:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.697+0000 7f9188a9d640 1 -- 172.21.15.100:0/2247189404 wait complete. 2024-03-12T05:17:19.704 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:17:19.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:17:19.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 589862 12.50 MB/sec execute 123 sec latency 32.545 ms 2024-03-12T05:17:19.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 -- 172.21.15.100:0/53857143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100c10 msgr2=0x7fc4a8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/53857143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100c10 0x7fc4a8104c50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc494009930 tx=0x7fc49402bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 -- 172.21.15.100:0/53857143 shutdown_connections 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/53857143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8105330 0x7fc4a810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/53857143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100c10 0x7fc4a8104c50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/53857143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a8100260 0x7fc4a8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 -- 172.21.15.100:0/53857143 >> 172.21.15.100:0/53857143 conn(0x7fc4a8077180 msgr2=0x7fc4a8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.818+0000 7fc4aefb6640 1 -- 172.21.15.100:0/53857143 shutdown_connections 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 -- 172.21.15.100:0/53857143 wait complete. 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 Processor -- start 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 -- start start 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 0x7fc4a810d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8105330 0x7fc4a810dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a810e260 0x7fc4a81a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81a5f70 con 0x7fc4a8100260 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81a60e0 con 0x7fc4a810e260 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4aefb6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81a62a0 con 0x7fc4a8105330 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4acd52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 0x7fc4a810d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4acd52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 0x7fc4a810d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51208/0 (socket says 172.21.15.100:51208) 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4acd52640 1 -- 172.21.15.100:0/190204952 learned_addr learned my addr 172.21.15.100:0/190204952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4ad553640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a810e260 0x7fc4a81a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.819+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8105330 0x7fc4a810dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4acd52640 1 -- 172.21.15.100:0/190204952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8105330 msgr2=0x7fc4a810dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4acd52640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8105330 0x7fc4a810dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4acd52640 1 -- 172.21.15.100:0/190204952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a810e260 msgr2=0x7fc4a81a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4acd52640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a810e260 0x7fc4a81a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4acd52640 1 -- 172.21.15.100:0/190204952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494009560 con 0x7fc4a8100260 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4ad553640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a810e260 0x7fc4a81a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4acd52640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 0x7fc4a810d7e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00e9f0 tx=0x7fc49c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/190204952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c009700 con 0x7fc4a8100260 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/190204952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc49c00cd80 con 0x7fc4a8100260 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a81a6580 con 0x7fc4a8100260 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/190204952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c018920 con 0x7fc4a8100260 2024-03-12T05:17:19.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.820+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a81a6a60 con 0x7fc4a8100260 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/190204952 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc49c018a80 con 0x7fc4a8100260 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 Processor -- start 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 -- start start 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e849630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e852210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e854b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e804960 con 0x55b13e852750 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4a5ffb640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4980a1bd0 0x7fc4980a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e7b8310 con 0x55b13e849b70 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e7b5840 con 0x55b13e849250 2024-03-12T05:17:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4ad553640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e849630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e854b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4acd52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e852210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4ad553640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e849630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40356/0 (socket says 172.21.15.100:40356) 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4ad553640 1 -- 172.21.15.100:0/3831364748 learned_addr learned my addr 172.21.15.100:0/3831364748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4ad553640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4980a1bd0 0x7fc4980a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.823+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/190204952 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc49c014070 con 0x7fc4a8100260 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4acd52640 1 -- 172.21.15.100:0/3831364748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 msgr2=0x55b13e849630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e849630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4acd52640 1 -- 172.21.15.100:0/3831364748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 msgr2=0x55b13e854b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e854b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4acd52640 1 -- 172.21.15.100:0/3831364748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13e773060 con 0x55b13e849b70 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4ad553640 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e849630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e854b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4ad553640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4980a1bd0 0x7fc4980a4090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005c40 tx=0x7fc4a0009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e852210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0cb3f0 tx=0x7fc49c010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f01d5f14738979 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc492ffd640 1 -- 172.21.15.100:0/3831364748 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c09a230 con 0x55b13e849b70 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc492ffd640 1 -- 172.21.15.100:0/3831364748 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc49c09afd0 con 0x55b13e849b70 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.824+0000 7fc492ffd640 1 -- 172.21.15.100:0/3831364748 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0951b0 con 0x55b13e849b70 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3831364748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 msgr2=0x55b13e852210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e852210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0cb3f0 tx=0x7fc49c010f50 comp rx=0 tx=0).stop 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3831364748 shutdown_connections 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e854b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e852210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3831364748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e849630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3831364748 >> 172.21.15.100:0/3831364748 conn(0x55b13e843980 msgr2=0x55b13e843d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3831364748 shutdown_connections 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3831364748 wait complete. 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 -- start start 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.825+0000 7fc4b1b87740 1 Processor -- start 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849250 0x55b13e84f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 0x55b13e84fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e850270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e7b1020 con 0x55b13e849b70 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e484430 con 0x55b13e849250 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e440e00 con 0x55b13e852750 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 10 client.0 _ll_register_callbacks cb 0x55b13e758cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 0x55b13e84fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 0x55b13e84fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51230/0 (socket says 172.21.15.100:51230) 2024-03-12T05:17:19.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 -- 172.21.15.100:0/3275239961 learned_addr learned my addr 172.21.15.100:0/3275239961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849250 0x55b13e84f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4ad553640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e850270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 -- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 msgr2=0x55b13e850270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e850270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 -- 172.21.15.100:0/3275239961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849250 msgr2=0x55b13e84f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849250 0x55b13e84f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 -- 172.21.15.100:0/3275239961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a81a6a60 con 0x55b13e849b70 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849250 0x55b13e84f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 0x55b13e84fd30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc494005ab0 tx=0x7fc49402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a2b76006e279e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc487fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc487fff640 1 -- 172.21.15.100:0/3275239961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494004040 con 0x55b13e849b70 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc487fff640 1 -- 172.21.15.100:0/3275239961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc4940041a0 con 0x55b13e849b70 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.826+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b13e4b07e0 con 0x55b13e849b70 2024-03-12T05:17:19.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc487fff640 1 -- 172.21.15.100:0/3275239961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc494004360 con 0x55b13e849b70 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 10 client.16596 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b13e7af6a0 con 0x55b13e849b70 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc487fff640 1 -- 172.21.15.100:0/3275239961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b13e4b07e0 con 0x55b13e849b70 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc487fff640 1 -- 172.21.15.100:0/3275239961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc49403c040 con 0x55b13e849b70 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 10 client.16596 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 10 client.16596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 20 client.16596 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e862b20 0x55b13e864f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 4 client.16596 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b13e4ed720 con 0x55b13e862b20 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.827+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e862b20 0x55b13e864f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4acd52640 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e862b20 0x55b13e864f40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c47642954e44d814 server_cookie=d91bef498d8936ef in_seq=0 out_seq=0 2024-03-12T05:17:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc487fff640 10 client.16596 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc487fff640 1 -- 172.21.15.100:0/3275239961 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b13e4ed720 con 0x55b13e862b20 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc487fff640 10 client.16596 handle_command_reply: tid=0 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 client.16596 shutdown 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 msgr2=0x55b13e84fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 0x55b13e84fd30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc494005ab0 tx=0x7fc49402ca10 comp rx=0 tx=0).stop 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 shutdown_connections 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e850270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849b70 0x55b13e84fd30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e862b20 0x55b13e864f40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/3275239961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849250 0x55b13e84f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc487fff640 0 client.16596 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 >> 172.21.15.100:0/3275239961 conn(0x55b13e843980 msgr2=0x55b13e8456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 shutdown_connections 2024-03-12T05:17:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 1 -- 172.21.15.100:0/3275239961 wait complete. 2024-03-12T05:17:19.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.828+0000 7fc4b1b87740 20 client.16596 trim_cache size 0 max 16384 2024-03-12T05:17:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 Processor -- start 2024-03-12T05:17:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 -- start start 2024-03-12T05:17:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 0x55b13e851f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e850090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e84ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e736130 con 0x55b13e849250 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e8489f0 con 0x55b13e849b70 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e8449b0 con 0x55b13e852750 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 0x55b13e851f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 0x55b13e851f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51242/0 (socket says 172.21.15.100:51242) 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 -- 172.21.15.100:0/996048290 learned_addr learned my addr 172.21.15.100:0/996048290 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e850090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4ad553640 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e84ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 -- 172.21.15.100:0/996048290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 msgr2=0x55b13e84ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e84ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 -- 172.21.15.100:0/996048290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 msgr2=0x55b13e850090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e850090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4acd52640 1 -- 172.21.15.100:0/996048290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13e74bc20 con 0x55b13e849250 2024-03-12T05:17:19.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.841+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e850090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4acd52640 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 0x55b13e851f20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc49c09a5d0 tx=0x7fc49c095040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bb90350f0520fd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4927fc640 1 -- 172.21.15.100:0/996048290 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c09eaa0 con 0x55b13e849250 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4927fc640 1 -- 172.21.15.100:0/996048290 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc49c0a3070 con 0x55b13e849250 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4927fc640 1 -- 172.21.15.100:0/996048290 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0a2be0 con 0x55b13e849250 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 -- 172.21.15.100:0/996048290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 msgr2=0x55b13e851f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 0x55b13e851f20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc49c09a5d0 tx=0x7fc49c095040 comp rx=0 tx=0).stop 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 -- 172.21.15.100:0/996048290 shutdown_connections 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e852750 0x55b13e84ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e850090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/996048290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e849250 0x55b13e851f20 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 -- 172.21.15.100:0/996048290 >> 172.21.15.100:0/996048290 conn(0x55b13e843980 msgr2=0x55b13e84aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 -- 172.21.15.100:0/996048290 shutdown_connections 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.842+0000 7fc4b1b87740 1 -- 172.21.15.100:0/996048290 wait complete. 2024-03-12T05:17:19.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 -- start start 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 Processor -- start 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e858280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e8587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e8b9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e858f80 con 0x55b13e852750 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e8590f0 con 0x55b13e849b70 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13e8b95b0 con 0x55b13e849250 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4b1b87740 10 client.0 _ll_register_callbacks cb 0x7fc49c096500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4ad553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e8b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4ad553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e8b9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51244/0 (socket says 172.21.15.100:51244) 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4ad553640 1 -- 172.21.15.100:0/1560964436 learned_addr learned my addr 172.21.15.100:0/1560964436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e8587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.843+0000 7fc4acd52640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e858280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4ad553640 1 -- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 msgr2=0x55b13e858280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4ad553640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e858280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4ad553640 1 -- 172.21.15.100:0/1560964436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 msgr2=0x55b13e8587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4ad553640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e8587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4ad553640 1 -- 172.21.15.100:0/1560964436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b13e7af6a0 con 0x55b13e852750 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4a7fff640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e8587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4ad553640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e8b9070 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0003e90 tx=0x7fc4a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ca81dfd8ff80b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc487fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc487fff640 1 -- 172.21.15.100:0/1560964436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0026070 con 0x55b13e852750 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc49c09afd0 con 0x55b13e852750 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc487fff640 1 -- 172.21.15.100:0/1560964436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc4a0004b20 con 0x55b13e852750 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc487fff640 1 -- 172.21.15.100:0/1560964436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4a0019040 con 0x55b13e852750 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 10 client.16602 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fc49c092040 con 0x55b13e852750 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc487fff640 1 -- 172.21.15.100:0/1560964436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4a0036080 con 0x55b13e852750 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc487fff640 1 -- 172.21.15.100:0/1560964436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc4a00051e0 con 0x55b13e852750 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 10 client.16602 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 10 client.16602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 20 client.16602 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e8baed0 0x55b13e8bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 4 client.16602 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.844+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b13e53c890 con 0x55b13e8baed0 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4acd52640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e8baed0 0x55b13e8bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc487fff640 10 client.16602 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4acd52640 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e8baed0 0x55b13e8bd2f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f83de9ff70c9901 server_cookie=32a09622cf400659 in_seq=0 out_seq=0 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc487fff640 1 -- 172.21.15.100:0/1560964436 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b13e53c890 con 0x55b13e8baed0 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc487fff640 10 client.16602 handle_command_reply: tid=0 2024-03-12T05:17:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 client.16602 shutdown 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 msgr2=0x55b13e8b9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e8b9070 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0003e90 tx=0x7fc4a0004ab0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 shutdown_connections 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b13e852750 0x55b13e8b9070 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55b13e8baed0 0x55b13e8bd2f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b13e849b70 0x55b13e8587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc487fff640 0 client.16602 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 --2- 172.21.15.100:0/1560964436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b13e849250 0x55b13e858280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 >> 172.21.15.100:0/1560964436 conn(0x55b13e843980 msgr2=0x55b13e8422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 shutdown_connections 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 1 -- 172.21.15.100:0/1560964436 wait complete. 2024-03-12T05:17:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.845+0000 7fc4b1b87740 20 client.16602 trim_cache size 0 max 16384 2024-03-12T05:17:19.849 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:17:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4980a1bd0 msgr2=0x7fc4980a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4980a1bd0 0x7fc4980a4090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005c40 tx=0x7fc4a0009500 comp rx=0 tx=0).stop 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 msgr2=0x7fc4a810d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 0x7fc4a810d7e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00e9f0 tx=0x7fc49c00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 shutdown_connections 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc4980a1bd0 0x7fc4980a4090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4a810e260 0x7fc4a81a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8105330 0x7fc4a810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 --2- 172.21.15.100:0/190204952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4a8100260 0x7fc4a810d7e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 >> 172.21.15.100:0/190204952 conn(0x7fc4a8077180 msgr2=0x7fc4a80fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 shutdown_connections 2024-03-12T05:17:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.846+0000 7fc4aefb6640 1 -- 172.21.15.100:0/190204952 wait complete. 2024-03-12T05:17:19.853 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:17:19.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:17:19.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 -- 172.21.15.100:0/1253991097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb0106430 msgr2=0x7f5fb010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 --2- 172.21.15.100:0/1253991097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb0106430 0x7f5fb010e000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009cd0 tx=0x7f5fa802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 -- 172.21.15.100:0/1253991097 shutdown_connections 2024-03-12T05:17:19.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 --2- 172.21.15.100:0/1253991097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb0106430 0x7f5fb010e000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 --2- 172.21.15.100:0/1253991097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0068f80 0x7f5fb0105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 --2- 172.21.15.100:0/1253991097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb00685d0 0x7f5fb00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 -- 172.21.15.100:0/1253991097 >> 172.21.15.100:0/1253991097 conn(0x7f5fb0077180 msgr2=0x7f5fb0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.967+0000 7f5fb5840640 1 -- 172.21.15.100:0/1253991097 shutdown_connections 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 -- 172.21.15.100:0/1253991097 wait complete. 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 Processor -- start 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 -- start start 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb00685d0 0x7f5fb019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0106430 0x7f5fb019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 0x7f5fb0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0197fd0 con 0x7f5fb01975e0 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0198140 con 0x7f5fb0106430 2024-03-12T05:17:19.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.968+0000 7f5fb5840640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0198300 con 0x7f5fb00685d0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 0x7f5fb0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 0x7f5fb0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51254/0 (socket says 172.21.15.100:51254) 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3058663724 learned_addr learned my addr 172.21.15.100:0/3058663724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0106430 0x7f5fb019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faeffd640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb00685d0 0x7f5fb019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3058663724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb00685d0 msgr2=0x7f5fb019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb00685d0 0x7f5fb019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3058663724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0106430 msgr2=0x7f5fb019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0106430 0x7f5fb019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3058663724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa8009900 con 0x7f5fb01975e0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0106430 0x7f5fb019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 0x7f5fb0197a90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004af0 tx=0x7f5fa8034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fb483e640 1 -- 172.21.15.100:0/3058663724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80328c0 con 0x7f5fb01975e0 2024-03-12T05:17:19.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb01a42f0 con 0x7f5fb01975e0 2024-03-12T05:17:19.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fb483e640 1 -- 172.21.15.100:0/3058663724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa8003e50 con 0x7f5fb01975e0 2024-03-12T05:17:19.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fb483e640 1 -- 172.21.15.100:0/3058663724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004860 con 0x7f5fb01975e0 2024-03-12T05:17:19.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.969+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb01a4750 con 0x7f5fb01975e0 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.971+0000 7f5fb483e640 1 -- 172.21.15.100:0/3058663724 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5fa800a040 con 0x7f5fb01975e0 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 Processor -- start 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 -- start start 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb483e640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f980a1ba0 0x7f5f980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 0x562ffc2c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc276960 con 0x562ffc2bbb70 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc22a310 con 0x562ffc2c4750 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb8413740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc227840 con 0x562ffc2bb250 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f980a1ba0 0x7f5f980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fb483e640 1 -- 172.21.15.100:0/3058663724 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5fa80f0400 con 0x7f5fb01975e0 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 0x562ffc2c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 0x562ffc2c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51270/0 (socket says 172.21.15.100:51270) 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 -- 172.21.15.100:0/3603681159 learned_addr learned my addr 172.21.15.100:0/3603681159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5faeffd640 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 -- 172.21.15.100:0/3603681159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 msgr2=0x562ffc2bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 -- 172.21.15.100:0/3603681159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 msgr2=0x562ffc2c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.972+0000 7f5fae7fc640 1 -- 172.21.15.100:0/3603681159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ffc1e5060 con 0x562ffc2bbb70 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f980a1ba0 0x7f5f980a4060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011cb0 tx=0x7f5fa0009ef0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 0x562ffc2c4210 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005bb0 tx=0x7f5fa001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150d46c4e10d9b31 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5f95ffb640 1 -- 172.21.15.100:0/3603681159 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0011da0 con 0x562ffc2bbb70 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5f95ffb640 1 -- 172.21.15.100:0/3603681159 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0020c80 con 0x562ffc2bbb70 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5f95ffb640 1 -- 172.21.15.100:0/3603681159 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0029460 con 0x562ffc2bbb70 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 -- 172.21.15.100:0/3603681159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 msgr2=0x562ffc2c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 0x562ffc2c4210 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005bb0 tx=0x7f5fa001d000 comp rx=0 tx=0).stop 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 -- 172.21.15.100:0/3603681159 shutdown_connections 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bbb70 0x562ffc2c4210 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3603681159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 -- 172.21.15.100:0/3603681159 >> 172.21.15.100:0/3603681159 conn(0x562ffc2b5980 msgr2=0x562ffc2b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 -- 172.21.15.100:0/3603681159 shutdown_connections 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.973+0000 7f5fb8413740 1 -- 172.21.15.100:0/3603681159 wait complete. 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 -- start start 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 Processor -- start 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc223020 con 0x562ffc2c4750 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffbef6430 con 0x562ffc2bbb70 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffbeb2e00 con 0x562ffc2bb250 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fb8413740 10 client.0 _ll_register_callbacks cb 0x562ffc1cacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5fae7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2c22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51276/0 (socket says 172.21.15.100:51276) 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3652298093 learned_addr learned my addr 172.21.15.100:0/3652298093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.974+0000 7f5faeffd640 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3652298093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 msgr2=0x562ffc2c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 msgr2=0x562ffc2c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5faf7fe640 1 -- 172.21.15.100:0/3652298093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa0019040 con 0x562ffc2c4750 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2c22a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009ca0 tx=0x7f5fa80bcd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924601351adcd9e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5f8effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5f8effd640 1 -- 172.21.15.100:0/3652298093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80c1280 con 0x562ffc2c4750 2024-03-12T05:17:19.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5f8effd640 1 -- 172.21.15.100:0/3652298093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5fa80c58d0 con 0x562ffc2c4750 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ffbf227e0 con 0x562ffc2c4750 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5f8effd640 1 -- 172.21.15.100:0/3652298093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa80b97a0 con 0x562ffc2c4750 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 10 client.16614 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ffc2216a0 con 0x562ffc2c4750 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5f8effd640 1 -- 172.21.15.100:0/3652298093 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa80c9020 con 0x562ffc2c4750 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5f8effd640 1 -- 172.21.15.100:0/3652298093 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fa80c5440 con 0x562ffc2c4750 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 10 client.16614 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 20 client.16614 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc2d4a20 0x562ffc2d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 4 client.16614 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.975+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ffbf5f720 con 0x562ffc2d4a20 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.976+0000 7f5faeffd640 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc2d4a20 0x562ffc2d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.976+0000 7f5f8effd640 10 client.16614 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.976+0000 7f5faeffd640 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc2d4a20 0x562ffc2d6e40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801da67db4c6a7bc server_cookie=4e4a7c5abf2b9890 in_seq=0 out_seq=0 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5f8effd640 1 -- 172.21.15.100:0/3652298093 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ffbf5f720 con 0x562ffc2d4a20 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5f8effd640 10 client.16614 handle_command_reply: tid=0 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 client.16614 shutdown 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 msgr2=0x562ffc2c22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2c22a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009ca0 tx=0x7f5fa80bcd30 comp rx=0 tx=0).stop 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 shutdown_connections 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc2d4a20 0x562ffc2d6e40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2c22a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 --2- 172.21.15.100:0/3652298093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5f8effd640 0 client.16614 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 >> 172.21.15.100:0/3652298093 conn(0x562ffc2b5980 msgr2=0x562ffc2b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 shutdown_connections 2024-03-12T05:17:19.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 1 -- 172.21.15.100:0/3652298093 wait complete. 2024-03-12T05:17:19.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.977+0000 7f5fb8413740 20 client.16614 trim_cache size 0 max 16384 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.988+0000 7f5fb8413740 1 Processor -- start 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.988+0000 7f5fb8413740 1 -- start start 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2befc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fb8413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc1a8130 con 0x562ffc2c4750 2024-03-12T05:17:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fb8413740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffbf5f720 con 0x562ffc2bbb70 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fb8413740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc2b15c0 con 0x562ffc2bb250 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2befc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2befc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51280/0 (socket says 172.21.15.100:51280) 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 -- 172.21.15.100:0/100404451 learned_addr learned my addr 172.21.15.100:0/100404451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 -- 172.21.15.100:0/100404451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 msgr2=0x562ffc2c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faeffd640 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 -- 172.21.15.100:0/100404451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 msgr2=0x562ffc2bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 -- 172.21.15.100:0/100404451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ffc1bdc20 con 0x562ffc2c4750 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2befc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80bd0c0 tx=0x7f5fa80f0390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91536aedb2026a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5f957fa640 1 -- 172.21.15.100:0/100404451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8032a20 con 0x562ffc2c4750 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5f957fa640 1 -- 172.21.15.100:0/100404451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa80c9030 con 0x562ffc2c4750 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.989+0000 7f5f957fa640 1 -- 172.21.15.100:0/100404451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80c5440 con 0x562ffc2c4750 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 -- 172.21.15.100:0/100404451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 msgr2=0x562ffc2befc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2befc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80bd0c0 tx=0x7f5fa80f0390 comp rx=0 tx=0).stop 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 -- 172.21.15.100:0/100404451 shutdown_connections 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2c4750 0x562ffc2befc0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2bbb70 0x562ffc2bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 --2- 172.21.15.100:0/100404451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bb250 0x562ffc2c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 -- 172.21.15.100:0/100404451 >> 172.21.15.100:0/100404451 conn(0x562ffc2b5980 msgr2=0x562ffc2b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 -- 172.21.15.100:0/100404451 shutdown_connections 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 -- 172.21.15.100:0/100404451 wait complete. 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 -- start start 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.990+0000 7f5fb8413740 1 Processor -- start 2024-03-12T05:17:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 0x562ffc2cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bbb70 0x562ffc2c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc2ca5b0 con 0x562ffc2bb250 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc2ca720 con 0x562ffc2c4750 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffc2ca8e0 con 0x562ffc2bbb70 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 10 client.0 _ll_register_callbacks cb 0x7f5fa0011800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 0x562ffc2cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 0x562ffc2cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51292/0 (socket says 172.21.15.100:51292) 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 -- 172.21.15.100:0/30147447 learned_addr learned my addr 172.21.15.100:0/30147447 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faf7fe640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bbb70 0x562ffc2c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 -- 172.21.15.100:0/30147447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bbb70 msgr2=0x562ffc2c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bbb70 0x562ffc2c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 -- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 msgr2=0x562ffc2ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 -- 172.21.15.100:0/30147447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ffc2216a0 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bbb70 0x562ffc2c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5faeffd640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 0x562ffc2cb740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4004700 tx=0x7f5fa4010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f607478b57badf53 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5f8effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5f8effd640 1 -- 172.21.15.100:0/30147447 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4015040 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5f8effd640 1 -- 172.21.15.100:0/30147447 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5fa4005440 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa0020c80 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5f8effd640 1 -- 172.21.15.100:0/30147447 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa4005600 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.991+0000 7f5fb8413740 10 client.16620 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ffc1e38c0 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5f8effd640 1 -- 172.21.15.100:0/30147447 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0020c80 con 0x562ffc2bb250 2024-03-12T05:17:19.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5f8effd640 1 -- 172.21.15.100:0/30147447 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fa401f660 con 0x562ffc2bb250 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 10 client.16620 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 20 client.16620 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc32c4e0 0x562ffc32e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 4 client.16620 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562ffbfae890 con 0x562ffc32c4e0 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc32c4e0 0x562ffc32e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5f8effd640 10 client.16620 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.992+0000 7f5fae7fc640 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc32c4e0 0x562ffc32e900 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909f40624bfd63dd server_cookie=b9f945df50d25d0f in_seq=0 out_seq=0 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5f8effd640 1 -- 172.21.15.100:0/30147447 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562ffbfae890 con 0x562ffc32c4e0 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5f8effd640 10 client.16620 handle_command_reply: tid=0 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 client.16620 shutdown 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 msgr2=0x562ffc2cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 0x562ffc2cb740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4004700 tx=0x7f5fa4010af0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 shutdown_connections 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ffc32c4e0 0x562ffc32e900 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ffc2c4750 0x562ffc2ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ffc2bbb70 0x562ffc2c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 --2- 172.21.15.100:0/30147447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffc2bb250 0x562ffc2cb740 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5f8effd640 0 client.16620 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 >> 172.21.15.100:0/30147447 conn(0x562ffc2b5980 msgr2=0x562ffc2c00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 shutdown_connections 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 1 -- 172.21.15.100:0/30147447 wait complete. 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb8413740 20 client.16620 trim_cache size 0 max 16384 2024-03-12T05:17:19.996 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:17:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f980a1ba0 msgr2=0x7f5f980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f980a1ba0 0x7f5f980a4060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011cb0 tx=0x7f5fa0009ef0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 msgr2=0x7f5fb0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 0x7f5fb0197a90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004af0 tx=0x7f5fa8034cc0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 shutdown_connections 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f980a1ba0 0x7f5f980a4060 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb01975e0 0x7f5fb0197a90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb0106430 0x7f5fb019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 --2- 172.21.15.100:0/3058663724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb00685d0 0x7f5fb019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.993+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 >> 172.21.15.100:0/3058663724 conn(0x7f5fb0077180 msgr2=0x7f5fb00fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.994+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 shutdown_connections 2024-03-12T05:17:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:19.994+0000 7f5fb5840640 1 -- 172.21.15.100:0/3058663724 wait complete. 2024-03-12T05:17:20.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:19 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3552742817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:17:20.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:19 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3552742817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:17:20.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:19 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3552742817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:17:20.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 591536 12.45 MB/sec execute 124 sec latency 24.006 ms 2024-03-12T05:17:21.001 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:17:21.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:17:21.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:17:21.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.117+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/2559416598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8100260 msgr2=0x7fb0f8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.117+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/2559416598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8100260 0x7fb0f8100640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009930 tx=0x7fb0e802bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/2559416598 shutdown_connections 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/2559416598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8105330 0x7fb0f810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/2559416598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f8100c10 0x7fb0f8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/2559416598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8100260 0x7fb0f8100640 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/2559416598 >> 172.21.15.100:0/2559416598 conn(0x7fb0f8077180 msgr2=0x7fb0f8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/2559416598 shutdown_connections 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/2559416598 wait complete. 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 Processor -- start 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- start start 2024-03-12T05:17:21.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8100260 0x7fb0f81966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 0x7fb0f8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f819afa0 0x7fb0f819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f819b990 con 0x7fb0f8105330 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f819bb00 con 0x7fb0f819afa0 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.118+0000 7fb0fcbd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f819bc70 con 0x7fb0f8100260 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 0x7fb0f8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 0x7fb0f8196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56268/0 (socket says 172.21.15.100:56268) 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 -- 172.21.15.100:0/1399934270 learned_addr learned my addr 172.21.15.100:0/1399934270 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f659c640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8100260 0x7fb0f81966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f819afa0 0x7fb0f819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 -- 172.21.15.100:0/1399934270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8100260 msgr2=0x7fb0f81966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8100260 0x7fb0f81966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 -- 172.21.15.100:0/1399934270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f819afa0 msgr2=0x7fb0f819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f819afa0 0x7fb0f819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 -- 172.21.15.100:0/1399934270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8009560 con 0x7fb0f8105330 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f659c640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8100260 0x7fb0f81966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 0x7fb0f8196c20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e9c0 tx=0x7fb0ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0e77fe640 1 -- 172.21.15.100:0/1399934270 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec009700 con 0x7fb0f8105330 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0e77fe640 1 -- 172.21.15.100:0/1399934270 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0ec00cd80 con 0x7fb0f8105330 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f81a3250 con 0x7fb0f8105330 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0e77fe640 1 -- 172.21.15.100:0/1399934270 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec010430 con 0x7fb0f8105330 2024-03-12T05:17:21.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.119+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f81a3700 con 0x7fb0f8105330 2024-03-12T05:17:21.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.121+0000 7fb0e77fe640 1 -- 172.21.15.100:0/1399934270 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb0ec005000 con 0x7fb0f8105330 2024-03-12T05:17:21.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 Processor -- start 2024-03-12T05:17:21.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 -- start start 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceaff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb0d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cea6bf90 con 0x5626ceaffba0 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cea694c0 con 0x5626ceaff280 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cea64ca0 con 0x5626ceb0ae20 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0f6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceaff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0e77fe640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb0e00a1ba0 0x7fb0e00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0f659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0f6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceaff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:35282/0 (socket says 172.21.15.100:35282) 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/3678296255 learned_addr learned my addr 172.21.15.100:0/3678296255 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb0d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.122+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb0e00a1ba0 0x7fb0e00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/3678296255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 msgr2=0x5626ceb0d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb0d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/3678296255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 msgr2=0x5626ceb0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb0a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0e77fe640 1 -- 172.21.15.100:0/1399934270 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb0ec014070 con 0x7fb0f8105330 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/3678296255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626cea270b0 con 0x5626ceaff280 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceaff660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0005c40 tx=0x7fb0f0013f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4dcf9929b9986ae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb0e00a1ba0 0x7fb0e00a4060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0f001d000 tx=0x7fb0f001f000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0e4ff9640 1 -- 172.21.15.100:0/3678296255 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000fde0 con 0x5626ceaff280 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0e4ff9640 1 -- 172.21.15.100:0/3678296255 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0f0012d00 con 0x5626ceaff280 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.123+0000 7fb0e4ff9640 1 -- 172.21.15.100:0/3678296255 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0010760 con 0x5626ceaff280 2024-03-12T05:17:21.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3678296255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 msgr2=0x5626ceaff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceaff660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0005c40 tx=0x7fb0f0013f50 comp rx=0 tx=0).stop 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3678296255 shutdown_connections 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb0d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3678296255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceaff660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3678296255 >> 172.21.15.100:0/3678296255 conn(0x5626ceaf9a50 msgr2=0x5626ceaf9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3678296255 shutdown_connections 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3678296255 wait complete. 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 -- start start 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.124+0000 7fb0ff79b740 1 Processor -- start 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceb07ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb08970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cea09f40 con 0x5626ceaffba0 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ce7384b0 con 0x5626ceaff280 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ce9ea670 con 0x5626ceb0ae20 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 10 client.0 _ll_register_callbacks cb 0x5626cea57f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceb07ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceb07ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:35296/0 (socket says 172.21.15.100:35296) 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb08430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56286/0 (socket says 172.21.15.100:56286) 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 -- 172.21.15.100:0/1189661425 learned_addr learned my addr 172.21.15.100:0/1189661425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb08970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 -- 172.21.15.100:0/1189661425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 msgr2=0x5626ceb08970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb08970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 -- 172.21.15.100:0/1189661425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 msgr2=0x5626ceb08430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb08430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 -- 172.21.15.100:0/1189661425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f00136a0 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0f659c640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceb07ef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005e00 tx=0x7fb0e802c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b6deb36cd239119 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ddffb640 1 -- 172.21.15.100:0/1189661425 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8004410 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626cea3ef40 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ddffb640 1 -- 172.21.15.100:0/1189661425 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb0e8004df0 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ddffb640 1 -- 172.21.15.100:0/1189661425 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e8004fb0 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 10 client.26012 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.125+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5626cea63320 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ddffb640 1 -- 172.21.15.100:0/1189661425 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626cea3ef40 con 0x5626ceaff280 2024-03-12T05:17:21.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ddffb640 1 -- 172.21.15.100:0/1189661425 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0e803c040 con 0x5626ceaff280 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ff79b740 10 client.26012 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ff79b740 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ff79b740 20 client.26012 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb1b2d0 0x5626ceb1d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ff79b740 4 client.26012 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626cea5b310 con 0x5626ceb1b2d0 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb1b2d0 0x5626ceb1d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb1b2d0 0x5626ceb1d6c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c0c9fb7b16d0a9 server_cookie=316769c633059b35 in_seq=0 out_seq=0 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.126+0000 7fb0ddffb640 10 client.26012 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.127+0000 7fb0ddffb640 1 -- 172.21.15.100:0/1189661425 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626cea5b310 con 0x5626ceb1b2d0 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.127+0000 7fb0ddffb640 10 client.26012 handle_command_reply: tid=0 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.127+0000 7fb0ff79b740 1 client.26012 shutdown 2024-03-12T05:17:21.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.127+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 msgr2=0x5626ceb07ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.127+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceb07ef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005e00 tx=0x7fb0e802c700 comp rx=0 tx=0).stop 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 shutdown_connections 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb1b2d0 0x5626ceb1d6c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceb0ae20 0x5626ceb08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaffba0 0x5626ceb08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1189661425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaff280 0x5626ceb07ef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ddffb640 0 client.26012 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 >> 172.21.15.100:0/1189661425 conn(0x5626ceaf9a50 msgr2=0x5626ceafb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 shutdown_connections 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1189661425 wait complete. 2024-03-12T05:17:21.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.128+0000 7fb0ff79b740 20 client.26012 trim_cache size 0 max 16384 2024-03-12T05:17:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 Processor -- start 2024-03-12T05:17:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 -- start start 2024-03-12T05:17:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaff280 0x5626ceb058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaffba0 0x5626ceb08790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 0x5626ceb11ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ceafae90 con 0x5626ceb0ae20 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ceaf98b0 con 0x5626ceaffba0 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ceaff0a0 con 0x5626ceaff280 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.143+0000 7fb0f6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 0x5626ceb11ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 0x5626ceb11ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56296/0 (socket says 172.21.15.100:56296) 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/1683786385 learned_addr learned my addr 172.21.15.100:0/1683786385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f659c640 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaff280 0x5626ceb058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaffba0 0x5626ceb08790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/1683786385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaff280 msgr2=0x5626ceb058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaff280 0x5626ceb058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/1683786385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaffba0 msgr2=0x5626ceb08790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaffba0 0x5626ceb08790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 -- 172.21.15.100:0/1683786385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ce9ffc00 con 0x5626ceb0ae20 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaffba0 0x5626ceb08790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 0x5626ceb11ca0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0024040 tx=0x7fb0f0012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80f84ec85853209 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0dffff640 1 -- 172.21.15.100:0/1683786385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00120d0 con 0x5626ceb0ae20 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0dffff640 1 -- 172.21.15.100:0/1683786385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0f0012820 con 0x5626ceb0ae20 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.144+0000 7fb0dffff640 1 -- 172.21.15.100:0/1683786385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0012230 con 0x5626ceb0ae20 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1683786385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 msgr2=0x5626ceb11ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 0x5626ceb11ca0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0024040 tx=0x7fb0f0012390 comp rx=0 tx=0).stop 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1683786385 shutdown_connections 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceb0ae20 0x5626ceb11ca0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceaffba0 0x5626ceb08790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/1683786385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaff280 0x5626ceb058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1683786385 >> 172.21.15.100:0/1683786385 conn(0x5626ceaf9a50 msgr2=0x5626ceaf7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1683786385 shutdown_connections 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- 172.21.15.100:0/1683786385 wait complete. 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- start start 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 Processor -- start 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 0x5626ceb10d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaffba0 0x5626ceb11250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceb0ae20 0x5626ceb717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ceb11950 con 0x5626ceaff280 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ceb71d10 con 0x5626ceb0ae20 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ceb71e50 con 0x5626ceaffba0 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0ff79b740 10 client.0 _ll_register_callbacks cb 0x7fb0f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0f659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 0x5626ceb10d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0f659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 0x5626ceb10d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56302/0 (socket says 172.21.15.100:56302) 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0f659c640 1 -- 172.21.15.100:0/3080188678 learned_addr learned my addr 172.21.15.100:0/3080188678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaffba0 0x5626ceb11250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.145+0000 7fb0f6d9d640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceb0ae20 0x5626ceb717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0f659c640 1 -- 172.21.15.100:0/3080188678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaffba0 msgr2=0x5626ceb11250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0f659c640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaffba0 0x5626ceb11250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0f659c640 1 -- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceb0ae20 msgr2=0x5626ceb717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0f659c640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceb0ae20 0x5626ceb717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0f659c640 1 -- 172.21.15.100:0/3080188678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626cea270b0 con 0x5626ceaff280 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0f659c640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 0x5626ceb10d10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005a50 tx=0x7fb0e802cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d95ba236bc70bd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ddffb640 1 -- 172.21.15.100:0/3080188678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8003fa0 con 0x5626ceaff280 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ddffb640 1 -- 172.21.15.100:0/3080188678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb0e8000e40 con 0x5626ceaff280 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626cea8a900 con 0x5626ceaff280 2024-03-12T05:17:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ddffb640 1 -- 172.21.15.100:0/3080188678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e8003e20 con 0x5626ceaff280 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 10 client.16638 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5626cea25910 con 0x5626ceaff280 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ddffb640 1 -- 172.21.15.100:0/3080188678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626cea8a900 con 0x5626ceaff280 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ddffb640 1 -- 172.21.15.100:0/3080188678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0e803c070 con 0x5626ceaff280 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 10 client.16638 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 20 client.16638 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb733e0 0x5626ceb75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 4 client.16638 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.146+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5626ce764860 con 0x5626ceb733e0 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.147+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb733e0 0x5626ceb75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.147+0000 7fb0ddffb640 10 client.16638 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.147+0000 7fb0f5d9b640 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb733e0 0x5626ceb75800 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c49ec53216554 server_cookie=7e4fd190db5190b9 in_seq=0 out_seq=0 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ddffb640 1 -- 172.21.15.100:0/3080188678 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5626ce764860 con 0x5626ceb733e0 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ddffb640 10 client.16638 handle_command_reply: tid=0 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 client.16638 shutdown 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 msgr2=0x5626ceb10d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 0x5626ceb10d10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005a50 tx=0x7fb0e802cc30 comp rx=0 tx=0).stop 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 shutdown_connections 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5626ceb733e0 0x5626ceb75800 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626ceb0ae20 0x5626ceb717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626ceaffba0 0x5626ceb11250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 --2- 172.21.15.100:0/3080188678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626ceaff280 0x5626ceb10d10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 >> 172.21.15.100:0/3080188678 conn(0x5626ceaf9a50 msgr2=0x5626ceaf6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ddffb640 0 client.16638 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 shutdown_connections 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 1 -- 172.21.15.100:0/3080188678 wait complete. 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.148+0000 7fb0ff79b740 20 client.16638 trim_cache size 0 max 16384 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "3d91e252-0b97-48cb-8e3e-67f9aee0398c", 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:17:21.153 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:17:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb0e00a1ba0 msgr2=0x7fb0e00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb0e00a1ba0 0x7fb0e00a4060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0f001d000 tx=0x7fb0f001f000 comp rx=0 tx=0).stop 2024-03-12T05:17:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 msgr2=0x7fb0f8196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 0x7fb0f8196c20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e9c0 tx=0x7fb0ec00ee80 comp rx=0 tx=0).stop 2024-03-12T05:17:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 shutdown_connections 2024-03-12T05:17:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb0e00a1ba0 0x7fb0e00a4060 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f819afa0 0x7fb0f819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f8105330 0x7fb0f8196c20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 --2- 172.21.15.100:0/1399934270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f8100260 0x7fb0f81966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 >> 172.21.15.100:0/1399934270 conn(0x7fb0f8077180 msgr2=0x7fb0f80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 shutdown_connections 2024-03-12T05:17:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.149+0000 7fb0fcbd0640 1 -- 172.21.15.100:0/1399934270 wait complete. 2024-03-12T05:17:21.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:17:21.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:17:21.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:20 smithi100 ceph-mon[32210]: pgmap v160: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.2 MiB/s wr, 78 op/s 2024-03-12T05:17:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:20 smithi156 ceph-mon[36270]: pgmap v160: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.2 MiB/s wr, 78 op/s 2024-03-12T05:17:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:20 smithi161 ceph-mon[36043]: pgmap v160: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.2 MiB/s wr, 78 op/s 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.270+0000 7fab4b6fb640 1 -- 172.21.15.100:0/728171268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44105340 msgr2=0x7fab4410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.270+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/728171268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44105340 0x7fab4410cf10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009cd0 tx=0x7fab3c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.270+0000 7fab4b6fb640 1 -- 172.21.15.100:0/728171268 shutdown_connections 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.270+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/728171268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44105340 0x7fab4410cf10 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/728171268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab44100c20 0x7fab44104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/728171268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab44100270 0x7fab44100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- 172.21.15.100:0/728171268 >> 172.21.15.100:0/728171268 conn(0x7fab44077230 msgr2=0x7fab44077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- 172.21.15.100:0/728171268 shutdown_connections 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- 172.21.15.100:0/728171268 wait complete. 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 Processor -- start 2024-03-12T05:17:21.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- start start 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 0x7fab4419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab44105340 0x7fab4419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab441975c0 0x7fab44197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44197fb0 con 0x7fab44100270 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44198120 con 0x7fab44105340 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab4b6fb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441982e0 con 0x7fab441975c0 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab49497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 0x7fab4419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab49497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 0x7fab4419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56320/0 (socket says 172.21.15.100:56320) 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.271+0000 7fab49497640 1 -- 172.21.15.100:0/2178091026 learned_addr learned my addr 172.21.15.100:0/2178091026 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab48c96640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab44105340 0x7fab4419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49c98640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab441975c0 0x7fab44197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49497640 1 -- 172.21.15.100:0/2178091026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab441975c0 msgr2=0x7fab44197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49497640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab441975c0 0x7fab44197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49497640 1 -- 172.21.15.100:0/2178091026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab44105340 msgr2=0x7fab4419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49497640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab44105340 0x7fab4419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49497640 1 -- 172.21.15.100:0/2178091026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c009900 con 0x7fab44100270 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49c98640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab441975c0 0x7fab44197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab49497640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 0x7fab4419d410 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fab4000d770 tx=0x7fab4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab3a7fc640 1 -- 172.21.15.100:0/2178091026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40009c30 con 0x7fab44100270 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab441a42d0 con 0x7fab44100270 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab3a7fc640 1 -- 172.21.15.100:0/2178091026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab4000de80 con 0x7fab44100270 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab3a7fc640 1 -- 172.21.15.100:0/2178091026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40004e80 con 0x7fab44100270 2024-03-12T05:17:21.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.272+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441a47e0 con 0x7fab44100270 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.274+0000 7fab3a7fc640 1 -- 172.21.15.100:0/2178091026 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fab40005020 con 0x7fab44100270 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 Processor -- start 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 -- start start 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x561499314690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x56149931d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561499283280 con 0x5614993142b0 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614992807b0 con 0x561499314bd0 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149927bf90 con 0x56149931d7b0 2024-03-12T05:17:21.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab3a7fc640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab300a1ba0 0x7fab300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab49c98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x561499314690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab49c98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x561499314690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56336/0 (socket says 172.21.15.100:56336) 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab49c98640 1 -- 172.21.15.100:0/3202713873 learned_addr learned my addr 172.21.15.100:0/3202713873 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab49497640 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x56149931d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab49c98640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab300a1ba0 0x7fab300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.275+0000 7fab48c96640 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab3a7fc640 1 -- 172.21.15.100:0/2178091026 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fab400cd260 con 0x7fab44100270 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49c98640 1 -- 172.21.15.100:0/3202713873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 msgr2=0x56149931fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49c98640 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49c98640 1 -- 172.21.15.100:0/3202713873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 msgr2=0x56149931d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49c98640 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x56149931d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49c98640 1 -- 172.21.15.100:0/3202713873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56149923a570 con 0x5614993142b0 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49497640 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x56149931d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab49c98640 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x561499314690 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fab3c02c700 tx=0x7fab3c032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b438b10c1aff1785 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab2f7fe640 1 -- 172.21.15.100:0/3202713873 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0347c0 con 0x5614993142b0 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab2f7fe640 1 -- 172.21.15.100:0/3202713873 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3c034e60 con 0x5614993142b0 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.276+0000 7fab2f7fe640 1 -- 172.21.15.100:0/3202713873 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004320 con 0x5614993142b0 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3202713873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 msgr2=0x561499314690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab49c98640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab300a1ba0 0x7fab300a4060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab3c04e000 tx=0x7fab3c04a000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x561499314690 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fab3c02c700 tx=0x7fab3c032610 comp rx=0 tx=0).stop 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3202713873 shutdown_connections 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x56149931d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3202713873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x561499314690 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3202713873 >> 172.21.15.100:0/3202713873 conn(0x56149930e990 msgr2=0x56149930ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3202713873 shutdown_connections 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3202713873 wait complete. 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 -- start start 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.277+0000 7fab4e3ad740 1 Processor -- start 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614993142b0 0x56149931d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 0x56149931a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561498f4f430 con 0x561499314bd0 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561498f0be00 con 0x5614993142b0 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561499200fd0 con 0x56149931d7b0 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 10 client.0 _ll_register_callbacks cb 0x561499256fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 0x56149931a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab49497640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614993142b0 0x56149931d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 0x56149931a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56350/0 (socket says 172.21.15.100:56350) 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 -- 172.21.15.100:0/849237624 learned_addr learned my addr 172.21.15.100:0/849237624 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab49c98640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 -- 172.21.15.100:0/849237624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 msgr2=0x56149931acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 -- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614993142b0 msgr2=0x56149931d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614993142b0 0x56149931d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 -- 172.21.15.100:0/849237624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab440682e0 con 0x561499314bd0 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab49497640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614993142b0 0x56149931d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab48c96640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 0x56149931a590 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fab4406a8f0 tx=0x7fab34012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8001482da84ff14 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab2cff9640 1 -- 172.21.15.100:0/849237624 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34004220 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab2cff9640 1 -- 172.21.15.100:0/849237624 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fab34004380 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.278+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561498f7b7e0 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab2cff9640 1 -- 172.21.15.100:0/849237624 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab34005020 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 10 client.16650 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56149927a610 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab2cff9640 1 -- 172.21.15.100:0/849237624 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561498f7b7e0 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab2cff9640 1 -- 172.21.15.100:0/849237624 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab34010710 con 0x561499314bd0 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 10 client.16650 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 20 client.16650 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149932da00 0x56149932fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 4 client.16650 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:21.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561498fb8720 con 0x56149932da00 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.279+0000 7fab49497640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149932da00 0x56149932fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab49497640 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149932da00 0x56149932fe20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b51a4090c8b5975 server_cookie=1dfd4621461cde7 in_seq=0 out_seq=0 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab2cff9640 10 client.16650 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab2cff9640 1 -- 172.21.15.100:0/849237624 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561498fb8720 con 0x56149932da00 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab2cff9640 10 client.16650 handle_command_reply: tid=0 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 client.16650 shutdown 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 msgr2=0x56149931a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 0x56149931a590 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fab4406a8f0 tx=0x7fab34012820 comp rx=0 tx=0).stop 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 shutdown_connections 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149932da00 0x56149932fe20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x56149931acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561499314bd0 0x56149931a590 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab2cff9640 0 client.16650 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/849237624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614993142b0 0x56149931d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 >> 172.21.15.100:0/849237624 conn(0x56149930e990 msgr2=0x561499310730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 shutdown_connections 2024-03-12T05:17:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 1 -- 172.21.15.100:0/849237624 wait complete. 2024-03-12T05:17:21.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.280+0000 7fab4e3ad740 20 client.16650 trim_cache size 0 max 16384 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 Processor -- start 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 -- start start 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x56149931ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561499314bd0 0x56149931d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149931d7b0 0x561499318e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149930f3e0 con 0x5614993142b0 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561498fb8750 con 0x56149931d7b0 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149930a0f0 con 0x561499314bd0 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x56149931ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x56149931ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56366/0 (socket says 172.21.15.100:56366) 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 -- 172.21.15.100:0/2653964488 learned_addr learned my addr 172.21.15.100:0/2653964488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49c98640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149931d7b0 0x561499318e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab48c96640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561499314bd0 0x56149931d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 -- 172.21.15.100:0/2653964488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561499314bd0 msgr2=0x56149931d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561499314bd0 0x56149931d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 -- 172.21.15.100:0/2653964488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149931d7b0 msgr2=0x561499318e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149931d7b0 0x561499318e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49497640 1 -- 172.21.15.100:0/2653964488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561499216ac0 con 0x5614993142b0 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab48c96640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561499314bd0 0x56149931d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.296+0000 7fab49c98640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149931d7b0 0x561499318e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab49497640 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x56149931ccc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fab40009610 tx=0x7fab400cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a396b07365f8c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab2effd640 1 -- 172.21.15.100:0/2653964488 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4009dfd0 con 0x5614993142b0 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab2effd640 1 -- 172.21.15.100:0/2653964488 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab400a2620 con 0x5614993142b0 2024-03-12T05:17:21.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab2effd640 1 -- 172.21.15.100:0/2653964488 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab400964f0 con 0x5614993142b0 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 -- 172.21.15.100:0/2653964488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 msgr2=0x56149931ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x56149931ccc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fab40009610 tx=0x7fab400cf040 comp rx=0 tx=0).stop 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 -- 172.21.15.100:0/2653964488 shutdown_connections 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149931d7b0 0x561499318e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561499314bd0 0x56149931d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/2653964488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x56149931ccc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 -- 172.21.15.100:0/2653964488 >> 172.21.15.100:0/2653964488 conn(0x56149930e990 msgr2=0x561499315850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.297+0000 7fab4e3ad740 1 -- 172.21.15.100:0/2653964488 shutdown_connections 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 -- 172.21.15.100:0/2653964488 wait complete. 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 -- start start 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 Processor -- start 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x5614993248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x561499320c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x5614993211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561499321770 con 0x5614993142b0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614993218e0 con 0x561499314bd0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561499321aa0 con 0x56149931d7b0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab4e3ad740 10 client.0 _ll_register_callbacks cb 0x7fab3c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab49497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x5614993248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab49497640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x5614993248f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56374/0 (socket says 172.21.15.100:56374) 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab49497640 1 -- 172.21.15.100:0/3206973003 learned_addr learned my addr 172.21.15.100:0/3206973003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab49c98640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x5614993211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.298+0000 7fab48c96640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x561499320c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab49497640 1 -- 172.21.15.100:0/3206973003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 msgr2=0x5614993211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab49497640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x5614993211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab49497640 1 -- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 msgr2=0x561499320c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab49497640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x561499320c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab49497640 1 -- 172.21.15.100:0/3206973003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab400cd040 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab49497640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x5614993248f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fab400047e0 tx=0x7fab40099ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a6d617bfc9eef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab2cff9640 1 -- 172.21.15.100:0/3206973003 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4009dfd0 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab2cff9640 1 -- 172.21.15.100:0/3206973003 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fab400cac20 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab3c034e60 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab2cff9640 1 -- 172.21.15.100:0/3206973003 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab400cade0 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 10 client.16656 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56149923c6a0 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab2cff9640 1 -- 172.21.15.100:0/3206973003 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3c034e60 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab2cff9640 1 -- 172.21.15.100:0/3206973003 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab400830c0 con 0x5614993142b0 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 10 client.16656 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:21.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 20 client.16656 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561499389570 0x56149938b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 4 client.16656 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.299+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561499007890 con 0x561499389570 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab48c96640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561499389570 0x56149938b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab48c96640 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561499389570 0x56149938b960 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25a067cff5a8adb server_cookie=fd41454a1b56fcf3 in_seq=0 out_seq=0 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab2cff9640 10 client.16656 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab2cff9640 1 -- 172.21.15.100:0/3206973003 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561499007890 con 0x561499389570 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab2cff9640 10 client.16656 handle_command_reply: tid=0 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 client.16656 shutdown 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 msgr2=0x5614993248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x5614993248f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fab400047e0 tx=0x7fab40099ac0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 shutdown_connections 2024-03-12T05:17:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561499389570 0x56149938b960 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149931d7b0 0x5614993211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab2cff9640 0 client.16656 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561499314bd0 0x561499320c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 --2- 172.21.15.100:0/3206973003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614993142b0 0x5614993248f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.300+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 >> 172.21.15.100:0/3206973003 conn(0x56149930e990 msgr2=0x561499318e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 shutdown_connections 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4e3ad740 1 -- 172.21.15.100:0/3206973003 wait complete. 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4e3ad740 20 client.16656 trim_cache size 0 max 16384 2024-03-12T05:17:21.303 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: "3d91e252-0b97-48cb-8e3e-67f9aee0398c": { 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "3d91e252-0b97-48cb-8e3e-67f9aee0398c", 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:17:21.304 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab300a1ba0 msgr2=0x7fab300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab300a1ba0 0x7fab300a4060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab3c04e000 tx=0x7fab3c04a000 comp rx=0 tx=0).stop 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 msgr2=0x7fab4419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 0x7fab4419d410 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fab4000d770 tx=0x7fab4000dc30 comp rx=0 tx=0).stop 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 shutdown_connections 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab300a1ba0 0x7fab300a4060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab441975c0 0x7fab44197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab44105340 0x7fab4419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 --2- 172.21.15.100:0/2178091026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab44100270 0x7fab4419d410 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:21.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 >> 172.21.15.100:0/2178091026 conn(0x7fab44077230 msgr2=0x7fab440fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:21.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 shutdown_connections 2024-03-12T05:17:21.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:21.301+0000 7fab4b6fb640 1 -- 172.21.15.100:0/2178091026 wait complete. 2024-03-12T05:17:21.308 INFO:tasks.cephfs.filesystem:scrub status for tag:3d91e252-0b97-48cb-8e3e-67f9aee0398c - {'path': '/', 'tag': '3d91e252-0b97-48cb-8e3e-67f9aee0398c', 'options': 'recursive,force'} 2024-03-12T05:17:21.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 593772 12.46 MB/sec execute 125 sec latency 16.182 ms 2024-03-12T05:17:22.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:21 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:17:22.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:21 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:21 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:17:22.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:21 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:17:22.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:21 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:21 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:21 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:17:22.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:21 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:21 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:17:22.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:21 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:17:22.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:21 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:21 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:21 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:17:22.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:21 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:21 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:17:22.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:21 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:17:22.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:21 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:21 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:22.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 596358 12.46 MB/sec execute 126 sec latency 15.394 ms 2024-03-12T05:17:23.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:22 smithi156 ceph-mon[36270]: pgmap v161: 97 pgs: 97 active+clean; 346 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 3.5 MiB/s wr, 146 op/s 2024-03-12T05:17:23.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:22 smithi161 ceph-mon[36043]: pgmap v161: 97 pgs: 97 active+clean; 346 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 3.5 MiB/s wr, 146 op/s 2024-03-12T05:17:23.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:22 smithi100 ceph-mon[32210]: pgmap v161: 97 pgs: 97 active+clean; 346 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 3.5 MiB/s wr, 146 op/s 2024-03-12T05:17:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 597969 12.43 MB/sec execute 127 sec latency 103.326 ms 2024-03-12T05:17:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 600625 12.46 MB/sec execute 128 sec latency 26.149 ms 2024-03-12T05:17:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:24 smithi156 ceph-mon[36270]: pgmap v162: 97 pgs: 97 active+clean; 349 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-03-12T05:17:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:24 smithi161 ceph-mon[36043]: pgmap v162: 97 pgs: 97 active+clean; 349 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-03-12T05:17:25.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:24 smithi100 ceph-mon[32210]: pgmap v162: 97 pgs: 97 active+clean; 349 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-03-12T05:17:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 602709 12.42 MB/sec execute 129 sec latency 20.530 ms 2024-03-12T05:17:26.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 604911 12.43 MB/sec execute 130 sec latency 18.667 ms 2024-03-12T05:17:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:26 smithi156 ceph-mon[36270]: pgmap v163: 97 pgs: 97 active+clean; 349 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-03-12T05:17:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:26 smithi161 ceph-mon[36043]: pgmap v163: 97 pgs: 97 active+clean; 349 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-03-12T05:17:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:26 smithi100 ceph-mon[32210]: pgmap v163: 97 pgs: 97 active+clean; 349 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-03-12T05:17:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 607510 12.46 MB/sec execute 131 sec latency 24.675 ms 2024-03-12T05:17:28.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 608953 12.41 MB/sec execute 132 sec latency 94.443 ms 2024-03-12T05:17:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:28 smithi156 ceph-mon[36270]: pgmap v164: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 233 op/s 2024-03-12T05:17:29.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:28 smithi161 ceph-mon[36043]: pgmap v164: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 233 op/s 2024-03-12T05:17:29.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:28 smithi100 ceph-mon[32210]: pgmap v164: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 233 op/s 2024-03-12T05:17:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 611349 12.43 MB/sec execute 133 sec latency 29.193 ms 2024-03-12T05:17:30.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:29 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:17:30.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:29 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:17:30.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:29 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:17:30.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 613489 12.39 MB/sec execute 134 sec latency 43.580 ms 2024-03-12T05:17:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:30 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:17:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:30 smithi156 ceph-mon[36270]: pgmap v165: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 195 op/s 2024-03-12T05:17:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:30 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:17:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:30 smithi161 ceph-mon[36043]: pgmap v165: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 195 op/s 2024-03-12T05:17:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:30 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:17:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:30 smithi100 ceph-mon[32210]: pgmap v165: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 195 op/s 2024-03-12T05:17:31.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 615658 12.40 MB/sec execute 135 sec latency 17.079 ms 2024-03-12T05:17:32.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:31 smithi156 ceph-mon[36270]: pgmap v166: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 303 op/s 2024-03-12T05:17:32.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:31 smithi161 ceph-mon[36043]: pgmap v166: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 303 op/s 2024-03-12T05:17:32.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:31 smithi100 ceph-mon[32210]: pgmap v166: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 303 op/s 2024-03-12T05:17:32.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 618293 12.43 MB/sec execute 136 sec latency 17.829 ms 2024-03-12T05:17:33.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 620282 12.39 MB/sec execute 137 sec latency 51.731 ms 2024-03-12T05:17:34.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 622441 12.40 MB/sec execute 138 sec latency 18.195 ms 2024-03-12T05:17:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:34 smithi156 ceph-mon[36270]: pgmap v167: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 236 op/s 2024-03-12T05:17:35.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:34 smithi161 ceph-mon[36043]: pgmap v167: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 236 op/s 2024-03-12T05:17:35.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:34 smithi100 ceph-mon[32210]: pgmap v167: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 236 op/s 2024-03-12T05:17:35.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 625295 12.44 MB/sec execute 139 sec latency 21.706 ms 2024-03-12T05:17:36.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 626992 12.39 MB/sec execute 140 sec latency 84.327 ms 2024-03-12T05:17:37.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:36 smithi156 ceph-mon[36270]: pgmap v168: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 214 op/s 2024-03-12T05:17:37.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:36 smithi161 ceph-mon[36043]: pgmap v168: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 214 op/s 2024-03-12T05:17:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:36 smithi100 ceph-mon[32210]: pgmap v168: 97 pgs: 97 active+clean; 363 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 214 op/s 2024-03-12T05:17:37.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 629417 12.40 MB/sec execute 141 sec latency 18.652 ms 2024-03-12T05:17:38.791 INFO:tasks.workunit.client.0.smithi100.stdout: 1 631589 12.40 MB/sec execute 142 sec latency 125.779 ms 2024-03-12T05:17:39.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:38 smithi156 ceph-mon[36270]: pgmap v169: 97 pgs: 97 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 312 op/s 2024-03-12T05:17:39.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:38 smithi161 ceph-mon[36043]: pgmap v169: 97 pgs: 97 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 312 op/s 2024-03-12T05:17:39.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:38 smithi100 ceph-mon[32210]: pgmap v169: 97 pgs: 97 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 312 op/s 2024-03-12T05:17:39.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 633659 12.39 MB/sec execute 143 sec latency 22.445 ms 2024-03-12T05:17:40.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 636185 12.41 MB/sec execute 144 sec latency 13.494 ms 2024-03-12T05:17:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:40 smithi156 ceph-mon[36270]: pgmap v170: 97 pgs: 97 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-03-12T05:17:41.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:40 smithi161 ceph-mon[36043]: pgmap v170: 97 pgs: 97 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-03-12T05:17:41.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:40 smithi100 ceph-mon[32210]: pgmap v170: 97 pgs: 97 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-03-12T05:17:41.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 638640 12.41 MB/sec execute 145 sec latency 17.405 ms 2024-03-12T05:17:42.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 640641 12.40 MB/sec execute 146 sec latency 68.788 ms 2024-03-12T05:17:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:42 smithi156 ceph-mon[36270]: pgmap v171: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-03-12T05:17:43.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:42 smithi161 ceph-mon[36043]: pgmap v171: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-03-12T05:17:43.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:42 smithi100 ceph-mon[32210]: pgmap v171: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-03-12T05:17:43.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 643170 12.41 MB/sec execute 147 sec latency 29.517 ms 2024-03-12T05:17:44.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 645395 12.38 MB/sec execute 148 sec latency 17.287 ms 2024-03-12T05:17:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:44 smithi156 ceph-mon[36270]: pgmap v172: 97 pgs: 97 active+clean; 379 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-03-12T05:17:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:44 smithi161 ceph-mon[36043]: pgmap v172: 97 pgs: 97 active+clean; 379 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-03-12T05:17:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:44 smithi100 ceph-mon[32210]: pgmap v172: 97 pgs: 97 active+clean; 379 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-03-12T05:17:45.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 647570 12.39 MB/sec execute 149 sec latency 16.853 ms 2024-03-12T05:17:46.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 650147 12.42 MB/sec execute 150 sec latency 27.226 ms 2024-03-12T05:17:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:46 smithi156 ceph-mon[36270]: pgmap v173: 97 pgs: 97 active+clean; 379 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-03-12T05:17:47.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:46 smithi161 ceph-mon[36043]: pgmap v173: 97 pgs: 97 active+clean; 379 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-03-12T05:17:47.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:46 smithi100 ceph-mon[32210]: pgmap v173: 97 pgs: 97 active+clean; 379 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-03-12T05:17:47.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 652095 12.38 MB/sec execute 151 sec latency 21.344 ms 2024-03-12T05:17:48.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 654231 12.38 MB/sec execute 152 sec latency 16.460 ms 2024-03-12T05:17:49.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:48 smithi156 ceph-mon[36270]: pgmap v174: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-12T05:17:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:48 smithi161 ceph-mon[36043]: pgmap v174: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-12T05:17:49.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:48 smithi100 ceph-mon[32210]: pgmap v174: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-12T05:17:49.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 656965 12.41 MB/sec execute 153 sec latency 25.703 ms 2024-03-12T05:17:50.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 658687 12.38 MB/sec execute 154 sec latency 18.161 ms 2024-03-12T05:17:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:50 smithi156 ceph-mon[36270]: pgmap v175: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-03-12T05:17:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:50 smithi161 ceph-mon[36043]: pgmap v175: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-03-12T05:17:51.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:17:51.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:17:51.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:50 smithi100 ceph-mon[32210]: pgmap v175: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-03-12T05:17:51.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3211022521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe4101d70 msgr2=0x7f6fe4105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3211022521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe4101d70 0x7f6fe4105db0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009930 tx=0x7f6fd802bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:51.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3211022521 shutdown_connections 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3211022521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 0x7f6fe410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3211022521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe4101d70 0x7f6fe4105db0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3211022521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe41013c0 0x7f6fe41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3211022521 >> 172.21.15.100:0/3211022521 conn(0x7f6fe40fcc20 msgr2=0x7f6fe40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3211022521 shutdown_connections 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3211022521 wait complete. 2024-03-12T05:17:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.422+0000 7f6fe9dbc640 1 Processor -- start 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 -- start start 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 0x7f6fe4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 0x7f6fe40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe4073f00 0x7f6fe41a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe41a5ff0 con 0x7f6fe41013c0 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe41a6160 con 0x7f6fe4106490 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe41a6320 con 0x7f6fe4073f00 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 0x7f6fe4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 0x7f6fe4073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59980/0 (socket says 172.21.15.100:59980) 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 -- 172.21.15.100:0/3196419851 learned_addr learned my addr 172.21.15.100:0/3196419851 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 0x7f6fe40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe4073f00 0x7f6fe41a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 -- 172.21.15.100:0/3196419851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe4073f00 msgr2=0x7f6fe41a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe4073f00 0x7f6fe41a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 -- 172.21.15.100:0/3196419851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 msgr2=0x7f6fe40739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 0x7f6fe40739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 -- 172.21.15.100:0/3196419851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc009840 con 0x7f6fe41013c0 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 0x7f6fe40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 0x7f6fe4073480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400e9f0 tx=0x7f6fd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe0ff9640 1 -- 172.21.15.100:0/3196419851 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4009720 con 0x7f6fe41013c0 2024-03-12T05:17:51.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.423+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd8009560 con 0x7f6fe41013c0 2024-03-12T05:17:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.424+0000 7f6fe0ff9640 1 -- 172.21.15.100:0/3196419851 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4005500 con 0x7f6fe41013c0 2024-03-12T05:17:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.424+0000 7f6fe0ff9640 1 -- 172.21.15.100:0/3196419851 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4010430 con 0x7f6fe41013c0 2024-03-12T05:17:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.424+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe41a6950 con 0x7f6fe41013c0 2024-03-12T05:17:51.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe0ff9640 1 -- 172.21.15.100:0/3196419851 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6fd4005020 con 0x7f6fe41013c0 2024-03-12T05:17:51.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 Processor -- start 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 -- start start 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9de2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9dedba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9d51280 con 0x55a5a9de22b0 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9d4e7b0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fec993740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9d49f90 con 0x55a5a9deb7b0 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9de2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9de2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59992/0 (socket says 172.21.15.100:59992) 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 -- 172.21.15.100:0/1779009548 learned_addr learned my addr 172.21.15.100:0/1779009548 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe0ff9640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fcc0a1bd0 0x7f6fcc0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9dedba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 -- 172.21.15.100:0/1779009548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 msgr2=0x55a5a9dedba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9dedba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 -- 172.21.15.100:0/1779009548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 msgr2=0x55a5a9deb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe3fff640 1 -- 172.21.15.100:0/1779009548 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a9d08570 con 0x55a5a9de22b0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9dedba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.427+0000 7f6fe0ff9640 1 -- 172.21.15.100:0/3196419851 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6fd4014070 con 0x7f6fe41013c0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.428+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fcc0a1bd0 0x7f6fcc0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.428+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9de2690 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0116b0 tx=0x7f6fdc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55704b2ba8ee5b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.428+0000 7f6fc9ffb640 1 -- 172.21.15.100:0/1779009548 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00f8c0 con 0x55a5a9de22b0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.428+0000 7f6fc9ffb640 1 -- 172.21.15.100:0/1779009548 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc01bdb0 con 0x55a5a9de22b0 2024-03-12T05:17:51.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.428+0000 7f6fc9ffb640 1 -- 172.21.15.100:0/1779009548 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc026d90 con 0x55a5a9de22b0 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.428+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fcc0a1bd0 0x7f6fcc0a4090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc023000 tx=0x7f6fdc024000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- 172.21.15.100:0/1779009548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 msgr2=0x55a5a9de2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9de2690 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0116b0 tx=0x7f6fdc011b70 comp rx=0 tx=0).stop 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- 172.21.15.100:0/1779009548 shutdown_connections 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9dedba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- 172.21.15.100:0/1779009548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9de2690 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- 172.21.15.100:0/1779009548 >> 172.21.15.100:0/1779009548 conn(0x55a5a9ddc990 msgr2=0x55a5a9ddcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- 172.21.15.100:0/1779009548 shutdown_connections 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- 172.21.15.100:0/1779009548 wait complete. 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- start start 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 Processor -- start 2024-03-12T05:17:51.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9deb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9de85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9a1d430 con 0x55a5a9de2bd0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a99d9e00 con 0x55a5a9de22b0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.429+0000 7f6fec993740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9ccefd0 con 0x55a5a9deb7b0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9de85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fec993740 10 client.0 _ll_register_callbacks cb 0x55a5a9d24fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9de85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60000/0 (socket says 172.21.15.100:60000) 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/2466443402 learned_addr learned my addr 172.21.15.100:0/2466443402 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9deb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/2466443402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 msgr2=0x55a5a9de8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 msgr2=0x55a5a9deb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9deb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/2466443402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc009df0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9deb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9de85d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009900 tx=0x7f6fd802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31821fd3e5e7655d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/2466443402 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80042d0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/2466443402 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6fd8004430 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5a9a497e0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/2466443402 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd8004df0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fec993740 10 client.16668 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.430+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a5a9d48610 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/2466443402 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a5a9a497e0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/2466443402 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fd803c040 con 0x55a5a9de2bd0 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fec993740 10 client.16668 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fec993740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fec993740 20 client.16668 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fec993740 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9dfb970 0x55a5a9dfdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fec993740 4 client.16668 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5a9a86720 con 0x55a5a9dfb970 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9dfb970 0x55a5a9dfdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fc2ffd640 10 client.16668 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.431+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9dfb970 0x55a5a9dfdd90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e696c123cc6cbd6c server_cookie=1271de1b19925960 in_seq=0 out_seq=0 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/2466443402 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5a9a86720 con 0x55a5a9dfb970 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fc2ffd640 10 client.16668 handle_command_reply: tid=0 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 client.16668 shutdown 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 msgr2=0x55a5a9de85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9de85d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009900 tx=0x7f6fd802c5d0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 shutdown_connections 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9dfb970 0x55a5a9dfdd90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9de85d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fc2ffd640 0 client.16668 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 --2- 172.21.15.100:0/2466443402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9deb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 >> 172.21.15.100:0/2466443402 conn(0x55a5a9ddc990 msgr2=0x55a5a9dde730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 shutdown_connections 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 1 -- 172.21.15.100:0/2466443402 wait complete. 2024-03-12T05:17:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.432+0000 7f6fec993740 20 client.16668 trim_cache size 0 max 16384 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 Processor -- start 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 -- start start 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9dead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9de15d0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9a86720 con 0x55a5a9de22b0 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.447+0000 7f6fec993740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9a497e0 con 0x55a5a9deb7b0 2024-03-12T05:17:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60004/0 (socket says 172.21.15.100:60004) 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/1955051790 learned_addr learned my addr 172.21.15.100:0/1955051790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9dead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/1955051790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 msgr2=0x55a5a9de7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/1955051790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 msgr2=0x55a5a9dead00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9dead00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 -- 172.21.15.100:0/1955051790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a9ce4ac0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb240 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8003bd0 tx=0x7f6fd802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2995e105b7b0ac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fc97fa640 1 -- 172.21.15.100:0/1955051790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd802c640 con 0x55a5a9de2bd0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fc97fa640 1 -- 172.21.15.100:0/1955051790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd802cbf0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.448+0000 7f6fc97fa640 1 -- 172.21.15.100:0/1955051790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8005460 con 0x55a5a9de2bd0 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 -- 172.21.15.100:0/1955051790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 msgr2=0x55a5a9deb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb240 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8003bd0 tx=0x7f6fd802bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 -- 172.21.15.100:0/1955051790 shutdown_connections 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9de7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de2bd0 0x55a5a9deb240 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 --2- 172.21.15.100:0/1955051790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de22b0 0x55a5a9dead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 -- 172.21.15.100:0/1955051790 >> 172.21.15.100:0/1955051790 conn(0x55a5a9ddc990 msgr2=0x55a5a9de6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 -- 172.21.15.100:0/1955051790 shutdown_connections 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 -- 172.21.15.100:0/1955051790 wait complete. 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 -- start start 2024-03-12T05:17:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.449+0000 7f6fec993740 1 Processor -- start 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9def420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9def960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9e51e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9df0130 con 0x55a5a9de22b0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9df02a0 con 0x55a5a9de2bd0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9e52400 con 0x55a5a9deb7b0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 10 client.0 _ll_register_callbacks cb 0x7f6fdc009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9def420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9def420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60020/0 (socket says 172.21.15.100:60020) 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 -- 172.21.15.100:0/241488684 learned_addr learned my addr 172.21.15.100:0/241488684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9def960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe3fff640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9e51e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 -- 172.21.15.100:0/241488684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 msgr2=0x55a5a9e51e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9e51e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 -- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 msgr2=0x55a5a9def960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9def960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 -- 172.21.15.100:0/241488684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a9d48610 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fe37fe640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9def420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40971c0 tx=0x7f6fd4096bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf21e517380a629a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/241488684 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4005670 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/241488684 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6fd409a7f0 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fdc01bdb0 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/241488684 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd409e940 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.450+0000 7f6fec993740 10 client.16674 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a5a9d0a6a0 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/241488684 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc01bdb0 con 0x55a5a9de22b0 2024-03-12T05:17:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/241488684 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fd401f060 con 0x55a5a9de22b0 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 10 client.16674 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 20 client.16674 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9e53c10 0x55a5a9e56000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 4 client.16674 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a5a9ad5890 con 0x55a5a9e53c10 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9e53c10 0x55a5a9e56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fc2ffd640 10 client.16674 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.451+0000 7f6fe2ffd640 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9e53c10 0x55a5a9e56000 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ad7cdb130c8112 server_cookie=47581d95b69d5d7e in_seq=0 out_seq=0 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fc2ffd640 1 -- 172.21.15.100:0/241488684 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a5a9ad5890 con 0x55a5a9e53c10 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fc2ffd640 10 client.16674 handle_command_reply: tid=0 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 client.16674 shutdown 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 msgr2=0x55a5a9def420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9def420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40971c0 tx=0x7f6fd4096bc0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 shutdown_connections 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a5a9e53c10 0x55a5a9e56000 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5a9deb7b0 0x55a5a9e51e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5a9de2bd0 0x55a5a9def960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fc2ffd640 0 client.16674 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 --2- 172.21.15.100:0/241488684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5a9de22b0 0x55a5a9def420 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 >> 172.21.15.100:0/241488684 conn(0x55a5a9ddc990 msgr2=0x55a5a9de6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 shutdown_connections 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 1 -- 172.21.15.100:0/241488684 wait complete. 2024-03-12T05:17:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fec993740 20 client.16674 trim_cache size 0 max 16384 2024-03-12T05:17:51.455 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:17:51.455 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:17:51.455 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:17:51.455 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fcc0a1bd0 msgr2=0x7f6fcc0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fcc0a1bd0 0x7f6fcc0a4090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc023000 tx=0x7f6fdc024000 comp rx=0 tx=0).stop 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 msgr2=0x7f6fe4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.452+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 0x7f6fe4073480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400e9f0 tx=0x7f6fd400edf0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 shutdown_connections 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fcc0a1bd0 0x7f6fcc0a4090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe4073f00 0x7f6fe41a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe4106490 0x7f6fe40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 --2- 172.21.15.100:0/3196419851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe41013c0 0x7f6fe4073480 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 >> 172.21.15.100:0/3196419851 conn(0x7f6fe40fcc20 msgr2=0x7f6fe40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 shutdown_connections 2024-03-12T05:17:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.453+0000 7f6fe9dbc640 1 -- 172.21.15.100:0/3196419851 wait complete. 2024-03-12T05:17:51.459 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:17:51.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:17:51.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- 172.21.15.100:0/867030588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4010b7c0 msgr2=0x7f0d4010dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/867030588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4010b7c0 0x7f0d4010dc60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009ca0 tx=0x7f0d3802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- 172.21.15.100:0/867030588 shutdown_connections 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/867030588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4010b7c0 0x7f0d4010dc60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/867030588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d40102d10 0x7f0d4010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/867030588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d401023f0 0x7f0d401027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- 172.21.15.100:0/867030588 >> 172.21.15.100:0/867030588 conn(0x7f0d40077180 msgr2=0x7f0d40077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- 172.21.15.100:0/867030588 shutdown_connections 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- 172.21.15.100:0/867030588 wait complete. 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 Processor -- start 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- start start 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 0x7f0d4019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d40102d10 0x7f0d4019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d4010b7c0 0x7f0d40197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40197c70 con 0x7f0d401023f0 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40197de0 con 0x7f0d40102d10 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d46bd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40197fa0 con 0x7f0d4010b7c0 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.613+0000 7f0d4496c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 0x7f0d4019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 0x7f0d4019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60048/0 (socket says 172.21.15.100:60048) 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 -- 172.21.15.100:0/118270955 learned_addr learned my addr 172.21.15.100:0/118270955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d3ffff640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d40102d10 0x7f0d4019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4516d640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d4010b7c0 0x7f0d40197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 -- 172.21.15.100:0/118270955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d4010b7c0 msgr2=0x7f0d40197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d4010b7c0 0x7f0d40197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 -- 172.21.15.100:0/118270955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d40102d10 msgr2=0x7f0d4019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d40102d10 0x7f0d4019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 -- 172.21.15.100:0/118270955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d38009900 con 0x7f0d401023f0 2024-03-12T05:17:51.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d4496c640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 0x7f0d4019d420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e9f0 tx=0x7f0d3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34009720 con 0x7f0d401023f0 2024-03-12T05:17:51.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d3400cd80 con 0x7f0d401023f0 2024-03-12T05:17:51.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d40198280 con 0x7f0d401023f0 2024-03-12T05:17:51.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34010690 con 0x7f0d401023f0 2024-03-12T05:17:51.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.614+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d401a44c0 con 0x7f0d401023f0 2024-03-12T05:17:51.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.615+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d40067f70 con 0x7f0d401023f0 2024-03-12T05:17:51.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.616+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0d340046c0 con 0x7f0d401023f0 2024-03-12T05:17:51.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.617+0000 7f0d3dffb640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d300a1ba0 0x7f0d300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.617+0000 7f0d3ffff640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d300a1ba0 0x7f0d300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.617+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0d34014070 con 0x7f0d401023f0 2024-03-12T05:17:51.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.617+0000 7f0d3ffff640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d300a1ba0 0x7f0d300a4060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d400ffa20 tx=0x7f0d2c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.621+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d34096a40 con 0x7f0d401023f0 2024-03-12T05:17:51.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 661227 12.39 MB/sec execute 155 sec latency 16.795 ms 2024-03-12T05:17:51.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.850+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d40198b00 con 0x7f0d401023f0 2024-03-12T05:17:51.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.851+0000 7f0d3dffb640 1 -- 172.21.15.100:0/118270955 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f0d3409a100 con 0x7f0d401023f0 2024-03-12T05:17:51.847 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:17:51.847 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:17:51.847 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d300a1ba0 msgr2=0x7f0d300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d300a1ba0 0x7f0d300a4060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d400ffa20 tx=0x7f0d2c005f90 comp rx=0 tx=0).stop 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 msgr2=0x7f0d4019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 0x7f0d4019d420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e9f0 tx=0x7f0d3400edf0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 shutdown_connections 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d300a1ba0 0x7f0d300a4060 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d4010b7c0 0x7f0d40197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d40102d10 0x7f0d4019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 --2- 172.21.15.100:0/118270955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d401023f0 0x7f0d4019d420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 >> 172.21.15.100:0/118270955 conn(0x7f0d40077180 msgr2=0x7f0d400fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 shutdown_connections 2024-03-12T05:17:51.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.853+0000 7f0d46bd0640 1 -- 172.21.15.100:0/118270955 wait complete. 2024-03-12T05:17:51.860 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:17:51.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:17:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.974+0000 7f12ec80f640 1 -- 172.21.15.100:0/2825819171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8101d30 msgr2=0x7f12e8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.974+0000 7f12ec80f640 1 --2- 172.21.15.100:0/2825819171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8101d30 0x7f12e8105d70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009930 tx=0x7f12dc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 -- 172.21.15.100:0/2825819171 shutdown_connections 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 --2- 172.21.15.100:0/2825819171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8106450 0x7f12e810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 --2- 172.21.15.100:0/2825819171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8101d30 0x7f12e8105d70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 --2- 172.21.15.100:0/2825819171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 0x7f12e8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 -- 172.21.15.100:0/2825819171 >> 172.21.15.100:0/2825819171 conn(0x7f12e8078540 msgr2=0x7f12e8078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 -- 172.21.15.100:0/2825819171 shutdown_connections 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 -- 172.21.15.100:0/2825819171 wait complete. 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 Processor -- start 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.975+0000 7f12ec80f640 1 -- start start 2024-03-12T05:17:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 0x7f12e819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 0x7f12e819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8197630 0x7f12e8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8198100 con 0x7f12e8106450 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8198270 con 0x7f12e8101380 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8198430 con 0x7f12e8197630 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 0x7f12e819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 0x7f12e819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 0x7f12e819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60054/0 (socket says 172.21.15.100:60054) 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 -- 172.21.15.100:0/1791595920 learned_addr learned my addr 172.21.15.100:0/1791595920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8197630 0x7f12e8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 -- 172.21.15.100:0/1791595920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8197630 msgr2=0x7f12e8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8197630 0x7f12e8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 -- 172.21.15.100:0/1791595920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 msgr2=0x7f12e819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 0x7f12e819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 -- 172.21.15.100:0/1791595920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc009560 con 0x7f12e8106450 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e659c640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 0x7f12e819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 0x7f12e819dab0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009230 tx=0x7f12dc004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12d77fe640 1 -- 172.21.15.100:0/1791595920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0043d0 con 0x7f12e8106450 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12d77fe640 1 -- 172.21.15.100:0/1791595920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12dc0052d0 con 0x7f12e8106450 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e81a4370 con 0x7f12e8106450 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.976+0000 7f12d77fe640 1 -- 172.21.15.100:0/1791595920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc003da0 con 0x7f12e8106450 2024-03-12T05:17:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.977+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e81a4850 con 0x7f12e8106450 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.979+0000 7f12d77fe640 1 -- 172.21.15.100:0/1791595920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f12dc005440 con 0x7f12e8106450 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.979+0000 7f12ef3dc740 1 Processor -- start 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 -- start start 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12d77fe640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12d00a1ba0 0x7f12d00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac8e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ac3c960 con 0x55f22ac81820 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22abf0310 con 0x55f22ac80f00 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22abed840 con 0x55f22ac8eb60 2024-03-12T05:17:51.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12d00a1ba0 0x7f12d00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac812e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34732/0 (socket says 172.21.15.100:34732) 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 -- 172.21.15.100:0/727417040 learned_addr learned my addr 172.21.15.100:0/727417040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12d77fe640 1 -- 172.21.15.100:0/1791595920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f12dc0ef8e0 con 0x7f12e8106450 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac8e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 -- 172.21.15.100:0/727417040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 msgr2=0x55f22ac90f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac90f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 -- 172.21.15.100:0/727417040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 msgr2=0x55f22ac8e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac8e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e659c640 1 -- 172.21.15.100:0/727417040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22abab060 con 0x55f22ac80f00 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.980+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12d00a1ba0 0x7f12d00a4060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55f22ac80bf0 tx=0x7f12e0005ce0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12e659c640 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac812e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005f40 tx=0x7f12d8011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931c0a15df468ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12d4ff9640 1 -- 172.21.15.100:0/727417040 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8011310 con 0x55f22ac80f00 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12d4ff9640 1 -- 172.21.15.100:0/727417040 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12d800fe80 con 0x55f22ac80f00 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12d4ff9640 1 -- 172.21.15.100:0/727417040 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80193e0 con 0x55f22ac80f00 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 -- 172.21.15.100:0/727417040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 msgr2=0x55f22ac812e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac812e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005f40 tx=0x7f12d8011c70 comp rx=0 tx=0).stop 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 -- 172.21.15.100:0/727417040 shutdown_connections 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac90f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac8e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/727417040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac812e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.981+0000 7f12ef3dc740 1 -- 172.21.15.100:0/727417040 >> 172.21.15.100:0/727417040 conn(0x55f22ac7b980 msgr2=0x55f22ac7bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 -- 172.21.15.100:0/727417040 shutdown_connections 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 -- 172.21.15.100:0/727417040 wait complete. 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 -- start start 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 Processor -- start 2024-03-12T05:17:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac80f00 0x55f22ac8ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac81820 0x55f22ac8bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ac8c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22abe9020 con 0x55f22ac8eb60 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22a8bc430 con 0x55f22ac81820 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22a878e00 con 0x55f22ac80f00 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12ef3dc740 10 client.0 _ll_register_callbacks cb 0x55f22ab90cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12e6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ac8c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12e6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ac8c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60074/0 (socket says 172.21.15.100:60074) 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12e6d9d640 1 -- 172.21.15.100:0/790589088 learned_addr learned my addr 172.21.15.100:0/790589088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac81820 0x55f22ac8bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.982+0000 7f12e659c640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac80f00 0x55f22ac8ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12e6d9d640 1 -- 172.21.15.100:0/790589088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac80f00 msgr2=0x55f22ac8ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac80f00 0x55f22ac8ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12e6d9d640 1 -- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac81820 msgr2=0x55f22ac8bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac81820 0x55f22ac8bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12e6d9d640 1 -- 172.21.15.100:0/790589088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d8011500 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ac8c500 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f12e000ce00 tx=0x7f12e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ebecb9c4591a26 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12cdffb640 1 -- 172.21.15.100:0/790589088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0020620 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12cdffb640 1 -- 172.21.15.100:0/790589088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f12e0020780 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f22a8e87e0 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12cdffb640 1 -- 172.21.15.100:0/790589088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e0020940 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12ef3dc740 10 client.16692 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f22abe76a0 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12cdffb640 1 -- 172.21.15.100:0/790589088 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f22a8e87e0 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.983+0000 7f12cdffb640 1 -- 172.21.15.100:0/790589088 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12e001e040 con 0x55f22ac8eb60 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12ef3dc740 10 client.16692 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12ef3dc740 10 client.16692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:17:51.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12ef3dc740 20 client.16692 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22ac9ef60 0x55f22aca1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12ef3dc740 4 client.16692 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f22a925720 con 0x55f22ac9ef60 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12e659c640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22ac9ef60 0x55f22aca1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12cdffb640 10 client.16692 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.984+0000 7f12e659c640 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22ac9ef60 0x55f22aca1350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbee01e83dc9a6bb server_cookie=73b8e2b60f76698e in_seq=0 out_seq=0 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12cdffb640 1 -- 172.21.15.100:0/790589088 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f22a925720 con 0x55f22ac9ef60 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12cdffb640 10 client.16692 handle_command_reply: tid=0 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 client.16692 shutdown 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 msgr2=0x55f22ac8c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ac8c500 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f12e000ce00 tx=0x7f12e00046c0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 shutdown_connections 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22ac9ef60 0x55f22aca1350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ac8c500 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac81820 0x55f22ac8bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12cdffb640 0 client.16692 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/790589088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac80f00 0x55f22ac8ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 >> 172.21.15.100:0/790589088 conn(0x55f22ac7b980 msgr2=0x55f22ac7d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 shutdown_connections 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 1 -- 172.21.15.100:0/790589088 wait complete. 2024-03-12T05:17:51.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.985+0000 7f12ef3dc740 20 client.16692 trim_cache size 0 max 16384 2024-03-12T05:17:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 Processor -- start 2024-03-12T05:17:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 -- start start 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac95ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac96010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ab6e130 con 0x55f22ac81820 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ac805a0 con 0x55f22ac80f00 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.997+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ac775c0 con 0x55f22ac8eb60 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac95ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac95ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60084/0 (socket says 172.21.15.100:60084) 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e5d9b640 1 -- 172.21.15.100:0/3322288660 learned_addr learned my addr 172.21.15.100:0/3322288660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac96010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 -- 172.21.15.100:0/3322288660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 msgr2=0x55f22ac96010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac96010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 -- 172.21.15.100:0/3322288660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 msgr2=0x55f22ac95ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac95ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 -- 172.21.15.100:0/3322288660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22ab83c20 con 0x55f22ac80f00 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac95ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12e659c640 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac7aa00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12d8000f80 tx=0x7f12d8010a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ca4f455f0ef4bc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12cffff640 1 -- 172.21.15.100:0/3322288660 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8010fb0 con 0x55f22ac80f00 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12cffff640 1 -- 172.21.15.100:0/3322288660 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12d8011390 con 0x55f22ac80f00 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.998+0000 7f12cffff640 1 -- 172.21.15.100:0/3322288660 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8010720 con 0x55f22ac80f00 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 -- 172.21.15.100:0/3322288660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 msgr2=0x55f22ac7aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac7aa00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12d8000f80 tx=0x7f12d8010a80 comp rx=0 tx=0).stop 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 -- 172.21.15.100:0/3322288660 shutdown_connections 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac8eb60 0x55f22ac96010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac81820 0x55f22ac95ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/3322288660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22ac7aa00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:51.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 -- 172.21.15.100:0/3322288660 >> 172.21.15.100:0/3322288660 conn(0x55f22ac7b980 msgr2=0x55f22ac7a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 -- 172.21.15.100:0/3322288660 shutdown_connections 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 -- 172.21.15.100:0/3322288660 wait complete. 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 -- start start 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:51.999+0000 7f12ef3dc740 1 Processor -- start 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22aceed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac81820 0x55f22acef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ace93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ace99d0 con 0x55f22ac8eb60 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ace9b40 con 0x55f22ac80f00 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22ace9d00 con 0x55f22ac81820 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 10 client.0 _ll_register_callbacks cb 0x7f12d8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ace93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22aceed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22aceed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34756/0 (socket says 172.21.15.100:34756) 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ace93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60086/0 (socket says 172.21.15.100:60086) 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e659c640 1 -- 172.21.15.100:0/589527311 learned_addr learned my addr 172.21.15.100:0/589527311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac81820 0x55f22acef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 -- 172.21.15.100:0/589527311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac81820 msgr2=0x55f22acef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac81820 0x55f22acef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 -- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 msgr2=0x55f22aceed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22aceed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 -- 172.21.15.100:0/589527311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e81a4850 con 0x55f22ac8eb60 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e659c640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22aceed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e5d9b640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac81820 0x55f22acef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12e6d9d640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ace93c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f12e0010ac0 tx=0x7f12e0010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c5e9825db65ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:52.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12cdffb640 1 -- 172.21.15.100:0/589527311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0010800 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12cdffb640 1 -- 172.21.15.100:0/589527311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f12e0013510 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.000+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f22ac770f0 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12cdffb640 1 -- 172.21.15.100:0/589527311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e000fce0 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 10 client.16698 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f12d801e040 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12cdffb640 1 -- 172.21.15.100:0/589527311 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12e0048080 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12cdffb640 1 -- 172.21.15.100:0/589527311 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12e001e050 con 0x55f22ac8eb60 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 10 client.16698 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 10 client.16698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 20 client.16698 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22acf6c20 0x55f22acf9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 4 client.16698 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:17:52.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f12d800fe80 con 0x55f22acf6c20 2024-03-12T05:17:52.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12e659c640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22acf6c20 0x55f22acf9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12e659c640 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22acf6c20 0x55f22acf9010 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ecc516b14b9fd3b server_cookie=e9605d01457626cf in_seq=0 out_seq=0 2024-03-12T05:17:52.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.001+0000 7f12cdffb640 10 client.16698 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12cdffb640 1 -- 172.21.15.100:0/589527311 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f12d800fe80 con 0x55f22acf6c20 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12cdffb640 10 client.16698 handle_command_reply: tid=0 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 client.16698 shutdown 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 msgr2=0x55f22ace93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ace93c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f12e0010ac0 tx=0x7f12e0010ee0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 shutdown_connections 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55f22acf6c20 0x55f22acf9010 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f22ac8eb60 0x55f22ace93c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f22ac81820 0x55f22acef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12cdffb640 0 client.16698 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 --2- 172.21.15.100:0/589527311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f22ac80f00 0x55f22aceed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.002+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 >> 172.21.15.100:0/589527311 conn(0x55f22ac7b980 msgr2=0x55f22ac7d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 shutdown_connections 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ef3dc740 1 -- 172.21.15.100:0/589527311 wait complete. 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ef3dc740 20 client.16698 trim_cache size 0 max 16384 2024-03-12T05:17:52.005 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:17:52.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12d00a1ba0 msgr2=0x7f12d00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12d00a1ba0 0x7f12d00a4060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55f22ac80bf0 tx=0x7f12e0005ce0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 msgr2=0x7f12e819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 0x7f12e819dab0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009230 tx=0x7f12dc004c00 comp rx=0 tx=0).stop 2024-03-12T05:17:52.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 shutdown_connections 2024-03-12T05:17:52.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12d00a1ba0 0x7f12d00a4060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8197630 0x7f12e8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e8106450 0x7f12e819dab0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 --2- 172.21.15.100:0/1791595920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101380 0x7f12e819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 >> 172.21.15.100:0/1791595920 conn(0x7f12e8078540 msgr2=0x7f12e80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 shutdown_connections 2024-03-12T05:17:52.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.003+0000 7f12ec80f640 1 -- 172.21.15.100:0/1791595920 wait complete. 2024-03-12T05:17:52.009 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:17:52.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:17:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 -- 172.21.15.100:0/2253123778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e30102ad0 msgr2=0x7f3e3010afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/2253123778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e30102ad0 0x7f3e3010afb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009900 tx=0x7f3e2002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 -- 172.21.15.100:0/2253123778 shutdown_connections 2024-03-12T05:17:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/2253123778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e3010b5d0 0x7f3e3010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/2253123778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e30102ad0 0x7f3e3010afb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/2253123778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301021b0 0x7f3e30102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 -- 172.21.15.100:0/2253123778 >> 172.21.15.100:0/2253123778 conn(0x7f3e300fb9d0 msgr2=0x7f3e300fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 -- 172.21.15.100:0/2253123778 shutdown_connections 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.128+0000 7f3e34f9e640 1 -- 172.21.15.100:0/2253123778 wait complete. 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 Processor -- start 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 -- start start 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 0x7f3e30197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e30102ad0 0x7f3e30197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e3010b5d0 0x7f3e3019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3019c510 con 0x7f3e301021b0 2024-03-12T05:17:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3019c680 con 0x7f3e30102ad0 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e34f9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3019c840 con 0x7f3e3010b5d0 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 0x7f3e30197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 0x7f3e30197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60108/0 (socket says 172.21.15.100:60108) 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e30102ad0 0x7f3e30197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e3010b5d0 0x7f3e3019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/1904604727 learned_addr learned my addr 172.21.15.100:0/1904604727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2ffff640 1 -- 172.21.15.100:0/1904604727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e30102ad0 msgr2=0x7f3e30197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e30102ad0 0x7f3e30197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2ffff640 1 -- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 msgr2=0x7f3e30197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 0x7f3e30197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2ffff640 1 -- 172.21.15.100:0/1904604727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20009560 con 0x7f3e3010b5d0 2024-03-12T05:17:52.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.129+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 0x7f3e30197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:52.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.130+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e3010b5d0 0x7f3e3019bf10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009fd0 tx=0x7f3e2800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.130+0000 7f3e2cff9640 1 -- 172.21.15.100:0/1904604727 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800ce20 con 0x7f3e3010b5d0 2024-03-12T05:17:52.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.130+0000 7f3e2cff9640 1 -- 172.21.15.100:0/1904604727 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e28005020 con 0x7f3e3010b5d0 2024-03-12T05:17:52.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.130+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3019cb20 con 0x7f3e3010b5d0 2024-03-12T05:17:52.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.130+0000 7f3e2cff9640 1 -- 172.21.15.100:0/1904604727 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28010690 con 0x7f3e3010b5d0 2024-03-12T05:17:52.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.130+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e301a43a0 con 0x7f3e3010b5d0 2024-03-12T05:17:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.132+0000 7f3e2cff9640 1 -- 172.21.15.100:0/1904604727 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3e28005500 con 0x7f3e3010b5d0 2024-03-12T05:17:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.132+0000 7f3e38dcd740 1 Processor -- start 2024-03-12T05:17:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.132+0000 7f3e38dcd740 1 -- start start 2024-03-12T05:17:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2cff9640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e180a1ba0 0x7f3e180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bc9c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bc57960 con 0x55799bc9c440 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bc0b310 con 0x55799bca5940 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bc08840 con 0x55799bc9cd60 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2effd640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e180a1ba0 0x7f3e180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bc9c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bc9c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60114/0 (socket says 172.21.15.100:60114) 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3371556519 learned_addr learned my addr 172.21.15.100:0/3371556519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2cff9640 1 -- 172.21.15.100:0/1904604727 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3e28014070 con 0x7f3e3010b5d0 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2effd640 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3371556519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 msgr2=0x55799bca5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3371556519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 msgr2=0x55799bca7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3371556519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55799bbc6060 con 0x55799bc9c440 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2effd640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e180a1ba0 0x7f3e180a4060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e20006010 tx=0x7f3e200048a0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bc9c820 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3e24011650 tx=0x7f3e24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbe4c53ce11d0f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e15ffb640 1 -- 172.21.15.100:0/3371556519 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400f6c0 con 0x55799bc9c440 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e15ffb640 1 -- 172.21.15.100:0/3371556519 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e2400fd00 con 0x55799bc9c440 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.133+0000 7f3e15ffb640 1 -- 172.21.15.100:0/3371556519 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24019e80 con 0x55799bc9c440 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3371556519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 msgr2=0x55799bc9c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bc9c820 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3e24011650 tx=0x7f3e24009f90 comp rx=0 tx=0).stop 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3371556519 shutdown_connections 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3371556519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bc9c820 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3371556519 >> 172.21.15.100:0/3371556519 conn(0x55799bc96b70 msgr2=0x55799bc96f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3371556519 shutdown_connections 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3371556519 wait complete. 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.134+0000 7f3e38dcd740 1 -- start start 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 Processor -- start 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bc04020 con 0x55799bc9c440 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799b8d7430 con 0x55799bca5940 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799b893e00 con 0x55799bc9cd60 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e38dcd740 10 client.0 _ll_register_callbacks cb 0x55799bbabcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60116/0 (socket says 172.21.15.100:60116) 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3883333591 learned_addr learned my addr 172.21.15.100:0/3883333591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2effd640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 msgr2=0x55799bca2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3883333591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 msgr2=0x55799bca3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3883333591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e24009e00 con 0x55799bc9c440 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca2990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3e2401ae80 tx=0x7f3e2401aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cda40a5664867d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.135+0000 7f3e0effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e0effd640 1 -- 172.21.15.100:0/3883333591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240197e0 con 0x55799bc9c440 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e0effd640 1 -- 172.21.15.100:0/3883333591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3e24004260 con 0x55799bc9c440 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55799b9037e0 con 0x55799bc9c440 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e0effd640 1 -- 172.21.15.100:0/3883333591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e24004420 con 0x55799bc9c440 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 10 client.16710 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55799bc026a0 con 0x55799bc9c440 2024-03-12T05:17:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e0effd640 1 -- 172.21.15.100:0/3883333591 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55799b9037e0 con 0x55799bc9c440 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e0effd640 1 -- 172.21.15.100:0/3883333591 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e240046c0 con 0x55799bc9c440 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 10 client.16710 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 20 client.16710 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bcb5c90 0x55799bcb80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 4 client.16710 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55799b940720 con 0x55799bcb5c90 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.136+0000 7f3e2effd640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bcb5c90 0x55799bcb80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.137+0000 7f3e0effd640 10 client.16710 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.137+0000 7f3e2effd640 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bcb5c90 0x55799bcb80b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fb6633abf80a4b5 server_cookie=48fc8b11542b3760 in_seq=0 out_seq=0 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.137+0000 7f3e0effd640 1 -- 172.21.15.100:0/3883333591 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55799b940720 con 0x55799bcb5c90 2024-03-12T05:17:52.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e0effd640 10 client.16710 handle_command_reply: tid=0 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 client.16710 shutdown 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 msgr2=0x55799bca2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca2990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3e2401ae80 tx=0x7f3e2401aeb0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 shutdown_connections 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bcb5c90 0x55799bcb80b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e0effd640 0 client.16710 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3883333591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca2990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 >> 172.21.15.100:0/3883333591 conn(0x55799bc96b70 msgr2=0x55799bc988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 shutdown_connections 2024-03-12T05:17:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3883333591 wait complete. 2024-03-12T05:17:52.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.138+0000 7f3e38dcd740 20 client.16710 trim_cache size 0 max 16384 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 Processor -- start 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 -- start start 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bb89130 con 0x55799bc9c440 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bca8300 con 0x55799bca5940 2024-03-12T05:17:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.150+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bc9b840 con 0x55799bc9cd60 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60128/0 (socket says 172.21.15.100:60128) 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3521606630 learned_addr learned my addr 172.21.15.100:0/3521606630 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2effd640 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3521606630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 msgr2=0x55799bca2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3521606630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 msgr2=0x55799bca3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3521606630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55799bb9ec20 con 0x55799bc9c440 2024-03-12T05:17:52.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca50c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3e24004a20 tx=0x7f3e240114d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5c42781eb14b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e157fa640 1 -- 172.21.15.100:0/3521606630 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24004d90 con 0x55799bc9c440 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e157fa640 1 -- 172.21.15.100:0/3521606630 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e2400f3c0 con 0x55799bc9c440 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.151+0000 7f3e157fa640 1 -- 172.21.15.100:0/3521606630 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240044c0 con 0x55799bc9c440 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3521606630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 msgr2=0x55799bca50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca50c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3e24004a20 tx=0x7f3e240114d0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3521606630 shutdown_connections 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bca5940 0x55799bca3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9cd60 0x55799bca2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/3521606630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bc9c440 0x55799bca50c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3521606630 >> 172.21.15.100:0/3521606630 conn(0x55799bc96b70 msgr2=0x55799bca1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3521606630 shutdown_connections 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 -- 172.21.15.100:0/3521606630 wait complete. 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.152+0000 7f3e38dcd740 1 -- start start 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 Processor -- start 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9c440 0x55799bcacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bc9cd60 0x55799bcaaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 0x55799bcab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bcabae0 con 0x55799bca5940 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bcabc50 con 0x55799bc9cd60 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55799bd0c1e0 con 0x55799bc9c440 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e38dcd740 10 client.0 _ll_register_callbacks cb 0x7f3e24009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 0x55799bcab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9c440 0x55799bcacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bc9cd60 0x55799bcaaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 0x55799bcab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60130/0 (socket says 172.21.15.100:60130) 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 -- 172.21.15.100:0/559360630 learned_addr learned my addr 172.21.15.100:0/559360630 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 -- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9c440 msgr2=0x55799bcacb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9c440 0x55799bcacb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 -- 172.21.15.100:0/559360630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bc9cd60 msgr2=0x55799bcaaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bc9cd60 0x55799bcaaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 -- 172.21.15.100:0/559360630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55799bc026a0 con 0x55799bca5940 2024-03-12T05:17:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2effd640 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bc9cd60 0x55799bcaaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e2ffff640 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 0x55799bcab410 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3e28096f60 tx=0x7f3e2809b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b241097ed0fada server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e0effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.153+0000 7f3e0effd640 1 -- 172.21.15.100:0/559360630 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280a3010 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e0effd640 1 -- 172.21.15.100:0/559360630 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3e2809ec50 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e2400fd00 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e0effd640 1 -- 172.21.15.100:0/559360630 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e280a32a0 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 10 client.16716 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55799bbc48c0 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e0effd640 1 -- 172.21.15.100:0/559360630 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e2400fd00 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e0effd640 1 -- 172.21.15.100:0/559360630 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e2801f070 con 0x55799bca5940 2024-03-12T05:17:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 10 client.16716 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 10 client.16716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 20 client.16716 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bd0d930 0x55799bd0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 4 client.16716 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55799b98f890 con 0x55799bd0d930 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.154+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bd0d930 0x55799bd0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e0effd640 10 client.16716 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bd0d930 0x55799bd0fd50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=719e0947366fc04f server_cookie=cd3dcb00f2c1944e in_seq=0 out_seq=0 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e0effd640 1 -- 172.21.15.100:0/559360630 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55799b98f890 con 0x55799bd0d930 2024-03-12T05:17:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e0effd640 10 client.16716 handle_command_reply: tid=0 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 client.16716 shutdown 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 msgr2=0x55799bcab410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 0x55799bcab410 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3e28096f60 tx=0x7f3e2809b2c0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 shutdown_connections 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55799bca5940 0x55799bcab410 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55799bd0d930 0x55799bd0fd50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55799bc9cd60 0x55799bcaaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 --2- 172.21.15.100:0/559360630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55799bc9c440 0x55799bcacb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e0effd640 0 client.16716 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 >> 172.21.15.100:0/559360630 conn(0x55799bc96b70 msgr2=0x55799bc95440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 shutdown_connections 2024-03-12T05:17:52.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 1 -- 172.21.15.100:0/559360630 wait complete. 2024-03-12T05:17:52.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.155+0000 7f3e38dcd740 20 client.16716 trim_cache size 0 max 16384 2024-03-12T05:17:52.160 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:17:52.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e180a1ba0 msgr2=0x7f3e180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e180a1ba0 0x7f3e180a4060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e20006010 tx=0x7f3e200048a0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e3010b5d0 msgr2=0x7f3e3019bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e3010b5d0 0x7f3e3019bf10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009fd0 tx=0x7f3e2800ef10 comp rx=0 tx=0).stop 2024-03-12T05:17:52.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 shutdown_connections 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e180a1ba0 0x7f3e180a4060 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e3010b5d0 0x7f3e3019bf10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e30102ad0 0x7f3e30197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 --2- 172.21.15.100:0/1904604727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e301021b0 0x7f3e30197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 >> 172.21.15.100:0/1904604727 conn(0x7f3e300fb9d0 msgr2=0x7f3e300fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 shutdown_connections 2024-03-12T05:17:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.156+0000 7f3e34f9e640 1 -- 172.21.15.100:0/1904604727 wait complete. 2024-03-12T05:17:52.162 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/118270955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:17:52.163 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:17:52.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:17:52.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/118270955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:17:52.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/118270955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 -- 172.21.15.100:0/4139462729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100c10 msgr2=0x7f7a94104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 --2- 172.21.15.100:0/4139462729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100c10 0x7f7a94104c50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009930 tx=0x7f7a8402bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 -- 172.21.15.100:0/4139462729 shutdown_connections 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 --2- 172.21.15.100:0/4139462729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a94105330 0x7f7a9410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 --2- 172.21.15.100:0/4139462729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100c10 0x7f7a94104c50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 --2- 172.21.15.100:0/4139462729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a94100260 0x7f7a94100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 -- 172.21.15.100:0/4139462729 >> 172.21.15.100:0/4139462729 conn(0x7f7a94077180 msgr2=0x7f7a94077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 -- 172.21.15.100:0/4139462729 shutdown_connections 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.278+0000 7f7a9b664640 1 -- 172.21.15.100:0/4139462729 wait complete. 2024-03-12T05:17:52.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 Processor -- start 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 -- start start 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 0x7f7a94196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a94105330 0x7f7a94196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a94196ea0 0x7f7a941a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941a5fe0 con 0x7f7a94100260 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941a6120 con 0x7f7a94196ea0 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a9b664640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941a6260 con 0x7f7a94105330 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 0x7f7a94196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 0x7f7a94196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60162/0 (socket says 172.21.15.100:60162) 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 -- 172.21.15.100:0/3755343256 learned_addr learned my addr 172.21.15.100:0/3755343256 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a98bff640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a94105330 0x7f7a94196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a94196ea0 0x7f7a941a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 -- 172.21.15.100:0/3755343256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a94105330 msgr2=0x7f7a94196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a94105330 0x7f7a94196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 -- 172.21.15.100:0/3755343256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a94196ea0 msgr2=0x7f7a941a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a94196ea0 0x7f7a941a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 -- 172.21.15.100:0/3755343256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009560 con 0x7f7a94100260 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.279+0000 7f7a99400640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 0x7f7a94196420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000ea20 tx=0x7f7a9000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.280+0000 7f7a8a7fc640 1 -- 172.21.15.100:0/3755343256 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90009700 con 0x7f7a94100260 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.280+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a941a6540 con 0x7f7a94100260 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.280+0000 7f7a8a7fc640 1 -- 172.21.15.100:0/3755343256 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a9000cd80 con 0x7f7a94100260 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.280+0000 7f7a8a7fc640 1 -- 172.21.15.100:0/3755343256 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90018840 con 0x7f7a94100260 2024-03-12T05:17:52.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.280+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941a6a70 con 0x7f7a94100260 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.282+0000 7f7a8a7fc640 1 -- 172.21.15.100:0/3755343256 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7a90010430 con 0x7f7a94100260 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.282+0000 7f7a9e23f740 1 Processor -- start 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.282+0000 7f7a8a7fc640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a800a1ba0 0x7f7a800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.282+0000 7f7a9e23f740 1 -- start start 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a800a1ba0 0x7f7a800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf53630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a8a7fc640 1 -- 172.21.15.100:0/3755343256 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a90014070 con 0x7f7a94100260 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf5c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cf5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cf0e960 con 0x564b9cf53b70 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cec2310 con 0x564b9cf53250 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cebf840 con 0x564b9cf5c750 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a98bff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf53630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf53630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34838/0 (socket says 172.21.15.100:34838) 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 -- 172.21.15.100:0/3318799413 learned_addr learned my addr 172.21.15.100:0/3318799413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cf5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 -- 172.21.15.100:0/3318799413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 msgr2=0x564b9cf5eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cf5eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 -- 172.21.15.100:0/3318799413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 msgr2=0x564b9cf5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf5c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 -- 172.21.15.100:0/3318799413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b9ce7d060 con 0x564b9cf53250 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a98bff640 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a800a1ba0 0x7f7a800a4060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009fd0 tx=0x7f7a8c014b70 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a99400640 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf53630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a900971f0 tx=0x7f7a9009a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8521511e8826626b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a7f7fe640 1 -- 172.21.15.100:0/3318799413 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9009e8c0 con 0x564b9cf53250 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a7f7fe640 1 -- 172.21.15.100:0/3318799413 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a900a2f10 con 0x564b9cf53250 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.283+0000 7f7a7f7fe640 1 -- 172.21.15.100:0/3318799413 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90097020 con 0x564b9cf53250 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3318799413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 msgr2=0x564b9cf53630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf53630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a900971f0 tx=0x7f7a9009a180 comp rx=0 tx=0).stop 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3318799413 shutdown_connections 2024-03-12T05:17:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cf5eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3318799413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf53630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3318799413 >> 172.21.15.100:0/3318799413 conn(0x564b9cf4d980 msgr2=0x564b9cf4dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3318799413 shutdown_connections 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.284+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3318799413 wait complete. 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 -- start start 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 Processor -- start 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf59870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cebb020 con 0x564b9cf5c750 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cb8e430 con 0x564b9cf53250 2024-03-12T05:17:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cb4ae00 con 0x564b9cf53b70 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a9e23f740 10 client.0 _ll_register_callbacks cb 0x564b9ce62cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf5a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60168/0 (socket says 172.21.15.100:60168) 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 -- 172.21.15.100:0/3063325977 learned_addr learned my addr 172.21.15.100:0/3063325977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99400640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf59870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a98bff640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 -- 172.21.15.100:0/3063325977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 msgr2=0x564b9cf59db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf59db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 -- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 msgr2=0x564b9cf59870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf59870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.285+0000 7f7a99c01640 1 -- 172.21.15.100:0/3063325977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941a6a70 con 0x564b9cf5c750 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf5a2f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c005cc0 tx=0x7f7a8c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffe0d6c80517aa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3063325977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c014270 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3063325977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a8c012450 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b9cbba7e0 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3063325977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a8c012610 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 10 client.16728 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b9ceb96a0 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3063325977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564b9cbba7e0 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3063325977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7a8c0299e0 con 0x564b9cf5c750 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 10 client.16728 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 20 client.16728 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cf6cbe0 0x564b9cf6f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 4 client.16728 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b9cbf7720 con 0x564b9cf6cbe0 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.286+0000 7f7a99400640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cf6cbe0 0x564b9cf6f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.287+0000 7f7a99400640 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cf6cbe0 0x564b9cf6f000 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d10c16b79d24b0a server_cookie=9ac7d5ea803d3f98 in_seq=0 out_seq=0 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.287+0000 7f7a7cff9640 10 client.16728 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.287+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3063325977 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b9cbf7720 con 0x564b9cf6cbe0 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.287+0000 7f7a7cff9640 10 client.16728 handle_command_reply: tid=0 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.287+0000 7f7a9e23f740 1 client.16728 shutdown 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 msgr2=0x564b9cf5a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf5a2f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c005cc0 tx=0x7f7a8c004ab0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 shutdown_connections 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cf6cbe0 0x564b9cf6f000 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf5a2f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a7cff9640 0 client.16728 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf59db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3063325977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf59870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 >> 172.21.15.100:0/3063325977 conn(0x564b9cf4d980 msgr2=0x564b9cf4f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 shutdown_connections 2024-03-12T05:17:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3063325977 wait complete. 2024-03-12T05:17:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.288+0000 7f7a9e23f740 20 client.16728 trim_cache size 0 max 16384 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 Processor -- start 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 -- start start 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf4ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf63540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9ce40130 con 0x564b9cf5c750 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cf69690 con 0x564b9cf53250 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.300+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cf52a20 con 0x564b9cf53b70 2024-03-12T05:17:52.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf63540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a98bff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf4ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf5a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34860/0 (socket says 172.21.15.100:34860) 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99400640 1 -- 172.21.15.100:0/1658364362 learned_addr learned my addr 172.21.15.100:0/1658364362 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 -- 172.21.15.100:0/1658364362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 msgr2=0x564b9cf4ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf4ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 -- 172.21.15.100:0/1658364362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 msgr2=0x564b9cf5a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf5a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 -- 172.21.15.100:0/1658364362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b9ce55c20 con 0x564b9cf5c750 2024-03-12T05:17:52.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a99c01640 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf63540 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0098b0 tx=0x7f7a8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9c954705a8332d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a7effd640 1 -- 172.21.15.100:0/1658364362 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c011c40 con 0x564b9cf5c750 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a7effd640 1 -- 172.21.15.100:0/1658364362 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a8c015e90 con 0x564b9cf5c750 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.301+0000 7f7a7effd640 1 -- 172.21.15.100:0/1658364362 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c012450 con 0x564b9cf5c750 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 -- 172.21.15.100:0/1658364362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 msgr2=0x564b9cf63540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf63540 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0098b0 tx=0x7f7a8c009840 comp rx=0 tx=0).stop 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 -- 172.21.15.100:0/1658364362 shutdown_connections 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf5c750 0x564b9cf63540 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf53b70 0x564b9cf4ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/1658364362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 -- 172.21.15.100:0/1658364362 >> 172.21.15.100:0/1658364362 conn(0x564b9cf4d980 msgr2=0x564b9cf4c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 -- 172.21.15.100:0/1658364362 shutdown_connections 2024-03-12T05:17:52.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 -- 172.21.15.100:0/1658364362 wait complete. 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 -- start start 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.302+0000 7f7a9e23f740 1 Processor -- start 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf62820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf62d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cfbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cfbd670 con 0x564b9cf53b70 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cfbd7e0 con 0x564b9cf53250 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9cfbd9a0 con 0x564b9cf5c750 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 10 client.0 _ll_register_callbacks cb 0x7f7a9000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf62d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf62d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60200/0 (socket says 172.21.15.100:60200) 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 -- 172.21.15.100:0/3766927222 learned_addr learned my addr 172.21.15.100:0/3766927222 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a99400640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf62820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a99c01640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cfbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 -- 172.21.15.100:0/3766927222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 msgr2=0x564b9cfbd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cfbd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 -- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 msgr2=0x564b9cf62820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf62820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 -- 172.21.15.100:0/3766927222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b9ceb96a0 con 0x564b9cf53b70 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a98bff640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf62d60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009900 tx=0x7f7a8402cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96df9479859d7dec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:52.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3766927222 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84045070 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3766927222 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a84004310 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.303+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a900a2f10 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3766927222 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a840044d0 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 10 client.16734 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b9ce7b8c0 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3766927222 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a900a2f10 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3766927222 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7a8403c040 con 0x564b9cf53b70 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 10 client.16734 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 20 client.16734 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:52.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cfc52d0 0x564b9cfc76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 4 client.16734 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b9cc46890 con 0x564b9cfc52d0 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a99400640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cfc52d0 0x564b9cfc76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a99400640 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cfc52d0 0x564b9cfc76f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6427edc06235a87a server_cookie=193639e7894e8eee in_seq=0 out_seq=0 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.304+0000 7f7a7cff9640 10 client.16734 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a7cff9640 1 -- 172.21.15.100:0/3766927222 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b9cc46890 con 0x564b9cfc52d0 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a7cff9640 10 client.16734 handle_command_reply: tid=0 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 client.16734 shutdown 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 msgr2=0x564b9cf62d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf62d60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009900 tx=0x7f7a8402cc90 comp rx=0 tx=0).stop 2024-03-12T05:17:52.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 shutdown_connections 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564b9cfc52d0 0x564b9cfc76f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b9cf5c750 0x564b9cfbd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b9cf53b70 0x564b9cf62d60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 --2- 172.21.15.100:0/3766927222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b9cf53250 0x564b9cf62820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 >> 172.21.15.100:0/3766927222 conn(0x564b9cf4d980 msgr2=0x564b9cf4f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a7cff9640 0 client.16734 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 shutdown_connections 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 1 -- 172.21.15.100:0/3766927222 wait complete. 2024-03-12T05:17:52.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9e23f740 20 client.16734 trim_cache size 0 max 16384 2024-03-12T05:17:52.314 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:17:52.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a800a1ba0 msgr2=0x7f7a800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a800a1ba0 0x7f7a800a4060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009fd0 tx=0x7f7a8c014b70 comp rx=0 tx=0).stop 2024-03-12T05:17:52.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 msgr2=0x7f7a94196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:52.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 0x7f7a94196420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000ea20 tx=0x7f7a9000eee0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 shutdown_connections 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7a800a1ba0 0x7f7a800a4060 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a94196ea0 0x7f7a941a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a94105330 0x7f7a94196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 --2- 172.21.15.100:0/3755343256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a94100260 0x7f7a94196420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.305+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 >> 172.21.15.100:0/3755343256 conn(0x7f7a94077180 msgr2=0x7f7a940fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.306+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 shutdown_connections 2024-03-12T05:17:52.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:52.306+0000 7f7a9b664640 1 -- 172.21.15.100:0/3755343256 wait complete. 2024-03-12T05:17:52.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 664020 12.42 MB/sec execute 156 sec latency 13.534 ms 2024-03-12T05:17:53.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:52 smithi156 ceph-mon[36270]: pgmap v176: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-03-12T05:17:53.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:52 smithi161 ceph-mon[36043]: pgmap v176: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-03-12T05:17:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:52 smithi100 ceph-mon[32210]: pgmap v176: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-03-12T05:17:53.317 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:17:53.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:17:53.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:17:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 -- 172.21.15.100:0/2437779691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7898068f80 msgr2=0x7f7898105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 --2- 172.21.15.100:0/2437779691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7898068f80 0x7f7898105d50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7880009930 tx=0x7f788002bc30 comp rx=0 tx=0).stop 2024-03-12T05:17:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 -- 172.21.15.100:0/2437779691 shutdown_connections 2024-03-12T05:17:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 --2- 172.21.15.100:0/2437779691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7898106430 0x7f789810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 --2- 172.21.15.100:0/2437779691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7898068f80 0x7f7898105d50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 --2- 172.21.15.100:0/2437779691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78980685d0 0x7f78980689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 -- 172.21.15.100:0/2437779691 >> 172.21.15.100:0/2437779691 conn(0x7f7898077180 msgr2=0x7f7898077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 -- 172.21.15.100:0/2437779691 shutdown_connections 2024-03-12T05:17:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.432+0000 7f789d8f2640 1 -- 172.21.15.100:0/2437779691 wait complete. 2024-03-12T05:17:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 Processor -- start 2024-03-12T05:17:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 -- start start 2024-03-12T05:17:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 0x7f7898100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7898106430 0x7f7898100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7898100ac0 0x7f78981a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78981a5f70 con 0x7f78980685d0 2024-03-12T05:17:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78981a60e0 con 0x7f7898100ac0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f789d8f2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78981a62a0 con 0x7f7898106430 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 0x7f7898100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 0x7f7898100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60226/0 (socket says 172.21.15.100:60226) 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 -- 172.21.15.100:0/649635975 learned_addr learned my addr 172.21.15.100:0/649635975 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f78977fe640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7898100ac0 0x7f78981a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f78967fc640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7898106430 0x7f7898100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 -- 172.21.15.100:0/649635975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7898106430 msgr2=0x7f7898100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7898106430 0x7f7898100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 -- 172.21.15.100:0/649635975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7898100ac0 msgr2=0x7f78981a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7898100ac0 0x7f78981a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.433+0000 7f7896ffd640 1 -- 172.21.15.100:0/649635975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009560 con 0x7f78980685d0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.434+0000 7f7896ffd640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 0x7f7898100040 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f789000ebc0 tx=0x7f789000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.434+0000 7f789c8f0640 1 -- 172.21.15.100:0/649635975 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900099b0 con 0x7f78980685d0 2024-03-12T05:17:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.434+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78981a6580 con 0x7f78980685d0 2024-03-12T05:17:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.434+0000 7f789c8f0640 1 -- 172.21.15.100:0/649635975 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f789000eec0 con 0x7f78980685d0 2024-03-12T05:17:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.434+0000 7f789c8f0640 1 -- 172.21.15.100:0/649635975 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890018b60 con 0x7f78980685d0 2024-03-12T05:17:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.434+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78981a6a60 con 0x7f78980685d0 2024-03-12T05:17:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.436+0000 7f789c8f0640 1 -- 172.21.15.100:0/649635975 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7890018cc0 con 0x7f78980685d0 2024-03-12T05:17:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.436+0000 7f78a059d740 1 Processor -- start 2024-03-12T05:17:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.436+0000 7f78a059d740 1 -- start start 2024-03-12T05:17:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.436+0000 7f789c8f0640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78840a1bd0 0x7f78840a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78a059d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01ba5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78a059d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78977fe640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78840a1bd0 0x7f78840a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78a059d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba648b0 0x55f01ba66d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78a059d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01b9c9f90 con 0x55f01ba5b380 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78a059d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01b9c74c0 con 0x55f01ba5bd30 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78a059d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01b9c2ca0 con 0x55f01ba648b0 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f789c8f0640 1 -- 172.21.15.100:0/649635975 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7890014070 con 0x7f78980685d0 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01ba5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78967fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78977fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba648b0 0x55f01ba66d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01ba5b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60238/0 (socket says 172.21.15.100:60238) 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78967fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34912/0 (socket says 172.21.15.100:34912) 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 -- 172.21.15.100:0/2551680619 learned_addr learned my addr 172.21.15.100:0/2551680619 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 -- 172.21.15.100:0/2551680619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba648b0 msgr2=0x55f01ba66d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba648b0 0x55f01ba66d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 -- 172.21.15.100:0/2551680619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 msgr2=0x55f01ba642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 -- 172.21.15.100:0/2551680619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f01b9850b0 con 0x55f01ba5b380 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f78977fe640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78840a1bd0 0x7f78840a4090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7888005f60 tx=0x7f7888013830 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01ba5b760 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f789009aa90 tx=0x7f789009a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e401b5d5488e4a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f788dffb640 1 -- 172.21.15.100:0/2551680619 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789009ab00 con 0x55f01ba5b380 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.437+0000 7f788dffb640 1 -- 172.21.15.100:0/2551680619 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f789009ec50 con 0x55f01ba5b380 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f788dffb640 1 -- 172.21.15.100:0/2551680619 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890099f50 con 0x55f01ba5b380 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 -- 172.21.15.100:0/2551680619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 msgr2=0x55f01ba5b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01ba5b760 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f789009aa90 tx=0x7f789009a460 comp rx=0 tx=0).stop 2024-03-12T05:17:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 -- 172.21.15.100:0/2551680619 shutdown_connections 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba648b0 0x55f01ba66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 --2- 172.21.15.100:0/2551680619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01ba5b760 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 -- 172.21.15.100:0/2551680619 >> 172.21.15.100:0/2551680619 conn(0x55f01ba55840 msgr2=0x55f01ba55c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 -- 172.21.15.100:0/2551680619 shutdown_connections 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.438+0000 7f78a059d740 1 -- 172.21.15.100:0/2551680619 wait complete. 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 -- start start 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 Processor -- start 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba61d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 0x55f01ba62280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01b967f40 con 0x55f01ba648b0 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01b6964b0 con 0x55f01ba5bd30 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01b948670 con 0x55f01ba5b380 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78a059d740 10 client.0 _ll_register_callbacks cb 0x55f01b9b5f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 0x55f01ba62280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 0x55f01ba62280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60250/0 (socket says 172.21.15.100:60250) 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 -- 172.21.15.100:0/2753856377 learned_addr learned my addr 172.21.15.100:0/2753856377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78967fc640 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba61d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 -- 172.21.15.100:0/2753856377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 msgr2=0x55f01ba61800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba61800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 -- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 msgr2=0x55f01ba61d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba61d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.439+0000 7f78977fe640 1 -- 172.21.15.100:0/2753856377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78981a6a60 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f78977fe640 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 0x55f01ba62280 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7888003820 tx=0x7f7888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653c53de225c8842 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f7876ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f7876ffd640 1 -- 172.21.15.100:0/2753856377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888018040 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f7876ffd640 1 -- 172.21.15.100:0/2753856377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7888003ee0 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f01b99cf40 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f7876ffd640 1 -- 172.21.15.100:0/2753856377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f788800fa40 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f78a059d740 10 client.16746 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f01b9c1320 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.440+0000 7f7876ffd640 1 -- 172.21.15.100:0/2753856377 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f01b99cf40 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f7876ffd640 1 -- 172.21.15.100:0/2753856377 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78880048c0 con 0x55f01ba648b0 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f78a059d740 10 client.16746 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f78a059d740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f78a059d740 20 client.16746 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f78a059d740 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01ba74d70 0x55f01ba77190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f78a059d740 4 client.16746 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f01b9b9310 con 0x55f01ba74d70 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01ba74d70 0x55f01ba77190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f7876ffd640 10 client.16746 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.441+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01ba74d70 0x55f01ba77190 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713db94451a757b4 server_cookie=c5c3cea25d59fc2c in_seq=0 out_seq=0 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f7876ffd640 1 -- 172.21.15.100:0/2753856377 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f01b9b9310 con 0x55f01ba74d70 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f7876ffd640 10 client.16746 handle_command_reply: tid=0 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 client.16746 shutdown 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 msgr2=0x55f01ba62280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 0x55f01ba62280 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7888003820 tx=0x7f7888004730 comp rx=0 tx=0).stop 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 shutdown_connections 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01ba74d70 0x55f01ba77190 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba648b0 0x55f01ba62280 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba5bd30 0x55f01ba61d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 --2- 172.21.15.100:0/2753856377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f7876ffd640 0 client.16746 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 >> 172.21.15.100:0/2753856377 conn(0x55f01ba55840 msgr2=0x55f01ba577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 shutdown_connections 2024-03-12T05:17:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 1 -- 172.21.15.100:0/2753856377 wait complete. 2024-03-12T05:17:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.442+0000 7f78a059d740 20 client.16746 trim_cache size 0 max 16384 2024-03-12T05:17:53.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 Processor -- start 2024-03-12T05:17:53.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 -- start start 2024-03-12T05:17:53.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 0x55f01ba57590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01ba5a410 con 0x55f01ba5bd30 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01ba5af10 con 0x55f01ba648b0 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78a059d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01ba4fcf0 con 0x55f01ba5b380 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 0x55f01ba57590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 0x55f01ba57590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60254/0 (socket says 172.21.15.100:60254) 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 -- 172.21.15.100:0/3196099827 learned_addr learned my addr 172.21.15.100:0/3196099827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78977fe640 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f7896ffd640 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 -- 172.21.15.100:0/3196099827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 msgr2=0x55f01ba620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 -- 172.21.15.100:0/3196099827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 msgr2=0x55f01ba6b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 -- 172.21.15.100:0/3196099827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f01b95dc00 con 0x55f01ba5bd30 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.459+0000 7f78967fc640 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 0x55f01ba57590 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f788002c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7592f848b18ae0f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f788d7fa640 1 -- 172.21.15.100:0/3196099827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880004880 con 0x55f01ba5bd30 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f788d7fa640 1 -- 172.21.15.100:0/3196099827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78800049e0 con 0x55f01ba5bd30 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f788d7fa640 1 -- 172.21.15.100:0/3196099827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880005170 con 0x55f01ba5bd30 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 -- 172.21.15.100:0/3196099827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 msgr2=0x55f01ba57590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 0x55f01ba57590 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f788002c2b0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 -- 172.21.15.100:0/3196099827 shutdown_connections 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5bd30 0x55f01ba57590 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 --2- 172.21.15.100:0/3196099827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5b380 0x55f01ba620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 -- 172.21.15.100:0/3196099827 >> 172.21.15.100:0/3196099827 conn(0x55f01ba55840 msgr2=0x55f01ba56e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 -- 172.21.15.100:0/3196099827 shutdown_connections 2024-03-12T05:17:53.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.460+0000 7f78a059d740 1 -- 172.21.15.100:0/3196099827 wait complete. 2024-03-12T05:17:53.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 -- start start 2024-03-12T05:17:53.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 Processor -- start 2024-03-12T05:17:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01bac4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5bd30 0x55f01bac4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01ba6b540 con 0x55f01ba5b380 2024-03-12T05:17:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01ba6b6b0 con 0x55f01ba648b0 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01ba6b870 con 0x55f01ba5bd30 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78a059d740 10 client.0 _ll_register_callbacks cb 0x7f7888009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01bac4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01bac4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60266/0 (socket says 172.21.15.100:60266) 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f7896ffd640 1 -- 172.21.15.100:0/2702796516 learned_addr learned my addr 172.21.15.100:0/2702796516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78977fe640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.461+0000 7f78967fc640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5bd30 0x55f01bac4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7896ffd640 1 -- 172.21.15.100:0/2702796516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5bd30 msgr2=0x55f01bac4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5bd30 0x55f01bac4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7896ffd640 1 -- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 msgr2=0x55f01ba6ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7896ffd640 1 -- 172.21.15.100:0/2702796516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789810e810 con 0x55f01ba5b380 2024-03-12T05:17:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7896ffd640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01bac4840 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f78900cbc20 tx=0x7f7890018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912fc8ac6b82a7d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7876ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7876ffd640 1 -- 172.21.15.100:0/2702796516 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890010dc0 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7876ffd640 1 -- 172.21.15.100:0/2702796516 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7890083070 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f01b9e8900 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7876ffd640 1 -- 172.21.15.100:0/2702796516 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78900a32a0 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f78a059d740 10 client.16752 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f01b983910 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.462+0000 7f7876ffd640 1 -- 172.21.15.100:0/2702796516 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f01b9e8900 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f7876ffd640 1 -- 172.21.15.100:0/2702796516 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7890079bf0 con 0x55f01ba5b380 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78a059d740 10 client.16752 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78a059d740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78a059d740 20 client.16752 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78a059d740 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01bacd4f0 0x55f01bacf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78a059d740 4 client.16752 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f01b6c2860 con 0x55f01bacd4f0 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78967fc640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01bacd4f0 0x55f01bacf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f7876ffd640 10 client.16752 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.463+0000 7f78967fc640 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01bacd4f0 0x55f01bacf8e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc5b822fc62de03 server_cookie=60a2038a56a64f41 in_seq=0 out_seq=0 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f7876ffd640 1 -- 172.21.15.100:0/2702796516 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f01b6c2860 con 0x55f01bacd4f0 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f7876ffd640 10 client.16752 handle_command_reply: tid=0 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 client.16752 shutdown 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 msgr2=0x55f01bac4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01bac4840 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f78900cbc20 tx=0x7f7890018420 comp rx=0 tx=0).stop 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 shutdown_connections 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f01bacd4f0 0x55f01bacf8e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f01ba648b0 0x55f01ba6ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f01ba5bd30 0x55f01bac4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f7876ffd640 0 client.16752 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 --2- 172.21.15.100:0/2702796516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f01ba5b380 0x55f01bac4840 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.464+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 >> 172.21.15.100:0/2702796516 conn(0x55f01ba55840 msgr2=0x55f01ba57630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 shutdown_connections 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f78a059d740 1 -- 172.21.15.100:0/2702796516 wait complete. 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f78a059d740 20 client.16752 trim_cache size 0 max 16384 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:17:53.467 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "008ecd66-13da-40c9-aa2c-5ee941594698", 2024-03-12T05:17:53.468 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:17:53.468 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78840a1bd0 msgr2=0x7f78840a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78840a1bd0 0x7f78840a4090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7888005f60 tx=0x7f7888013830 comp rx=0 tx=0).stop 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 msgr2=0x7f7898100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 0x7f7898100040 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f789000ebc0 tx=0x7f789000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 shutdown_connections 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f78840a1bd0 0x7f78840a4090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7898100ac0 0x7f78981a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7898106430 0x7f7898100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 --2- 172.21.15.100:0/649635975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78980685d0 0x7f7898100040 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 >> 172.21.15.100:0/649635975 conn(0x7f7898077180 msgr2=0x7f78980fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 shutdown_connections 2024-03-12T05:17:53.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.465+0000 7f789d8f2640 1 -- 172.21.15.100:0/649635975 wait complete. 2024-03-12T05:17:53.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:17:53.472 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 -- 172.21.15.100:0/3698236728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 msgr2=0x7f3b0c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 --2- 172.21.15.100:0/3698236728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c10ce40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ca0 tx=0x7f3b0402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 -- 172.21.15.100:0/3698236728 shutdown_connections 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 --2- 172.21.15.100:0/3698236728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c10ce40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 --2- 172.21.15.100:0/3698236728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c1002d0 0x7f3b0c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 --2- 172.21.15.100:0/3698236728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0c106110 0x7f3b0c1064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 -- 172.21.15.100:0/3698236728 >> 172.21.15.100:0/3698236728 conn(0x7f3b0c076e20 msgr2=0x7f3b0c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 -- 172.21.15.100:0/3698236728 shutdown_connections 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 -- 172.21.15.100:0/3698236728 wait complete. 2024-03-12T05:17:53.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 Processor -- start 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.601+0000 7f3b1196c640 1 -- start start 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0c1002d0 0x7f3b0c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c106110 0x7f3b0c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c199e90 con 0x7f3b0c100c90 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c19a000 con 0x7f3b0c106110 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c19a1c0 con 0x7f3b0c1002d0 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60290/0 (socket says 172.21.15.100:60290) 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 -- 172.21.15.100:0/2891976706 learned_addr learned my addr 172.21.15.100:0/2891976706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c106110 0x7f3b0c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b0affd640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0c1002d0 0x7f3b0c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 -- 172.21.15.100:0/2891976706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0c1002d0 msgr2=0x7f3b0c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0c1002d0 0x7f3b0c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 -- 172.21.15.100:0/2891976706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c106110 msgr2=0x7f3b0c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c106110 0x7f3b0c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 -- 172.21.15.100:0/2891976706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009900 con 0x7f3b0c100c90 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b03fff640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c19fb80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3af400e9f0 tx=0x7f3af400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1096a640 1 -- 172.21.15.100:0/2891976706 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4009700 con 0x7f3b0c100c90 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c19a4a0 con 0x7f3b0c100c90 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1096a640 1 -- 172.21.15.100:0/2891976706 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3af400cd80 con 0x7f3b0c100c90 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1096a640 1 -- 172.21.15.100:0/2891976706 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4018920 con 0x7f3b0c100c90 2024-03-12T05:17:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.602+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1a65c0 con 0x7f3b0c100c90 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b1096a640 1 -- 172.21.15.100:0/2891976706 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3af4018a80 con 0x7f3b0c100c90 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 Processor -- start 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 -- start start 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 0x55ad9a24f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24fbd0 0x55ad9a258270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a1be280 con 0x55ad9a2587b0 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a1bb7b0 con 0x55ad9a24fbd0 2024-03-12T05:17:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b14618740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a1b6f90 con 0x55ad9a24f2b0 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 0x55ad9a24f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24fbd0 0x55ad9a258270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 0x55ad9a24f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51902/0 (socket says 172.21.15.100:51902) 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1351049357 learned_addr learned my addr 172.21.15.100:0/1351049357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60300/0 (socket says 172.21.15.100:60300) 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24fbd0 0x55ad9a258270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34978/0 (socket says 172.21.15.100:34978) 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b1096a640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af80a1ba0 0x7f3af80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af80a1ba0 0x7f3af80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1351049357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24fbd0 msgr2=0x55ad9a258270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24fbd0 0x55ad9a258270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1351049357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 msgr2=0x55ad9a25aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1351049357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9a175570 con 0x55ad9a24f2b0 2024-03-12T05:17:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b1096a640 1 -- 172.21.15.100:0/2891976706 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3af4014070 con 0x7f3b0c100c90 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.605+0000 7f3b03fff640 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.606+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 0x55ad9a24f690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009430 tx=0x7f3b04032610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900be7eabb206895 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.606+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af80a1ba0 0x7f3af80a4060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b04048000 tx=0x7f3b0404b000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.606+0000 7f3b01ffb640 1 -- 172.21.15.100:0/1351049357 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b040326b0 con 0x55ad9a24f2b0 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.606+0000 7f3b01ffb640 1 -- 172.21.15.100:0/1351049357 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b04003e50 con 0x55ad9a24f2b0 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.606+0000 7f3b01ffb640 1 -- 172.21.15.100:0/1351049357 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b040043a0 con 0x55ad9a24f2b0 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- 172.21.15.100:0/1351049357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 msgr2=0x55ad9a24f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 0x55ad9a24f690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009430 tx=0x7f3b04032610 comp rx=0 tx=0).stop 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- 172.21.15.100:0/1351049357 shutdown_connections 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24fbd0 0x55ad9a258270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- 172.21.15.100:0/1351049357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24f2b0 0x55ad9a24f690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- 172.21.15.100:0/1351049357 >> 172.21.15.100:0/1351049357 conn(0x55ad9a249990 msgr2=0x55ad9a249d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- 172.21.15.100:0/1351049357 shutdown_connections 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- 172.21.15.100:0/1351049357 wait complete. 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- start start 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 Processor -- start 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 0x55ad9a255640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a255b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a2587b0 0x55ad9a2560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad99e8a430 con 0x55ad9a24f2b0 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad99e46e00 con 0x55ad9a2587b0 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a13bfd0 con 0x55ad9a24fbd0 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.607+0000 7f3b14618740 10 client.0 _ll_register_callbacks cb 0x55ad9a191fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 0x55ad9a255640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 0x55ad9a255640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60308/0 (socket says 172.21.15.100:60308) 2024-03-12T05:17:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 -- 172.21.15.100:0/351640865 learned_addr learned my addr 172.21.15.100:0/351640865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a2587b0 0x55ad9a2560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b03fff640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a255b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 -- 172.21.15.100:0/351640865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 msgr2=0x55ad9a255b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a255b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 -- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a2587b0 msgr2=0x55ad9a2560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a2587b0 0x55ad9a2560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 -- 172.21.15.100:0/351640865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c10d790 con 0x55ad9a24f2b0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b0affd640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 0x55ad9a255640 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00bf00 tx=0x7f3afc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1298c3cf0c5bdb90 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3aeaffd640 1 -- 172.21.15.100:0/351640865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0108a0 con 0x55ad9a24f2b0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3aeaffd640 1 -- 172.21.15.100:0/351640865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3afc010a00 con 0x55ad9a24f2b0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad99eb67e0 con 0x55ad9a24f2b0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.608+0000 7f3aeaffd640 1 -- 172.21.15.100:0/351640865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3afc010bc0 con 0x55ad9a24f2b0 2024-03-12T05:17:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 10 client.16764 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ad9a1b5610 con 0x55ad9a24f2b0 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3aeaffd640 1 -- 172.21.15.100:0/351640865 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3afc027020 con 0x55ad9a24f2b0 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3aeaffd640 1 -- 172.21.15.100:0/351640865 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3afc00f3c0 con 0x55ad9a24f2b0 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 10 client.16764 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 20 client.16764 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a268a50 0x55ad9a26ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 4 client.16764 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad99ef3720 con 0x55ad9a268a50 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.609+0000 7f3b03fff640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a268a50 0x55ad9a26ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3aeaffd640 10 client.16764 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b03fff640 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a268a50 0x55ad9a26ae70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e3dcff0ff6c0ed server_cookie=29ef9bd2bc0070a9 in_seq=0 out_seq=0 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3aeaffd640 1 -- 172.21.15.100:0/351640865 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad99ef3720 con 0x55ad9a268a50 2024-03-12T05:17:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3aeaffd640 10 client.16764 handle_command_reply: tid=0 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 client.16764 shutdown 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 msgr2=0x55ad9a255640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 0x55ad9a255640 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00bf00 tx=0x7f3afc012820 comp rx=0 tx=0).stop 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 shutdown_connections 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a268a50 0x55ad9a26ae70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a2587b0 0x55ad9a2560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a255b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 --2- 172.21.15.100:0/351640865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24f2b0 0x55ad9a255640 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.610+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 >> 172.21.15.100:0/351640865 conn(0x55ad9a249990 msgr2=0x55ad9a24b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.611+0000 7f3aeaffd640 0 client.16764 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.611+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 shutdown_connections 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.611+0000 7f3b14618740 1 -- 172.21.15.100:0/351640865 wait complete. 2024-03-12T05:17:53.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.611+0000 7f3b14618740 20 client.16764 trim_cache size 0 max 16384 2024-03-12T05:17:53.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 Processor -- start 2024-03-12T05:17:53.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 -- start start 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a257d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a2587b0 0x55ad9a25fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a24a9e0 con 0x55ad9a24fbd0 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a25af20 con 0x55ad9a24f2b0 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b14618740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a24a780 con 0x55ad9a2587b0 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60324/0 (socket says 172.21.15.100:60324) 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 -- 172.21.15.100:0/1254247572 learned_addr learned my addr 172.21.15.100:0/1254247572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b0affd640 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a257d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a2587b0 0x55ad9a25fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 -- 172.21.15.100:0/1254247572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a2587b0 msgr2=0x55ad9a25fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a2587b0 0x55ad9a25fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 -- 172.21.15.100:0/1254247572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 msgr2=0x55ad9a257d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a257d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 -- 172.21.15.100:0/1254247572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9a151ac0 con 0x55ad9a24fbd0 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a2587b0 0x55ad9a25fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.626+0000 7f3b03fff640 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25f5a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3af4096ca0 tx=0x7f3af40cff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2071810bf5d8d466 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b017fa640 1 -- 172.21.15.100:0/1254247572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af409af40 con 0x55ad9a24fbd0 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b017fa640 1 -- 172.21.15.100:0/1254247572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3af409a840 con 0x55ad9a24fbd0 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b017fa640 1 -- 172.21.15.100:0/1254247572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40caba0 con 0x55ad9a24fbd0 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 -- 172.21.15.100:0/1254247572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 msgr2=0x55ad9a25f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25f5a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3af4096ca0 tx=0x7f3af40cff30 comp rx=0 tx=0).stop 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 -- 172.21.15.100:0/1254247572 shutdown_connections 2024-03-12T05:17:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a2587b0 0x55ad9a25fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25f5a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 --2- 172.21.15.100:0/1254247572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a257d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 -- 172.21.15.100:0/1254247572 >> 172.21.15.100:0/1254247572 conn(0x55ad9a249990 msgr2=0x55ad9a255f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 -- 172.21.15.100:0/1254247572 shutdown_connections 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.627+0000 7f3b14618740 1 -- 172.21.15.100:0/1254247572 wait complete. 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 -- start start 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 Processor -- start 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a25dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a25fe80 con 0x55ad9a2587b0 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a2bef80 con 0x55ad9a24f2b0 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9a2bf0c0 con 0x55ad9a24fbd0 2024-03-12T05:17:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b14618740 10 client.0 _ll_register_callbacks cb 0x7f3af4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60328/0 (socket says 172.21.15.100:60328) 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1221063122 learned_addr learned my addr 172.21.15.100:0/1221063122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0affd640 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a25dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1221063122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 msgr2=0x55ad9a25e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 msgr2=0x55ad9a25dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.628+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a25dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b0b7fe640 1 -- 172.21.15.100:0/1221063122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c19a4a0 con 0x55ad9a2587b0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b0b7fe640 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25e960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3b04032c60 tx=0x7f3b04032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7610e382c7fd05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3aeaffd640 1 -- 172.21.15.100:0/1221063122 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04009540 con 0x55ad9a2587b0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3aeaffd640 1 -- 172.21.15.100:0/1221063122 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3b0404a070 con 0x55ad9a2587b0 2024-03-12T05:17:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b04003e50 con 0x55ad9a2587b0 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3aeaffd640 1 -- 172.21.15.100:0/1221063122 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b040040b0 con 0x55ad9a2587b0 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 10 client.16770 fetch_fsmap learned FSMap version 25 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ad9a1776a0 con 0x55ad9a2587b0 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3aeaffd640 1 -- 172.21.15.100:0/1221063122 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b04003e50 con 0x55ad9a2587b0 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3aeaffd640 1 -- 172.21.15.100:0/1221063122 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b0403e050 con 0x55ad9a2587b0 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 10 client.16770 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 20 client.16770 populate_metadata read hostname 'smithi100' 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a2c0690 0x55ad9a2c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 4 client.16770 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.629+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad99f42890 con 0x55ad9a2c0690 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b0affd640 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a2c0690 0x55ad9a2c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3aeaffd640 10 client.16770 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b0affd640 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a2c0690 0x55ad9a2c2ab0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16e3a8ff1d804f0 server_cookie=1cad92571c0f72dd in_seq=0 out_seq=0 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3aeaffd640 1 -- 172.21.15.100:0/1221063122 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ad99f42890 con 0x55ad9a2c0690 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3aeaffd640 10 client.16770 handle_command_reply: tid=0 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 client.16770 shutdown 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 msgr2=0x55ad9a25e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25e960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3b04032c60 tx=0x7f3b04032b50 comp rx=0 tx=0).stop 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 shutdown_connections 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ad9a2c0690 0x55ad9a2c2ab0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad9a2587b0 0x55ad9a25e960 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad9a24fbd0 0x55ad9a25e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3aeaffd640 0 client.16770 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 --2- 172.21.15.100:0/1221063122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9a24f2b0 0x55ad9a25dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 >> 172.21.15.100:0/1221063122 conn(0x55ad9a249990 msgr2=0x55ad9a255f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 shutdown_connections 2024-03-12T05:17:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 1 -- 172.21.15.100:0/1221063122 wait complete. 2024-03-12T05:17:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.630+0000 7f3b14618740 20 client.16770 trim_cache size 0 max 16384 2024-03-12T05:17:53.634 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:17:53.634 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:17:53.634 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:17:53.634 INFO:teuthology.orchestra.run.smithi100.stdout: "008ecd66-13da-40c9-aa2c-5ee941594698": { 2024-03-12T05:17:53.634 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:17:53.635 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "008ecd66-13da-40c9-aa2c-5ee941594698", 2024-03-12T05:17:53.635 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:17:53.635 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:17:53.635 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:17:53.635 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af80a1ba0 msgr2=0x7f3af80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af80a1ba0 0x7f3af80a4060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b04048000 tx=0x7f3b0404b000 comp rx=0 tx=0).stop 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 msgr2=0x7f3b0c19fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c19fb80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3af400e9f0 tx=0x7f3af400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 shutdown_connections 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af80a1ba0 0x7f3af80a4060 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c106110 0x7f3b0c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c100c90 0x7f3b0c19fb80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 --2- 172.21.15.100:0/2891976706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0c1002d0 0x7f3b0c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:17:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 >> 172.21.15.100:0/2891976706 conn(0x7f3b0c076e20 msgr2=0x7f3b0c0697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:17:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 shutdown_connections 2024-03-12T05:17:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:17:53.631+0000 7f3b1196c640 1 -- 172.21.15.100:0/2891976706 wait complete. 2024-03-12T05:17:53.638 INFO:tasks.cephfs.filesystem:scrub status for tag:008ecd66-13da-40c9-aa2c-5ee941594698 - {'path': '/', 'tag': '008ecd66-13da-40c9-aa2c-5ee941594698', 'options': 'recursive,force'} 2024-03-12T05:17:53.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 665582 12.38 MB/sec execute 157 sec latency 98.787 ms 2024-03-12T05:17:54.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:53 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:17:54.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:53 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:53 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:17:54.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:53 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:17:54.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:53 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:53 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:53 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:17:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:53 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:53 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:17:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:53 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:17:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:53 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:53 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:53 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:17:54.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:53 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:53 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:17:54.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:53 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:17:54.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:53 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:53 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:17:54.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 668010 12.40 MB/sec execute 158 sec latency 45.321 ms 2024-03-12T05:17:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:54 smithi156 ceph-mon[36270]: pgmap v177: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-03-12T05:17:55.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:54 smithi100 ceph-mon[32210]: pgmap v177: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-03-12T05:17:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:54 smithi161 ceph-mon[36043]: pgmap v177: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-03-12T05:17:55.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 670390 12.39 MB/sec execute 159 sec latency 18.122 ms 2024-03-12T05:17:56.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 672617 12.39 MB/sec execute 160 sec latency 22.866 ms 2024-03-12T05:17:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:56 smithi156 ceph-mon[36270]: pgmap v178: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-03-12T05:17:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:56 smithi161 ceph-mon[36043]: pgmap v178: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-03-12T05:17:57.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:56 smithi100 ceph-mon[32210]: pgmap v178: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-03-12T05:17:57.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 675159 12.40 MB/sec execute 161 sec latency 26.235 ms 2024-03-12T05:17:58.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 677215 12.37 MB/sec execute 162 sec latency 61.628 ms 2024-03-12T05:17:59.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:58 smithi156 ceph-mon[36270]: pgmap v179: 97 pgs: 97 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 255 op/s 2024-03-12T05:17:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:58 smithi161 ceph-mon[36043]: pgmap v179: 97 pgs: 97 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 255 op/s 2024-03-12T05:17:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:58 smithi100 ceph-mon[32210]: pgmap v179: 97 pgs: 97 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 255 op/s 2024-03-12T05:17:59.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 679224 12.37 MB/sec execute 163 sec latency 18.160 ms 2024-03-12T05:18:00.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:17:59 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:18:00.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:17:59 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:18:00.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:17:59 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:18:00.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 682026 12.40 MB/sec execute 164 sec latency 21.116 ms 2024-03-12T05:18:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:00 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:18:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:00 smithi156 ceph-mon[36270]: pgmap v180: 97 pgs: 97 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-03-12T05:18:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:00 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:18:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:00 smithi161 ceph-mon[36043]: pgmap v180: 97 pgs: 97 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-03-12T05:18:01.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:00 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:18:01.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:00 smithi100 ceph-mon[32210]: pgmap v180: 97 pgs: 97 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-03-12T05:18:01.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 683345 12.36 MB/sec execute 165 sec latency 21.673 ms 2024-03-12T05:18:02.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 685787 12.37 MB/sec execute 166 sec latency 27.904 ms 2024-03-12T05:18:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:02 smithi156 ceph-mon[36270]: pgmap v181: 97 pgs: 97 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 295 op/s 2024-03-12T05:18:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:02 smithi161 ceph-mon[36043]: pgmap v181: 97 pgs: 97 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 295 op/s 2024-03-12T05:18:03.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:02 smithi100 ceph-mon[32210]: pgmap v181: 97 pgs: 97 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 295 op/s 2024-03-12T05:18:03.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 687777 12.34 MB/sec execute 167 sec latency 52.015 ms 2024-03-12T05:18:04.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 689642 12.34 MB/sec execute 168 sec latency 69.650 ms 2024-03-12T05:18:05.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:04 smithi156 ceph-mon[36270]: pgmap v182: 97 pgs: 97 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-03-12T05:18:05.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:04 smithi161 ceph-mon[36043]: pgmap v182: 97 pgs: 97 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-03-12T05:18:05.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:04 smithi100 ceph-mon[32210]: pgmap v182: 97 pgs: 97 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-03-12T05:18:05.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 692171 12.35 MB/sec execute 169 sec latency 20.516 ms 2024-03-12T05:18:06.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 694129 12.34 MB/sec execute 170 sec latency 14.334 ms 2024-03-12T05:18:07.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:06 smithi156 ceph-mon[36270]: pgmap v183: 97 pgs: 97 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-03-12T05:18:07.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:06 smithi161 ceph-mon[36043]: pgmap v183: 97 pgs: 97 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-03-12T05:18:07.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:06 smithi100 ceph-mon[32210]: pgmap v183: 97 pgs: 97 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-03-12T05:18:07.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 696581 12.35 MB/sec execute 171 sec latency 13.529 ms 2024-03-12T05:18:08.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 698954 12.35 MB/sec execute 172 sec latency 16.481 ms 2024-03-12T05:18:09.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:08 smithi156 ceph-mon[36270]: pgmap v184: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 351 op/s 2024-03-12T05:18:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:08 smithi161 ceph-mon[36043]: pgmap v184: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 351 op/s 2024-03-12T05:18:09.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:08 smithi100 ceph-mon[32210]: pgmap v184: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 351 op/s 2024-03-12T05:18:09.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 700456 12.32 MB/sec execute 173 sec latency 75.240 ms 2024-03-12T05:18:10.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 703135 12.35 MB/sec execute 174 sec latency 12.805 ms 2024-03-12T05:18:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:10 smithi156 ceph-mon[36270]: pgmap v185: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-03-12T05:18:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:10 smithi161 ceph-mon[36043]: pgmap v185: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-03-12T05:18:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:10 smithi100 ceph-mon[32210]: pgmap v185: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-03-12T05:18:11.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 704822 12.31 MB/sec execute 175 sec latency 54.990 ms 2024-03-12T05:18:12.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 707267 12.32 MB/sec execute 176 sec latency 17.520 ms 2024-03-12T05:18:13.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:12 smithi156 ceph-mon[36270]: pgmap v186: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-03-12T05:18:13.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:12 smithi161 ceph-mon[36043]: pgmap v186: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-03-12T05:18:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:12 smithi100 ceph-mon[32210]: pgmap v186: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-03-12T05:18:13.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 709879 12.33 MB/sec execute 177 sec latency 35.229 ms 2024-03-12T05:18:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 711475 12.30 MB/sec execute 178 sec latency 80.090 ms 2024-03-12T05:18:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:14 smithi156 ceph-mon[36270]: pgmap v187: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-03-12T05:18:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:14 smithi161 ceph-mon[36043]: pgmap v187: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-03-12T05:18:15.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:14 smithi100 ceph-mon[32210]: pgmap v187: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-03-12T05:18:15.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 714075 12.33 MB/sec execute 179 sec latency 26.547 ms 2024-03-12T05:18:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:15 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:18:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:15 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:18:16.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:15 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:18:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 715759 12.29 MB/sec execute 180 sec latency 57.513 ms 2024-03-12T05:18:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:16 smithi156 ceph-mon[36270]: pgmap v188: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-12T05:18:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:16 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:18:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:16 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:18:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:16 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:18:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:16 smithi161 ceph-mon[36043]: pgmap v188: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-12T05:18:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:16 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:18:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:16 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:18:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:16 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:18:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:16 smithi100 ceph-mon[32210]: pgmap v188: 97 pgs: 97 active+clean; 399 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-12T05:18:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:16 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:18:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:16 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:18:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:16 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:18:17.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 718077 12.30 MB/sec execute 181 sec latency 15.235 ms 2024-03-12T05:18:18.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 720857 12.33 MB/sec execute 182 sec latency 18.144 ms 2024-03-12T05:18:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:18 smithi156 ceph-mon[36270]: pgmap v189: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-03-12T05:18:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:18 smithi161 ceph-mon[36043]: pgmap v189: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-03-12T05:18:19.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:18 smithi100 ceph-mon[32210]: pgmap v189: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-03-12T05:18:19.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 722386 12.30 MB/sec execute 183 sec latency 103.399 ms 2024-03-12T05:18:20.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 724812 12.31 MB/sec execute 184 sec latency 15.654 ms 2024-03-12T05:18:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:20 smithi156 ceph-mon[36270]: pgmap v190: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-12T05:18:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:20 smithi161 ceph-mon[36043]: pgmap v190: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-12T05:18:21.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:20 smithi100 ceph-mon[32210]: pgmap v190: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-12T05:18:21.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 726860 12.28 MB/sec execute 185 sec latency 35.345 ms 2024-03-12T05:18:22.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 728975 12.29 MB/sec execute 186 sec latency 18.214 ms 2024-03-12T05:18:23.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:22 smithi156 ceph-mon[36270]: pgmap v191: 97 pgs: 97 active+clean; 381 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-03-12T05:18:23.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:22 smithi161 ceph-mon[36043]: pgmap v191: 97 pgs: 97 active+clean; 381 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-03-12T05:18:23.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:22 smithi100 ceph-mon[32210]: pgmap v191: 97 pgs: 97 active+clean; 381 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-03-12T05:18:23.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:18:23.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:18:23.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 -- 172.21.15.100:0/149839313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 msgr2=0x7f0f9410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 --2- 172.21.15.100:0/149839313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f9410cf10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009cd0 tx=0x7f0f8c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 -- 172.21.15.100:0/149839313 shutdown_connections 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 --2- 172.21.15.100:0/149839313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f9410cf10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 --2- 172.21.15.100:0/149839313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100c20 0x7f0f94104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 --2- 172.21.15.100:0/149839313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94100270 0x7f0f94100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.751+0000 7f0f9972e640 1 -- 172.21.15.100:0/149839313 >> 172.21.15.100:0/149839313 conn(0x7f0f94077230 msgr2=0x7f0f94077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 -- 172.21.15.100:0/149839313 shutdown_connections 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 -- 172.21.15.100:0/149839313 wait complete. 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 Processor -- start 2024-03-12T05:18:23.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 -- start start 2024-03-12T05:18:23.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100270 0x7f0f94196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f94196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9419aef0 0x7f0f9419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9419b8e0 con 0x7f0f94105340 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9419ba50 con 0x7f0f94100270 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.752+0000 7f0f9972e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9419bbc0 con 0x7f0f9419aef0 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f94196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f94196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60038/0 (socket says 172.21.15.100:60038) 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 -- 172.21.15.100:0/1285957929 learned_addr learned my addr 172.21.15.100:0/1285957929 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100270 0x7f0f94196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f937fe640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9419aef0 0x7f0f9419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 -- 172.21.15.100:0/1285957929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9419aef0 msgr2=0x7f0f9419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9419aef0 0x7f0f9419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 -- 172.21.15.100:0/1285957929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100270 msgr2=0x7f0f94196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100270 0x7f0f94196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 -- 172.21.15.100:0/1285957929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c009900 con 0x7f0f94105340 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100270 0x7f0f94196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f927fc640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f94196b70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800e9c0 tx=0x7f0f8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f7bfff640 1 -- 172.21.15.100:0/1285957929 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88009700 con 0x7f0f94105340 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941a31a0 con 0x7f0f94105340 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f7bfff640 1 -- 172.21.15.100:0/1285957929 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f8800cd80 con 0x7f0f94105340 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f7bfff640 1 -- 172.21.15.100:0/1285957929 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88010430 con 0x7f0f94105340 2024-03-12T05:18:23.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.753+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941a36a0 con 0x7f0f94105340 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.756+0000 7f0f7bfff640 1 -- 172.21.15.100:0/1285957929 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0f88005000 con 0x7f0f94105340 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.756+0000 7f0f9c3d7740 1 Processor -- start 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.756+0000 7f0f9c3d7740 1 -- start start 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f7bfff640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0f7c0a1ba0 0x7f0f7c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 0x55e81f881480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8819c0 0x55e81f883db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f7e5280 con 0x55e81f8819c0 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f7e27b0 con 0x55e81f878de0 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f7ddf90 con 0x55e81f8784c0 2024-03-12T05:18:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f7bfff640 1 -- 172.21.15.100:0/1285957929 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0f88014070 con 0x7f0f94105340 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f937fe640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0f7c0a1ba0 0x7f0f7c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 0x55e81f881480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 0x55e81f881480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59478/0 (socket says 172.21.15.100:59478) 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 -- 172.21.15.100:0/997071237 learned_addr learned my addr 172.21.15.100:0/997071237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f937fe640 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8819c0 0x55e81f883db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 -- 172.21.15.100:0/997071237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 msgr2=0x55e81f8788a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8788a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 -- 172.21.15.100:0/997071237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8819c0 msgr2=0x55e81f883db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8819c0 0x55e81f883db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 -- 172.21.15.100:0/997071237 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81f79c570 con 0x55e81f878de0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f937fe640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0f7c0a1ba0 0x7f0f7c0a4060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006010 tx=0x7f0f8c032ea0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f927fc640 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 0x55e81f881480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f88000c00 tx=0x7f0f8809a390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b168e32bf786e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.757+0000 7f0f797fa640 1 -- 172.21.15.100:0/997071237 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8809af50 con 0x55e81f878de0 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f797fa640 1 -- 172.21.15.100:0/997071237 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f88019480 con 0x55e81f878de0 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f797fa640 1 -- 172.21.15.100:0/997071237 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88098be0 con 0x55e81f878de0 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/997071237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 msgr2=0x55e81f881480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 0x55e81f881480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f88000c00 tx=0x7f0f8809a390 comp rx=0 tx=0).stop 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/997071237 shutdown_connections 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8819c0 0x55e81f883db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f878de0 0x55e81f881480 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/997071237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/997071237 >> 172.21.15.100:0/997071237 conn(0x55e81f872ba0 msgr2=0x55e81f872f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/997071237 shutdown_connections 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.758+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/997071237 wait complete. 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 -- start start 2024-03-12T05:18:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 Processor -- start 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8784c0 0x55e81f87c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8819c0 0x55e81f87d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f4b1430 con 0x55e81f878de0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f46de00 con 0x55e81f8784c0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f762fd0 con 0x55e81f8819c0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f9c3d7740 10 client.0 _ll_register_callbacks cb 0x55e81f7b8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f92ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8784c0 0x55e81f87c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60044/0 (socket says 172.21.15.100:60044) 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 -- 172.21.15.100:0/2041624529 learned_addr learned my addr 172.21.15.100:0/2041624529 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f937fe640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8819c0 0x55e81f87d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 -- 172.21.15.100:0/2041624529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8819c0 msgr2=0x55e81f87d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8819c0 0x55e81f87d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 -- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8784c0 msgr2=0x55e81f87c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8784c0 0x55e81f87c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 -- 172.21.15.100:0/2041624529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941a36a0 con 0x55e81f878de0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8784c0 0x55e81f87c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f927fc640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87ceb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0f88096cb0 tx=0x7f0f88004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcccacbc8fa595f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.759+0000 7f0f76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f76ff5640 1 -- 172.21.15.100:0/2041624529 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8809a850 con 0x55e81f878de0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e81f4dd7e0 con 0x55e81f878de0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f76ff5640 1 -- 172.21.15.100:0/2041624529 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0f8809e9a0 con 0x55e81f878de0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f76ff5640 1 -- 172.21.15.100:0/2041624529 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f880b3270 con 0x55e81f878de0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 10 client.16782 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e81f7dc610 con 0x55e81f878de0 2024-03-12T05:18:23.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f76ff5640 1 -- 172.21.15.100:0/2041624529 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e81f4dd7e0 con 0x55e81f878de0 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f76ff5640 1 -- 172.21.15.100:0/2041624529 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f8801f060 con 0x55e81f878de0 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 10 client.16782 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 20 client.16782 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f893a00 0x55e81f895e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 4 client.16782 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e81f51a720 con 0x55e81f893a00 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.760+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f893a00 0x55e81f895e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f76ff5640 10 client.16782 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f893a00 0x55e81f895e40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e73c17fe97bc612 server_cookie=e0078134194cacf7 in_seq=0 out_seq=0 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f76ff5640 1 -- 172.21.15.100:0/2041624529 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e81f51a720 con 0x55e81f893a00 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f76ff5640 10 client.16782 handle_command_reply: tid=0 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f9c3d7740 1 client.16782 shutdown 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 msgr2=0x55e81f87ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.761+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87ceb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0f88096cb0 tx=0x7f0f88004a50 comp rx=0 tx=0).stop 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 shutdown_connections 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f893a00 0x55e81f895e40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8819c0 0x55e81f87d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87ceb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f76ff5640 0 client.16782 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/2041624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8784c0 0x55e81f87c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 >> 172.21.15.100:0/2041624529 conn(0x55e81f872ba0 msgr2=0x55e81f874940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 shutdown_connections 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/2041624529 wait complete. 2024-03-12T05:18:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.762+0000 7f0f9c3d7740 20 client.16782 trim_cache size 0 max 16384 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 Processor -- start 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 -- start start 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f87a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f890850 con 0x55e81f878de0 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f877f00 con 0x55e81f8819c0 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f8777e0 con 0x55e81f8784c0 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f92ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60056/0 (socket says 172.21.15.100:60056) 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f927fc640 1 -- 172.21.15.100:0/1031382148 learned_addr learned my addr 172.21.15.100:0/1031382148 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f937fe640 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f87a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f927fc640 1 -- 172.21.15.100:0/1031382148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 msgr2=0x55e81f8810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.778+0000 7f0f927fc640 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f927fc640 1 -- 172.21.15.100:0/1031382148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 msgr2=0x55e81f87a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f927fc640 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f87a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f927fc640 1 -- 172.21.15.100:0/1031382148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81f778ac0 con 0x55e81f878de0 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f927fc640 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87d210 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0f88097220 tx=0x7f0f8809a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868ed3dc7bb568ce server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f78ff9640 1 -- 172.21.15.100:0/1031382148 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8809e9a0 con 0x55e81f878de0 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f78ff9640 1 -- 172.21.15.100:0/1031382148 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f8809a850 con 0x55e81f878de0 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f78ff9640 1 -- 172.21.15.100:0/1031382148 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880052b0 con 0x55e81f878de0 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/1031382148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 msgr2=0x55e81f87d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87d210 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0f88097220 tx=0x7f0f8809a390 comp rx=0 tx=0).stop 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/1031382148 shutdown_connections 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.779+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f87a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f878de0 0x55e81f87d210 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/1031382148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f8784c0 0x55e81f8810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/1031382148 >> 172.21.15.100:0/1031382148 conn(0x55e81f872ba0 msgr2=0x55e81f881fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/1031382148 shutdown_connections 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/1031382148 wait complete. 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- start start 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 Processor -- start 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 0x55e81f88c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f878de0 0x55e81f88cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f88aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f88b170 con 0x55e81f8784c0 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f88b2e0 con 0x55e81f8819c0 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f88b4a0 con 0x55e81f878de0 2024-03-12T05:18:23.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f9c3d7740 10 client.0 _ll_register_callbacks cb 0x7f0f88005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f92ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 0x55e81f88c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f92ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 0x55e81f88c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60068/0 (socket says 172.21.15.100:60068) 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f88aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f92ffd640 1 -- 172.21.15.100:0/90952098 learned_addr learned my addr 172.21.15.100:0/90952098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.780+0000 7f0f927fc640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f878de0 0x55e81f88cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f92ffd640 1 -- 172.21.15.100:0/90952098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f878de0 msgr2=0x55e81f88cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f878de0 0x55e81f88cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f92ffd640 1 -- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 msgr2=0x55e81f88aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f88aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f92ffd640 1 -- 172.21.15.100:0/90952098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e81f7dc610 con 0x55e81f8784c0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f937fe640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f88aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f92ffd640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 0x55e81f88c670 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x55e81f896790 tx=0x7f0f8401f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988355bdcad96ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f76ff5640 1 -- 172.21.15.100:0/90952098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84017040 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f76ff5640 1 -- 172.21.15.100:0/90952098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0f84004250 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f88019480 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f76ff5640 1 -- 172.21.15.100:0/90952098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f84004410 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 10 client.16788 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e81f79e6a0 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f76ff5640 1 -- 172.21.15.100:0/90952098 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f88019480 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f76ff5640 1 -- 172.21.15.100:0/90952098 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f84003710 con 0x55e81f8784c0 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 10 client.16788 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 20 client.16788 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f8eb190 0x55e81f8ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 4 client.16788 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.781+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e81f569890 con 0x55e81f8eb190 2024-03-12T05:18:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f927fc640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f8eb190 0x55e81f8ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f76ff5640 10 client.16788 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f927fc640 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f8eb190 0x55e81f8ed5b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e1d49c2b170b22 server_cookie=4a90c5e823204025 in_seq=0 out_seq=0 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f76ff5640 1 -- 172.21.15.100:0/90952098 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e81f569890 con 0x55e81f8eb190 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f76ff5640 10 client.16788 handle_command_reply: tid=0 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 client.16788 shutdown 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 msgr2=0x55e81f88c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 0x55e81f88c670 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x55e81f896790 tx=0x7f0f8401f7c0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 shutdown_connections 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e81f8eb190 0x55e81f8ed5b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81f8819c0 0x55e81f88aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e81f878de0 0x55e81f88cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 --2- 172.21.15.100:0/90952098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e81f8784c0 0x55e81f88c670 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f76ff5640 0 client.16788 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 >> 172.21.15.100:0/90952098 conn(0x55e81f872ba0 msgr2=0x55e81f87d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 shutdown_connections 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 1 -- 172.21.15.100:0/90952098 wait complete. 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.782+0000 7f0f9c3d7740 20 client.16788 trim_cache size 0 max 16384 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:18:23.786 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:18:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0f7c0a1ba0 msgr2=0x7f0f7c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0f7c0a1ba0 0x7f0f7c0a4060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006010 tx=0x7f0f8c032ea0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 msgr2=0x7f0f94196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f94196b70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800e9c0 tx=0x7f0f8800ee80 comp rx=0 tx=0).stop 2024-03-12T05:18:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 shutdown_connections 2024-03-12T05:18:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0f7c0a1ba0 0x7f0f7c0a4060 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9419aef0 0x7f0f9419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f94105340 0x7f0f94196b70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 --2- 172.21.15.100:0/1285957929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f94100270 0x7f0f94196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 >> 172.21.15.100:0/1285957929 conn(0x7f0f94077230 msgr2=0x7f0f940fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 shutdown_connections 2024-03-12T05:18:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.783+0000 7f0f9972e640 1 -- 172.21.15.100:0/1285957929 wait complete. 2024-03-12T05:18:23.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:18:23.790 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:18:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 731703 12.31 MB/sec execute 187 sec latency 14.205 ms 2024-03-12T05:18:23.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- 172.21.15.100:0/3867928587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef90100ca0 msgr2=0x7fef90104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- 172.21.15.100:0/3867928587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef90100ca0 0x7fef90104ce0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fef80009930 tx=0x7fef8002bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:23.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- 172.21.15.100:0/3867928587 shutdown_connections 2024-03-12T05:18:23.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- 172.21.15.100:0/3867928587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef901053c0 0x7fef9010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- 172.21.15.100:0/3867928587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef90100ca0 0x7fef90104ce0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- 172.21.15.100:0/3867928587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901002f0 0x7fef901006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- 172.21.15.100:0/3867928587 >> 172.21.15.100:0/3867928587 conn(0x7fef90077180 msgr2=0x7fef90077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- 172.21.15.100:0/3867928587 shutdown_connections 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- 172.21.15.100:0/3867928587 wait complete. 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 Processor -- start 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- start start 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 0x7fef9019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901053c0 0x7fef9019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef90197690 0x7fef90197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90198160 con 0x7fef901002f0 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901982d0 con 0x7fef90197690 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.899+0000 7fef96a9d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90198490 con 0x7fef901053c0 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 0x7fef9019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 0x7fef9019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60094/0 (socket says 172.21.15.100:60094) 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 -- 172.21.15.100:0/2306772329 learned_addr learned my addr 172.21.15.100:0/2306772329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef87fff640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901053c0 0x7fef9019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef9503a640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef90197690 0x7fef90197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 -- 172.21.15.100:0/2306772329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901053c0 msgr2=0x7fef9019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901053c0 0x7fef9019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 -- 172.21.15.100:0/2306772329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef90197690 msgr2=0x7fef90197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef90197690 0x7fef90197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 -- 172.21.15.100:0/2306772329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c009490 con 0x7fef901002f0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef87fff640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901053c0 0x7fef9019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef94839640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 0x7fef9019d5d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00ea60 tx=0x7fef8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00cde0 con 0x7fef901002f0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8c0035f0 con 0x7fef901002f0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef80009560 con 0x7fef901002f0 2024-03-12T05:18:23.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c010690 con 0x7fef901002f0 2024-03-12T05:18:23.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.900+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901a4780 con 0x7fef901002f0 2024-03-12T05:18:23.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.901+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef901017c0 con 0x7fef901002f0 2024-03-12T05:18:23.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.902+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fef8c003af0 con 0x7fef901002f0 2024-03-12T05:18:23.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.903+0000 7fef85ffb640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef780a1ba0 0x7fef780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:23.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.903+0000 7fef87fff640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef780a1ba0 0x7fef780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:23.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.903+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fef8c014070 con 0x7fef901002f0 2024-03-12T05:18:23.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.904+0000 7fef87fff640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef780a1ba0 0x7fef780a4060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fef80009900 tx=0x7fef80005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:23.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:23.908+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef8c096bd0 con 0x7fef901002f0 2024-03-12T05:18:24.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.138+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef90067f70 con 0x7fef901002f0 2024-03-12T05:18:24.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.139+0000 7fef85ffb640 1 -- 172.21.15.100:0/2306772329 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fef8c09a290 con 0x7fef901002f0 2024-03-12T05:18:24.136 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:18:24.136 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:18:24.136 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef780a1ba0 msgr2=0x7fef780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef780a1ba0 0x7fef780a4060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fef80009900 tx=0x7fef80005fd0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 msgr2=0x7fef9019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 0x7fef9019d5d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00ea60 tx=0x7fef8c00ef20 comp rx=0 tx=0).stop 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 shutdown_connections 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef780a1ba0 0x7fef780a4060 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef90197690 0x7fef90197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef901053c0 0x7fef9019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 --2- 172.21.15.100:0/2306772329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef901002f0 0x7fef9019d5d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 >> 172.21.15.100:0/2306772329 conn(0x7fef90077180 msgr2=0x7fef90104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 shutdown_connections 2024-03-12T05:18:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.141+0000 7fef96a9d640 1 -- 172.21.15.100:0/2306772329 wait complete. 2024-03-12T05:18:24.149 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:18:24.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 -- 172.21.15.100:0/674710371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068c80 msgr2=0x7f7d4c10d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 --2- 172.21.15.100:0/674710371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068c80 0x7f7d4c10d5e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009ca0 tx=0x7f7d4002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 -- 172.21.15.100:0/674710371 shutdown_connections 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 --2- 172.21.15.100:0/674710371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d4c10db20 0x7f7d4c10ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 --2- 172.21.15.100:0/674710371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068c80 0x7f7d4c10d5e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 --2- 172.21.15.100:0/674710371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d4c068360 0x7f7d4c068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 -- 172.21.15.100:0/674710371 >> 172.21.15.100:0/674710371 conn(0x7f7d4c077180 msgr2=0x7f7d4c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 -- 172.21.15.100:0/674710371 shutdown_connections 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.287+0000 7f7d541e2640 1 -- 172.21.15.100:0/674710371 wait complete. 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 Processor -- start 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 -- start start 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 0x7f7d4c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d4c068c80 0x7f7d4c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d4c10db20 0x7f7d4c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c199ea0 con 0x7f7d4c068360 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c19a010 con 0x7f7d4c068c80 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d541e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c19a1d0 con 0x7f7d4c10db20 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 0x7f7d4c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 0x7f7d4c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60118/0 (socket says 172.21.15.100:60118) 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 -- 172.21.15.100:0/1624731937 learned_addr learned my addr 172.21.15.100:0/1624731937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d5177d640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d4c068c80 0x7f7d4c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 -- 172.21.15.100:0/1624731937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d4c10db20 msgr2=0x7f7d4c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d5277f640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d4c10db20 0x7f7d4c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d4c10db20 0x7f7d4c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 -- 172.21.15.100:0/1624731937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d4c068c80 msgr2=0x7f7d4c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d4c068c80 0x7f7d4c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 -- 172.21.15.100:0/1624731937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d40009900 con 0x7f7d4c068360 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.288+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 0x7f7d4c19f690 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e9f0 tx=0x7f7d4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.289+0000 7f7d3effd640 1 -- 172.21.15.100:0/1624731937 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44009720 con 0x7f7d4c068360 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.289+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4c19a4b0 con 0x7f7d4c068360 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.289+0000 7f7d3effd640 1 -- 172.21.15.100:0/1624731937 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d4400cd80 con 0x7f7d4c068360 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.289+0000 7f7d3effd640 1 -- 172.21.15.100:0/1624731937 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44010690 con 0x7f7d4c068360 2024-03-12T05:18:24.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.289+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c1a66c0 con 0x7f7d4c068360 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.291+0000 7f7d3effd640 1 -- 172.21.15.100:0/1624731937 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7d440046c0 con 0x7f7d4c068360 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.291+0000 7f7d56db4740 1 Processor -- start 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.291+0000 7f7d3effd640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7d380a1ba0 0x7f7d380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.291+0000 7f7d56db4740 1 -- start start 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d3effd640 1 -- 172.21.15.100:0/1624731937 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7d44014070 con 0x7f7d4c068360 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d5277f640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7d380a1ba0 0x7f7d380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d56db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b70960 con 0x5631e8bb55e0 2024-03-12T05:18:24.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d56db4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b24310 con 0x5631e8bc6f60 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d56db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b21840 con 0x5631e8bb4c30 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bb5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51342/0 (socket says 172.21.15.100:51342) 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 -- 172.21.15.100:0/4129469784 learned_addr learned my addr 172.21.15.100:0/4129469784 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d5277f640 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d5177d640 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 -- 172.21.15.100:0/4129469784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 msgr2=0x5631e8bc9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 -- 172.21.15.100:0/4129469784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 msgr2=0x5631e8bc6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 -- 172.21.15.100:0/4129469784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e8adf060 con 0x5631e8bb4c30 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d5277f640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7d380a1ba0 0x7f7d380a4060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005d80 tx=0x7f7d480157f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bb5010 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d4409a560 tx=0x7f7d4409a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f7a7862a535943 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.292+0000 7f7d37fff640 1 -- 172.21.15.100:0/4129469784 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4409e9b0 con 0x5631e8bb4c30 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d37fff640 1 -- 172.21.15.100:0/4129469784 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d440a3000 con 0x5631e8bb4c30 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d37fff640 1 -- 172.21.15.100:0/4129469784 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440a2b70 con 0x5631e8bb4c30 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 -- 172.21.15.100:0/4129469784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 msgr2=0x5631e8bb5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bb5010 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d4409a560 tx=0x7f7d4409a1c0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 -- 172.21.15.100:0/4129469784 shutdown_connections 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4129469784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bb5010 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 -- 172.21.15.100:0/4129469784 >> 172.21.15.100:0/4129469784 conn(0x5631e8baf980 msgr2=0x5631e8bafd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 -- 172.21.15.100:0/4129469784 shutdown_connections 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.293+0000 7f7d56db4740 1 -- 172.21.15.100:0/4129469784 wait complete. 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 -- start start 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 Processor -- start 2024-03-12T05:18:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bc0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b1d020 con 0x5631e8bb55e0 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e87f0430 con 0x5631e8bc6f60 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e87ace00 con 0x5631e8bb4c30 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.294+0000 7f7d56db4740 10 client.0 _ll_register_callbacks cb 0x5631e8ac4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d51f7e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bc0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60134/0 (socket says 172.21.15.100:60134) 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5277f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 -- 172.21.15.100:0/2296382628 learned_addr learned my addr 172.21.15.100:0/2296382628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 -- 172.21.15.100:0/2296382628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 msgr2=0x5631e8bc0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bc0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 -- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 msgr2=0x5631e8bc66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 -- 172.21.15.100:0/2296382628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c1a66c0 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5277f640 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d5177d640 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc0ed0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7d4002c700 tx=0x7f7d40034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2781b58ecb9eb84 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d357fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d357fa640 1 -- 172.21.15.100:0/2296382628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d400328b0 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631e881c7e0 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d357fa640 1 -- 172.21.15.100:0/2296382628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7d40046070 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d357fa640 1 -- 172.21.15.100:0/2296382628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d40032a10 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d56db4740 10 client.16806 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.295+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5631e8b1b6a0 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d357fa640 1 -- 172.21.15.100:0/2296382628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5631e881c7e0 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d357fa640 1 -- 172.21.15.100:0/2296382628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7d40004600 con 0x5631e8bb55e0 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d56db4740 10 client.16806 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d56db4740 10 client.16806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:18:24.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d56db4740 20 client.16806 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d56db4740 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8bdfc30 0x5631e8be2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d56db4740 4 client.16806 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631e8859720 con 0x5631e8bdfc30 2024-03-12T05:18:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8bdfc30 0x5631e8be2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d357fa640 10 client.16806 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.296+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8bdfc30 0x5631e8be2050 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9a9f5e8a3fa1bfd server_cookie=712be12faaec228f in_seq=0 out_seq=0 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d357fa640 1 -- 172.21.15.100:0/2296382628 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631e8859720 con 0x5631e8bdfc30 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d357fa640 10 client.16806 handle_command_reply: tid=0 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 client.16806 shutdown 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 msgr2=0x5631e8bc0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc0ed0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7d4002c700 tx=0x7f7d40034670 comp rx=0 tx=0).stop 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 shutdown_connections 2024-03-12T05:18:24.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8bdfc30 0x5631e8be2050 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bc6f60 0x5631e8bc66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bc0ed0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 --2- 172.21.15.100:0/2296382628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8bc0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 >> 172.21.15.100:0/2296382628 conn(0x5631e8baf980 msgr2=0x5631e8bb16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d357fa640 0 client.16806 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 shutdown_connections 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 1 -- 172.21.15.100:0/2296382628 wait complete. 2024-03-12T05:18:24.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.299+0000 7f7d56db4740 20 client.16806 trim_cache size 0 max 16384 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 Processor -- start 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 -- start start 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb4c30 0x5631e8bc65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bb8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bc6f60 0x5631e8bb9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8aa2130 con 0x5631e8bb55e0 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8ba1a40 con 0x5631e8bb4c30 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d56db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8bc6db0 con 0x5631e8bc6f60 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d5177d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bb8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d5177d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bb8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60142/0 (socket says 172.21.15.100:60142) 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d5177d640 1 -- 172.21.15.100:0/4250697350 learned_addr learned my addr 172.21.15.100:0/4250697350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.311+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb4c30 0x5631e8bc65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5277f640 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bc6f60 0x5631e8bb9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5177d640 1 -- 172.21.15.100:0/4250697350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bc6f60 msgr2=0x5631e8bb9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5177d640 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bc6f60 0x5631e8bb9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5177d640 1 -- 172.21.15.100:0/4250697350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb4c30 msgr2=0x5631e8bc65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5177d640 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb4c30 0x5631e8bc65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5177d640 1 -- 172.21.15.100:0/4250697350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e8ab7c20 con 0x5631e8bb55e0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb4c30 0x5631e8bc65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d5177d640 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bb8ff0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7d40002230 tx=0x7f7d4002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12edd4242c4e1379 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d377fe640 1 -- 172.21.15.100:0/4250697350 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d400328b0 con 0x5631e8bb55e0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d377fe640 1 -- 172.21.15.100:0/4250697350 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d40044070 con 0x5631e8bb55e0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.312+0000 7f7d377fe640 1 -- 172.21.15.100:0/4250697350 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40053440 con 0x5631e8bb55e0 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 -- 172.21.15.100:0/4250697350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 msgr2=0x5631e8bb8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bb8ff0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7d40002230 tx=0x7f7d4002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 -- 172.21.15.100:0/4250697350 shutdown_connections 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bc6f60 0x5631e8bb9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bb55e0 0x5631e8bb8ff0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 --2- 172.21.15.100:0/4250697350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb4c30 0x5631e8bc65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 -- 172.21.15.100:0/4250697350 >> 172.21.15.100:0/4250697350 conn(0x5631e8baf980 msgr2=0x5631e8bba660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 -- 172.21.15.100:0/4250697350 shutdown_connections 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 -- 172.21.15.100:0/4250697350 wait complete. 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 -- start start 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.313+0000 7f7d56db4740 1 Processor -- start 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8c2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb55e0 0x5631e8c2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 0x5631e8c2a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8c2a6d0 con 0x5631e8bc6f60 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8c2a840 con 0x5631e8bb55e0 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8c2aa00 con 0x5631e8bb4c30 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 10 client.0 _ll_register_callbacks cb 0x7f7d44096440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5177d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb55e0 0x5631e8c2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 0x5631e8c2a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 0x5631e8c2a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60144/0 (socket says 172.21.15.100:60144) 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5177d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb55e0 0x5631e8c2fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59602/0 (socket says 172.21.15.100:59602) 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 -- 172.21.15.100:0/882801904 learned_addr learned my addr 172.21.15.100:0/882801904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8c2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 -- 172.21.15.100:0/882801904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 msgr2=0x5631e8c2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8c2f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 -- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb55e0 msgr2=0x5631e8c2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb55e0 0x5631e8c2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 -- 172.21.15.100:0/882801904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631e8adf060 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d5277f640 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 0x5631e8c2a0c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d48020b10 tx=0x7f7d48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd5e06addad0181 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d357fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d357fa640 1 -- 172.21.15.100:0/882801904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48025070 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d357fa640 1 -- 172.21.15.100:0/882801904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7d48012ba0 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d440a3000 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.314+0000 7f7d357fa640 1 -- 172.21.15.100:0/882801904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d48012d60 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 10 client.16812 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5631e8b1b6a0 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d357fa640 1 -- 172.21.15.100:0/882801904 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d48036020 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d357fa640 1 -- 172.21.15.100:0/882801904 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7d48010460 con 0x5631e8bc6f60 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 10 client.16812 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 10 client.16812 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 20 client.16812 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:24.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8c37750 0x5631e8c39b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 4 client.16812 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:18:24.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631e88a8890 con 0x5631e8c37750 2024-03-12T05:18:24.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8c37750 0x5631e8c39b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d357fa640 10 client.16812 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:24.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.315+0000 7f7d51f7e640 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8c37750 0x5631e8c39b70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17b7ce78e9c9f3f5 server_cookie=f9c0c9cf590356cd in_seq=0 out_seq=0 2024-03-12T05:18:24.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d357fa640 1 -- 172.21.15.100:0/882801904 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631e88a8890 con 0x5631e8c37750 2024-03-12T05:18:24.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d357fa640 10 client.16812 handle_command_reply: tid=0 2024-03-12T05:18:24.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d56db4740 1 client.16812 shutdown 2024-03-12T05:18:24.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 msgr2=0x5631e8c2a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d56db4740 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 0x5631e8c2a0c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d48020b10 tx=0x7f7d48004730 comp rx=0 tx=0).stop 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 shutdown_connections 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.318+0000 7f7d56db4740 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5631e8c37750 0x5631e8c39b70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e8bc6f60 0x5631e8c2a0c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d357fa640 0 client.16812 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e8bb55e0 0x5631e8c2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 1 --2- 172.21.15.100:0/882801904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e8bb4c30 0x5631e8c2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 >> 172.21.15.100:0/882801904 conn(0x5631e8baf980 msgr2=0x5631e8bb0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 shutdown_connections 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 1 -- 172.21.15.100:0/882801904 wait complete. 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d56db4740 20 client.16812 trim_cache size 0 max 16384 2024-03-12T05:18:24.320 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:18:24.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7d380a1ba0 msgr2=0x7f7d380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7d380a1ba0 0x7f7d380a4060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005d80 tx=0x7f7d480157f0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 msgr2=0x7f7d4c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 0x7f7d4c19f690 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e9f0 tx=0x7f7d4400edf0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 shutdown_connections 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7d380a1ba0 0x7f7d380a4060 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d4c10db20 0x7f7d4c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d4c068c80 0x7f7d4c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 --2- 172.21.15.100:0/1624731937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c068360 0x7f7d4c19f690 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 >> 172.21.15.100:0/1624731937 conn(0x7f7d4c077180 msgr2=0x7f7d4c10f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 shutdown_connections 2024-03-12T05:18:24.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.319+0000 7f7d541e2640 1 -- 172.21.15.100:0/1624731937 wait complete. 2024-03-12T05:18:24.326 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:18:24.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:18:24.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 -- 172.21.15.100:0/4212602083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101d40 msgr2=0x7f1798105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 --2- 172.21.15.100:0/4212602083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101d40 0x7f1798105d80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1788009930 tx=0x7f178802bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 -- 172.21.15.100:0/4212602083 shutdown_connections 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 --2- 172.21.15.100:0/4212602083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798106460 0x7f179810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 --2- 172.21.15.100:0/4212602083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101d40 0x7f1798105d80 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 --2- 172.21.15.100:0/4212602083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1798101390 0x7f1798101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 -- 172.21.15.100:0/4212602083 >> 172.21.15.100:0/4212602083 conn(0x7f17980fcbb0 msgr2=0x7f17980fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 -- 172.21.15.100:0/4212602083 shutdown_connections 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 -- 172.21.15.100:0/4212602083 wait complete. 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.437+0000 7f179fd7a640 1 Processor -- start 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 -- start start 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 0x7f179819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1798106460 0x7f179819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798197640 0x7f1798197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1798198030 con 0x7f1798101390 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981981a0 con 0x7f1798197640 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179fd7a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1798198360 con 0x7f1798106460 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 0x7f179819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 0x7f179819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60164/0 (socket says 172.21.15.100:60164) 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 -- 172.21.15.100:0/3333718804 learned_addr learned my addr 172.21.15.100:0/3333718804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179e317640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798197640 0x7f1798197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179d315640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1798106460 0x7f179819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 -- 172.21.15.100:0/3333718804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1798106460 msgr2=0x7f179819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1798106460 0x7f179819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 -- 172.21.15.100:0/3333718804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798197640 msgr2=0x7f1798197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798197640 0x7f1798197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 -- 172.21.15.100:0/3333718804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009560 con 0x7f1798101390 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179e317640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798197640 0x7f1798197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f179db16640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 0x7f179819d440 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f179400d950 tx=0x7f179400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.438+0000 7f178effd640 1 -- 172.21.15.100:0/3333718804 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794009a30 con 0x7f1798101390 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.439+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17981a4300 con 0x7f1798101390 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.439+0000 7f178effd640 1 -- 172.21.15.100:0/3333718804 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1794009b90 con 0x7f1798101390 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.439+0000 7f178effd640 1 -- 172.21.15.100:0/3333718804 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794004a00 con 0x7f1798101390 2024-03-12T05:18:24.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.439+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17981a47e0 con 0x7f1798101390 2024-03-12T05:18:24.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.441+0000 7f178effd640 1 -- 172.21.15.100:0/3333718804 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1794009d00 con 0x7f1798101390 2024-03-12T05:18:24.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 Processor -- start 2024-03-12T05:18:24.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 -- start start 2024-03-12T05:18:24.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f178effd640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17840a1ba0 0x7f17840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b11360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5acc960 con 0x55e4d5b1eb60 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5a80310 con 0x55e4d5b10f80 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5a7d840 con 0x55e4d5b118a0 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f178effd640 1 -- 172.21.15.100:0/3333718804 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f17940cd250 con 0x7f1798101390 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f179e317640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17840a1ba0 0x7f17840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f179d315640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f179db16640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b11360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f179d315640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51386/0 (socket says 172.21.15.100:51386) 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f179d315640 1 -- 172.21.15.100:0/2856953325 learned_addr learned my addr 172.21.15.100:0/2856953325 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.442+0000 7f179e317640 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179d315640 1 -- 172.21.15.100:0/2856953325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 msgr2=0x55e4d5b11360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179d315640 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b11360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179d315640 1 -- 172.21.15.100:0/2856953325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 msgr2=0x55e4d5b20f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179d315640 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b20f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179d315640 1 -- 172.21.15.100:0/2856953325 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4d5a3b060 con 0x55e4d5b118a0 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179db16640 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179e317640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17840a1ba0 0x7f17840a4060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1790005fd0 tx=0x7f1790013a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f179d315640 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1e620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1788000c00 tx=0x7f178802c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe1ef4af4c34118 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f1783fff640 1 -- 172.21.15.100:0/2856953325 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788004a00 con 0x55e4d5b118a0 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f1783fff640 1 -- 172.21.15.100:0/2856953325 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1788004030 con 0x55e4d5b118a0 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.443+0000 7f1783fff640 1 -- 172.21.15.100:0/2856953325 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17880052b0 con 0x55e4d5b118a0 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2856953325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 msgr2=0x55e4d5b1e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1e620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1788000c00 tx=0x7f178802c5d0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2856953325 shutdown_connections 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b20f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1e620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2856953325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2856953325 >> 172.21.15.100:0/2856953325 conn(0x55e4d5b0b980 msgr2=0x55e4d5b0bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2856953325 shutdown_connections 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.444+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2856953325 wait complete. 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 -- start start 2024-03-12T05:18:24.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 Processor -- start 2024-03-12T05:18:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 0x55e4d5b1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5a79020 con 0x55e4d5b118a0 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d574c430 con 0x55e4d5b1eb60 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5708e00 con 0x55e4d5b10f80 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f179d315640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f17a2a2b740 10 client.0 _ll_register_callbacks cb 0x55e4d5a20cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f179db16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 0x55e4d5b1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f179d315640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60182/0 (socket says 172.21.15.100:60182) 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f179db16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 0x55e4d5b1bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51388/0 (socket says 172.21.15.100:51388) 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f179d315640 1 -- 172.21.15.100:0/2360437886 learned_addr learned my addr 172.21.15.100:0/2360437886 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.445+0000 7f179e317640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179db16640 1 -- 172.21.15.100:0/2360437886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b1eb60 msgr2=0x55e4d5b1c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179db16640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b1c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179db16640 1 -- 172.21.15.100:0/2360437886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 msgr2=0x55e4d5b1c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179db16640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179db16640 1 -- 172.21.15.100:0/2360437886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179810e840 con 0x55e4d5b10f80 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179d315640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f179db16640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 0x55e4d5b1bb40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f17940ca990 tx=0x7f179409a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ebb6abe29edfdb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17817fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17817fa640 1 -- 172.21.15.100:0/2360437886 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794005680 con 0x55e4d5b10f80 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4d57787e0 con 0x55e4d5b10f80 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17817fa640 1 -- 172.21.15.100:0/2360437886 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f17940992c0 con 0x55e4d5b10f80 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17817fa640 1 -- 172.21.15.100:0/2360437886 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17940b8310 con 0x55e4d5b10f80 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17a2a2b740 10 client.26239 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4d5a776a0 con 0x55e4d5b10f80 2024-03-12T05:18:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.446+0000 7f17817fa640 1 -- 172.21.15.100:0/2360437886 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1794063080 con 0x55e4d5b10f80 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17817fa640 1 -- 172.21.15.100:0/2360437886 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17940b2c30 con 0x55e4d5b10f80 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17a2a2b740 10 client.26239 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17a2a2b740 10 client.26239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17a2a2b740 20 client.26239 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b2ef60 0x55e4d5b313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17a2a2b740 4 client.26239 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4d57b5720 con 0x55e4d5b2ef60 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f179d315640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b2ef60 0x55e4d5b313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f17817fa640 10 client.26239 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.447+0000 7f179d315640 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b2ef60 0x55e4d5b313a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c48da8214dfed6a4 server_cookie=fa27fbd3456fd85f in_seq=0 out_seq=0 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17817fa640 1 -- 172.21.15.100:0/2360437886 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4d57b5720 con 0x55e4d5b2ef60 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17817fa640 10 client.26239 handle_command_reply: tid=0 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 client.26239 shutdown 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 msgr2=0x55e4d5b1bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 0x55e4d5b1bb40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f17940ca990 tx=0x7f179409a6a0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 shutdown_connections 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b2ef60 0x55e4d5b313a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b1c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b118a0 0x55e4d5b1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/2360437886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b10f80 0x55e4d5b1bb40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17817fa640 0 client.26239 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 >> 172.21.15.100:0/2360437886 conn(0x55e4d5b0b980 msgr2=0x55e4d5b0d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 shutdown_connections 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 1 -- 172.21.15.100:0/2360437886 wait complete. 2024-03-12T05:18:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.448+0000 7f17a2a2b740 20 client.26239 trim_cache size 0 max 16384 2024-03-12T05:18:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 Processor -- start 2024-03-12T05:18:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 -- start start 2024-03-12T05:18:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b0aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b25f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d59fe130 con 0x55e4d5b1eb60 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5b21520 con 0x55e4d5b10f80 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5b365f0 con 0x55e4d5b118a0 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179e317640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b25f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179e317640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b25f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60198/0 (socket says 172.21.15.100:60198) 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179e317640 1 -- 172.21.15.100:0/226725191 learned_addr learned my addr 172.21.15.100:0/226725191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179d315640 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179db16640 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b0aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179d315640 1 -- 172.21.15.100:0/226725191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 msgr2=0x55e4d5b0aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179d315640 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b0aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179d315640 1 -- 172.21.15.100:0/226725191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 msgr2=0x55e4d5b25f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179d315640 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b25f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179d315640 1 -- 172.21.15.100:0/226725191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4d5a13c20 con 0x55e4d5b118a0 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.461+0000 7f179e317640 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f179d315640 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b259c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55e4d5b076e0 tx=0x7f178802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c93d0f726b4d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17837fe640 1 -- 172.21.15.100:0/226725191 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788059070 con 0x55e4d5b118a0 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17837fe640 1 -- 172.21.15.100:0/226725191 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178804f040 con 0x55e4d5b118a0 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17837fe640 1 -- 172.21.15.100:0/226725191 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178804d040 con 0x55e4d5b118a0 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 -- 172.21.15.100:0/226725191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 msgr2=0x55e4d5b259c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b259c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55e4d5b076e0 tx=0x7f178802bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 -- 172.21.15.100:0/226725191 shutdown_connections 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b259c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/226725191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b0aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 -- 172.21.15.100:0/226725191 >> 172.21.15.100:0/226725191 conn(0x55e4d5b0b980 msgr2=0x55e4d5b0a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.462+0000 7f17a2a2b740 1 -- 172.21.15.100:0/226725191 shutdown_connections 2024-03-12T05:18:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 -- 172.21.15.100:0/226725191 wait complete. 2024-03-12T05:18:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 -- start start 2024-03-12T05:18:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 Processor -- start 2024-03-12T05:18:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b24380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5b26280 con 0x55e4d5b1eb60 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5b854d0 con 0x55e4d5b10f80 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4d5b85610 con 0x55e4d5b118a0 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f17a2a2b740 10 client.0 _ll_register_callbacks cb 0x7f17900093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.463+0000 7f179e317640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179e317640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b24e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60202/0 (socket says 172.21.15.100:60202) 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b248c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51404/0 (socket says 172.21.15.100:51404) 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179e317640 1 -- 172.21.15.100:0/514709159 learned_addr learned my addr 172.21.15.100:0/514709159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179db16640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b24380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 -- 172.21.15.100:0/514709159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 msgr2=0x55e4d5b24380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b24380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 -- 172.21.15.100:0/514709159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 msgr2=0x55e4d5b24e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b24e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 -- 172.21.15.100:0/514709159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179001b040 con 0x55e4d5b118a0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179e317640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f179d315640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b248c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55e4d5b1a380 tx=0x7f178802c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af18d2f6513821d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17817fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17817fa640 1 -- 172.21.15.100:0/514709159 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788059050 con 0x55e4d5b118a0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17817fa640 1 -- 172.21.15.100:0/514709159 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1788054550 con 0x55e4d5b118a0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1788004030 con 0x55e4d5b118a0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17817fa640 1 -- 172.21.15.100:0/514709159 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f178804d040 con 0x55e4d5b118a0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17a2a2b740 10 client.26245 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.464+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4d5a3b060 con 0x55e4d5b118a0 2024-03-12T05:18:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17817fa640 1 -- 172.21.15.100:0/514709159 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1788069080 con 0x55e4d5b118a0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17817fa640 1 -- 172.21.15.100:0/514709159 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1788064070 con 0x55e4d5b118a0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 10 client.26245 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 10 client.26245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 20 client.26245 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b869c0 0x55e4d5b88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 4 client.26245 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4d5804890 con 0x55e4d5b869c0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f179db16640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b869c0 0x55e4d5b88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17817fa640 10 client.26245 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f179db16640 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b869c0 0x55e4d5b88db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10f8ccc4b8b4115c server_cookie=e3e3acc7613f4e0 in_seq=0 out_seq=0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17817fa640 1 -- 172.21.15.100:0/514709159 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4d5804890 con 0x55e4d5b869c0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17817fa640 10 client.26245 handle_command_reply: tid=0 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.465+0000 7f17a2a2b740 1 client.26245 shutdown 2024-03-12T05:18:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 msgr2=0x55e4d5b248c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b248c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55e4d5b1a380 tx=0x7f178802c740 comp rx=0 tx=0).stop 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 shutdown_connections 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4d5b1eb60 0x55e4d5b24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4d5b118a0 0x55e4d5b248c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55e4d5b869c0 0x55e4d5b88db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 --2- 172.21.15.100:0/514709159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4d5b10f80 0x55e4d5b24380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17817fa640 0 client.26245 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 >> 172.21.15.100:0/514709159 conn(0x55e4d5b0b980 msgr2=0x55e4d5b0ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 shutdown_connections 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 1 -- 172.21.15.100:0/514709159 wait complete. 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f17a2a2b740 20 client.26245 trim_cache size 0 max 16384 2024-03-12T05:18:24.469 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:18:24.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17840a1ba0 msgr2=0x7f17840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17840a1ba0 0x7f17840a4060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1790005fd0 tx=0x7f1790013a40 comp rx=0 tx=0).stop 2024-03-12T05:18:24.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 msgr2=0x7f179819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 0x7f179819d440 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f179400d950 tx=0x7f179400de10 comp rx=0 tx=0).stop 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 shutdown_connections 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17840a1ba0 0x7f17840a4060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1798197640 0x7f1798197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1798106460 0x7f179819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 --2- 172.21.15.100:0/3333718804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1798101390 0x7f179819d440 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 >> 172.21.15.100:0/3333718804 conn(0x7f17980fcbb0 msgr2=0x7f17980fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 shutdown_connections 2024-03-12T05:18:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.466+0000 7f179fd7a640 1 -- 172.21.15.100:0/3333718804 wait complete. 2024-03-12T05:18:24.474 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:18:24.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 -- 172.21.15.100:0/3609084669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 msgr2=0x7f1420100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 --2- 172.21.15.100:0/3609084669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f1420100640 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f141c009930 tx=0x7f141c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 -- 172.21.15.100:0/3609084669 shutdown_connections 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 --2- 172.21.15.100:0/3609084669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1420105330 0x7f142010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 --2- 172.21.15.100:0/3609084669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420100c10 0x7f1420104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 --2- 172.21.15.100:0/3609084669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f1420100640 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 -- 172.21.15.100:0/3609084669 >> 172.21.15.100:0/3609084669 conn(0x7f1420077180 msgr2=0x7f1420077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.588+0000 7f14279e2640 1 -- 172.21.15.100:0/3609084669 shutdown_connections 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 -- 172.21.15.100:0/3609084669 wait complete. 2024-03-12T05:18:24.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 Processor -- start 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 -- start start 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f142019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1420105330 0x7f142019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420197670 0x7f1420197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420198140 con 0x7f1420100260 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14201982b0 con 0x7f1420105330 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f14279e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420198470 con 0x7f1420197670 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f142577e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f142019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f142577e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f142019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60210/0 (socket says 172.21.15.100:60210) 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f142577e640 1 -- 172.21.15.100:0/1749812669 learned_addr learned my addr 172.21.15.100:0/1749812669 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f1425f7f640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420197670 0x7f1420197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.589+0000 7f1424f7d640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1420105330 0x7f142019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f142577e640 1 -- 172.21.15.100:0/1749812669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420197670 msgr2=0x7f1420197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f142577e640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420197670 0x7f1420197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f142577e640 1 -- 172.21.15.100:0/1749812669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1420105330 msgr2=0x7f142019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f142577e640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1420105330 0x7f142019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f142577e640 1 -- 172.21.15.100:0/1749812669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c009560 con 0x7f1420100260 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f142577e640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f142019d5b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f141c009230 tx=0x7f141c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f14167fc640 1 -- 172.21.15.100:0/1749812669 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c004440 con 0x7f1420100260 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f14167fc640 1 -- 172.21.15.100:0/1749812669 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f141c0052d0 con 0x7f1420100260 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14201a43b0 con 0x7f1420100260 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f14167fc640 1 -- 172.21.15.100:0/1749812669 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c003d70 con 0x7f1420100260 2024-03-12T05:18:24.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.590+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14201a48e0 con 0x7f1420100260 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.592+0000 7f14167fc640 1 -- 172.21.15.100:0/1749812669 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f141c005440 con 0x7f1420100260 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 Processor -- start 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 -- start start 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f14167fc640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f140c0a1c20 0x7f140c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ebb60 0x55812e5edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e599960 con 0x55812e5ebb60 2024-03-12T05:18:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e54d310 con 0x55812e5de8a0 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142a5b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e54a840 con 0x55812e5ddf80 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f1425f7f640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f140c0a1c20 0x7f140c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f14167fc640 1 -- 172.21.15.100:0/1749812669 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f141c0ef830 con 0x7f1420100260 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142577e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142577e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5de360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51422/0 (socket says 172.21.15.100:51422) 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f142577e640 1 -- 172.21.15.100:0/2890809262 learned_addr learned my addr 172.21.15.100:0/2890809262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f1425f7f640 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ebb60 0x55812e5edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.593+0000 7f1424f7d640 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f142577e640 1 -- 172.21.15.100:0/2890809262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 msgr2=0x55812e5eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f142577e640 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f142577e640 1 -- 172.21.15.100:0/2890809262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ebb60 msgr2=0x55812e5edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f142577e640 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ebb60 0x55812e5edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f142577e640 1 -- 172.21.15.100:0/2890809262 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55812e508060 con 0x55812e5ddf80 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f1425f7f640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f140c0a1c20 0x7f140c0a40e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1418005fd0 tx=0x7f1418012ad0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f142577e640 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5de360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f141c009430 tx=0x7f141c003840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be0e6de2527fca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f140b7fe640 1 -- 172.21.15.100:0/2890809262 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c037120 con 0x55812e5ddf80 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f140b7fe640 1 -- 172.21.15.100:0/2890809262 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f141c049ce0 con 0x55812e5ddf80 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.594+0000 7f140b7fe640 1 -- 172.21.15.100:0/2890809262 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c043df0 con 0x55812e5ddf80 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 -- 172.21.15.100:0/2890809262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 msgr2=0x55812e5de360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5de360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f141c009430 tx=0x7f141c003840 comp rx=0 tx=0).stop 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 -- 172.21.15.100:0/2890809262 shutdown_connections 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ebb60 0x55812e5edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2890809262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5de360 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 -- 172.21.15.100:0/2890809262 >> 172.21.15.100:0/2890809262 conn(0x55812e5d8980 msgr2=0x55812e5d8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 -- 172.21.15.100:0/2890809262 shutdown_connections 2024-03-12T05:18:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 -- 172.21.15.100:0/2890809262 wait complete. 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 -- start start 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.595+0000 7f142a5b4740 1 Processor -- start 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 0x55812e5e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5ebb60 0x55812e5e9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e546020 con 0x55812e5de8a0 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e219430 con 0x55812e5ebb60 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e1d5e00 con 0x55812e5ddf80 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 10 client.0 _ll_register_callbacks cb 0x55812e4edcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 0x55812e5e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 0x55812e5e90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60228/0 (socket says 172.21.15.100:60228) 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 -- 172.21.15.100:0/2757936572 learned_addr learned my addr 172.21.15.100:0/2757936572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142577e640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1425f7f640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5ebb60 0x55812e5e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 -- 172.21.15.100:0/2757936572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 msgr2=0x55812e5e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 -- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5ebb60 msgr2=0x55812e5e9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5ebb60 0x55812e5e9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 -- 172.21.15.100:0/2757936572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14201a48e0 con 0x55812e5de8a0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142577e640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1424f7d640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 0x55812e5e90d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f141000b950 tx=0x7f141000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06b479d44aabc1f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1408ff9640 1 -- 172.21.15.100:0/2757936572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141001b440 con 0x55812e5de8a0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1408ff9640 1 -- 172.21.15.100:0/2757936572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1410011440 con 0x55812e5de8a0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55812e2457e0 con 0x55812e5de8a0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f1408ff9640 1 -- 172.21.15.100:0/2757936572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1410011600 con 0x55812e5de8a0 2024-03-12T05:18:24.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 10 client.16842 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.596+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55812e5446a0 con 0x55812e5de8a0 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f1408ff9640 1 -- 172.21.15.100:0/2757936572 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55812e2457e0 con 0x55812e5de8a0 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f1408ff9640 1 -- 172.21.15.100:0/2757936572 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f141001baa0 con 0x55812e5de8a0 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142a5b4740 10 client.16842 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142a5b4740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142a5b4740 20 client.16842 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e5fbd90 0x55812e5fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142a5b4740 4 client.16842 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55812e282720 con 0x55812e5fbd90 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142577e640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e5fbd90 0x55812e5fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f1408ff9640 10 client.16842 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.597+0000 7f142577e640 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e5fbd90 0x55812e5fe1b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b0aa73c04e0f44 server_cookie=ac4e8f70821ec5f4 in_seq=0 out_seq=0 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f1408ff9640 1 -- 172.21.15.100:0/2757936572 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55812e282720 con 0x55812e5fbd90 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f1408ff9640 10 client.16842 handle_command_reply: tid=0 2024-03-12T05:18:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 client.16842 shutdown 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 msgr2=0x55812e5e90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 0x55812e5e90d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f141000b950 tx=0x7f141000bd50 comp rx=0 tx=0).stop 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 shutdown_connections 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e5fbd90 0x55812e5fe1b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5ebb60 0x55812e5e9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5de8a0 0x55812e5e90d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 --2- 172.21.15.100:0/2757936572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ddf80 0x55812e5e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 >> 172.21.15.100:0/2757936572 conn(0x55812e5d8980 msgr2=0x55812e5da6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.598+0000 7f1408ff9640 0 client.16842 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.599+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 shutdown_connections 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.599+0000 7f142a5b4740 1 -- 172.21.15.100:0/2757936572 wait complete. 2024-03-12T05:18:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.599+0000 7f142a5b4740 20 client.16842 trim_cache size 0 max 16384 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 Processor -- start 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 -- start start 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e4cb130 con 0x55812e5ddf80 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e282750 con 0x55812e5de8a0 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142a5b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e5caa40 con 0x55812e5ebb60 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60236/0 (socket says 172.21.15.100:60236) 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 -- 172.21.15.100:0/1124148065 learned_addr learned my addr 172.21.15.100:0/1124148065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f1425f7f640 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f1424f7d640 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 -- 172.21.15.100:0/1124148065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 msgr2=0x55812e5e5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5e5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 -- 172.21.15.100:0/1124148065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 msgr2=0x55812e5e9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5e9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 -- 172.21.15.100:0/1124148065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55812e4e0c20 con 0x55812e5ddf80 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f142577e640 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5eb2c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f141c0b8f30 tx=0x7f141c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d406ac3eb418850 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f140affd640 1 -- 172.21.15.100:0/1124148065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0bcc60 con 0x55812e5ddf80 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f140affd640 1 -- 172.21.15.100:0/1124148065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f141c0bc560 con 0x55812e5ddf80 2024-03-12T05:18:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.612+0000 7f140affd640 1 -- 172.21.15.100:0/1124148065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0ba860 con 0x55812e5ddf80 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- 172.21.15.100:0/1124148065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 msgr2=0x55812e5eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5eb2c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f141c0b8f30 tx=0x7f141c0dc040 comp rx=0 tx=0).stop 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- 172.21.15.100:0/1124148065 shutdown_connections 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5e5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5e9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- 172.21.15.100:0/1124148065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5eb2c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- 172.21.15.100:0/1124148065 >> 172.21.15.100:0/1124148065 conn(0x55812e5d8980 msgr2=0x55812e5e7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- 172.21.15.100:0/1124148065 shutdown_connections 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- 172.21.15.100:0/1124148065 wait complete. 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- start start 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 Processor -- start 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e5f1090 con 0x55812e5ddf80 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e5f1200 con 0x55812e5de8a0 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812e5f13c0 con 0x55812e5ebb60 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.613+0000 7f142a5b4740 10 client.0 _ll_register_callbacks cb 0x7f141c0045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60252/0 (socket says 172.21.15.100:60252) 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 -- 172.21.15.100:0/3350097906 learned_addr learned my addr 172.21.15.100:0/3350097906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f1424f7d640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f1425f7f640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 -- 172.21.15.100:0/3350097906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 msgr2=0x55812e5f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5f0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 -- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 msgr2=0x55812e5f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 -- 172.21.15.100:0/3350097906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55812e508060 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142577e640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5f2550 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f141c006010 tx=0x7f141c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4166dfce8e2266 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f1408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f1408ff9640 1 -- 172.21.15.100:0/3350097906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0a0070 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f1408ff9640 1 -- 172.21.15.100:0/3350097906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f141c0c06b0 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f141c049ce0 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f1408ff9640 1 -- 172.21.15.100:0/3350097906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f141c0bcc60 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142a5b4740 10 client.16848 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.614+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55812e5068c0 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f1408ff9640 1 -- 172.21.15.100:0/3350097906 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f141c049ce0 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f1408ff9640 1 -- 172.21.15.100:0/3350097906 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f141c0a3030 con 0x55812e5ddf80 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f142a5b4740 10 client.16848 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:24.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f142a5b4740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f142a5b4740 20 client.16848 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f142a5b4740 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e653230 0x55812e655650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f142a5b4740 4 client.16848 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55812e2d1890 con 0x55812e653230 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f1424f7d640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e653230 0x55812e655650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f1408ff9640 10 client.16848 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.615+0000 7f1424f7d640 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e653230 0x55812e655650 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7efa8762eb94c0 server_cookie=15738c56d4376dce in_seq=0 out_seq=0 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f1408ff9640 1 -- 172.21.15.100:0/3350097906 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55812e2d1890 con 0x55812e653230 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f1408ff9640 10 client.16848 handle_command_reply: tid=0 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 client.16848 shutdown 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 msgr2=0x55812e5f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5f2550 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f141c006010 tx=0x7f141c003ed0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 shutdown_connections 2024-03-12T05:18:24.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55812e653230 0x55812e655650 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55812e5ebb60 0x55812e5f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55812e5de8a0 0x55812e5f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 --2- 172.21.15.100:0/3350097906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55812e5ddf80 0x55812e5f2550 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f1408ff9640 0 client.16848 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 >> 172.21.15.100:0/3350097906 conn(0x55812e5d8980 msgr2=0x55812e5d72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 shutdown_connections 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 1 -- 172.21.15.100:0/3350097906 wait complete. 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f142a5b4740 20 client.16848 trim_cache size 0 max 16384 2024-03-12T05:18:24.619 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f140c0a1c20 msgr2=0x7f140c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f140c0a1c20 0x7f140c0a40e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1418005fd0 tx=0x7f1418012ad0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 msgr2=0x7f142019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f142019d5b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f141c009230 tx=0x7f141c004040 comp rx=0 tx=0).stop 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 shutdown_connections 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f140c0a1c20 0x7f140c0a40e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420197670 0x7f1420197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1420105330 0x7f142019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 --2- 172.21.15.100:0/1749812669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1420100260 0x7f142019d5b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:24.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.616+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 >> 172.21.15.100:0/1749812669 conn(0x7f1420077180 msgr2=0x7f14200fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:24.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.617+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 shutdown_connections 2024-03-12T05:18:24.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:24.617+0000 7f14279e2640 1 -- 172.21.15.100:0/1749812669 wait complete. 2024-03-12T05:18:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 733144 12.28 MB/sec execute 188 sec latency 57.104 ms 2024-03-12T05:18:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:24 smithi156 ceph-mon[36270]: pgmap v192: 97 pgs: 97 active+clean; 385 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 212 op/s 2024-03-12T05:18:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:24 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2306772329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:18:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:24 smithi161 ceph-mon[36043]: pgmap v192: 97 pgs: 97 active+clean; 385 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 212 op/s 2024-03-12T05:18:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:24 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2306772329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:18:25.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:24 smithi100 ceph-mon[32210]: pgmap v192: 97 pgs: 97 active+clean; 385 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 212 op/s 2024-03-12T05:18:25.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:24 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2306772329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:18:25.625 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:18:25.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:18:25.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:18:25.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1351899165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd4106430 msgr2=0x7f5fd410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1351899165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd4106430 0x7f5fd410e000 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009cd0 tx=0x7f5fcc02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1351899165 shutdown_connections 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1351899165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd4106430 0x7f5fd410e000 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1351899165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4068f80 0x7f5fd4105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1351899165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 0x7f5fd40689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1351899165 >> 172.21.15.100:0/1351899165 conn(0x7f5fd4077180 msgr2=0x7f5fd4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1351899165 shutdown_connections 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1351899165 wait complete. 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 Processor -- start 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.739+0000 7f5fdc7bc640 1 -- start start 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 0x7f5fd419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4106430 0x7f5fd419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 0x7f5fd4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4197fd0 con 0x7f5fd41975e0 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4198140 con 0x7f5fd4106430 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4198300 con 0x7f5fd40685d0 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 0x7f5fd4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 0x7f5fd4197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60270/0 (socket says 172.21.15.100:60270) 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 -- 172.21.15.100:0/1361098216 learned_addr learned my addr 172.21.15.100:0/1361098216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4106430 0x7f5fd419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fda558640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 0x7f5fd419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 -- 172.21.15.100:0/1361098216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 msgr2=0x7f5fd419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 0x7f5fd419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 -- 172.21.15.100:0/1361098216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4106430 msgr2=0x7f5fd419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4106430 0x7f5fd419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 -- 172.21.15.100:0/1361098216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009900 con 0x7f5fd41975e0 2024-03-12T05:18:25.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fda558640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 0x7f5fd419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:18:25.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdad59640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 0x7f5fd4197a90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc004af0 tx=0x7f5fcc034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fcb7fe640 1 -- 172.21.15.100:0/1361098216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0328c0 con 0x7f5fd41975e0 2024-03-12T05:18:25.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fcb7fe640 1 -- 172.21.15.100:0/1361098216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc003e50 con 0x7f5fd41975e0 2024-03-12T05:18:25.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd41a42f0 con 0x7f5fd41975e0 2024-03-12T05:18:25.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fcb7fe640 1 -- 172.21.15.100:0/1361098216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc004860 con 0x7f5fd41975e0 2024-03-12T05:18:25.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.740+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd41a4770 con 0x7f5fd41975e0 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.743+0000 7f5fcb7fe640 1 -- 172.21.15.100:0/1361098216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5fcc00a040 con 0x7f5fd41975e0 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.743+0000 7f5fdf391740 1 Processor -- start 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.743+0000 7f5fdf391740 1 -- start start 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.743+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0dfbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdf391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0d5ef90 con 0x560cd0df0380 2024-03-12T05:18:25.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdf391740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0d5c4c0 con 0x560cd0df0d30 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdf391740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0d57ca0 con 0x560cd0df98b0 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fcb7fe640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5fc00a1ba0 0x7f5fc00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60284/0 (socket says 172.21.15.100:60284) 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fda558640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 -- 172.21.15.100:0/252011185 learned_addr learned my addr 172.21.15.100:0/252011185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdad59640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5fc00a1ba0 0x7f5fc00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0dfbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fcb7fe640 1 -- 172.21.15.100:0/1361098216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5fcc0f0480 con 0x7f5fd41975e0 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 -- 172.21.15.100:0/252011185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 msgr2=0x560cd0dfbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0dfbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 -- 172.21.15.100:0/252011185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 msgr2=0x560cd0df92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 -- 172.21.15.100:0/252011185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cd0d1a0b0 con 0x560cd0df0380 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fda558640 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df0760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009f90 tx=0x7f5fd00098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700a8c1048baa971 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fdad59640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5fc00a1ba0 0x7f5fc00a4060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0edd80 tx=0x7f5fcc0b2880 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fc8ff9640 1 -- 172.21.15.100:0/252011185 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0004250 con 0x560cd0df0380 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fc8ff9640 1 -- 172.21.15.100:0/252011185 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd00043b0 con 0x560cd0df0380 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.744+0000 7f5fc8ff9640 1 -- 172.21.15.100:0/252011185 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011890 con 0x560cd0df0380 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 -- 172.21.15.100:0/252011185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 msgr2=0x560cd0df0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df0760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009f90 tx=0x7f5fd00098f0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 -- 172.21.15.100:0/252011185 shutdown_connections 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0dfbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 --2- 172.21.15.100:0/252011185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df0760 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 -- 172.21.15.100:0/252011185 >> 172.21.15.100:0/252011185 conn(0x560cd0dea840 msgr2=0x560cd0deac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 -- 172.21.15.100:0/252011185 shutdown_connections 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.745+0000 7f5fdf391740 1 -- 172.21.15.100:0/252011185 wait complete. 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 -- start start 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 Processor -- start 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0cfcf40 con 0x560cd0df0380 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0a2b4b0 con 0x560cd0df0d30 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0cdd670 con 0x560cd0df98b0 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fdf391740 10 client.0 _ll_register_callbacks cb 0x560cd0d4af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60288/0 (socket says 172.21.15.100:60288) 2024-03-12T05:18:25.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 -- 172.21.15.100:0/225582841 learned_addr learned my addr 172.21.15.100:0/225582841 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fda558640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 -- 172.21.15.100:0/225582841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 msgr2=0x560cd0df72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 -- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 msgr2=0x560cd0df6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 -- 172.21.15.100:0/225582841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd0009cb0 con 0x560cd0df0380 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df6820 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0011570 tx=0x7f5fd0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44723792bb7554a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.746+0000 7f5fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fbdffb640 1 -- 172.21.15.100:0/225582841 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00035f0 con 0x560cd0df0380 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fbdffb640 1 -- 172.21.15.100:0/225582841 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5fd0003750 con 0x560cd0df0380 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cd0d31f40 con 0x560cd0df0380 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fbdffb640 1 -- 172.21.15.100:0/225582841 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd0003910 con 0x560cd0df0380 2024-03-12T05:18:25.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 10 client.16860 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560cd0d56320 con 0x560cd0df0380 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fbdffb640 1 -- 172.21.15.100:0/225582841 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560cd0d31f40 con 0x560cd0df0380 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fbdffb640 1 -- 172.21.15.100:0/225582841 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fd0011610 con 0x560cd0df0380 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 10 client.16860 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 20 client.16860 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e09c90 0x560cd0e0c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 4 client.16860 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cd0d4e310 con 0x560cd0e09c90 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.747+0000 7f5fda558640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e09c90 0x560cd0e0c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fbdffb640 10 client.16860 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fda558640 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e09c90 0x560cd0e0c0b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa19e4d78b9a3f78 server_cookie=c3feb190fc7fe072 in_seq=0 out_seq=0 2024-03-12T05:18:25.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fbdffb640 1 -- 172.21.15.100:0/225582841 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cd0d4e310 con 0x560cd0e09c90 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fbdffb640 10 client.16860 handle_command_reply: tid=0 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 client.16860 shutdown 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 msgr2=0x560cd0df6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df6820 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0011570 tx=0x7f5fd0005d50 comp rx=0 tx=0).stop 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 shutdown_connections 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e09c90 0x560cd0e0c0b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fbdffb640 0 client.16860 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 --2- 172.21.15.100:0/225582841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df6820 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.748+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 >> 172.21.15.100:0/225582841 conn(0x560cd0dea840 msgr2=0x560cd0dec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.749+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 shutdown_connections 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.749+0000 7f5fdf391740 1 -- 172.21.15.100:0/225582841 wait complete. 2024-03-12T05:18:25.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.749+0000 7f5fdf391740 20 client.16860 trim_cache size 0 max 16384 2024-03-12T05:18:25.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 Processor -- start 2024-03-12T05:18:25.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 -- start start 2024-03-12T05:18:25.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0df77f0 con 0x560cd0df0380 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0e06900 con 0x560cd0df0d30 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fdf391740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0e09a90 con 0x560cd0df98b0 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60294/0 (socket says 172.21.15.100:60294) 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 -- 172.21.15.100:0/1501998884 learned_addr learned my addr 172.21.15.100:0/1501998884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fda558640 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 -- 172.21.15.100:0/1501998884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 msgr2=0x560cd0df3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 -- 172.21.15.100:0/1501998884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 msgr2=0x560cd0df3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 -- 172.21.15.100:0/1501998884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cd0cf2c00 con 0x560cd0df0380 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df8f50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00059f0 tx=0x7f5fd0014f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9707878002861176 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fbffff640 1 -- 172.21.15.100:0/1501998884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0009ed0 con 0x560cd0df0380 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fbffff640 1 -- 172.21.15.100:0/1501998884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd0005390 con 0x560cd0df0380 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.764+0000 7f5fbffff640 1 -- 172.21.15.100:0/1501998884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0003d30 con 0x560cd0df0380 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 -- 172.21.15.100:0/1501998884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 msgr2=0x560cd0df8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df8f50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00059f0 tx=0x7f5fd0014f60 comp rx=0 tx=0).stop 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 -- 172.21.15.100:0/1501998884 shutdown_connections 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0df3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0d30 0x560cd0df3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1501998884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0380 0x560cd0df8f50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 -- 172.21.15.100:0/1501998884 >> 172.21.15.100:0/1501998884 conn(0x560cd0dea840 msgr2=0x560cd0df50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 -- 172.21.15.100:0/1501998884 shutdown_connections 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 -- 172.21.15.100:0/1501998884 wait complete. 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.765+0000 7f5fdf391740 1 -- start start 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 Processor -- start 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0380 0x560cd0dffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0d30 0x560cd0e001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0e5a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0e5a800 con 0x560cd0df0d30 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0e5a940 con 0x560cd0df0380 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cd0e5ab00 con 0x560cd0df98b0 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fdf391740 10 client.0 _ll_register_callbacks cb 0x7f5fd00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0380 0x560cd0dffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0d30 0x560cd0e001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fd9d57640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0380 0x560cd0dffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59780/0 (socket says 172.21.15.100:59780) 2024-03-12T05:18:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fd9d57640 1 -- 172.21.15.100:0/1457794625 learned_addr learned my addr 172.21.15.100:0/1457794625 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0e5a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 -- 172.21.15.100:0/1457794625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 msgr2=0x560cd0e5a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0e5a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 -- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0380 msgr2=0x560cd0dffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0380 0x560cd0dffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 -- 172.21.15.100:0/1457794625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cd0d56320 con 0x560cd0df0d30 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fda558640 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0d30 0x560cd0e001c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x560cd0df6080 tx=0x7f5fc401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db770816d98f8d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.766+0000 7f5fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fbdffb640 1 -- 172.21.15.100:0/1457794625 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4017040 con 0x560cd0df0d30 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fbdffb640 1 -- 172.21.15.100:0/1457794625 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5fc4011c80 con 0x560cd0df0d30 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cd0d7d900 con 0x560cd0df0d30 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fbdffb640 1 -- 172.21.15.100:0/1457794625 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc4011e40 con 0x560cd0df0d30 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 10 client.16866 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560cd0d18910 con 0x560cd0df0d30 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fbdffb640 1 -- 172.21.15.100:0/1457794625 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc4030080 con 0x560cd0df0d30 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fbdffb640 1 -- 172.21.15.100:0/1457794625 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fc4024430 con 0x560cd0df0d30 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 10 client.16866 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 20 client.16866 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e62460 0x560cd0e64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 4 client.16866 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560cd0a57860 con 0x560cd0e62460 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.767+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e62460 0x560cd0e64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.768+0000 7f5fd9d57640 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e62460 0x560cd0e64880 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa7154ac6323cdc server_cookie=8887941fd7ff4731 in_seq=0 out_seq=0 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.768+0000 7f5fbdffb640 10 client.16866 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fbdffb640 1 -- 172.21.15.100:0/1457794625 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560cd0a57860 con 0x560cd0e62460 2024-03-12T05:18:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fbdffb640 10 client.16866 handle_command_reply: tid=0 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 client.16866 shutdown 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0d30 msgr2=0x560cd0e001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0d30 0x560cd0e001c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x560cd0df6080 tx=0x7f5fc401c740 comp rx=0 tx=0).stop 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 shutdown_connections 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cd0e62460 0x560cd0e64880 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd0df98b0 0x560cd0e5a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cd0df0d30 0x560cd0e001c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 --2- 172.21.15.100:0/1457794625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cd0df0380 0x560cd0dffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 >> 172.21.15.100:0/1457794625 conn(0x560cd0dea840 msgr2=0x560cd0df5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fbdffb640 0 client.16866 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 shutdown_connections 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 1 -- 172.21.15.100:0/1457794625 wait complete. 2024-03-12T05:18:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdf391740 20 client.16866 trim_cache size 0 max 16384 2024-03-12T05:18:25.772 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:18:25.772 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:18:25.772 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "cdd0ad1d-52d4-40fe-a61b-c46bc8841a33", 2024-03-12T05:18:25.772 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:18:25.772 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5fc00a1ba0 msgr2=0x7f5fc00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5fc00a1ba0 0x7f5fc00a4060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0edd80 tx=0x7f5fcc0b2880 comp rx=0 tx=0).stop 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 msgr2=0x7f5fd4197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 0x7f5fd4197a90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc004af0 tx=0x7f5fcc034cc0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 shutdown_connections 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5fc00a1ba0 0x7f5fc00a4060 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd41975e0 0x7f5fd4197a90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4106430 0x7f5fd419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 --2- 172.21.15.100:0/1361098216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40685d0 0x7f5fd419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.769+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 >> 172.21.15.100:0/1361098216 conn(0x7f5fd4077180 msgr2=0x7f5fd40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.770+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 shutdown_connections 2024-03-12T05:18:25.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.770+0000 7f5fdc7bc640 1 -- 172.21.15.100:0/1361098216 wait complete. 2024-03-12T05:18:25.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:18:25.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:18:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 735509 12.28 MB/sec execute 189 sec latency 16.347 ms 2024-03-12T05:18:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 -- 172.21.15.100:0/3680190865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38006a190 msgr2=0x7fe38010dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 --2- 172.21.15.100:0/3680190865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38006a190 0x7fe38010dfc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe378009ca0 tx=0x7fe37802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 -- 172.21.15.100:0/3680190865 shutdown_connections 2024-03-12T05:18:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 --2- 172.21.15.100:0/3680190865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38006a190 0x7fe38010dfc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 --2- 172.21.15.100:0/3680190865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3800697f0 0x7fe380069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 --2- 172.21.15.100:0/3680190865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe380107290 0x7fe380107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 -- 172.21.15.100:0/3680190865 >> 172.21.15.100:0/3680190865 conn(0x7fe380076e60 msgr2=0x7fe380077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 -- 172.21.15.100:0/3680190865 shutdown_connections 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.889+0000 7fe387b25640 1 -- 172.21.15.100:0/3680190865 wait complete. 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 Processor -- start 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 -- start start 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3800697f0 0x7fe3801976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe38006a190 0x7fe380197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 0x7fe38019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38019c550 con 0x7fe380107290 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38019c6c0 con 0x7fe38006a190 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe387b25640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38019c880 con 0x7fe3800697f0 2024-03-12T05:18:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe3860c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 0x7fe38019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe3860c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 0x7fe38019bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60326/0 (socket says 172.21.15.100:60326) 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe3860c2640 1 -- 172.21.15.100:0/4195030280 learned_addr learned my addr 172.21.15.100:0/4195030280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.890+0000 7fe3850c0640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe38006a190 0x7fe380197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3858c1640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3800697f0 0x7fe3801976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3860c2640 1 -- 172.21.15.100:0/4195030280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3800697f0 msgr2=0x7fe3801976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3860c2640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3800697f0 0x7fe3801976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3860c2640 1 -- 172.21.15.100:0/4195030280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe38006a190 msgr2=0x7fe380197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3860c2640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe38006a190 0x7fe380197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3860c2640 1 -- 172.21.15.100:0/4195030280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe378009900 con 0x7fe380107290 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3850c0640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe38006a190 0x7fe380197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe3860c2640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 0x7fe38019bf50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe378005e30 tx=0x7fe378035aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe376ffd640 1 -- 172.21.15.100:0/4195030280 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378035ea0 con 0x7fe380107290 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe376ffd640 1 -- 172.21.15.100:0/4195030280 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe378003e50 con 0x7fe380107290 2024-03-12T05:18:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38019cb00 con 0x7fe380107290 2024-03-12T05:18:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe376ffd640 1 -- 172.21.15.100:0/4195030280 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004830 con 0x7fe380107290 2024-03-12T05:18:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.891+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38019cdd0 con 0x7fe380107290 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.893+0000 7fe376ffd640 1 -- 172.21.15.100:0/4195030280 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe37803b040 con 0x7fe380107290 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 Processor -- start 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 -- start start 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe376ffd640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe36c0a1ba0 0x7fe36c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e6fe0 0x562ee74e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee7456280 con 0x562ee74f4bc0 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee74537b0 con 0x562ee74e6fe0 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee744ef90 con 0x562ee74e7900 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe3850c0640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe36c0a1ba0 0x7fe36c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe3860c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e6fe0 0x562ee74e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe376ffd640 1 -- 172.21.15.100:0/4195030280 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe3780f0010 con 0x7fe380107290 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe3850c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe3860c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60332/0 (socket says 172.21.15.100:60332) 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.894+0000 7fe3860c2640 1 -- 172.21.15.100:0/660603972 learned_addr learned my addr 172.21.15.100:0/660603972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3860c2640 1 -- 172.21.15.100:0/660603972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 msgr2=0x562ee74f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3860c2640 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3860c2640 1 -- 172.21.15.100:0/660603972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e6fe0 msgr2=0x562ee74e73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3860c2640 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e6fe0 0x562ee74e73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3860c2640 1 -- 172.21.15.100:0/660603972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee740d570 con 0x562ee74f4bc0 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3858c1640 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e6fe0 0x562ee74e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3850c0640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe36c0a1ba0 0x7fe36c0a4060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe370009730 tx=0x7fe370009ed0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe3860c2640 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f6fb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe3780b91a0 tx=0x7fe3780b90a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c4231060a5a346 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe36bfff640 1 -- 172.21.15.100:0/660603972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780edea0 con 0x562ee74f4bc0 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe36bfff640 1 -- 172.21.15.100:0/660603972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3780bd440 con 0x562ee74f4bc0 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.895+0000 7fe36bfff640 1 -- 172.21.15.100:0/660603972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004040 con 0x562ee74f4bc0 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 -- 172.21.15.100:0/660603972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 msgr2=0x562ee74f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f6fb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe3780b91a0 tx=0x7fe3780b90a0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 -- 172.21.15.100:0/660603972 shutdown_connections 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f6fb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/660603972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e6fe0 0x562ee74e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 -- 172.21.15.100:0/660603972 >> 172.21.15.100:0/660603972 conn(0x562ee74e1990 msgr2=0x562ee74e1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 -- 172.21.15.100:0/660603972 shutdown_connections 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 -- 172.21.15.100:0/660603972 wait complete. 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 -- start start 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.896+0000 7fe38a7d2740 1 Processor -- start 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74f4bc0 0x562ee74f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee7122430 con 0x562ee74e6fe0 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee70dee00 con 0x562ee74f4bc0 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee73d3fd0 con 0x562ee74e7900 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 10 client.0 _ll_register_callbacks cb 0x562ee7429fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:25.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60342/0 (socket says 172.21.15.100:60342) 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 -- 172.21.15.100:0/753280120 learned_addr learned my addr 172.21.15.100:0/753280120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3860c2640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74f4bc0 0x562ee74f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3850c0640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 -- 172.21.15.100:0/753280120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 msgr2=0x562ee74f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 -- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74f4bc0 msgr2=0x562ee74f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74f4bc0 0x562ee74f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 -- 172.21.15.100:0/753280120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38019cdd0 con 0x562ee74e6fe0 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3858c1640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f45b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe37c012840 tx=0x7fe37c012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31e1da42fe2426 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3697fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:25.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3697fa640 1 -- 172.21.15.100:0/753280120 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c017070 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3697fa640 1 -- 172.21.15.100:0/753280120 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe37c004050 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ee714e7e0 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe3697fa640 1 -- 172.21.15.100:0/753280120 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe37c004210 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 10 client.16878 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.897+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ee744d610 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe3697fa640 1 -- 172.21.15.100:0/753280120 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ee714e7e0 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe3697fa640 1 -- 172.21.15.100:0/753280120 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe37c003a30 con 0x562ee74e6fe0 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe38a7d2740 10 client.16878 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe38a7d2740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe38a7d2740 20 client.16878 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee7507150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe38a7d2740 4 client.16878 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ee718b720 con 0x562ee7504d30 2024-03-12T05:18:25.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe3850c0640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee7507150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe3697fa640 10 client.16878 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.898+0000 7fe3850c0640 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee7507150 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5708830483828cac server_cookie=7ea2aef5da903f40 in_seq=0 out_seq=0 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe3697fa640 1 -- 172.21.15.100:0/753280120 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ee718b720 con 0x562ee7504d30 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe3697fa640 10 client.16878 handle_command_reply: tid=0 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 client.16878 shutdown 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 msgr2=0x562ee74f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f45b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe37c012840 tx=0x7fe37c012d00 comp rx=0 tx=0).stop 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 shutdown_connections 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee7507150 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74f4bc0 0x562ee74f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e7900 0x562ee74f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe3697fa640 0 client.16878 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/753280120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f45b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 >> 172.21.15.100:0/753280120 conn(0x562ee74e1990 msgr2=0x562ee74e3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 shutdown_connections 2024-03-12T05:18:25.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 1 -- 172.21.15.100:0/753280120 wait complete. 2024-03-12T05:18:25.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.899+0000 7fe38a7d2740 20 client.16878 trim_cache size 0 max 16384 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 Processor -- start 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 -- start start 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74f4bc0 0x562ee74f0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee7501110 con 0x562ee74e6fe0 2024-03-12T05:18:25.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee74dd0f0 con 0x562ee74e7900 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee74dd320 con 0x562ee74f4bc0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60358/0 (socket says 172.21.15.100:60358) 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe3858c1640 1 -- 172.21.15.100:0/4241850137 learned_addr learned my addr 172.21.15.100:0/4241850137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.914+0000 7fe3850c0640 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3860c2640 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74f4bc0 0x562ee74f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3858c1640 1 -- 172.21.15.100:0/4241850137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74f4bc0 msgr2=0x562ee74f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3858c1640 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74f4bc0 0x562ee74f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3858c1640 1 -- 172.21.15.100:0/4241850137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 msgr2=0x562ee74f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3858c1640 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3858c1640 1 -- 172.21.15.100:0/4241850137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee73e9ac0 con 0x562ee74e6fe0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe3858c1640 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f4070 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009ee0 tx=0x7fe37c0123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdf068f98d6127e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe36b7fe640 1 -- 172.21.15.100:0/4241850137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c017040 con 0x562ee74e6fe0 2024-03-12T05:18:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe36b7fe640 1 -- 172.21.15.100:0/4241850137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37c0104f0 con 0x562ee74e6fe0 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe36b7fe640 1 -- 172.21.15.100:0/4241850137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00f790 con 0x562ee74e6fe0 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.915+0000 7fe38a7d2740 1 -- 172.21.15.100:0/4241850137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 msgr2=0x562ee74f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f4070 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009ee0 tx=0x7fe37c0123a0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- 172.21.15.100:0/4241850137 shutdown_connections 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74f4bc0 0x562ee74f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/4241850137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74e6fe0 0x562ee74f4070 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- 172.21.15.100:0/4241850137 >> 172.21.15.100:0/4241850137 conn(0x562ee74e1990 msgr2=0x562ee74ecbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- 172.21.15.100:0/4241850137 shutdown_connections 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- 172.21.15.100:0/4241850137 wait complete. 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- start start 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 Processor -- start 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e6fe0 0x562ee74fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee74f8ae0 con 0x562ee74f4bc0 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee74f8c50 con 0x562ee74e7900 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee74f8e10 con 0x562ee74e6fe0 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.916+0000 7fe38a7d2740 10 client.0 _ll_register_callbacks cb 0x7fe37c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e6fe0 0x562ee74fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3850c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60360/0 (socket says 172.21.15.100:60360) 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3850c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59850/0 (socket says 172.21.15.100:59850) 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3858c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e6fe0 0x562ee74fbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51582/0 (socket says 172.21.15.100:51582) 2024-03-12T05:18:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 -- 172.21.15.100:0/390183998 learned_addr learned my addr 172.21.15.100:0/390183998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 -- 172.21.15.100:0/390183998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e6fe0 msgr2=0x562ee74fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e6fe0 0x562ee74fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 -- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 msgr2=0x562ee74f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 -- 172.21.15.100:0/390183998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ee744d610 con 0x562ee74f4bc0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3860c2640 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f84d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe3780bca40 tx=0x7fe3780bd5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea4d602752aee5f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3697fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3697fa640 1 -- 172.21.15.100:0/390183998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780edea0 con 0x562ee74f4bc0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3697fa640 1 -- 172.21.15.100:0/390183998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe3780c54e0 con 0x562ee74f4bc0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3780bd440 con 0x562ee74f4bc0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3697fa640 1 -- 172.21.15.100:0/390183998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3780bcd40 con 0x562ee74f4bc0 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe38a7d2740 10 client.16884 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ee740f6a0 con 0x562ee74f4bc0 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.917+0000 7fe3697fa640 1 -- 172.21.15.100:0/390183998 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3780bd440 con 0x562ee74f4bc0 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe3697fa640 1 -- 172.21.15.100:0/390183998 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe3780bd0a0 con 0x562ee74f4bc0 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 10 client.16884 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 20 client.16884 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee755e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 4 client.16884 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ee71da890 con 0x562ee7504d30 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe3858c1640 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee755e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe3858c1640 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee755e570 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2925c999bb168060 server_cookie=c8c88170a4355a76 in_seq=0 out_seq=0 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe3697fa640 10 client.16884 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe3697fa640 1 -- 172.21.15.100:0/390183998 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562ee71da890 con 0x562ee7504d30 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe3697fa640 10 client.16884 handle_command_reply: tid=0 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 1 client.16884 shutdown 2024-03-12T05:18:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 msgr2=0x562ee74f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.918+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f84d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe3780bca40 tx=0x7fe3780bd5b0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 shutdown_connections 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562ee7504d30 0x562ee755e570 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee74f4bc0 0x562ee74f84d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ee74e7900 0x562ee74f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe3697fa640 0 client.16884 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 --2- 172.21.15.100:0/390183998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ee74e6fe0 0x562ee74fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 >> 172.21.15.100:0/390183998 conn(0x562ee74e1990 msgr2=0x562ee74f01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 shutdown_connections 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 1 -- 172.21.15.100:0/390183998 wait complete. 2024-03-12T05:18:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe38a7d2740 20 client.16884 trim_cache size 0 max 16384 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: "cdd0ad1d-52d4-40fe-a61b-c46bc8841a33": { 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "cdd0ad1d-52d4-40fe-a61b-c46bc8841a33", 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:18:25.922 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:18:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe36c0a1ba0 msgr2=0x7fe36c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe36c0a1ba0 0x7fe36c0a4060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe370009730 tx=0x7fe370009ed0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 msgr2=0x7fe38019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 0x7fe38019bf50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe378005e30 tx=0x7fe378035aa0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 shutdown_connections 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe36c0a1ba0 0x7fe36c0a4060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe380107290 0x7fe38019bf50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe38006a190 0x7fe380197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 --2- 172.21.15.100:0/4195030280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3800697f0 0x7fe3801976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 >> 172.21.15.100:0/4195030280 conn(0x7fe380076e60 msgr2=0x7fe3800fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 shutdown_connections 2024-03-12T05:18:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:25.919+0000 7fe387b25640 1 -- 172.21.15.100:0/4195030280 wait complete. 2024-03-12T05:18:25.925 INFO:tasks.cephfs.filesystem:scrub status for tag:cdd0ad1d-52d4-40fe-a61b-c46bc8841a33 - {'path': '/', 'tag': 'cdd0ad1d-52d4-40fe-a61b-c46bc8841a33', 'options': 'recursive,force'} 2024-03-12T05:18:26.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 737645 12.26 MB/sec execute 190 sec latency 27.025 ms 2024-03-12T05:18:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:18:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:18:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:18:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.204 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:26 smithi156 ceph-mon[36270]: pgmap v193: 97 pgs: 97 active+clean; 385 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 211 op/s 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:26 smithi161 ceph-mon[36043]: pgmap v193: 97 pgs: 97 active+clean; 385 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 211 op/s 2024-03-12T05:18:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:18:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:18:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:18:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:27.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:26 smithi100 ceph-mon[32210]: pgmap v193: 97 pgs: 97 active+clean; 385 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 211 op/s 2024-03-12T05:18:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: 1 739807 12.27 MB/sec execute 191 sec latency 16.127 ms 2024-03-12T05:18:28.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 742415 12.29 MB/sec execute 192 sec latency 17.990 ms 2024-03-12T05:18:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:28 smithi156 ceph-mon[36270]: pgmap v194: 97 pgs: 97 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-03-12T05:18:29.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:28 smithi161 ceph-mon[36043]: pgmap v194: 97 pgs: 97 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-03-12T05:18:29.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:28 smithi100 ceph-mon[32210]: pgmap v194: 97 pgs: 97 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-03-12T05:18:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 743820 12.26 MB/sec execute 193 sec latency 107.096 ms 2024-03-12T05:18:30.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 746223 12.26 MB/sec execute 194 sec latency 21.659 ms 2024-03-12T05:18:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:30 smithi156 ceph-mon[36270]: pgmap v195: 97 pgs: 97 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-12T05:18:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:30 smithi161 ceph-mon[36043]: pgmap v195: 97 pgs: 97 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-12T05:18:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:30 smithi100 ceph-mon[32210]: pgmap v195: 97 pgs: 97 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-12T05:18:31.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 748674 12.27 MB/sec execute 195 sec latency 16.424 ms 2024-03-12T05:18:32.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 750712 12.26 MB/sec execute 196 sec latency 19.060 ms 2024-03-12T05:18:33.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:32 smithi156 ceph-mon[36270]: pgmap v196: 97 pgs: 97 active+clean; 398 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-03-12T05:18:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:32 smithi161 ceph-mon[36043]: pgmap v196: 97 pgs: 97 active+clean; 398 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-03-12T05:18:33.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:32 smithi100 ceph-mon[32210]: pgmap v196: 97 pgs: 97 active+clean; 398 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-03-12T05:18:33.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 753225 12.27 MB/sec execute 197 sec latency 17.482 ms 2024-03-12T05:18:34.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 755194 12.25 MB/sec execute 198 sec latency 81.834 ms 2024-03-12T05:18:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:34 smithi156 ceph-mon[36270]: pgmap v197: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 232 op/s 2024-03-12T05:18:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:34 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:18:35.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:34 smithi161 ceph-mon[36043]: pgmap v197: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 232 op/s 2024-03-12T05:18:35.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:34 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:18:35.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:34 smithi100 ceph-mon[32210]: pgmap v197: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 232 op/s 2024-03-12T05:18:35.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:34 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:18:35.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 757267 12.25 MB/sec execute 199 sec latency 23.973 ms 2024-03-12T05:18:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:35 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:18:36.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:35 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:18:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:35 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:18:36.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 759816 12.27 MB/sec execute 200 sec latency 67.011 ms 2024-03-12T05:18:37.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:36 smithi156 ceph-mon[36270]: pgmap v198: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-03-12T05:18:37.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:36 smithi161 ceph-mon[36043]: pgmap v198: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-03-12T05:18:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:36 smithi100 ceph-mon[32210]: pgmap v198: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-03-12T05:18:37.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 761642 12.24 MB/sec execute 201 sec latency 17.198 ms 2024-03-12T05:18:38.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:37 smithi156 ceph-mon[36270]: pgmap v199: 97 pgs: 97 active+clean; 411 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 308 op/s 2024-03-12T05:18:38.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:37 smithi161 ceph-mon[36043]: pgmap v199: 97 pgs: 97 active+clean; 411 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 308 op/s 2024-03-12T05:18:38.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:37 smithi100 ceph-mon[32210]: pgmap v199: 97 pgs: 97 active+clean; 411 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 308 op/s 2024-03-12T05:18:38.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 764003 12.25 MB/sec execute 202 sec latency 13.220 ms 2024-03-12T05:18:39.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 766302 12.25 MB/sec execute 203 sec latency 58.302 ms 2024-03-12T05:18:40.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 768382 12.25 MB/sec execute 204 sec latency 20.116 ms 2024-03-12T05:18:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:40 smithi156 ceph-mon[36270]: pgmap v200: 97 pgs: 97 active+clean; 411 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 203 op/s 2024-03-12T05:18:41.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:40 smithi161 ceph-mon[36043]: pgmap v200: 97 pgs: 97 active+clean; 411 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 203 op/s 2024-03-12T05:18:41.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:40 smithi100 ceph-mon[32210]: pgmap v200: 97 pgs: 97 active+clean; 411 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 203 op/s 2024-03-12T05:18:41.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 770909 12.26 MB/sec execute 205 sec latency 12.447 ms 2024-03-12T05:18:42.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 773070 12.24 MB/sec execute 206 sec latency 21.207 ms 2024-03-12T05:18:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:42 smithi156 ceph-mon[36270]: pgmap v201: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.9 MiB/s wr, 302 op/s 2024-03-12T05:18:43.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:42 smithi161 ceph-mon[36043]: pgmap v201: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.9 MiB/s wr, 302 op/s 2024-03-12T05:18:43.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:42 smithi100 ceph-mon[32210]: pgmap v201: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.9 MiB/s wr, 302 op/s 2024-03-12T05:18:43.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 775231 12.24 MB/sec execute 207 sec latency 38.332 ms 2024-03-12T05:18:44.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 777724 12.26 MB/sec execute 208 sec latency 84.804 ms 2024-03-12T05:18:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:44 smithi156 ceph-mon[36270]: pgmap v202: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-03-12T05:18:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:44 smithi161 ceph-mon[36043]: pgmap v202: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-03-12T05:18:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:44 smithi100 ceph-mon[32210]: pgmap v202: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-03-12T05:18:45.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 779249 12.23 MB/sec execute 209 sec latency 133.103 ms 2024-03-12T05:18:46.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 781738 12.24 MB/sec execute 210 sec latency 17.073 ms 2024-03-12T05:18:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:46 smithi156 ceph-mon[36270]: pgmap v203: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-03-12T05:18:47.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:46 smithi161 ceph-mon[36043]: pgmap v203: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-03-12T05:18:47.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:46 smithi100 ceph-mon[32210]: pgmap v203: 97 pgs: 97 active+clean; 419 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-03-12T05:18:47.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 784326 12.24 MB/sec execute 211 sec latency 29.888 ms 2024-03-12T05:18:48.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 786200 12.24 MB/sec execute 212 sec latency 16.101 ms 2024-03-12T05:18:49.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:48 smithi156 ceph-mon[36270]: pgmap v204: 97 pgs: 97 active+clean; 427 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 303 op/s 2024-03-12T05:18:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:48 smithi161 ceph-mon[36043]: pgmap v204: 97 pgs: 97 active+clean; 427 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 303 op/s 2024-03-12T05:18:49.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:48 smithi100 ceph-mon[32210]: pgmap v204: 97 pgs: 97 active+clean; 427 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 303 op/s 2024-03-12T05:18:49.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 788661 12.25 MB/sec execute 213 sec latency 15.757 ms 2024-03-12T05:18:50.794 INFO:tasks.workunit.client.0.smithi100.stdout: 1 790576 12.22 MB/sec execute 214 sec latency 76.419 ms 2024-03-12T05:18:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:50 smithi156 ceph-mon[36270]: pgmap v205: 97 pgs: 97 active+clean; 427 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-03-12T05:18:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:50 smithi161 ceph-mon[36043]: pgmap v205: 97 pgs: 97 active+clean; 427 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-03-12T05:18:51.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:50 smithi100 ceph-mon[32210]: pgmap v205: 97 pgs: 97 active+clean; 427 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-03-12T05:18:51.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 792681 12.23 MB/sec execute 215 sec latency 24.129 ms 2024-03-12T05:18:52.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 795475 12.25 MB/sec execute 216 sec latency 13.241 ms 2024-03-12T05:18:53.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:52 smithi156 ceph-mon[36270]: pgmap v206: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 305 op/s 2024-03-12T05:18:53.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:52 smithi161 ceph-mon[36043]: pgmap v206: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 305 op/s 2024-03-12T05:18:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:52 smithi100 ceph-mon[32210]: pgmap v206: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 305 op/s 2024-03-12T05:18:53.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 797301 12.22 MB/sec execute 217 sec latency 16.769 ms 2024-03-12T05:18:54.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 799653 12.23 MB/sec execute 218 sec latency 13.657 ms 2024-03-12T05:18:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:54 smithi156 ceph-mon[36270]: pgmap v207: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-03-12T05:18:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:54 smithi161 ceph-mon[36043]: pgmap v207: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-03-12T05:18:55.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:54 smithi100 ceph-mon[32210]: pgmap v207: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-03-12T05:18:55.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 801843 12.23 MB/sec execute 219 sec latency 100.852 ms 2024-03-12T05:18:55.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:18:55.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:18:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 -- 172.21.15.100:0/796964922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec102cc0 msgr2=0x7f97ec10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 --2- 172.21.15.100:0/796964922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec102cc0 0x7f97ec10b1a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 -- 172.21.15.100:0/796964922 shutdown_connections 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 --2- 172.21.15.100:0/796964922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec10b6e0 0x7f97ec10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 --2- 172.21.15.100:0/796964922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec102cc0 0x7f97ec10b1a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 --2- 172.21.15.100:0/796964922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec1023a0 0x7f97ec102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 -- 172.21.15.100:0/796964922 >> 172.21.15.100:0/796964922 conn(0x7f97ec077230 msgr2=0x7f97ec077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 -- 172.21.15.100:0/796964922 shutdown_connections 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.041+0000 7f97f26c0640 1 -- 172.21.15.100:0/796964922 wait complete. 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 Processor -- start 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 -- start start 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec1023a0 0x7f97ec0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec102cc0 0x7f97ec100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 0x7f97ec100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1a5bd0 con 0x7f97ec10b6e0 2024-03-12T05:18:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1a5d40 con 0x7f97ec1023a0 2024-03-12T05:18:56.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f26c0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1a5f00 con 0x7f97ec102cc0 2024-03-12T05:18:56.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f0c5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 0x7f97ec100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec1023a0 0x7f97ec0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f0c5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 0x7f97ec100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34518/0 (socket says 172.21.15.100:34518) 2024-03-12T05:18:56.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3004616448 learned_addr learned my addr 172.21.15.100:0/3004616448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec102cc0 0x7f97ec100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3004616448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec102cc0 msgr2=0x7f97ec100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec102cc0 0x7f97ec100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.042+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3004616448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec1023a0 msgr2=0x7f97ec0ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec1023a0 0x7f97ec0ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3004616448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009560 con 0x7f97ec10b6e0 2024-03-12T05:18:56.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97ebfff640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec1023a0 0x7f97ec0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 0x7f97ec100a70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f97e400eb10 tx=0x7f97e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97e97fa640 1 -- 172.21.15.100:0/3004616448 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4004b20 con 0x7f97ec10b6e0 2024-03-12T05:18:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97e97fa640 1 -- 172.21.15.100:0/3004616448 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97e4004c80 con 0x7f97ec10b6e0 2024-03-12T05:18:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec1a61e0 con 0x7f97ec10b6e0 2024-03-12T05:18:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97e97fa640 1 -- 172.21.15.100:0/3004616448 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4010650 con 0x7f97ec10b6e0 2024-03-12T05:18:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.043+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec1a66c0 con 0x7f97ec10b6e0 2024-03-12T05:18:56.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.045+0000 7f97e97fa640 1 -- 172.21.15.100:0/3004616448 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f97e4005020 con 0x7f97ec10b6e0 2024-03-12T05:18:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.045+0000 7f97f5369740 1 Processor -- start 2024-03-12T05:18:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.045+0000 7f97f5369740 1 -- start start 2024-03-12T05:18:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97e97fa640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97d40a1c20 0x7f97d40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f5369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe96690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f5369740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f5369740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfea1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f5369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfe05280 con 0x55e1cfe962b0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f5369740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfe027b0 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f5369740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfdfdf90 con 0x55e1cfe96bd0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe96690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97d40a1c20 0x7f97d40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe96690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34522/0 (socket says 172.21.15.100:34522) 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 -- 172.21.15.100:0/1707083860 learned_addr learned my addr 172.21.15.100:0/1707083860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfea1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97e97fa640 1 -- 172.21.15.100:0/3004616448 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f97e4014070 con 0x7f97ec10b6e0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 -- 172.21.15.100:0/1707083860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 msgr2=0x55e1cfe9f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 -- 172.21.15.100:0/1707083860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe9f7b0 msgr2=0x55e1cfea1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfea1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 -- 172.21.15.100:0/1707083860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1cfdbc570 con 0x55e1cfe962b0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97d40a1c20 0x7f97d40a40e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f97e0006010 tx=0x7f97e0004790 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.046+0000 7f97ebfff640 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe96690 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0115c0 tx=0x7f97dc011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bdee9bd8022eb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97d27fc640 1 -- 172.21.15.100:0/1707083860 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00f5c0 con 0x55e1cfe962b0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97d27fc640 1 -- 172.21.15.100:0/1707083860 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97dc00fc00 con 0x55e1cfe962b0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97d27fc640 1 -- 172.21.15.100:0/1707083860 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc019c30 con 0x55e1cfe962b0 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 -- 172.21.15.100:0/1707083860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 msgr2=0x55e1cfe96690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe96690 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0115c0 tx=0x7f97dc011a80 comp rx=0 tx=0).stop 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 -- 172.21.15.100:0/1707083860 shutdown_connections 2024-03-12T05:18:56.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfea1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 --2- 172.21.15.100:0/1707083860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe96690 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.047+0000 7f97f5369740 1 -- 172.21.15.100:0/1707083860 >> 172.21.15.100:0/1707083860 conn(0x55e1cfe90990 msgr2=0x55e1cfe90d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 -- 172.21.15.100:0/1707083860 shutdown_connections 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 -- 172.21.15.100:0/1707083860 wait complete. 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 -- start start 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 Processor -- start 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfad1430 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfa8de00 con 0x55e1cfe962b0 2024-03-12T05:18:56.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfd82fd0 con 0x55e1cfe96bd0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f5369740 10 client.0 _ll_register_callbacks cb 0x55e1cfdd8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f0c5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f0c5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34526/0 (socket says 172.21.15.100:34526) 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.048+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3158446975 learned_addr learned my addr 172.21.15.100:0/3158446975 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3158446975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 msgr2=0x55e1cfe9c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 msgr2=0x55e1cfe9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f0c5d640 1 -- 172.21.15.100:0/3158446975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc009d40 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97ebfff640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9cd00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f97e40cb9d0 tx=0x7f97e409b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10945a82b568178 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97cb7fe640 1 -- 172.21.15.100:0/3158446975 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40a3050 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97cb7fe640 1 -- 172.21.15.100:0/3158446975 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f97e4096f20 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1cfafd7e0 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97cb7fe640 1 -- 172.21.15.100:0/3158446975 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e4097100 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f5369740 10 client.16896 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e1cfdfc610 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97cb7fe640 1 -- 172.21.15.100:0/3158446975 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97e40b7020 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.049+0000 7f97cb7fe640 1 -- 172.21.15.100:0/3158446975 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f97e401f060 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97f5369740 10 client.16896 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97f5369740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97f5369740 20 client.16896 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97f5369740 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cfeafa00 0x55e1cfeb1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97f5369740 4 client.16896 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1cfb3a720 con 0x55e1cfeafa00 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97ebfff640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cfeafa00 0x55e1cfeb1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97ebfff640 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cfeafa00 0x55e1cfeb1e20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6ab5cb5be27ecf server_cookie=9ba5ad78c5ab7b2d in_seq=0 out_seq=0 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.050+0000 7f97cb7fe640 10 client.16896 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97cb7fe640 1 -- 172.21.15.100:0/3158446975 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1cfb3a720 con 0x55e1cfeafa00 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97cb7fe640 10 client.16896 handle_command_reply: tid=0 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 client.16896 shutdown 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 msgr2=0x55e1cfe9cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9cd00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f97e40cb9d0 tx=0x7f97e409b070 comp rx=0 tx=0).stop 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 shutdown_connections 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cfeafa00 0x55e1cfeb1e20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9cd00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97cb7fe640 0 client.16896 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 --2- 172.21.15.100:0/3158446975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 >> 172.21.15.100:0/3158446975 conn(0x55e1cfe90990 msgr2=0x55e1cfe92730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 shutdown_connections 2024-03-12T05:18:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 1 -- 172.21.15.100:0/3158446975 wait complete. 2024-03-12T05:18:56.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.051+0000 7f97f5369740 20 client.16896 trim_cache size 0 max 16384 2024-03-12T05:18:56.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 Processor -- start 2024-03-12T05:18:56.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 -- start start 2024-03-12T05:18:56.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfe91770 con 0x55e1cfe962b0 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfe95f10 con 0x55e1cfe96bd0 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f5369740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cfe919e0 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34528/0 (socket says 172.21.15.100:34528) 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 -- 172.21.15.100:0/1686356626 learned_addr learned my addr 172.21.15.100:0/1686356626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 -- 172.21.15.100:0/1686356626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 msgr2=0x55e1cfe9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 -- 172.21.15.100:0/1686356626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe96bd0 msgr2=0x55e1cfe9ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.067+0000 7f97ebfff640 1 -- 172.21.15.100:0/1686356626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1cfd98ac0 con 0x55e1cfe962b0 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.068+0000 7f97ebfff640 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9c560 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009380 tx=0x7f97dc0114f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388739a289958dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.068+0000 7f97d1ffb640 1 -- 172.21.15.100:0/1686356626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc029070 con 0x55e1cfe962b0 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.068+0000 7f97d1ffb640 1 -- 172.21.15.100:0/1686356626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97dc00f5c0 con 0x55e1cfe962b0 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.068+0000 7f97d1ffb640 1 -- 172.21.15.100:0/1686356626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc009be0 con 0x55e1cfe962b0 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- 172.21.15.100:0/1686356626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 msgr2=0x55e1cfe9c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9c560 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009380 tx=0x7f97dc0114f0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- 172.21.15.100:0/1686356626 shutdown_connections 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfe9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe96bd0 0x55e1cfe9ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- 172.21.15.100:0/1686356626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe962b0 0x55e1cfe9c560 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- 172.21.15.100:0/1686356626 >> 172.21.15.100:0/1686356626 conn(0x55e1cfe90990 msgr2=0x55e1cfe9be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- 172.21.15.100:0/1686356626 shutdown_connections 2024-03-12T05:18:56.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- 172.21.15.100:0/1686356626 wait complete. 2024-03-12T05:18:56.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- start start 2024-03-12T05:18:56.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 Processor -- start 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfea3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 0x55e1cfea3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfeffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cff00410 con 0x55e1cfe96bd0 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cff00580 con 0x55e1cfe962b0 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.069+0000 7f97f5369740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1cff00740 con 0x55e1cfe9f7b0 2024-03-12T05:18:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97f5369740 10 client.0 _ll_register_callbacks cb 0x7f97dc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 0x55e1cfea3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfea3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 0x55e1cfea3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34534/0 (socket says 172.21.15.100:34534) 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97eb7fe640 1 -- 172.21.15.100:0/2985816846 learned_addr learned my addr 172.21.15.100:0/2985816846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97f0c5d640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfeffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 -- 172.21.15.100:0/2985816846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 msgr2=0x55e1cfeffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfeffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 -- 172.21.15.100:0/2985816846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 msgr2=0x55e1cfea3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 0x55e1cfea3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 -- 172.21.15.100:0/2985816846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1cfdbc570 con 0x55e1cfe962b0 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 0x55e1cfea3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97ebfff640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfea3870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005890 tx=0x7f97dc00f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d370c269d5f21e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97cb7fe640 1 -- 172.21.15.100:0/2985816846 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc029070 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97dc00fc00 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97cb7fe640 1 -- 172.21.15.100:0/2985816846 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f97dc005440 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97cb7fe640 1 -- 172.21.15.100:0/2985816846 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97dc009be0 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.070+0000 7f97f5369740 10 client.26282 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e1cfdbe6a0 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97cb7fe640 1 -- 172.21.15.100:0/2985816846 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97dc00fc00 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97cb7fe640 1 -- 172.21.15.100:0/2985816846 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f97dc0038f0 con 0x55e1cfe962b0 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 10 client.26282 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 20 client.26282 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cff07c50 0x55e1cff0a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 4 client.26282 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:18:56.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e1cfb89890 con 0x55e1cff07c50 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.071+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cff07c50 0x55e1cff0a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97cb7fe640 10 client.26282 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97eb7fe640 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cff07c50 0x55e1cff0a040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6b66cbeca9f006 server_cookie=e99acfac3f08cd79 in_seq=0 out_seq=0 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97cb7fe640 1 -- 172.21.15.100:0/2985816846 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e1cfb89890 con 0x55e1cff07c50 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97cb7fe640 10 client.26282 handle_command_reply: tid=0 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 client.26282 shutdown 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 msgr2=0x55e1cfea3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfea3870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005890 tx=0x7f97dc00f9e0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 shutdown_connections 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e1cff07c50 0x55e1cff0a040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1cfe9f7b0 0x55e1cfeffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97cb7fe640 0 client.26282 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1cfe96bd0 0x55e1cfea3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 --2- 172.21.15.100:0/2985816846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1cfe962b0 0x55e1cfea3870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 >> 172.21.15.100:0/2985816846 conn(0x55e1cfe90990 msgr2=0x55e1cfe8fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 shutdown_connections 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 1 -- 172.21.15.100:0/2985816846 wait complete. 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f5369740 20 client.26282 trim_cache size 0 max 16384 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:18:56.075 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:18:56.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97d40a1c20 msgr2=0x7f97d40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f26c0640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97d40a1c20 0x7f97d40a40e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f97e0006010 tx=0x7f97e0004790 comp rx=0 tx=0).stop 2024-03-12T05:18:56.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 msgr2=0x7f97ec100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.072+0000 7f97f26c0640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 0x7f97ec100a70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f97e400eb10 tx=0x7f97e400c2d0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 shutdown_connections 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97d40a1c20 0x7f97d40a40e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97ec10b6e0 0x7f97ec100a70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec102cc0 0x7f97ec100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 --2- 172.21.15.100:0/3004616448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97ec1023a0 0x7f97ec0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 >> 172.21.15.100:0/3004616448 conn(0x7f97ec077230 msgr2=0x7f97ec0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 shutdown_connections 2024-03-12T05:18:56.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.073+0000 7f97f26c0640 1 -- 172.21.15.100:0/3004616448 wait complete. 2024-03-12T05:18:56.079 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:18:56.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:18:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 -- 172.21.15.100:0/1677763478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a3006a450 msgr2=0x7f0a30105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 --2- 172.21.15.100:0/1677763478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a3006a450 0x7f0a30105dc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009930 tx=0x7f0a2402bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 -- 172.21.15.100:0/1677763478 shutdown_connections 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 --2- 172.21.15.100:0/1677763478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a301064a0 0x7f0a3010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 --2- 172.21.15.100:0/1677763478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a3006a450 0x7f0a30105dc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 --2- 172.21.15.100:0/1677763478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a30069aa0 0x7f0a30069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 -- 172.21.15.100:0/1677763478 >> 172.21.15.100:0/1677763478 conn(0x7f0a30077120 msgr2=0x7f0a30077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 -- 172.21.15.100:0/1677763478 shutdown_connections 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.190+0000 7f0a36125640 1 -- 172.21.15.100:0/1677763478 wait complete. 2024-03-12T05:18:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 Processor -- start 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 -- start start 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a30069aa0 0x7f0a3019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 0x7f0a3019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a301976c0 0x7f0a30197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30198190 con 0x7f0a301064a0 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30198300 con 0x7f0a301976c0 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a36125640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a301984c0 con 0x7f0a30069aa0 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a2effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 0x7f0a3019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a2effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 0x7f0a3019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34562/0 (socket says 172.21.15.100:34562) 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a2effd640 1 -- 172.21.15.100:0/3924778942 learned_addr learned my addr 172.21.15.100:0/3924778942 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.191+0000 7f0a2ffff640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a301976c0 0x7f0a30197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2f7fe640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a30069aa0 0x7f0a3019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2effd640 1 -- 172.21.15.100:0/3924778942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a30069aa0 msgr2=0x7f0a3019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2effd640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a30069aa0 0x7f0a3019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2effd640 1 -- 172.21.15.100:0/3924778942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a301976c0 msgr2=0x7f0a30197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2effd640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a301976c0 0x7f0a30197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2effd640 1 -- 172.21.15.100:0/3924778942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a24009560 con 0x7f0a301064a0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2effd640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 0x7f0a3019db40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0a24000c00 tx=0x7f0a24005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24004350 con 0x7f0a301064a0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a301a4400 con 0x7f0a301064a0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a240044b0 con 0x7f0a301064a0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24003cd0 con 0x7f0a301064a0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.192+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a301a48e0 con 0x7f0a301064a0 2024-03-12T05:18:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.193+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a301028a0 con 0x7f0a301064a0 2024-03-12T05:18:56.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.194+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0a24004df0 con 0x7f0a301064a0 2024-03-12T05:18:56.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.194+0000 7f0a2cff9640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a180a1ba0 0x7f0a180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.195+0000 7f0a2f7fe640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a180a1ba0 0x7f0a180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.195+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a240ef840 con 0x7f0a301064a0 2024-03-12T05:18:56.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.195+0000 7f0a2f7fe640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a180a1ba0 0x7f0a180a4060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a20005fd0 tx=0x7f0a20005e20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.199+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a240b87b0 con 0x7f0a301064a0 2024-03-12T05:18:56.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.439+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a300683c0 con 0x7f0a301064a0 2024-03-12T05:18:56.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.440+0000 7f0a2cff9640 1 -- 172.21.15.100:0/3924778942 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f0a240bbe70 con 0x7f0a301064a0 2024-03-12T05:18:56.435 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:18:56.435 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:18:56.436 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:18:56.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.441+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a180a1ba0 msgr2=0x7f0a180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.441+0000 7f0a36125640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a180a1ba0 0x7f0a180a4060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a20005fd0 tx=0x7f0a20005e20 comp rx=0 tx=0).stop 2024-03-12T05:18:56.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.441+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 msgr2=0x7f0a3019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.441+0000 7f0a36125640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 0x7f0a3019db40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0a24000c00 tx=0x7f0a24005e70 comp rx=0 tx=0).stop 2024-03-12T05:18:56.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 shutdown_connections 2024-03-12T05:18:56.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a180a1ba0 0x7f0a180a4060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a301976c0 0x7f0a30197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a301064a0 0x7f0a3019db40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 --2- 172.21.15.100:0/3924778942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a30069aa0 0x7f0a3019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 >> 172.21.15.100:0/3924778942 conn(0x7f0a30077120 msgr2=0x7f0a300fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 shutdown_connections 2024-03-12T05:18:56.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.442+0000 7f0a36125640 1 -- 172.21.15.100:0/3924778942 wait complete. 2024-03-12T05:18:56.448 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:18:56.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:18:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 -- 172.21.15.100:0/1214878600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 msgr2=0x7fe85010c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 --2- 172.21.15.100:0/1214878600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 0x7fe85010c590 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe848009cd0 tx=0x7fe84802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 -- 172.21.15.100:0/1214878600 shutdown_connections 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 --2- 172.21.15.100:0/1214878600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 0x7fe85010c590 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 --2- 172.21.15.100:0/1214878600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8500ff970 0x7fe8500ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 --2- 172.21.15.100:0/1214878600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8501056d0 0x7fe850105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 -- 172.21.15.100:0/1214878600 >> 172.21.15.100:0/1214878600 conn(0x7fe8500752d0 msgr2=0x7fe8500756e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 -- 172.21.15.100:0/1214878600 shutdown_connections 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 -- 172.21.15.100:0/1214878600 wait complete. 2024-03-12T05:18:56.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.567+0000 7fe855802640 1 Processor -- start 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 -- start start 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 0x7fe8501976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 0x7fe850197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8501056d0 0x7fe85019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019c560 con 0x7fe850100330 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019c6d0 con 0x7fe8500ff970 2024-03-12T05:18:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe855802640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019c890 con 0x7fe8501056d0 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 0x7fe8501976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 0x7fe850197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8501056d0 0x7fe85019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 0x7fe8501976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45062/0 (socket says 172.21.15.100:45062) 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 -- 172.21.15.100:0/1533378341 learned_addr learned my addr 172.21.15.100:0/1533378341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 -- 172.21.15.100:0/1533378341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8501056d0 msgr2=0x7fe85019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8501056d0 0x7fe85019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 -- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 msgr2=0x7fe850197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 0x7fe850197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 -- 172.21.15.100:0/1533378341 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848009900 con 0x7fe8500ff970 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.568+0000 7fe84effd640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 0x7fe8501976a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e9f0 tx=0x7fe84000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.569+0000 7fe854800640 1 -- 172.21.15.100:0/1533378341 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840009700 con 0x7fe8500ff970 2024-03-12T05:18:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.569+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85019cb70 con 0x7fe8500ff970 2024-03-12T05:18:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.569+0000 7fe854800640 1 -- 172.21.15.100:0/1533378341 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe84000cd80 con 0x7fe8500ff970 2024-03-12T05:18:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.569+0000 7fe854800640 1 -- 172.21.15.100:0/1533378341 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840018920 con 0x7fe8500ff970 2024-03-12T05:18:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.569+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8501a43f0 con 0x7fe8500ff970 2024-03-12T05:18:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 Processor -- start 2024-03-12T05:18:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe854800640 1 -- 172.21.15.100:0/1533378341 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe840018a80 con 0x7fe8500ff970 2024-03-12T05:18:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 -- start start 2024-03-12T05:18:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43d60 0x56297ab4c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297aafe960 con 0x56297ab43440 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297aab2310 con 0x56297ab4c940 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe8583d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297aaaf840 con 0x56297ab43d60 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.571+0000 7fe84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab43820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34606/0 (socket says 172.21.15.100:34606) 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 -- 172.21.15.100:0/1008612152 learned_addr learned my addr 172.21.15.100:0/1008612152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84effd640 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43d60 0x56297ab4c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 -- 172.21.15.100:0/1008612152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43d60 msgr2=0x56297ab4c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43d60 0x56297ab4c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 -- 172.21.15.100:0/1008612152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 msgr2=0x56297ab4ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 -- 172.21.15.100:0/1008612152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297aa6d060 con 0x56297ab43440 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe854800640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8380a1b90 0x7fe8380a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84effd640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8380a1b90 0x7fe8380a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab43820 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe84802c700 tx=0x7fe848032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5271c3578bd998f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe854800640 1 -- 172.21.15.100:0/1533378341 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe840014070 con 0x7fe8500ff970 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe835ffb640 1 -- 172.21.15.100:0/1008612152 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848034440 con 0x56297ab43440 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe835ffb640 1 -- 172.21.15.100:0/1008612152 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8480345a0 con 0x56297ab43440 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.572+0000 7fe835ffb640 1 -- 172.21.15.100:0/1008612152 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848003bd0 con 0x56297ab43440 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 -- 172.21.15.100:0/1008612152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 msgr2=0x56297ab43820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab43820 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe84802c700 tx=0x7fe848032d70 comp rx=0 tx=0).stop 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe84effd640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8380a1b90 0x7fe8380a4050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8400cb2e0 tx=0x7fe840085040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 -- 172.21.15.100:0/1008612152 shutdown_connections 2024-03-12T05:18:56.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43d60 0x56297ab4c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 --2- 172.21.15.100:0/1008612152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab43820 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 -- 172.21.15.100:0/1008612152 >> 172.21.15.100:0/1008612152 conn(0x56297ab3db70 msgr2=0x56297ab3df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 -- 172.21.15.100:0/1008612152 shutdown_connections 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 -- 172.21.15.100:0/1008612152 wait complete. 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 -- start start 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.573+0000 7fe8583d3740 1 Processor -- start 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43440 0x56297ab499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab49f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297aaab020 con 0x56297ab43d60 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297a77e430 con 0x56297ab4c940 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297a73ae00 con 0x56297ab43440 2024-03-12T05:18:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe8583d3740 10 client.0 _ll_register_callbacks cb 0x56297aa52cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab49f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab49f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34614/0 (socket says 172.21.15.100:34614) 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 -- 172.21.15.100:0/652317962 learned_addr learned my addr 172.21.15.100:0/652317962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43440 0x56297ab499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 -- 172.21.15.100:0/652317962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43440 msgr2=0x56297ab499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43440 0x56297ab499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 -- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 msgr2=0x56297ab4a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 -- 172.21.15.100:0/652317962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85010ce90 con 0x56297ab43d60 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab49f00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe8440126e0 tx=0x7fe844012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6786ec8df83a976 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe82effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.574+0000 7fe82effd640 1 -- 172.21.15.100:0/652317962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844017070 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe82effd640 1 -- 172.21.15.100:0/652317962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe84400f920 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56297a7aa7e0 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe82effd640 1 -- 172.21.15.100:0/652317962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe84400fae0 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 10 client.16920 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56297aaa96a0 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe82effd640 1 -- 172.21.15.100:0/652317962 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe844028080 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe82effd640 1 -- 172.21.15.100:0/652317962 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe84401c420 con 0x56297ab43d60 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 10 client.16920 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 20 client.16920 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297ab5ced0 0x56297ab5f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 4 client.16920 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:56.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56297a7e7720 con 0x56297ab5ced0 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.575+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297ab5ced0 0x56297ab5f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe82effd640 10 client.16920 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297ab5ced0 0x56297ab5f2f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=396121e5ca6d3ca4 server_cookie=df022e39944c8002 in_seq=0 out_seq=0 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe82effd640 1 -- 172.21.15.100:0/652317962 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56297a7e7720 con 0x56297ab5ced0 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe82effd640 10 client.16920 handle_command_reply: tid=0 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 client.16920 shutdown 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 msgr2=0x56297ab49f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab49f00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe8440126e0 tx=0x7fe844012ba0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 shutdown_connections 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297ab5ced0 0x56297ab5f2f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab4c940 0x56297ab4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab49f00 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe82effd640 0 client.16920 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:56.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 --2- 172.21.15.100:0/652317962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab43440 0x56297ab499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.576+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 >> 172.21.15.100:0/652317962 conn(0x56297ab3db70 msgr2=0x56297ab3f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.577+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 shutdown_connections 2024-03-12T05:18:56.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.577+0000 7fe8583d3740 1 -- 172.21.15.100:0/652317962 wait complete. 2024-03-12T05:18:56.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.577+0000 7fe8583d3740 20 client.16920 trim_cache size 0 max 16384 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.588+0000 7fe8583d3740 1 Processor -- start 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.588+0000 7fe8583d3740 1 -- start start 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43440 0x56297ab4c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab4a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab46ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8583d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297aa30130 con 0x56297ab43d60 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8583d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ab39520 con 0x56297ab43440 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8583d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ab43bc0 con 0x56297ab4c940 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab4a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab4a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34616/0 (socket says 172.21.15.100:34616) 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 -- 172.21.15.100:0/2207072576 learned_addr learned my addr 172.21.15.100:0/2207072576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43440 0x56297ab4c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab46ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 -- 172.21.15.100:0/2207072576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 msgr2=0x56297ab46ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab46ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 -- 172.21.15.100:0/2207072576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43440 msgr2=0x56297ab4c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43440 0x56297ab4c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 -- 172.21.15.100:0/2207072576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297aa45c20 con 0x56297ab43d60 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab4a280 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe8440059f0 tx=0x7fe84400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9333febe286894f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8357fa640 1 -- 172.21.15.100:0/2207072576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844017040 con 0x56297ab43d60 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8357fa640 1 -- 172.21.15.100:0/2207072576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56297a7aa7e0 con 0x56297ab43d60 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.589+0000 7fe8357fa640 1 -- 172.21.15.100:0/2207072576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844005340 con 0x56297ab43d60 2024-03-12T05:18:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 -- 172.21.15.100:0/2207072576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 msgr2=0x56297ab4a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab4a280 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe8440059f0 tx=0x7fe84400f810 comp rx=0 tx=0).stop 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 -- 172.21.15.100:0/2207072576 shutdown_connections 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab46ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43d60 0x56297ab4a280 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 --2- 172.21.15.100:0/2207072576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43440 0x56297ab4c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 -- 172.21.15.100:0/2207072576 >> 172.21.15.100:0/2207072576 conn(0x56297ab3db70 msgr2=0x56297ab482f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 -- 172.21.15.100:0/2207072576 shutdown_connections 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.590+0000 7fe8583d3740 1 -- 172.21.15.100:0/2207072576 wait complete. 2024-03-12T05:18:56.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 -- start start 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 Processor -- start 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab52230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43d60 0x56297ab52770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab52cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297abb33e0 con 0x56297ab43440 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297abb3520 con 0x56297ab43d60 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297abb36e0 con 0x56297ab4c940 2024-03-12T05:18:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe8583d3740 10 client.0 _ll_register_callbacks cb 0x7fe840005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab52230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab52230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34632/0 (socket says 172.21.15.100:34632) 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe84f7fe640 1 -- 172.21.15.100:0/730678609 learned_addr learned my addr 172.21.15.100:0/730678609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43d60 0x56297ab52770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.591+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab52cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84f7fe640 1 -- 172.21.15.100:0/730678609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 msgr2=0x56297ab52cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab52cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84f7fe640 1 -- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43d60 msgr2=0x56297ab52770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43d60 0x56297ab52770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84f7fe640 1 -- 172.21.15.100:0/730678609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56297aaa96a0 con 0x56297ab43440 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43d60 0x56297ab52770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe84f7fe640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab52230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe848009890 tx=0x7fe848041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab30607e3d5f1b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe82effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe82effd640 1 -- 172.21.15.100:0/730678609 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848050040 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe82effd640 1 -- 172.21.15.100:0/730678609 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe848034a90 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8480345a0 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe82effd640 1 -- 172.21.15.100:0/730678609 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8480327f0 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 10 client.16926 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56297aa6b8c0 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe82effd640 1 -- 172.21.15.100:0/730678609 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8480345a0 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe82effd640 1 -- 172.21.15.100:0/730678609 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe848004040 con 0x56297ab43440 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 10 client.16926 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 10 client.16926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 20 client.16926 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297abb4b10 0x56297abb6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 4 client.16926 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.592+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56297a836890 con 0x56297abb4b10 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.593+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297abb4b10 0x56297abb6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.593+0000 7fe82effd640 10 client.16926 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.593+0000 7fe84e7fc640 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297abb4b10 0x56297abb6f00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c55502720ad665db server_cookie=97762c55c0db39c4 in_seq=0 out_seq=0 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe82effd640 1 -- 172.21.15.100:0/730678609 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56297a836890 con 0x56297abb4b10 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe82effd640 10 client.16926 handle_command_reply: tid=0 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 client.16926 shutdown 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 msgr2=0x56297ab52230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab52230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe848009890 tx=0x7fe848041040 comp rx=0 tx=0).stop 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 shutdown_connections 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56297abb4b10 0x56297abb6f00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56297ab4c940 0x56297ab52cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56297ab43d60 0x56297ab52770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe82effd640 0 client.16926 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:18:56.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 --2- 172.21.15.100:0/730678609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56297ab43440 0x56297ab52230 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 >> 172.21.15.100:0/730678609 conn(0x56297ab3db70 msgr2=0x56297ab3c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 shutdown_connections 2024-03-12T05:18:56.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 1 -- 172.21.15.100:0/730678609 wait complete. 2024-03-12T05:18:56.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.595+0000 7fe8583d3740 20 client.16926 trim_cache size 0 max 16384 2024-03-12T05:18:56.596 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8380a1b90 msgr2=0x7fe8380a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8380a1b90 0x7fe8380a4050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8400cb2e0 tx=0x7fe840085040 comp rx=0 tx=0).stop 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 msgr2=0x7fe8501976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 0x7fe8501976a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e9f0 tx=0x7fe84000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 shutdown_connections 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8380a1b90 0x7fe8380a4050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8501056d0 0x7fe85019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe850100330 0x7fe850197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 --2- 172.21.15.100:0/1533378341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500ff970 0x7fe8501976a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 >> 172.21.15.100:0/1533378341 conn(0x7fe8500752d0 msgr2=0x7fe850101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 shutdown_connections 2024-03-12T05:18:56.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.596+0000 7fe855802640 1 -- 172.21.15.100:0/1533378341 wait complete. 2024-03-12T05:18:56.602 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:18:56.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:18:56.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 -- 172.21.15.100:0/2408504414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 msgr2=0x7fb44c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2408504414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c100640 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb448009900 tx=0x7fb44802bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:56.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 -- 172.21.15.100:0/2408504414 shutdown_connections 2024-03-12T05:18:56.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2408504414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c105330 0x7fb44c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2408504414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c100c10 0x7fb44c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2408504414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c100640 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 -- 172.21.15.100:0/2408504414 >> 172.21.15.100:0/2408504414 conn(0x7fb44c077180 msgr2=0x7fb44c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 -- 172.21.15.100:0/2408504414 shutdown_connections 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.716+0000 7fb453b3a640 1 -- 172.21.15.100:0/2408504414 wait complete. 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 Processor -- start 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 -- start start 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c105330 0x7fb44c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c197620 0x7fb44c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c198010 con 0x7fb44c100260 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c198180 con 0x7fb44c197620 2024-03-12T05:18:56.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb453b3a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c198340 con 0x7fb44c105330 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34660/0 (socket says 172.21.15.100:34660) 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 -- 172.21.15.100:0/2657337189 learned_addr learned my addr 172.21.15.100:0/2657337189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4520d7640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c197620 0x7fb44c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c105330 0x7fb44c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 -- 172.21.15.100:0/2657337189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c105330 msgr2=0x7fb44c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c105330 0x7fb44c19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 -- 172.21.15.100:0/2657337189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c197620 msgr2=0x7fb44c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c197620 0x7fb44c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 -- 172.21.15.100:0/2657337189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448009560 con 0x7fb44c100260 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c105330 0x7fb44c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.717+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c19d470 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb448005940 tx=0x7fb44802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.718+0000 7fb442ffd640 1 -- 172.21.15.100:0/2657337189 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448004350 con 0x7fb44c100260 2024-03-12T05:18:56.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.718+0000 7fb442ffd640 1 -- 172.21.15.100:0/2657337189 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4480044b0 con 0x7fb44c100260 2024-03-12T05:18:56.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.718+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c1a4330 con 0x7fb44c100260 2024-03-12T05:18:56.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.718+0000 7fb442ffd640 1 -- 172.21.15.100:0/2657337189 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003d30 con 0x7fb44c100260 2024-03-12T05:18:56.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.718+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c1a4830 con 0x7fb44c100260 2024-03-12T05:18:56.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.720+0000 7fb442ffd640 1 -- 172.21.15.100:0/2657337189 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb448004df0 con 0x7fb44c100260 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 Processor -- start 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 -- start start 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 0x5642c3136630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb442ffd640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4380a1c20 0x7fb4380a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c313f750 0x5642c3141b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c30f1960 con 0x5642c313f750 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c30a5310 con 0x5642c3136250 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4567e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c30a2840 con 0x5642c3136b70 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4380a1c20 0x7fb4380a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4520d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 0x5642c3136630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4520d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 0x5642c3136630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45114/0 (socket says 172.21.15.100:45114) 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4520d7640 1 -- 172.21.15.100:0/795386696 learned_addr learned my addr 172.21.15.100:0/795386696 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb442ffd640 1 -- 172.21.15.100:0/2657337189 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb4480ef790 con 0x7fb44c100260 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4520d7640 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c313f750 0x5642c3141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.721+0000 7fb4518d6640 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4520d7640 1 -- 172.21.15.100:0/795386696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 msgr2=0x5642c313f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4520d7640 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4520d7640 1 -- 172.21.15.100:0/795386696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c313f750 msgr2=0x5642c3141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4520d7640 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c313f750 0x5642c3141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4520d7640 1 -- 172.21.15.100:0/795386696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642c3060060 con 0x5642c3136250 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4380a1c20 0x7fb4380a40e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0ffc00 tx=0x7fb43c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb4520d7640 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 0x5642c3136630 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb444005f40 tx=0x7fb444005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b52c52c30d2c7c65 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb437fff640 1 -- 172.21.15.100:0/795386696 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444004b20 con 0x5642c3136250 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb437fff640 1 -- 172.21.15.100:0/795386696 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb444004c80 con 0x5642c3136250 2024-03-12T05:18:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.722+0000 7fb437fff640 1 -- 172.21.15.100:0/795386696 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444004030 con 0x5642c3136250 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 -- 172.21.15.100:0/795386696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 msgr2=0x5642c3136630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 0x5642c3136630 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb444005f40 tx=0x7fb444005bd0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 -- 172.21.15.100:0/795386696 shutdown_connections 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c313f750 0x5642c3141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 --2- 172.21.15.100:0/795386696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136250 0x5642c3136630 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 -- 172.21.15.100:0/795386696 >> 172.21.15.100:0/795386696 conn(0x5642c3130980 msgr2=0x5642c3130d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 -- 172.21.15.100:0/795386696 shutdown_connections 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 -- 172.21.15.100:0/795386696 wait complete. 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 -- start start 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.723+0000 7fb4567e9740 1 Processor -- start 2024-03-12T05:18:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c313f750 0x5642c313d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c309e020 con 0x5642c3136250 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c2d71430 con 0x5642c313f750 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c2d2de00 con 0x5642c3136b70 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4567e9740 10 client.0 _ll_register_callbacks cb 0x5642c3045cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34668/0 (socket says 172.21.15.100:34668) 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4518d6640 1 -- 172.21.15.100:0/2511580124 learned_addr learned my addr 172.21.15.100:0/2511580124 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4520d7640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c313f750 0x5642c313d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4520d7640 1 -- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 msgr2=0x5642c313cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4520d7640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4520d7640 1 -- 172.21.15.100:0/2511580124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 msgr2=0x5642c313c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4520d7640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4520d7640 1 -- 172.21.15.100:0/2511580124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb444019040 con 0x5642c313f750 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.724+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4520d7640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c313f750 0x5642c313d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb444009a70 tx=0x7fb444011560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25738750f1a43a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4357fa640 1 -- 172.21.15.100:0/2511580124 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444024070 con 0x5642c313f750 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642c2d9d7e0 con 0x5642c313f750 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4357fa640 1 -- 172.21.15.100:0/2511580124 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb444004a40 con 0x5642c313f750 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4357fa640 1 -- 172.21.15.100:0/2511580124 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb444004e70 con 0x5642c313f750 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 10 client.26318 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5642c309c6a0 con 0x5642c313f750 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4357fa640 1 -- 172.21.15.100:0/2511580124 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5642c2d9d7e0 con 0x5642c313f750 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4357fa640 1 -- 172.21.15.100:0/2511580124 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb44401d070 con 0x5642c313f750 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 10 client.26318 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 10 client.26318 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 20 client.26318 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c314faa0 0x5642c3151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 4 client.26318 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.725+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642c2dda720 con 0x5642c314faa0 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c314faa0 0x5642c3151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c314faa0 0x5642c3151ec0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b8009be38000ce6 server_cookie=3475fe938b61ca3f in_seq=0 out_seq=0 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4357fa640 10 client.26318 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4357fa640 1 -- 172.21.15.100:0/2511580124 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642c2dda720 con 0x5642c314faa0 2024-03-12T05:18:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4357fa640 10 client.26318 handle_command_reply: tid=0 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4567e9740 1 client.26318 shutdown 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.726+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c313f750 msgr2=0x5642c313d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c313f750 0x5642c313d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb444009a70 tx=0x7fb444011560 comp rx=0 tx=0).stop 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 shutdown_connections 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c314faa0 0x5642c3151ec0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c313f750 0x5642c313d1c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c3136b70 0x5642c313cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4357fa640 0 client.26318 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2511580124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 >> 172.21.15.100:0/2511580124 conn(0x5642c3130980 msgr2=0x5642c31326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 shutdown_connections 2024-03-12T05:18:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 1 -- 172.21.15.100:0/2511580124 wait complete. 2024-03-12T05:18:56.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.727+0000 7fb4567e9740 20 client.26318 trim_cache size 0 max 16384 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 Processor -- start 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 -- start start 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c31399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c3139f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c3023130 con 0x5642c3136250 2024-03-12T05:18:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c3135cc0 con 0x5642c3136b70 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4567e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c3131550 con 0x5642c313f750 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34682/0 (socket says 172.21.15.100:34682) 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 -- 172.21.15.100:0/2882682833 learned_addr learned my addr 172.21.15.100:0/2882682833 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4520d7640 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c3139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4510d5640 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c31399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 -- 172.21.15.100:0/2882682833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 msgr2=0x5642c3139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c3139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 -- 172.21.15.100:0/2882682833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 msgr2=0x5642c31399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c31399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 -- 172.21.15.100:0/2882682833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642c3038c20 con 0x5642c3136250 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4518d6640 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313ee70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb448006010 tx=0x7fb4480ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd8e26ddc918721 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4377fe640 1 -- 172.21.15.100:0/2882682833 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480dc0b0 con 0x5642c3136250 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4377fe640 1 -- 172.21.15.100:0/2882682833 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4480bc4c0 con 0x5642c3136250 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.739+0000 7fb4377fe640 1 -- 172.21.15.100:0/2882682833 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480ba8e0 con 0x5642c3136250 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 -- 172.21.15.100:0/2882682833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 msgr2=0x5642c313ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313ee70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb448006010 tx=0x7fb4480ef040 comp rx=0 tx=0).stop 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 -- 172.21.15.100:0/2882682833 shutdown_connections 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c3139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c31399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 --2- 172.21.15.100:0/2882682833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c313ee70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 -- 172.21.15.100:0/2882682833 >> 172.21.15.100:0/2882682833 conn(0x5642c3130980 msgr2=0x5642c3137a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 -- 172.21.15.100:0/2882682833 shutdown_connections 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 -- 172.21.15.100:0/2882682833 wait complete. 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.740+0000 7fb4567e9740 1 -- start start 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 Processor -- start 2024-03-12T05:18:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c31456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c3145c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c31a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c3146340 con 0x5642c3136250 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c31a64f0 con 0x5642c3136b70 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642c31a6680 con 0x5642c313f750 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4567e9740 10 client.0 _ll_register_callbacks cb 0x7fb444009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c31456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c31456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34688/0 (socket says 172.21.15.100:34688) 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 -- 172.21.15.100:0/652503285 learned_addr learned my addr 172.21.15.100:0/652503285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4510d5640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c3145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4520d7640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c31a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 -- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 msgr2=0x5642c31a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c31a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 -- 172.21.15.100:0/652503285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 msgr2=0x5642c3145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c3145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 -- 172.21.15.100:0/652503285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c1a4830 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4518d6640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c31456c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb4480c4730 tx=0x7fb4480cf360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c380817f4585630 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.741+0000 7fb4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4357fa640 1 -- 172.21.15.100:0/652503285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480a70c0 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4357fa640 1 -- 172.21.15.100:0/652503285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb4480c4c60 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb444004c80 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4357fa640 1 -- 172.21.15.100:0/652503285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4480dc0b0 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 10 client.16944 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5642c305e8c0 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4357fa640 1 -- 172.21.15.100:0/652503285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb444004c80 con 0x5642c3136250 2024-03-12T05:18:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4357fa640 1 -- 172.21.15.100:0/652503285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb44803c070 con 0x5642c3136250 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 10 client.16944 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 10 client.16944 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 20 client.16944 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c31a7a30 0x5642c31a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 4 client.16944 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5642c2e29890 con 0x5642c31a7a30 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.742+0000 7fb4510d5640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c31a7a30 0x5642c31a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4357fa640 10 client.16944 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4510d5640 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c31a7a30 0x5642c31a9e20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6d163ad7e8b615e server_cookie=8546fe855ac5f5b2 in_seq=0 out_seq=0 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4357fa640 1 -- 172.21.15.100:0/652503285 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5642c2e29890 con 0x5642c31a7a30 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4357fa640 10 client.16944 handle_command_reply: tid=0 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 client.16944 shutdown 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 msgr2=0x5642c31456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c31456c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb4480c4730 tx=0x7fb4480cf360 comp rx=0 tx=0).stop 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 shutdown_connections 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642c313f750 0x5642c31a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5642c31a7a30 0x5642c31a9e20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5642c3136b70 0x5642c3145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 --2- 172.21.15.100:0/652503285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642c3136250 0x5642c31456c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4357fa640 0 client.16944 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 >> 172.21.15.100:0/652503285 conn(0x5642c3130980 msgr2=0x5642c312f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 shutdown_connections 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 1 -- 172.21.15.100:0/652503285 wait complete. 2024-03-12T05:18:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.743+0000 7fb4567e9740 20 client.16944 trim_cache size 0 max 16384 2024-03-12T05:18:56.747 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:18:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4380a1c20 msgr2=0x7fb4380a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4380a1c20 0x7fb4380a40e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0ffc00 tx=0x7fb43c005f90 comp rx=0 tx=0).stop 2024-03-12T05:18:56.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 msgr2=0x7fb44c19d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c19d470 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb448005940 tx=0x7fb44802c670 comp rx=0 tx=0).stop 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 shutdown_connections 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4380a1c20 0x7fb4380a40e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c197620 0x7fb44c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb44c105330 0x7fb44c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 --2- 172.21.15.100:0/2657337189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb44c100260 0x7fb44c19d470 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 >> 172.21.15.100:0/2657337189 conn(0x7fb44c077180 msgr2=0x7fb44c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 shutdown_connections 2024-03-12T05:18:56.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.744+0000 7fb453b3a640 1 -- 172.21.15.100:0/2657337189 wait complete. 2024-03-12T05:18:56.751 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:18:56.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:18:56.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 803836 12.23 MB/sec execute 220 sec latency 23.168 ms 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 -- 172.21.15.100:0/2762797582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48101d40 msgr2=0x7f5d48105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 --2- 172.21.15.100:0/2762797582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48101d40 0x7f5d48105d80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009930 tx=0x7f5d3c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 -- 172.21.15.100:0/2762797582 shutdown_connections 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 --2- 172.21.15.100:0/2762797582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48106460 0x7f5d4810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 --2- 172.21.15.100:0/2762797582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48101d40 0x7f5d48105d80 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 --2- 172.21.15.100:0/2762797582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d48101390 0x7f5d48101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 -- 172.21.15.100:0/2762797582 >> 172.21.15.100:0/2762797582 conn(0x7f5d480fcbb0 msgr2=0x7f5d480fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 -- 172.21.15.100:0/2762797582 shutdown_connections 2024-03-12T05:18:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 -- 172.21.15.100:0/2762797582 wait complete. 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 Processor -- start 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.864+0000 7f5d4db35640 1 -- start start 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48101390 0x7f5d4819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 0x7f5d4819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d48197720 0x7f5d48197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d481981f0 con 0x7f5d48106460 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48198360 con 0x7f5d48101390 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48198520 con 0x7f5d48197720 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 0x7f5d4819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48101390 0x7f5d4819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 0x7f5d4819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34716/0 (socket says 172.21.15.100:34716) 2024-03-12T05:18:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48101390 0x7f5d4819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45156/0 (socket says 172.21.15.100:45156) 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 -- 172.21.15.100:0/3951268488 learned_addr learned my addr 172.21.15.100:0/3951268488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d47fff640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d48197720 0x7f5d48197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 -- 172.21.15.100:0/3951268488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d48197720 msgr2=0x7f5d48197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d48197720 0x7f5d48197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 -- 172.21.15.100:0/3951268488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48101390 msgr2=0x7f5d4819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48101390 0x7f5d4819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 -- 172.21.15.100:0/3951268488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c009560 con 0x7f5d48106460 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 0x7f5d4819db50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009230 tx=0x7f5d3c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d44ff9640 1 -- 172.21.15.100:0/3951268488 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004350 con 0x7f5d48106460 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d44ff9640 1 -- 172.21.15.100:0/3951268488 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c0044b0 con 0x7f5d48106460 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d481a4410 con 0x7f5d48106460 2024-03-12T05:18:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d44ff9640 1 -- 172.21.15.100:0/3951268488 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c003da0 con 0x7f5d48106460 2024-03-12T05:18:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.865+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d481a4940 con 0x7f5d48106460 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.867+0000 7f5d44ff9640 1 -- 172.21.15.100:0/3951268488 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5d3c004df0 con 0x7f5d48106460 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 Processor -- start 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 -- start start 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d44ff9640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d300a1c20 0x7f5d300a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x557067494630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x55706749d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 0x55706749fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706744f960 con 0x55706749d750 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557067403310 con 0x557067494250 2024-03-12T05:18:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d507e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557067400840 con 0x557067494b70 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d47fff640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d300a1c20 0x7f5d300a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d44ff9640 1 -- 172.21.15.100:0/3951268488 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5d3c0ef720 con 0x7f5d48106460 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x55706749d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x557067494630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 0x55706749fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 0x55706749fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34724/0 (socket says 172.21.15.100:34724) 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x557067494630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45166/0 (socket says 172.21.15.100:45166) 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x55706749d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:46166/0 (socket says 172.21.15.100:46166) 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.868+0000 7f5d47fff640 1 -- 172.21.15.100:0/2389726372 learned_addr learned my addr 172.21.15.100:0/2389726372 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d46ffd640 1 -- 172.21.15.100:0/2389726372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 msgr2=0x557067494630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x557067494630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d46ffd640 1 -- 172.21.15.100:0/2389726372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 msgr2=0x55706749fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 0x55706749fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d46ffd640 1 -- 172.21.15.100:0/2389726372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570673be060 con 0x557067494b70 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d477fe640 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x557067494630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d47fff640 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 0x55706749fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d47fff640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d300a1c20 0x7f5d300a40e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009fd0 tx=0x7f5d40009f40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x55706749d210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0ecf70 tx=0x7f5d3c003840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5923460ed41445 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d2dffb640 1 -- 172.21.15.100:0/2389726372 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0bc450 con 0x557067494b70 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d2dffb640 1 -- 172.21.15.100:0/2389726372 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c0c05a0 con 0x557067494b70 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.869+0000 7f5d2dffb640 1 -- 172.21.15.100:0/2389726372 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0ed0e0 con 0x557067494b70 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 -- 172.21.15.100:0/2389726372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 msgr2=0x55706749d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x55706749d210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0ecf70 tx=0x7f5d3c003840 comp rx=0 tx=0).stop 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 -- 172.21.15.100:0/2389726372 shutdown_connections 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55706749d750 0x55706749fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x55706749d210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 --2- 172.21.15.100:0/2389726372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x557067494630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 -- 172.21.15.100:0/2389726372 >> 172.21.15.100:0/2389726372 conn(0x55706748e980 msgr2=0x55706748ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 -- 172.21.15.100:0/2389726372 shutdown_connections 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 -- 172.21.15.100:0/2389726372 wait complete. 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 -- start start 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 Processor -- start 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x55706749a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.870+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706749d750 0x55706749b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570673fc020 con 0x557067494b70 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570670cf430 con 0x557067494250 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706708be00 con 0x55706749d750 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 10 client.0 _ll_register_callbacks cb 0x5570673a3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34728/0 (socket says 172.21.15.100:34728) 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4269710688 learned_addr learned my addr 172.21.15.100:0/4269710688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d477fe640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x55706749a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d47fff640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706749d750 0x55706749b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4269710688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706749d750 msgr2=0x55706749b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706749d750 0x55706749b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 msgr2=0x55706749a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x55706749a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4269710688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d481a4940 con 0x557067494b70 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749acb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0ed460 tx=0x7f5d3c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ad9904e77d8645 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d26ffd640 1 -- 172.21.15.100:0/4269710688 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0ba4f0 con 0x557067494b70 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d26ffd640 1 -- 172.21.15.100:0/4269710688 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5d3c0bc450 con 0x557067494b70 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570670fb7e0 con 0x557067494b70 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d26ffd640 1 -- 172.21.15.100:0/4269710688 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3c0c4bf0 con 0x557067494b70 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 10 client.16956 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.871+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5570673fa6a0 con 0x557067494b70 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d26ffd640 1 -- 172.21.15.100:0/4269710688 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5570670fb7e0 con 0x557067494b70 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d26ffd640 1 -- 172.21.15.100:0/4269710688 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5d3c03c070 con 0x557067494b70 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d507e0740 10 client.16956 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d507e0740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d507e0740 20 client.16956 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5570674ad440 0x5570674af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d507e0740 4 client.16956 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557067138720 con 0x5570674ad440 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d477fe640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5570674ad440 0x5570674af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d477fe640 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5570674ad440 0x5570674af830 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cb9287a2d500e2 server_cookie=402b22029ee064a4 in_seq=0 out_seq=0 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.872+0000 7f5d26ffd640 10 client.16956 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d26ffd640 1 -- 172.21.15.100:0/4269710688 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557067138720 con 0x5570674ad440 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d26ffd640 10 client.16956 handle_command_reply: tid=0 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 client.16956 shutdown 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 msgr2=0x55706749acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749acb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0ed460 tx=0x7f5d3c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 shutdown_connections 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5570674ad440 0x5570674af830 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55706749d750 0x55706749b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749acb0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4269710688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557067494250 0x55706749a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 >> 172.21.15.100:0/4269710688 conn(0x55706748e980 msgr2=0x5570674906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d26ffd640 0 client.16956 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 shutdown_connections 2024-03-12T05:18:56.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 1 -- 172.21.15.100:0/4269710688 wait complete. 2024-03-12T05:18:56.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.873+0000 7f5d507e0740 20 client.16956 trim_cache size 0 max 16384 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 Processor -- start 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 -- start start 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494250 0x55706749cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x557067495ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557067381130 con 0x557067494b70 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570674aa3b0 con 0x55706749d750 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d507e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557067493c90 con 0x557067494250 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34740/0 (socket says 172.21.15.100:34740) 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4154377493 learned_addr learned my addr 172.21.15.100:0/4154377493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d47fff640 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x557067495ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.885+0000 7f5d477fe640 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494250 0x55706749cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4154377493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494250 msgr2=0x55706749cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494250 0x55706749cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4154377493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 msgr2=0x557067495ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x557067495ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d46ffd640 1 -- 172.21.15.100:0/4154377493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557067396c20 con 0x557067494b70 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d47fff640 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x557067495ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749b010 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0a7200 tx=0x7f5d3c003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc8f30da677d1d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d2d7fa640 1 -- 172.21.15.100:0/4154377493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0bb8f0 con 0x557067494b70 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d2d7fa640 1 -- 172.21.15.100:0/4154377493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c0ba4f0 con 0x557067494b70 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d2d7fa640 1 -- 172.21.15.100:0/4154377493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0c4c80 con 0x557067494b70 2024-03-12T05:18:56.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d507e0740 1 -- 172.21.15.100:0/4154377493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 msgr2=0x55706749b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749b010 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0a7200 tx=0x7f5d3c003840 comp rx=0 tx=0).stop 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d507e0740 1 -- 172.21.15.100:0/4154377493 shutdown_connections 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x557067495ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494b70 0x55706749b010 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.886+0000 7f5d507e0740 1 --2- 172.21.15.100:0/4154377493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494250 0x55706749cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- 172.21.15.100:0/4154377493 >> 172.21.15.100:0/4154377493 conn(0x55706748e980 msgr2=0x5570674b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- 172.21.15.100:0/4154377493 shutdown_connections 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- 172.21.15.100:0/4154377493 wait complete. 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- start start 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 Processor -- start 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 0x5570674a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x5570674a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x5570674a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570674a2e60 con 0x557067494250 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570674a2fd0 con 0x55706749d750 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570674a3190 con 0x557067494b70 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d507e0740 10 client.0 _ll_register_callbacks cb 0x7f5d400117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 0x5570674a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 0x5570674a4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34752/0 (socket says 172.21.15.100:34752) 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d477fe640 1 -- 172.21.15.100:0/3252582303 learned_addr learned my addr 172.21.15.100:0/3252582303 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d47fff640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x5570674a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.887+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x5570674a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d477fe640 1 -- 172.21.15.100:0/3252582303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 msgr2=0x5570674a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d477fe640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x5570674a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d477fe640 1 -- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 msgr2=0x5570674a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d477fe640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x5570674a27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d477fe640 1 -- 172.21.15.100:0/3252582303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d40019040 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d477fe640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 0x5570674a4150 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x5570674b0150 tx=0x7f5d380044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ca5f880964e76f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d26ffd640 1 -- 172.21.15.100:0/3252582303 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38017070 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d26ffd640 1 -- 172.21.15.100:0/3252582303 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5d38005220 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d3c0c05a0 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d26ffd640 1 -- 172.21.15.100:0/3252582303 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d380053e0 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d507e0740 10 client.16962 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5570673fa6a0 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.888+0000 7f5d26ffd640 1 -- 172.21.15.100:0/3252582303 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d38032080 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d26ffd640 1 -- 172.21.15.100:0/3252582303 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5d3801d040 con 0x557067494250 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 10 client.16962 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 20 client.16962 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:56.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557067504c90 0x5570675070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 4 client.16962 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557067187890 con 0x557067504c90 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557067504c90 0x5570675070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d26ffd640 10 client.16962 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d46ffd640 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557067504c90 0x5570675070b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac39c3ead7f6a0c8 server_cookie=d11ec99f34160bf4 in_seq=0 out_seq=0 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d26ffd640 1 -- 172.21.15.100:0/3252582303 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557067187890 con 0x557067504c90 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d26ffd640 10 client.16962 handle_command_reply: tid=0 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.889+0000 7f5d507e0740 1 client.16962 shutdown 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 msgr2=0x5570674a4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 0x5570674a4150 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x5570674b0150 tx=0x7f5d380044d0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 shutdown_connections 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557067504c90 0x5570675070b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55706749d750 0x5570674a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557067494b70 0x5570674a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 --2- 172.21.15.100:0/3252582303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557067494250 0x5570674a4150 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d26ffd640 0 client.16962 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 >> 172.21.15.100:0/3252582303 conn(0x55706748e980 msgr2=0x557067495ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 shutdown_connections 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 1 -- 172.21.15.100:0/3252582303 wait complete. 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d507e0740 20 client.16962 trim_cache size 0 max 16384 2024-03-12T05:18:56.894 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:18:56.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d300a1c20 msgr2=0x7f5d300a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d300a1c20 0x7f5d300a40e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009fd0 tx=0x7f5d40009f40 comp rx=0 tx=0).stop 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 msgr2=0x7f5d4819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 0x7f5d4819db50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009230 tx=0x7f5d3c02c680 comp rx=0 tx=0).stop 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 shutdown_connections 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d300a1c20 0x7f5d300a40e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d48197720 0x7f5d48197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d48106460 0x7f5d4819db50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 --2- 172.21.15.100:0/3951268488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48101390 0x7f5d4819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 >> 172.21.15.100:0/3951268488 conn(0x7f5d480fcbb0 msgr2=0x7f5d480fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:56.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.890+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 shutdown_connections 2024-03-12T05:18:56.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:56.891+0000 7f5d4db35640 1 -- 172.21.15.100:0/3951268488 wait complete. 2024-03-12T05:18:57.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:56 smithi100 ceph-mon[32210]: pgmap v208: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-03-12T05:18:57.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3924778942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:18:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:56 smithi156 ceph-mon[36270]: pgmap v208: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-03-12T05:18:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3924778942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:18:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:56 smithi161 ceph-mon[36043]: pgmap v208: 97 pgs: 97 active+clean; 436 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-03-12T05:18:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3924778942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:18:57.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 806364 12.24 MB/sec execute 221 sec latency 13.769 ms 2024-03-12T05:18:57.898 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:18:57.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:18:57.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.012+0000 7f9a122f2640 1 -- 172.21.15.100:0/2761471030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102410 msgr2=0x7f9a0c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.012+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2761471030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102410 0x7f9a0c1027f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009900 tx=0x7f99fc02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- 172.21.15.100:0/2761471030 shutdown_connections 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2761471030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a0c10b750 0x7f9a0c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2761471030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a0c102d30 0x7f9a0c10b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2761471030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102410 0x7f9a0c1027f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- 172.21.15.100:0/2761471030 >> 172.21.15.100:0/2761471030 conn(0x7f9a0c0771e0 msgr2=0x7f9a0c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- 172.21.15.100:0/2761471030 shutdown_connections 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- 172.21.15.100:0/2761471030 wait complete. 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 Processor -- start 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- start start 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a0c102410 0x7f9a0c100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 0x7f9a0c1005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a0c10b750 0x7f9a0c100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1a5b90 con 0x7f9a0c102d30 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1a5d00 con 0x7f9a0c102410 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.013+0000 7f9a122f2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1a5ec0 con 0x7f9a0c10b750 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 0x7f9a0c1005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 0x7f9a0c1005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34778/0 (socket says 172.21.15.100:34778) 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/2289567642 learned_addr learned my addr 172.21.15.100:0/2289567642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a0c102410 0x7f9a0c100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a1088f640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a0c10b750 0x7f9a0c100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/2289567642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a0c10b750 msgr2=0x7f9a0c100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a0c10b750 0x7f9a0c100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/2289567642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a0c102410 msgr2=0x7f9a0c100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a0c102410 0x7f9a0c100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/2289567642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc009560 con 0x7f9a0c102d30 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 0x7f9a0c1005a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e9f0 tx=0x7f9a0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a097fa640 1 -- 172.21.15.100:0/2289567642 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00009700 con 0x7f9a0c102d30 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c1a61a0 con 0x7f9a0c102d30 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a097fa640 1 -- 172.21.15.100:0/2289567642 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a0000cd80 con 0x7f9a0c102d30 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a097fa640 1 -- 172.21.15.100:0/2289567642 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00018920 con 0x7f9a0c102d30 2024-03-12T05:18:58.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.014+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c1a6700 con 0x7f9a0c102d30 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a097fa640 1 -- 172.21.15.100:0/2289567642 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9a00018a80 con 0x7f9a0c102d30 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 Processor -- start 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 -- start start 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a097fa640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99f40a1ba0 0x7f99f40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689380 0x55c63b689760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 0x55c63b6922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b6928b0 0x55c63b694d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b5f7f90 con 0x55c63b689380 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b5f54c0 con 0x55c63b6928b0 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b5f0ca0 con 0x55c63b689d30 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a097fa640 1 -- 172.21.15.100:0/2289567642 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9a00014070 con 0x7f9a0c102d30 2024-03-12T05:18:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a1088f640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99f40a1ba0 0x7f99f40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689380 0x55c63b689760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 0x55c63b6922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 0x55c63b6922e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:46232/0 (socket says 172.21.15.100:46232) 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689380 0x55c63b689760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34792/0 (socket says 172.21.15.100:34792) 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.017+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/46785815 learned_addr learned my addr 172.21.15.100:0/46785815 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a1088f640 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b6928b0 0x55c63b694d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/46785815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b6928b0 msgr2=0x55c63b694d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b6928b0 0x55c63b694d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/46785815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689380 msgr2=0x55c63b689760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689380 0x55c63b689760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/46785815 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c63b5b30b0 con 0x55c63b689d30 2024-03-12T05:18:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a1088f640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99f40a1ba0 0x7f99f40a4060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009fd0 tx=0x7f9a04009a50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 0x55c63b6922e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9a000a2a20 tx=0x7f9a0009a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de632e05095f37e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f99f27fc640 1 -- 172.21.15.100:0/46785815 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00098b60 con 0x55c63b689d30 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f99f27fc640 1 -- 172.21.15.100:0/46785815 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a00010b80 con 0x55c63b689d30 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.018+0000 7f99f27fc640 1 -- 172.21.15.100:0/46785815 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000cb790 con 0x55c63b689d30 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 -- 172.21.15.100:0/46785815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 msgr2=0x55c63b6922e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 0x55c63b6922e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9a000a2a20 tx=0x7f9a0009a280 comp rx=0 tx=0).stop 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 -- 172.21.15.100:0/46785815 shutdown_connections 2024-03-12T05:18:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b6928b0 0x55c63b694d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689d30 0x55c63b6922e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/46785815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689380 0x55c63b689760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 -- 172.21.15.100:0/46785815 >> 172.21.15.100:0/46785815 conn(0x55c63b683840 msgr2=0x55c63b683c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 -- 172.21.15.100:0/46785815 shutdown_connections 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 -- 172.21.15.100:0/46785815 wait complete. 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 -- start start 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 Processor -- start 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.019+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 0x55c63b68f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b68fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b6928b0 0x55c63b690320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b595f40 con 0x55c63b689d30 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b2c44b0 con 0x55c63b689380 2024-03-12T05:18:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b576670 con 0x55c63b6928b0 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 10 client.0 _ll_register_callbacks cb 0x55c63b5e3f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 0x55c63b68f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b68fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a1088f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b6928b0 0x55c63b690320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 0x55c63b68f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45242/0 (socket says 172.21.15.100:45242) 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0bfff640 1 -- 172.21.15.100:0/84193411 learned_addr learned my addr 172.21.15.100:0/84193411 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/84193411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b6928b0 msgr2=0x55c63b690320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b6928b0 0x55c63b690320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 msgr2=0x55c63b68f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 0x55c63b68f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 -- 172.21.15.100:0/84193411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c1a6700 con 0x55c63b689d30 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 0x55c63b68f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b68fde0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x55c63b688e50 tx=0x7f9a0000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20791553d8ea893e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f99eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f99eb7fe640 1 -- 172.21.15.100:0/84193411 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00096ed0 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f99eb7fe640 1 -- 172.21.15.100:0/84193411 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9a000b3220 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c63b5caf40 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f99eb7fe640 1 -- 172.21.15.100:0/84193411 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a00097030 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 10 client.16974 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.020+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c63b5ef320 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f99eb7fe640 1 -- 172.21.15.100:0/84193411 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c63b5caf40 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f99eb7fe640 1 -- 172.21.15.100:0/84193411 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a000ba2a0 con 0x55c63b689d30 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a14f9d740 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a14f9d740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a14f9d740 20 client.16974 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6a2af0 0x55c63b6a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a14f9d740 4 client.16974 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c63b5e7310 con 0x55c63b6a2af0 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6a2af0 0x55c63b6a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f99eb7fe640 10 client.16974 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.021+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6a2af0 0x55c63b6a4ee0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75cd86e739733680 server_cookie=34d247a6df73e4c1 in_seq=0 out_seq=0 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f99eb7fe640 1 -- 172.21.15.100:0/84193411 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c63b5e7310 con 0x55c63b6a2af0 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f99eb7fe640 10 client.16974 handle_command_reply: tid=0 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 client.16974 shutdown 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 msgr2=0x55c63b68fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b68fde0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x55c63b688e50 tx=0x7f9a0000c7d0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 shutdown_connections 2024-03-12T05:18:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6a2af0 0x55c63b6a4ee0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b6928b0 0x55c63b690320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b68fde0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f99eb7fe640 0 client.16974 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/84193411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b689380 0x55c63b68f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 >> 172.21.15.100:0/84193411 conn(0x55c63b683840 msgr2=0x55c63b6857a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 shutdown_connections 2024-03-12T05:18:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 1 -- 172.21.15.100:0/84193411 wait complete. 2024-03-12T05:18:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.022+0000 7f9a14f9d740 20 client.16974 trim_cache size 0 max 16384 2024-03-12T05:18:58.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 Processor -- start 2024-03-12T05:18:58.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 -- start start 2024-03-12T05:18:58.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b690140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b699e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b6886f0 con 0x55c63b689d30 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b6953c0 con 0x55c63b683840 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b68ddf0 con 0x55c63b689380 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a1088f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b699e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a1088f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b699e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34806/0 (socket says 172.21.15.100:34806) 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b690140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.039+0000 7f9a1088f640 1 -- 172.21.15.100:0/1909758581 learned_addr learned my addr 172.21.15.100:0/1909758581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a1088f640 1 -- 172.21.15.100:0/1909758581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 msgr2=0x55c63b6998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a1088f640 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a1088f640 1 -- 172.21.15.100:0/1909758581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 msgr2=0x55c63b690140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a1088f640 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b690140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a1088f640 1 -- 172.21.15.100:0/1909758581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c63b58bc00 con 0x55c63b689d30 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a0b7fe640 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b690140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f9a1088f640 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b699e20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9a04013010 tx=0x7f9a04004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8a9134de5d7cd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f99f1ffb640 1 -- 172.21.15.100:0/1909758581 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04017070 con 0x55c63b689d30 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f99f1ffb640 1 -- 172.21.15.100:0/1909758581 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a0400eed0 con 0x55c63b689d30 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.040+0000 7f99f1ffb640 1 -- 172.21.15.100:0/1909758581 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04021da0 con 0x55c63b689d30 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1909758581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 msgr2=0x55c63b699e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b699e20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9a04013010 tx=0x7f9a04004ab0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1909758581 shutdown_connections 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b699e20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1909758581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b690140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1909758581 >> 172.21.15.100:0/1909758581 conn(0x55c63b68a480 msgr2=0x55c63b682050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1909758581 shutdown_connections 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1909758581 wait complete. 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 -- start start 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.041+0000 7f9a14f9d740 1 Processor -- start 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b6f3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b698c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b699350 con 0x55c63b689d30 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b6994c0 con 0x55c63b683840 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63b699680 con 0x55c63b689380 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 10 client.0 _ll_register_callbacks cb 0x7f9a000963e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b698c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b698c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34814/0 (socket says 172.21.15.100:34814) 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 -- 172.21.15.100:0/1239430129 learned_addr learned my addr 172.21.15.100:0/1239430129 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b6f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 -- 172.21.15.100:0/1239430129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 msgr2=0x55c63b6f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 -- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 msgr2=0x55c63b6f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b6f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 -- 172.21.15.100:0/1239430129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c10e840 con 0x55c63b689d30 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a1088f640 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b698c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009370 tx=0x7f9a04009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833f7fadeeb302d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f99eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f99eb7fe640 1 -- 172.21.15.100:0/1239430129 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04021740 con 0x55c63b689d30 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f99eb7fe640 1 -- 172.21.15.100:0/1239430129 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9a04021440 con 0x55c63b689d30 2024-03-12T05:18:58.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.042+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c63b616900 con 0x55c63b689d30 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f99eb7fe640 1 -- 172.21.15.100:0/1239430129 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a04010bb0 con 0x55c63b689d30 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 10 client.16980 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c63b5ef320 con 0x55c63b689d30 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f99eb7fe640 1 -- 172.21.15.100:0/1239430129 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a04040020 con 0x55c63b689d30 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f99eb7fe640 1 -- 172.21.15.100:0/1239430129 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a0402abe0 con 0x55c63b689d30 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 10 client.16980 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 20 client.16980 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6928b0 0x55c63b6fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 4 client.16980 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c63b2f0860 con 0x55c63b6928b0 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6928b0 0x55c63b6fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f99eb7fe640 10 client.16980 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.043+0000 7f9a0bfff640 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6928b0 0x55c63b6fd410 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c8458c117d666 server_cookie=2f651f1b7226c45d in_seq=0 out_seq=0 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f99eb7fe640 1 -- 172.21.15.100:0/1239430129 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c63b2f0860 con 0x55c63b6928b0 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f99eb7fe640 10 client.16980 handle_command_reply: tid=0 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 client.16980 shutdown 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 msgr2=0x55c63b698c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b698c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009370 tx=0x7f9a04009ac0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 shutdown_connections 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c63b6928b0 0x55c63b6fd410 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c63b689d30 0x55c63b698c80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c63b689380 0x55c63b6f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 --2- 172.21.15.100:0/1239430129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c63b683840 0x55c63b6f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 >> 172.21.15.100:0/1239430129 conn(0x55c63b68a480 msgr2=0x55c63b6828e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f99eb7fe640 0 client.16980 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 shutdown_connections 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 1 -- 172.21.15.100:0/1239430129 wait complete. 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a14f9d740 20 client.16980 trim_cache size 0 max 16384 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:18:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:18:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "bf378c44-c00e-4337-84e2-fb438ba8fa17", 2024-03-12T05:18:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:18:58.048 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:18:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99f40a1ba0 msgr2=0x7f99f40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99f40a1ba0 0x7f99f40a4060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009fd0 tx=0x7f9a04009a50 comp rx=0 tx=0).stop 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 msgr2=0x7f9a0c1005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.045+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 0x7f9a0c1005a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e9f0 tx=0x7f9a0000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 shutdown_connections 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99f40a1ba0 0x7f99f40a4060 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a0c10b750 0x7f9a0c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c102d30 0x7f9a0c1005a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 --2- 172.21.15.100:0/2289567642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a0c102410 0x7f9a0c100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 >> 172.21.15.100:0/2289567642 conn(0x7f9a0c0771e0 msgr2=0x7f9a0c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 shutdown_connections 2024-03-12T05:18:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.046+0000 7f9a122f2640 1 -- 172.21.15.100:0/2289567642 wait complete. 2024-03-12T05:18:58.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:18:58.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:18:58.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 -- 172.21.15.100:0/2482313177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4100270 msgr2=0x7fbfe4100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 --2- 172.21.15.100:0/2482313177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4100270 0x7fbfe4100650 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009930 tx=0x7fbfd402bc30 comp rx=0 tx=0).stop 2024-03-12T05:18:58.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 -- 172.21.15.100:0/2482313177 shutdown_connections 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 --2- 172.21.15.100:0/2482313177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfe4105340 0x7fbfe410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 --2- 172.21.15.100:0/2482313177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100c20 0x7fbfe4104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 --2- 172.21.15.100:0/2482313177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4100270 0x7fbfe4100650 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.166+0000 7fbfea93e640 1 -- 172.21.15.100:0/2482313177 >> 172.21.15.100:0/2482313177 conn(0x7fbfe4077230 msgr2=0x7fbfe4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 -- 172.21.15.100:0/2482313177 shutdown_connections 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 -- 172.21.15.100:0/2482313177 wait complete. 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 Processor -- start 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 -- start start 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100270 0x7fbfe419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfe4105340 0x7fbfe419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 0x7fbfe4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe4197ff0 con 0x7fbfe4197600 2024-03-12T05:18:58.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe4198160 con 0x7fbfe4105340 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfea93e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe4198320 con 0x7fbfe4100270 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfe8edb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 0x7fbfe4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100270 0x7fbfe419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfe8edb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 0x7fbfe4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34846/0 (socket says 172.21.15.100:34846) 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfe4105340 0x7fbfe419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.167+0000 7fbfe8edb640 1 -- 172.21.15.100:0/682012973 learned_addr learned my addr 172.21.15.100:0/682012973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe8edb640 1 -- 172.21.15.100:0/682012973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100270 msgr2=0x7fbfe419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe8edb640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100270 0x7fbfe419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe8edb640 1 -- 172.21.15.100:0/682012973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfe4105340 msgr2=0x7fbfe419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe8edb640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfe4105340 0x7fbfe419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe8edb640 1 -- 172.21.15.100:0/682012973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009490 con 0x7fbfe4197600 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100270 0x7fbfe419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:58.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe8edb640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 0x7fbfe4197ab0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00efd0 tx=0x7fbfdc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe1ffb640 1 -- 172.21.15.100:0/682012973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc009b40 con 0x7fbfe4197600 2024-03-12T05:18:58.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe1ffb640 1 -- 172.21.15.100:0/682012973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc003b70 con 0x7fbfe4197600 2024-03-12T05:18:58.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc009840 con 0x7fbfe4197600 2024-03-12T05:18:58.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfe1ffb640 1 -- 172.21.15.100:0/682012973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0108f0 con 0x7fbfe4197600 2024-03-12T05:18:58.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.168+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4009560 con 0x7fbfe4197600 2024-03-12T05:18:58.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfe1ffb640 1 -- 172.21.15.100:0/682012973 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fbfdc003ce0 con 0x7fbfe4197600 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 Processor -- start 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 -- start start 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb6c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb79680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb7bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcadb280 con 0x5609fcb79bc0 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcad87b0 con 0x5609fcb6c900 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcad3f90 con 0x5609fcb6bfe0 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb7bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfe1ffb640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbfd00a1c20 0x7fbfd00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.171+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb7bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34862/0 (socket says 172.21.15.100:34862) 2024-03-12T05:18:58.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 -- 172.21.15.100:0/3422549233 learned_addr learned my addr 172.21.15.100:0/3422549233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb79680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb6c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfe8edb640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbfd00a1c20 0x7fbfd00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 -- 172.21.15.100:0/3422549233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 msgr2=0x5609fcb6c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb6c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 -- 172.21.15.100:0/3422549233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 msgr2=0x5609fcb79680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb79680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 -- 172.21.15.100:0/3422549233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609fca92570 con 0x5609fcb79bc0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfe1ffb640 1 -- 172.21.15.100:0/682012973 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbfdc014070 con 0x7fbfe4197600 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb7bfb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0093e0 tx=0x7fbfcc009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e5190e660f24f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfe8edb640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbfd00a1c20 0x7fbfd00a40e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01b000 tx=0x7fbfdc096c20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfda7fc640 1 -- 172.21.15.100:0/3422549233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00f7c0 con 0x5609fcb79bc0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfda7fc640 1 -- 172.21.15.100:0/3422549233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc00fe00 con 0x5609fcb79bc0 2024-03-12T05:18:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.172+0000 7fbfda7fc640 1 -- 172.21.15.100:0/3422549233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc01fea0 con 0x5609fcb79bc0 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 -- 172.21.15.100:0/3422549233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 msgr2=0x5609fcb7bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb7bfb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0093e0 tx=0x7fbfcc009370 comp rx=0 tx=0).stop 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 -- 172.21.15.100:0/3422549233 shutdown_connections 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb7bfb0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb79680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/3422549233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb6c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 -- 172.21.15.100:0/3422549233 >> 172.21.15.100:0/3422549233 conn(0x5609fcb66990 msgr2=0x5609fcb66d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 -- 172.21.15.100:0/3422549233 shutdown_connections 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 -- 172.21.15.100:0/3422549233 wait complete. 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 -- start start 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.173+0000 7fbfed5ec740 1 Processor -- start 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb79680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb76a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb77150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fc7a7430 con 0x5609fcb79bc0 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fc763e00 con 0x5609fcb6c900 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fca58fd0 con 0x5609fcb6bfe0 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfed5ec740 10 client.0 _ll_register_callbacks cb 0x5609fcaaefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb77150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb77150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34872/0 (socket says 172.21.15.100:34872) 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 -- 172.21.15.100:0/533161936 learned_addr learned my addr 172.21.15.100:0/533161936 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb76a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb79680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 -- 172.21.15.100:0/533161936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 msgr2=0x5609fcb79680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb79680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 -- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 msgr2=0x5609fcb76a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb76a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 -- 172.21.15.100:0/533161936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc019040 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb76a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb77150 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc01f970 tx=0x7fbfcc01faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c005916a5b089c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.174+0000 7fbfc37fe640 1 -- 172.21.15.100:0/533161936 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00fa60 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfc37fe640 1 -- 172.21.15.100:0/533161936 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbfcc00fbc0 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609fc7d37e0 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfc37fe640 1 -- 172.21.15.100:0/533161936 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfcc005050 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 10 client.16992 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5609fcad2610 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfc37fe640 1 -- 172.21.15.100:0/533161936 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5609fc7d37e0 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfc37fe640 1 -- 172.21.15.100:0/533161936 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbfcc004cd0 con 0x5609fcb79bc0 2024-03-12T05:18:58.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 10 client.16992 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 20 client.16992 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcb89c90 0x5609fcb8c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 4 client.16992 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609fc810720 con 0x5609fcb89c90 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.175+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcb89c90 0x5609fcb8c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.176+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcb89c90 0x5609fcb8c080 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ed27ee5bfda425 server_cookie=6ecddf0d60a81c4e in_seq=0 out_seq=0 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.176+0000 7fbfc37fe640 10 client.16992 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.176+0000 7fbfc37fe640 1 -- 172.21.15.100:0/533161936 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609fc810720 con 0x5609fcb89c90 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.176+0000 7fbfc37fe640 10 client.16992 handle_command_reply: tid=0 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 client.16992 shutdown 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 msgr2=0x5609fcb77150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb77150 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc01f970 tx=0x7fbfcc01faa0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 shutdown_connections 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcb89c90 0x5609fcb8c080 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb77150 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6c900 0x5609fcb76a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/533161936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6bfe0 0x5609fcb79680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfc37fe640 0 client.16992 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 >> 172.21.15.100:0/533161936 conn(0x5609fcb66990 msgr2=0x5609fcb68730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 shutdown_connections 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 1 -- 172.21.15.100:0/533161936 wait complete. 2024-03-12T05:18:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.177+0000 7fbfed5ec740 20 client.16992 trim_cache size 0 max 16384 2024-03-12T05:18:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 Processor -- start 2024-03-12T05:18:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 -- start start 2024-03-12T05:18:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6c900 0x5609fcb79140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb79680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcb62320 con 0x5609fcb79bc0 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcb6b620 con 0x5609fcb6bfe0 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcb89a90 con 0x5609fcb6c900 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb79680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb79680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34886/0 (socket says 172.21.15.100:34886) 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 -- 172.21.15.100:0/357752993 learned_addr learned my addr 172.21.15.100:0/357752993 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6c900 0x5609fcb79140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 -- 172.21.15.100:0/357752993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6c900 msgr2=0x5609fcb79140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6c900 0x5609fcb79140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 -- 172.21.15.100:0/357752993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 msgr2=0x5609fcb909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb909a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 -- 172.21.15.100:0/357752993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609fca6eac0 con 0x5609fcb79bc0 2024-03-12T05:18:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb79680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009cf0 tx=0x7fbfcc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d649719f87f3b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfd9ffb640 1 -- 172.21.15.100:0/357752993 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc009e50 con 0x5609fcb79bc0 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfd9ffb640 1 -- 172.21.15.100:0/357752993 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc026070 con 0x5609fcb79bc0 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.192+0000 7fbfd9ffb640 1 -- 172.21.15.100:0/357752993 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc01fd20 con 0x5609fcb79bc0 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 -- 172.21.15.100:0/357752993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 msgr2=0x5609fcb79680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb79680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009cf0 tx=0x7fbfcc005970 comp rx=0 tx=0).stop 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 -- 172.21.15.100:0/357752993 shutdown_connections 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb79bc0 0x5609fcb79680 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb6c900 0x5609fcb79140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/357752993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 -- 172.21.15.100:0/357752993 >> 172.21.15.100:0/357752993 conn(0x5609fcb66990 msgr2=0x5609fcb66d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 -- 172.21.15.100:0/357752993 shutdown_connections 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 -- 172.21.15.100:0/357752993 wait complete. 2024-03-12T05:18:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.193+0000 7fbfed5ec740 1 -- start start 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 Processor -- start 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb80cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 0x5609fcb7d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb79bc0 0x5609fcb7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcb7db50 con 0x5609fcb6c900 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcb7dcc0 con 0x5609fcb6bfe0 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fcb7de80 con 0x5609fcb79bc0 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfed5ec740 10 client.0 _ll_register_callbacks cb 0x7fbfcc009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 0x5609fcb7d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 0x5609fcb7d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34898/0 (socket says 172.21.15.100:34898) 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 -- 172.21.15.100:0/346130370 learned_addr learned my addr 172.21.15.100:0/346130370 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb80cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb79bc0 0x5609fcb7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 -- 172.21.15.100:0/346130370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb79bc0 msgr2=0x5609fcb7d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb79bc0 0x5609fcb7d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 -- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 msgr2=0x5609fcb80cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb80cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 -- 172.21.15.100:0/346130370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609fcad2610 con 0x5609fcb6c900 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb79bc0 0x5609fcb7d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfe3fff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 0x5609fcb7d0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009900 tx=0x7fbfd40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6598513e4fd308d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.194+0000 7fbfc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfc37fe640 1 -- 172.21.15.100:0/346130370 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4049be0 con 0x5609fcb6c900 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfc37fe640 1 -- 172.21.15.100:0/346130370 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbfd4046040 con 0x5609fcb6c900 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfcc00fe00 con 0x5609fcb6c900 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfc37fe640 1 -- 172.21.15.100:0/346130370 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd4049d40 con 0x5609fcb6c900 2024-03-12T05:18:58.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 10 client.16998 fetch_fsmap learned FSMap version 25 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5609fca946a0 con 0x5609fcb6c900 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfc37fe640 1 -- 172.21.15.100:0/346130370 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc00fe00 con 0x5609fcb6c900 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfc37fe640 1 -- 172.21.15.100:0/346130370 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbfd403c040 con 0x5609fcb6c900 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 10 client.16998 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 20 client.16998 populate_metadata read hostname 'smithi100' 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcbe17d0 0x5609fcbe3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 4 client.16998 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609fc85f890 con 0x5609fcbe17d0 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.195+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcbe17d0 0x5609fcbe3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfdbfff640 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcbe17d0 0x5609fcbe3bf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a659da4510ad1ec5 server_cookie=9c181e4676793f89 in_seq=0 out_seq=0 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfc37fe640 10 client.16998 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfc37fe640 1 -- 172.21.15.100:0/346130370 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5609fc85f890 con 0x5609fcbe17d0 2024-03-12T05:18:58.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfc37fe640 10 client.16998 handle_command_reply: tid=0 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 client.16998 shutdown 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 msgr2=0x5609fcb7d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 0x5609fcb7d0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009900 tx=0x7fbfd40041a0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 shutdown_connections 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5609fcbe17d0 0x5609fcbe3bf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609fcb79bc0 0x5609fcb7d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609fcb6c900 0x5609fcb7d0a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfc37fe640 0 client.16998 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 --2- 172.21.15.100:0/346130370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5609fcb6bfe0 0x5609fcb80cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 >> 172.21.15.100:0/346130370 conn(0x5609fcb66990 msgr2=0x5609fcb909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 shutdown_connections 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 1 -- 172.21.15.100:0/346130370 wait complete. 2024-03-12T05:18:58.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.196+0000 7fbfed5ec740 20 client.16998 trim_cache size 0 max 16384 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bf378c44-c00e-4337-84e2-fb438ba8fa17": { 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "bf378c44-c00e-4337-84e2-fb438ba8fa17", 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:18:58.200 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:18:58.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbfd00a1c20 msgr2=0x7fbfd00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbfd00a1c20 0x7fbfd00a40e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01b000 tx=0x7fbfdc096c20 comp rx=0 tx=0).stop 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 msgr2=0x7fbfe4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 0x7fbfe4197ab0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00efd0 tx=0x7fbfdc00c6a0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 shutdown_connections 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbfd00a1c20 0x7fbfd00a40e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfe4197600 0x7fbfe4197ab0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfe4105340 0x7fbfe419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 --2- 172.21.15.100:0/682012973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe4100270 0x7fbfe419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 >> 172.21.15.100:0/682012973 conn(0x7fbfe4077230 msgr2=0x7fbfe40fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 shutdown_connections 2024-03-12T05:18:58.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:18:58.197+0000 7fbfea93e640 1 -- 172.21.15.100:0/682012973 wait complete. 2024-03-12T05:18:58.203 INFO:tasks.cephfs.filesystem:scrub status for tag:bf378c44-c00e-4337-84e2-fb438ba8fa17 - {'path': '/', 'tag': 'bf378c44-c00e-4337-84e2-fb438ba8fa17', 'options': 'recursive,force'} 2024-03-12T05:18:58.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 808530 12.22 MB/sec execute 222 sec latency 15.963 ms 2024-03-12T05:18:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: pgmap v209: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-03-12T05:18:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:18:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:18:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:18:59.205 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.206 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:18:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: pgmap v209: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:18:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: pgmap v209: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-03-12T05:18:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:18:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:18:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:18:59.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:18:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:18:59.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 810494 12.22 MB/sec execute 223 sec latency 15.243 ms 2024-03-12T05:19:00.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 812669 12.22 MB/sec execute 224 sec latency 130.931 ms 2024-03-12T05:19:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:00 smithi156 ceph-mon[36270]: pgmap v210: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 203 op/s 2024-03-12T05:19:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:00 smithi161 ceph-mon[36043]: pgmap v210: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 203 op/s 2024-03-12T05:19:01.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:00 smithi100 ceph-mon[32210]: pgmap v210: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 203 op/s 2024-03-12T05:19:01.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 814013 12.19 MB/sec execute 225 sec latency 26.310 ms 2024-03-12T05:19:02.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 816801 12.22 MB/sec execute 226 sec latency 15.906 ms 2024-03-12T05:19:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:02 smithi156 ceph-mon[36270]: pgmap v211: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 337 op/s 2024-03-12T05:19:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:02 smithi161 ceph-mon[36043]: pgmap v211: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 337 op/s 2024-03-12T05:19:03.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:02 smithi100 ceph-mon[32210]: pgmap v211: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 337 op/s 2024-03-12T05:19:03.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 818370 12.19 MB/sec execute 227 sec latency 40.012 ms 2024-03-12T05:19:04.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 820728 12.20 MB/sec execute 228 sec latency 13.247 ms 2024-03-12T05:19:05.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:04 smithi156 ceph-mon[36270]: pgmap v212: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2024-03-12T05:19:05.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:04 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:19:05.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:04 smithi161 ceph-mon[36043]: pgmap v212: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2024-03-12T05:19:05.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:04 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:19:05.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:04 smithi100 ceph-mon[32210]: pgmap v212: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2024-03-12T05:19:05.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:04 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:19:05.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 822706 12.18 MB/sec execute 229 sec latency 141.453 ms 2024-03-12T05:19:06.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:05 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:19:06.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:05 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:19:06.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:05 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:19:06.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 824567 12.18 MB/sec execute 230 sec latency 94.280 ms 2024-03-12T05:19:07.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:06 smithi156 ceph-mon[36270]: pgmap v213: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2024-03-12T05:19:07.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:06 smithi161 ceph-mon[36043]: pgmap v213: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2024-03-12T05:19:07.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:06 smithi100 ceph-mon[32210]: pgmap v213: 97 pgs: 97 active+clean; 420 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2024-03-12T05:19:07.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 827300 12.20 MB/sec execute 231 sec latency 16.265 ms 2024-03-12T05:19:08.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 828931 12.17 MB/sec execute 232 sec latency 17.396 ms 2024-03-12T05:19:09.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:08 smithi156 ceph-mon[36270]: pgmap v214: 97 pgs: 97 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 334 op/s 2024-03-12T05:19:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:08 smithi161 ceph-mon[36043]: pgmap v214: 97 pgs: 97 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 334 op/s 2024-03-12T05:19:09.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:08 smithi100 ceph-mon[32210]: pgmap v214: 97 pgs: 97 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 334 op/s 2024-03-12T05:19:09.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 831368 12.18 MB/sec execute 233 sec latency 15.244 ms 2024-03-12T05:19:10.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 833273 12.16 MB/sec execute 234 sec latency 119.340 ms 2024-03-12T05:19:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:10 smithi156 ceph-mon[36270]: pgmap v215: 97 pgs: 97 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-03-12T05:19:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:10 smithi156 ceph-mon[36270]: osdmap e90: 12 total, 12 up, 12 in 2024-03-12T05:19:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:10 smithi161 ceph-mon[36043]: pgmap v215: 97 pgs: 97 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-03-12T05:19:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:10 smithi161 ceph-mon[36043]: osdmap e90: 12 total, 12 up, 12 in 2024-03-12T05:19:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:10 smithi100 ceph-mon[32210]: pgmap v215: 97 pgs: 97 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-03-12T05:19:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:10 smithi100 ceph-mon[32210]: osdmap e90: 12 total, 12 up, 12 in 2024-03-12T05:19:11.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1 835387 12.17 MB/sec execute 235 sec latency 24.627 ms 2024-03-12T05:19:12.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 838138 12.19 MB/sec execute 236 sec latency 15.561 ms 2024-03-12T05:19:13.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:12 smithi156 ceph-mon[36270]: pgmap v217: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-03-12T05:19:13.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:12 smithi161 ceph-mon[36043]: pgmap v217: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-03-12T05:19:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:12 smithi100 ceph-mon[32210]: pgmap v217: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-03-12T05:19:13.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 839798 12.16 MB/sec execute 237 sec latency 39.910 ms 2024-03-12T05:19:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 842113 12.17 MB/sec execute 238 sec latency 18.999 ms 2024-03-12T05:19:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:14 smithi156 ceph-mon[36270]: pgmap v218: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-03-12T05:19:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:14 smithi161 ceph-mon[36043]: pgmap v218: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-03-12T05:19:15.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:14 smithi100 ceph-mon[32210]: pgmap v218: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-03-12T05:19:15.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 844387 12.17 MB/sec execute 239 sec latency 92.767 ms 2024-03-12T05:19:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 846430 12.17 MB/sec execute 240 sec latency 19.685 ms 2024-03-12T05:19:17.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:16 smithi100 ceph-mon[32210]: osdmap e91: 12 total, 12 up, 12 in 2024-03-12T05:19:17.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:16 smithi100 ceph-mon[32210]: pgmap v220: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-03-12T05:19:17.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:16 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:19:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:16 smithi156 ceph-mon[36270]: osdmap e91: 12 total, 12 up, 12 in 2024-03-12T05:19:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:16 smithi156 ceph-mon[36270]: pgmap v220: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-03-12T05:19:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:16 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:19:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:16 smithi161 ceph-mon[36043]: osdmap e91: 12 total, 12 up, 12 in 2024-03-12T05:19:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:16 smithi161 ceph-mon[36043]: pgmap v220: 97 pgs: 97 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-03-12T05:19:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:16 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:19:17.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 848938 12.17 MB/sec execute 241 sec latency 17.205 ms 2024-03-12T05:19:18.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:17 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:19:18.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:17 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:19:18.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:17 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:19:18.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:17 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:19:18.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:17 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:19:18.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:17 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:19:18.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:17 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:19:18.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:17 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:19:18.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:17 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:19:18.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 851042 12.15 MB/sec execute 242 sec latency 28.777 ms 2024-03-12T05:19:19.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:18 smithi100 ceph-mon[32210]: pgmap v221: 97 pgs: 97 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 308 op/s 2024-03-12T05:19:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:18 smithi156 ceph-mon[36270]: pgmap v221: 97 pgs: 97 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 308 op/s 2024-03-12T05:19:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:18 smithi161 ceph-mon[36043]: pgmap v221: 97 pgs: 97 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 308 op/s 2024-03-12T05:19:19.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 853253 12.16 MB/sec execute 243 sec latency 17.254 ms 2024-03-12T05:19:20.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 855662 12.17 MB/sec execute 244 sec latency 63.647 ms 2024-03-12T05:19:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:20 smithi156 ceph-mon[36270]: pgmap v222: 97 pgs: 97 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 249 op/s 2024-03-12T05:19:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:20 smithi161 ceph-mon[36043]: pgmap v222: 97 pgs: 97 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 249 op/s 2024-03-12T05:19:21.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:20 smithi100 ceph-mon[32210]: pgmap v222: 97 pgs: 97 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 249 op/s 2024-03-12T05:19:21.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 857275 12.15 MB/sec execute 245 sec latency 57.196 ms 2024-03-12T05:19:22.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 859581 12.16 MB/sec execute 246 sec latency 15.013 ms 2024-03-12T05:19:23.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:22 smithi156 ceph-mon[36270]: pgmap v223: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-03-12T05:19:23.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:22 smithi161 ceph-mon[36043]: pgmap v223: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-03-12T05:19:23.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:22 smithi100 ceph-mon[32210]: pgmap v223: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-03-12T05:19:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 861604 12.14 MB/sec execute 247 sec latency 18.690 ms 2024-03-12T05:19:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 863723 12.14 MB/sec execute 248 sec latency 17.058 ms 2024-03-12T05:19:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:24 smithi156 ceph-mon[36270]: pgmap v224: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-03-12T05:19:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:24 smithi161 ceph-mon[36043]: pgmap v224: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-03-12T05:19:25.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:24 smithi100 ceph-mon[32210]: pgmap v224: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-03-12T05:19:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 866272 12.16 MB/sec execute 249 sec latency 113.193 ms 2024-03-12T05:19:26.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 867812 12.14 MB/sec execute 250 sec latency 42.768 ms 2024-03-12T05:19:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:26 smithi156 ceph-mon[36270]: pgmap v225: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 247 op/s 2024-03-12T05:19:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:26 smithi161 ceph-mon[36043]: pgmap v225: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 247 op/s 2024-03-12T05:19:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:26 smithi100 ceph-mon[32210]: pgmap v225: 97 pgs: 97 active+clean; 458 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 247 op/s 2024-03-12T05:19:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 870226 12.15 MB/sec execute 251 sec latency 16.799 ms 2024-03-12T05:19:28.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:19:28.204 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 -- 172.21.15.100:0/141515011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100c20 msgr2=0x7f406c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 --2- 172.21.15.100:0/141515011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100c20 0x7f406c104c60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f405c009930 tx=0x7f405c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 -- 172.21.15.100:0/141515011 shutdown_connections 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 --2- 172.21.15.100:0/141515011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406c105340 0x7f406c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 --2- 172.21.15.100:0/141515011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100c20 0x7f406c104c60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 --2- 172.21.15.100:0/141515011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c100270 0x7f406c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 -- 172.21.15.100:0/141515011 >> 172.21.15.100:0/141515011 conn(0x7f406c077230 msgr2=0x7f406c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 -- 172.21.15.100:0/141515011 shutdown_connections 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.319+0000 7f4073313640 1 -- 172.21.15.100:0/141515011 wait complete. 2024-03-12T05:19:28.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 Processor -- start 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 -- start start 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 0x7f406c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406c105340 0x7f406c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197630 0x7f406c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c198100 con 0x7f406c100270 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c198270 con 0x7f406c197630 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f4073313640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c198430 con 0x7f406c105340 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 0x7f406c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 0x7f406c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52700/0 (socket says 172.21.15.100:52700) 2024-03-12T05:19:28.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 -- 172.21.15.100:0/4087980284 learned_addr learned my addr 172.21.15.100:0/4087980284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40718b0640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197630 0x7f406c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 -- 172.21.15.100:0/4087980284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406c105340 msgr2=0x7f406c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40708ae640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406c105340 0x7f406c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406c105340 0x7f406c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 -- 172.21.15.100:0/4087980284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197630 msgr2=0x7f406c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197630 0x7f406c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 -- 172.21.15.100:0/4087980284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c009560 con 0x7f406c100270 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40710af640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 0x7f406c19d570 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f406800d950 tx=0x7f406800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.320+0000 7f40627fc640 1 -- 172.21.15.100:0/4087980284 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068009a30 con 0x7f406c100270 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.321+0000 7f40627fc640 1 -- 172.21.15.100:0/4087980284 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4068009b90 con 0x7f406c100270 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.321+0000 7f40627fc640 1 -- 172.21.15.100:0/4087980284 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068004a00 con 0x7f406c100270 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.321+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c1a43d0 con 0x7f406c100270 2024-03-12T05:19:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.321+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c1a48b0 con 0x7f406c100270 2024-03-12T05:19:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40627fc640 1 -- 172.21.15.100:0/4087980284 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4068009d00 con 0x7f406c100270 2024-03-12T05:19:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 Processor -- start 2024-03-12T05:19:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 -- start start 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d88fe0 0x562640d893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 0x562640d98fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640cf8280 con 0x562640d96bc0 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640cf57b0 con 0x562640d88fe0 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f4075fc7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640cf0f90 con 0x562640d89900 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40708ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 0x562640d98fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40718b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d88fe0 0x562640d893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40708ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 0x562640d98fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52702/0 (socket says 172.21.15.100:52702) 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40708ae640 1 -- 172.21.15.100:0/2499866154 learned_addr learned my addr 172.21.15.100:0/2499866154 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40710af640 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40627fc640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40580a1ba0 0x7f40580a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40718b0640 1 -- 172.21.15.100:0/2499866154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 msgr2=0x562640d96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40718b0640 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d96680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40718b0640 1 -- 172.21.15.100:0/2499866154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 msgr2=0x562640d98fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40718b0640 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 0x562640d98fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40718b0640 1 -- 172.21.15.100:0/2499866154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562640caf570 con 0x562640d88fe0 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40708ae640 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 0x562640d98fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.324+0000 7f40627fc640 1 -- 172.21.15.100:0/4087980284 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40680cd230 con 0x7f406c100270 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.325+0000 7f40718b0640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40580a1ba0 0x7f40580a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.325+0000 7f40718b0640 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d88fe0 0x562640d893c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f406400a400 tx=0x7f406400a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a3fbfe2b5d7eb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.325+0000 7f40577fe640 1 -- 172.21.15.100:0/2499866154 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640177a0 con 0x562640d88fe0 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.325+0000 7f40577fe640 1 -- 172.21.15.100:0/2499866154 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4064017de0 con 0x562640d88fe0 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.325+0000 7f40577fe640 1 -- 172.21.15.100:0/2499866154 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064010070 con 0x562640d88fe0 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 -- 172.21.15.100:0/2499866154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d88fe0 msgr2=0x562640d893c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f40718b0640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40580a1ba0 0x7f40580a4060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4064020000 tx=0x7f4064021000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d88fe0 0x562640d893c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f406400a400 tx=0x7f406400a8c0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 -- 172.21.15.100:0/2499866154 shutdown_connections 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d96bc0 0x562640d98fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d96680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 --2- 172.21.15.100:0/2499866154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d88fe0 0x562640d893c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 -- 172.21.15.100:0/2499866154 >> 172.21.15.100:0/2499866154 conn(0x562640d83990 msgr2=0x562640d83d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 -- 172.21.15.100:0/2499866154 shutdown_connections 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 -- 172.21.15.100:0/2499866154 wait complete. 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 -- start start 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.326+0000 7f4075fc7740 1 Processor -- start 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d96630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626409c4430 con 0x562640d88fe0 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640980e00 con 0x562640d89900 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640c75fd0 con 0x562640d96bc0 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 10 client.0 _ll_register_callbacks cb 0x562640ccbfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d96630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40708ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d96630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52712/0 (socket says 172.21.15.100:52712) 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 -- 172.21.15.100:0/866614284 learned_addr learned my addr 172.21.15.100:0/866614284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40718b0640 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 -- 172.21.15.100:0/866614284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 msgr2=0x562640d94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d94100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 -- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 msgr2=0x562640d939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 -- 172.21.15.100:0/866614284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4064009bf0 con 0x562640d88fe0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f40710af640 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d96630 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4068099b20 tx=0x7f4068005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed6fbdd2d0afe11 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4054ff9640 1 -- 172.21.15.100:0/866614284 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068099780 con 0x562640d88fe0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4054ff9640 1 -- 172.21.15.100:0/866614284 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f406809a520 con 0x562640d88fe0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626409f07e0 con 0x562640d88fe0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4054ff9640 1 -- 172.21.15.100:0/866614284 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4068099e20 con 0x562640d88fe0 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 10 client.17010 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.327+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562640cef610 con 0x562640d88fe0 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4054ff9640 1 -- 172.21.15.100:0/866614284 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40680b1070 con 0x562640d88fe0 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4054ff9640 1 -- 172.21.15.100:0/866614284 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40680d0b90 con 0x562640d88fe0 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4075fc7740 10 client.17010 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4075fc7740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4075fc7740 20 client.17010 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4075fc7740 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640da6d80 0x562640da9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4075fc7740 4 client.17010 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562640a2d720 con 0x562640da6d80 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f40708ae640 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640da6d80 0x562640da9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f4054ff9640 10 client.17010 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.328+0000 7f40708ae640 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640da6d80 0x562640da9170 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da542b282e7f557 server_cookie=e1a378afe0066ad0 in_seq=0 out_seq=0 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4054ff9640 1 -- 172.21.15.100:0/866614284 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562640a2d720 con 0x562640da6d80 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4054ff9640 10 client.17010 handle_command_reply: tid=0 2024-03-12T05:19:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 client.17010 shutdown 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 msgr2=0x562640d96630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d96630 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4068099b20 tx=0x7f4068005cd0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 shutdown_connections 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640da6d80 0x562640da9170 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4054ff9640 0 client.17010 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 --2- 172.21.15.100:0/866614284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d96630 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 >> 172.21.15.100:0/866614284 conn(0x562640d83990 msgr2=0x562640d85730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 shutdown_connections 2024-03-12T05:19:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 1 -- 172.21.15.100:0/866614284 wait complete. 2024-03-12T05:19:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.329+0000 7f4075fc7740 20 client.17010 trim_cache size 0 max 16384 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 Processor -- start 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 -- start start 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640d7f120 con 0x562640d88fe0 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640d847a0 con 0x562640d89900 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f4075fc7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640d88ba0 con 0x562640d96bc0 2024-03-12T05:19:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d929a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52720/0 (socket says 172.21.15.100:52720) 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40710af640 1 -- 172.21.15.100:0/1556034034 learned_addr learned my addr 172.21.15.100:0/1556034034 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40708ae640 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40718b0640 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d9dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40710af640 1 -- 172.21.15.100:0/1556034034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 msgr2=0x562640d9dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40710af640 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d9dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.345+0000 7f40710af640 1 -- 172.21.15.100:0/1556034034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 msgr2=0x562640d9da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f40710af640 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d9da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f40710af640 1 -- 172.21.15.100:0/1556034034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562640c8bac0 con 0x562640d88fe0 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f40718b0640 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f40710af640 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d929a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f40680967f0 tx=0x7f406800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693857f83f4f09dc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f4056ffd640 1 -- 172.21.15.100:0/1556034034 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068099bf0 con 0x562640d88fe0 2024-03-12T05:19:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f4056ffd640 1 -- 172.21.15.100:0/1556034034 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40680b1070 con 0x562640d88fe0 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.346+0000 7f4056ffd640 1 -- 172.21.15.100:0/1556034034 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068005680 con 0x562640d88fe0 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- 172.21.15.100:0/1556034034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 msgr2=0x562640d929a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d929a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f40680967f0 tx=0x7f406800df40 comp rx=0 tx=0).stop 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- 172.21.15.100:0/1556034034 shutdown_connections 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d96bc0 0x562640d9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d89900 0x562640d9da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1556034034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d929a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- 172.21.15.100:0/1556034034 >> 172.21.15.100:0/1556034034 conn(0x562640d83990 msgr2=0x562640d92290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- 172.21.15.100:0/1556034034 shutdown_connections 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- 172.21.15.100:0/1556034034 wait complete. 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- start start 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 Processor -- start 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d9c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d9cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d96bc0 0x562640d9d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640dfd3c0 con 0x562640d88fe0 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640dfd530 con 0x562640d96bc0 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562640dfd6f0 con 0x562640d89900 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f4075fc7740 10 client.0 _ll_register_callbacks cb 0x7f4064009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.347+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d9c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d9c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52734/0 (socket says 172.21.15.100:52734) 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 -- 172.21.15.100:0/1617170684 learned_addr learned my addr 172.21.15.100:0/1617170684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40718b0640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d96bc0 0x562640d9d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40708ae640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d9cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 -- 172.21.15.100:0/1617170684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 msgr2=0x562640d9cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d9cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 -- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d96bc0 msgr2=0x562640d9d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d96bc0 0x562640d9d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 -- 172.21.15.100:0/1617170684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562640cef610 con 0x562640d88fe0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f40710af640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d9c740 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f4068005c60 tx=0x7f40680d09a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd9026bd8f9be80 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4054ff9640 1 -- 172.21.15.100:0/1617170684 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680d0d10 con 0x562640d88fe0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4054ff9640 1 -- 172.21.15.100:0/1617170684 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f406809a520 con 0x562640d88fe0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4064017de0 con 0x562640d88fe0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4054ff9640 1 -- 172.21.15.100:0/1617170684 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4068005680 con 0x562640d88fe0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4075fc7740 10 client.17016 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562640cb16a0 con 0x562640d88fe0 2024-03-12T05:19:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4054ff9640 1 -- 172.21.15.100:0/1617170684 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4064017de0 con 0x562640d88fe0 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.348+0000 7f4054ff9640 1 -- 172.21.15.100:0/1617170684 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4068099270 con 0x562640d88fe0 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 10 client.17016 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 20 client.17016 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640dfe990 0x562640e00d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 4 client.17016 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562640a7c890 con 0x562640dfe990 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f40708ae640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640dfe990 0x562640e00d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4054ff9640 10 client.17016 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f40708ae640 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640dfe990 0x562640e00d80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c325210b68a786d6 server_cookie=2c8c505dadafc485 in_seq=0 out_seq=0 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4054ff9640 1 -- 172.21.15.100:0/1617170684 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562640a7c890 con 0x562640dfe990 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4054ff9640 10 client.17016 handle_command_reply: tid=0 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 client.17016 shutdown 2024-03-12T05:19:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 msgr2=0x562640d9c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d9c740 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f4068005c60 tx=0x7f40680d09a0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 shutdown_connections 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562640dfe990 0x562640e00d80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562640d96bc0 0x562640d9d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4054ff9640 0 client.17016 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562640d89900 0x562640d9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 --2- 172.21.15.100:0/1617170684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562640d88fe0 0x562640d9c740 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.349+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 >> 172.21.15.100:0/1617170684 conn(0x562640d83990 msgr2=0x562640d84ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 shutdown_connections 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4075fc7740 1 -- 172.21.15.100:0/1617170684 wait complete. 2024-03-12T05:19:28.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4075fc7740 20 client.17016 trim_cache size 0 max 16384 2024-03-12T05:19:28.354 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:19:28.354 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:19:28.354 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:19:28.354 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40580a1ba0 msgr2=0x7f40580a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40580a1ba0 0x7f40580a4060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4064020000 tx=0x7f4064021000 comp rx=0 tx=0).stop 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 msgr2=0x7f406c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 0x7f406c19d570 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f406800d950 tx=0x7f406800de10 comp rx=0 tx=0).stop 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 shutdown_connections 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40580a1ba0 0x7f40580a4060 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197630 0x7f406c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406c105340 0x7f406c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 --2- 172.21.15.100:0/4087980284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f406c100270 0x7f406c19d570 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.350+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 >> 172.21.15.100:0/4087980284 conn(0x7f406c077230 msgr2=0x7f406c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.351+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 shutdown_connections 2024-03-12T05:19:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.351+0000 7f4073313640 1 -- 172.21.15.100:0/4087980284 wait complete. 2024-03-12T05:19:28.361 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:19:28.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:19:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 -- 172.21.15.100:0/721259534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 msgr2=0x7f149c10e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 --2- 172.21.15.100:0/721259534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c10e030 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1494009cd0 tx=0x7f149402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 -- 172.21.15.100:0/721259534 shutdown_connections 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 --2- 172.21.15.100:0/721259534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c10e030 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 --2- 172.21.15.100:0/721259534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c101d40 0x7f149c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 --2- 172.21.15.100:0/721259534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f149c101390 0x7f149c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 -- 172.21.15.100:0/721259534 >> 172.21.15.100:0/721259534 conn(0x7f149c0fcc10 msgr2=0x7f149c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 -- 172.21.15.100:0/721259534 shutdown_connections 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 -- 172.21.15.100:0/721259534 wait complete. 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 Processor -- start 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.475+0000 7f14a33c7640 1 -- start start 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f149c101390 0x7f149c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c197630 0x7f149c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c198100 con 0x7f149c106460 2024-03-12T05:19:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c198270 con 0x7f149c197630 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c198430 con 0x7f149c101390 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52740/0 (socket says 172.21.15.100:52740) 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 -- 172.21.15.100:0/1270157988 learned_addr learned my addr 172.21.15.100:0/1270157988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a1163640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f149c101390 0x7f149c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a1964640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c197630 0x7f149c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 -- 172.21.15.100:0/1270157988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f149c101390 msgr2=0x7f149c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f149c101390 0x7f149c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 -- 172.21.15.100:0/1270157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c197630 msgr2=0x7f149c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c197630 0x7f149c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 -- 172.21.15.100:0/1270157988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009900 con 0x7f149c106460 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a1964640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c197630 0x7f149c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a0962640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c19dab0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e9c0 tx=0x7f148c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c009700 con 0x7f149c106460 2024-03-12T05:19:28.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c1a43d0 con 0x7f149c106460 2024-03-12T05:19:28.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f148c00cd80 con 0x7f149c106460 2024-03-12T05:19:28.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c010430 con 0x7f149c106460 2024-03-12T05:19:28.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.476+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c1a4900 con 0x7f149c106460 2024-03-12T05:19:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.477+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c102860 con 0x7f149c106460 2024-03-12T05:19:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.478+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f148c005000 con 0x7f149c106460 2024-03-12T05:19:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.479+0000 7f14927fc640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f14880a1ba0 0x7f14880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.479+0000 7f14a1163640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f14880a1ba0 0x7f14880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.479+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f148c014070 con 0x7f149c106460 2024-03-12T05:19:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.480+0000 7f14a1163640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f14880a1ba0 0x7f14880a4060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1498009bd0 tx=0x7f1498005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.484+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f148c096b00 con 0x7f149c106460 2024-03-12T05:19:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.708+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f149c1096f0 con 0x7f149c106460 2024-03-12T05:19:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.709+0000 7f14927fc640 1 -- 172.21.15.100:0/1270157988 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f148c09a1c0 con 0x7f149c106460 2024-03-12T05:19:28.705 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:19:28.705 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:19:28.705 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:19:28.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f14880a1ba0 msgr2=0x7f14880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f14880a1ba0 0x7f14880a4060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1498009bd0 tx=0x7f1498005f90 comp rx=0 tx=0).stop 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 msgr2=0x7f149c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c19dab0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e9c0 tx=0x7f148c00ee80 comp rx=0 tx=0).stop 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 shutdown_connections 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f14880a1ba0 0x7f14880a4060 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c197630 0x7f149c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c106460 0x7f149c19dab0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 --2- 172.21.15.100:0/1270157988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f149c101390 0x7f149c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 >> 172.21.15.100:0/1270157988 conn(0x7f149c0fcc10 msgr2=0x7f149c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 shutdown_connections 2024-03-12T05:19:28.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.711+0000 7f14a33c7640 1 -- 172.21.15.100:0/1270157988 wait complete. 2024-03-12T05:19:28.717 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:19:28.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:19:28.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 872316 12.13 MB/sec execute 252 sec latency 20.377 ms 2024-03-12T05:19:28.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 -- 172.21.15.100:0/4036817860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0100c10 msgr2=0x7f28f0104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 --2- 172.21.15.100:0/4036817860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0100c10 0x7f28f0104c50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009930 tx=0x7f28dc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:19:28.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 -- 172.21.15.100:0/4036817860 shutdown_connections 2024-03-12T05:19:28.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 --2- 172.21.15.100:0/4036817860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f0105330 0x7f28f010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 --2- 172.21.15.100:0/4036817860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0100c10 0x7f28f0104c50 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 --2- 172.21.15.100:0/4036817860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f0100260 0x7f28f0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 -- 172.21.15.100:0/4036817860 >> 172.21.15.100:0/4036817860 conn(0x7f28f0077180 msgr2=0x7f28f0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 -- 172.21.15.100:0/4036817860 shutdown_connections 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.829+0000 7f28f6c88640 1 -- 172.21.15.100:0/4036817860 wait complete. 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 Processor -- start 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 -- start start 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f0100260 0x7f28f01966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 0x7f28f0196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f019af90 0x7f28f019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019b980 con 0x7f28f0105330 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019baf0 con 0x7f28f019af90 2024-03-12T05:19:28.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019bc60 con 0x7f28f0100260 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 0x7f28f0196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 0x7f28f0196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52766/0 (socket says 172.21.15.100:52766) 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f4a24640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f0100260 0x7f28f01966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 -- 172.21.15.100:0/1558092262 learned_addr learned my addr 172.21.15.100:0/1558092262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f5225640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f019af90 0x7f28f019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 -- 172.21.15.100:0/1558092262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f0100260 msgr2=0x7f28f01966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f0100260 0x7f28f01966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 -- 172.21.15.100:0/1558092262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f019af90 msgr2=0x7f28f019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f019af90 0x7f28f019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 -- 172.21.15.100:0/1558092262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc009560 con 0x7f28f0105330 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28effff640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 0x7f28f0196c10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009230 tx=0x7f28dc02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28edffb640 1 -- 172.21.15.100:0/1558092262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004350 con 0x7f28f0105330 2024-03-12T05:19:28.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28edffb640 1 -- 172.21.15.100:0/1558092262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28dc0044b0 con 0x7f28f0105330 2024-03-12T05:19:28.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f01a3240 con 0x7f28f0105330 2024-03-12T05:19:28.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.830+0000 7f28edffb640 1 -- 172.21.15.100:0/1558092262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc003da0 con 0x7f28f0105330 2024-03-12T05:19:28.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.831+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01a36f0 con 0x7f28f0105330 2024-03-12T05:19:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28edffb640 1 -- 172.21.15.100:0/1558092262 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f28dc004df0 con 0x7f28f0105330 2024-03-12T05:19:28.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 Processor -- start 2024-03-12T05:19:28.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 -- start start 2024-03-12T05:19:28.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a78210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37a2a960 con 0x556d37a6fb70 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d379de310 con 0x556d37a78750 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f9864740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d379db840 con 0x556d37a6f250 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28edffb640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f28e00a1c20 0x7f28e00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f4a24640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a78210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f4a24640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a78210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52778/0 (socket says 172.21.15.100:52778) 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f4a24640 1 -- 172.21.15.100:0/2089266225 learned_addr learned my addr 172.21.15.100:0/2089266225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28f5225640 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.833+0000 7f28effff640 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f28e00a1c20 0x7f28e00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28edffb640 1 -- 172.21.15.100:0/1558092262 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f28dc0ef7a0 con 0x7f28f0105330 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 -- 172.21.15.100:0/2089266225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 msgr2=0x556d37a7ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 -- 172.21.15.100:0/2089266225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 msgr2=0x556d37a78210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a78210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 -- 172.21.15.100:0/2089266225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d37999060 con 0x556d37a6f250 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f4a24640 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a78210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a6f630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f28e8011cc0 tx=0x7f28e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5055c9191ef0406 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28f5225640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f28e00a1c20 0x7f28e00a40e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28e800fc90 tx=0x7f28e801e000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28daffd640 1 -- 172.21.15.100:0/2089266225 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e801ddf0 con 0x556d37a6f250 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28daffd640 1 -- 172.21.15.100:0/2089266225 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8011e70 con 0x556d37a6f250 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.834+0000 7f28daffd640 1 -- 172.21.15.100:0/2089266225 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8005770 con 0x556d37a6f250 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 -- 172.21.15.100:0/2089266225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 msgr2=0x556d37a6f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a6f630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f28e8011cc0 tx=0x7f28e800f790 comp rx=0 tx=0).stop 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 -- 172.21.15.100:0/2089266225 shutdown_connections 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a78210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 --2- 172.21.15.100:0/2089266225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a6f630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 -- 172.21.15.100:0/2089266225 >> 172.21.15.100:0/2089266225 conn(0x556d37a69980 msgr2=0x556d37a69d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 -- 172.21.15.100:0/2089266225 shutdown_connections 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 -- 172.21.15.100:0/2089266225 wait complete. 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 -- start start 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.835+0000 7f28f9864740 1 Processor -- start 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a6f250 0x556d37a757a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6fb70 0x556d37a75ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 0x556d37a76220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d379d7020 con 0x556d37a78750 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d376aa430 con 0x556d37a6f250 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37666e00 con 0x556d37a6fb70 2024-03-12T05:19:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 10 client.0 _ll_register_callbacks cb 0x556d3797ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:28.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 0x556d37a76220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 0x556d37a76220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52786/0 (socket says 172.21.15.100:52786) 2024-03-12T05:19:28.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 -- 172.21.15.100:0/3823953786 learned_addr learned my addr 172.21.15.100:0/3823953786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28effff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6fb70 0x556d37a75ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f4a24640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a6f250 0x556d37a757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 -- 172.21.15.100:0/3823953786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6fb70 msgr2=0x556d37a75ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6fb70 0x556d37a75ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 -- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a6f250 msgr2=0x556d37a757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a6f250 0x556d37a757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 -- 172.21.15.100:0/3823953786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e8019040 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28effff640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6fb70 0x556d37a75ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f5225640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 0x556d37a76220 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009fc0 tx=0x7f28e8011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62393742abd2adb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28cffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28cffff640 1 -- 172.21.15.100:0/3823953786 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8011220 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28cffff640 1 -- 172.21.15.100:0/3823953786 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f28e8005700 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d376d67e0 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28cffff640 1 -- 172.21.15.100:0/3823953786 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e80058c0 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 10 client.17034 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.836+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556d379d56a0 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28cffff640 1 -- 172.21.15.100:0/3823953786 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8040020 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28cffff640 1 -- 172.21.15.100:0/3823953786 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f28e8024030 con 0x556d37a78750 2024-03-12T05:19:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f9864740 10 client.17034 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f9864740 10 client.17034 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f9864740 20 client.17034 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f9864740 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37a88bb0 0x556d37a8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f9864740 4 client.17034 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d37713720 con 0x556d37a88bb0 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f4a24640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37a88bb0 0x556d37a8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28cffff640 10 client.17034 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.837+0000 7f28f4a24640 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37a88bb0 0x556d37a8afa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37ea8041a96b95bc server_cookie=bf64329698dcb629 in_seq=0 out_seq=0 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28cffff640 1 -- 172.21.15.100:0/3823953786 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d37713720 con 0x556d37a88bb0 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28cffff640 10 client.17034 handle_command_reply: tid=0 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 client.17034 shutdown 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 msgr2=0x556d37a76220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 0x556d37a76220 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009fc0 tx=0x7f28e8011480 comp rx=0 tx=0).stop 2024-03-12T05:19:28.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 shutdown_connections 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37a88bb0 0x556d37a8afa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a78750 0x556d37a76220 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6fb70 0x556d37a75ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28cffff640 0 client.17034 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 --2- 172.21.15.100:0/3823953786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a6f250 0x556d37a757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.838+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 >> 172.21.15.100:0/3823953786 conn(0x556d37a69980 msgr2=0x556d37a6b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.839+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 shutdown_connections 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.839+0000 7f28f9864740 1 -- 172.21.15.100:0/3823953786 wait complete. 2024-03-12T05:19:28.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.839+0000 7f28f9864740 20 client.17034 trim_cache size 0 max 16384 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 Processor -- start 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 -- start start 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a77ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a70a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3795c130 con 0x556d37a6fb70 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37a6e5d0 con 0x556d37a78750 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28f9864740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37a6abe0 con 0x556d37a6f250 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.850+0000 7f28effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a76040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52798/0 (socket says 172.21.15.100:52798) 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 -- 172.21.15.100:0/515846812 learned_addr learned my addr 172.21.15.100:0/515846812 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28f5225640 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a70a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28f4a24640 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a77ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 -- 172.21.15.100:0/515846812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 msgr2=0x556d37a77ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a77ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 -- 172.21.15.100:0/515846812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 msgr2=0x556d37a70a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a70a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 -- 172.21.15.100:0/515846812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d37971c20 con 0x556d37a6fb70 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28f4a24640 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a77ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28effff640 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a76040 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006010 tx=0x7f28dc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477f82456a868887 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28da7fc640 1 -- 172.21.15.100:0/515846812 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0dc0b0 con 0x556d37a6fb70 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28da7fc640 1 -- 172.21.15.100:0/515846812 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28dc0a70b0 con 0x556d37a6fb70 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.851+0000 7f28da7fc640 1 -- 172.21.15.100:0/515846812 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0b8b10 con 0x556d37a6fb70 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- 172.21.15.100:0/515846812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 msgr2=0x556d37a76040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a76040 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006010 tx=0x7f28dc0dc040 comp rx=0 tx=0).stop 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- 172.21.15.100:0/515846812 shutdown_connections 2024-03-12T05:19:28.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a76040 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- 172.21.15.100:0/515846812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a77ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- 172.21.15.100:0/515846812 >> 172.21.15.100:0/515846812 conn(0x556d37a69980 msgr2=0x556d37a64a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- 172.21.15.100:0/515846812 shutdown_connections 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- 172.21.15.100:0/515846812 wait complete. 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- start start 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 Processor -- start 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a7e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a7e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37adf1b0 con 0x556d37a6fb70 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37adf320 con 0x556d37a78750 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d37adf4e0 con 0x556d37a6f250 2024-03-12T05:19:28.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28f9864740 10 client.0 _ll_register_callbacks cb 0x7f28e8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.852+0000 7f28effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a7e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a7e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52808/0 (socket says 172.21.15.100:52808) 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 -- 172.21.15.100:0/13027114 learned_addr learned my addr 172.21.15.100:0/13027114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f4a24640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a7e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f5225640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 -- 172.21.15.100:0/13027114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 msgr2=0x556d37a7e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a7e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 -- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 msgr2=0x556d37a7ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 -- 172.21.15.100:0/13027114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01a36f0 con 0x556d37a6fb70 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f5225640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28effff640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a7e6a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe3638993aac641 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28cffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:28.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28cffff640 1 -- 172.21.15.100:0/13027114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0c4c40 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28cffff640 1 -- 172.21.15.100:0/13027114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f28dc0dc0b0 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e8011e70 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28cffff640 1 -- 172.21.15.100:0/13027114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28dc0bcba0 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f9864740 10 client.17040 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556d379978c0 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28cffff640 1 -- 172.21.15.100:0/13027114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8011e70 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28cffff640 1 -- 172.21.15.100:0/13027114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f28dc03c070 con 0x556d37a6fb70 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f9864740 10 client.17040 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.853+0000 7f28f9864740 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28f9864740 20 client.17040 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28f9864740 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37ae0940 0x556d37ae2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28f9864740 4 client.17040 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d37762890 con 0x556d37ae0940 2024-03-12T05:19:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28f4a24640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37ae0940 0x556d37ae2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28cffff640 10 client.17040 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.854+0000 7f28f4a24640 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37ae0940 0x556d37ae2d30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d12398b3ab86ae4 server_cookie=52b11155830f1ea2 in_seq=0 out_seq=0 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28cffff640 1 -- 172.21.15.100:0/13027114 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d37762890 con 0x556d37ae0940 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28cffff640 10 client.17040 handle_command_reply: tid=0 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 client.17040 shutdown 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 msgr2=0x556d37a7e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a7e6a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 shutdown_connections 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556d37ae0940 0x556d37ae2d30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d37a78750 0x556d37a7ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28cffff640 0 client.17040 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d37a6fb70 0x556d37a7e6a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 --2- 172.21.15.100:0/13027114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d37a6f250 0x556d37a7e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 >> 172.21.15.100:0/13027114 conn(0x556d37a69980 msgr2=0x556d37a73c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 shutdown_connections 2024-03-12T05:19:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 1 -- 172.21.15.100:0/13027114 wait complete. 2024-03-12T05:19:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.855+0000 7f28f9864740 20 client.17040 trim_cache size 0 max 16384 2024-03-12T05:19:28.862 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:19:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f28e00a1c20 msgr2=0x7f28e00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f28e00a1c20 0x7f28e00a40e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28e800fc90 tx=0x7f28e801e000 comp rx=0 tx=0).stop 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 msgr2=0x7f28f0196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 0x7f28f0196c10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009230 tx=0x7f28dc02c680 comp rx=0 tx=0).stop 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 shutdown_connections 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f28e00a1c20 0x7f28e00a40e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f019af90 0x7f28f019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28f0105330 0x7f28f0196c10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 --2- 172.21.15.100:0/1558092262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f0100260 0x7f28f01966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 >> 172.21.15.100:0/1558092262 conn(0x7f28f0077180 msgr2=0x7f28f00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 shutdown_connections 2024-03-12T05:19:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.856+0000 7f28f6c88640 1 -- 172.21.15.100:0/1558092262 wait complete. 2024-03-12T05:19:28.865 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:19:28.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 -- 172.21.15.100:0/741501962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 msgr2=0x7f124010e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 --2- 172.21.15.100:0/741501962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124010e090 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1238009cd0 tx=0x7f123802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 -- 172.21.15.100:0/741501962 shutdown_connections 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 --2- 172.21.15.100:0/741501962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124010e090 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 --2- 172.21.15.100:0/741501962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f124006a450 0x7f1240105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 --2- 172.21.15.100:0/741501962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240069aa0 0x7f1240069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 -- 172.21.15.100:0/741501962 >> 172.21.15.100:0/741501962 conn(0x7f1240077150 msgr2=0x7f1240077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 -- 172.21.15.100:0/741501962 shutdown_connections 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.978+0000 7f1248288640 1 -- 172.21.15.100:0/741501962 wait complete. 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 Processor -- start 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 -- start start 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1240069aa0 0x7f124019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240197610 0x7f1240197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240198000 con 0x7f12401064e0 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240198170 con 0x7f1240069aa0 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1248288640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240198330 con 0x7f1240197610 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124019d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52824/0 (socket says 172.21.15.100:52824) 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 -- 172.21.15.100:0/2468863098 learned_addr learned my addr 172.21.15.100:0/2468863098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1246024640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1240069aa0 0x7f124019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1246825640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240197610 0x7f1240197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 -- 172.21.15.100:0/2468863098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240197610 msgr2=0x7f1240197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240197610 0x7f1240197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 -- 172.21.15.100:0/2468863098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1240069aa0 msgr2=0x7f124019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1240069aa0 0x7f124019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 -- 172.21.15.100:0/2468863098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238009900 con 0x7f12401064e0 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1246825640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240197610 0x7f1240197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.979+0000 7f1245823640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124019d950 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f123000e9f0 tx=0x7f123000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.980+0000 7f12377fe640 1 -- 172.21.15.100:0/2468863098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300096a0 con 0x7f12401064e0 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.980+0000 7f12377fe640 1 -- 172.21.15.100:0/2468863098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f123000ccc0 con 0x7f12401064e0 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.980+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12401a42d0 con 0x7f12401064e0 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.980+0000 7f12377fe640 1 -- 172.21.15.100:0/2468863098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230018860 con 0x7f12401064e0 2024-03-12T05:19:28.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.980+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12401a47e0 con 0x7f12401064e0 2024-03-12T05:19:28.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.982+0000 7f12377fe640 1 -- 172.21.15.100:0/2468863098 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1230010aa0 con 0x7f12401064e0 2024-03-12T05:19:28.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.982+0000 7f124ae63740 1 Processor -- start 2024-03-12T05:19:28.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.982+0000 7f124ae63740 1 -- start start 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f124ae63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c007630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f124ae63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 0x555c6c010210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f124ae63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c010750 0x555c6c012b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f124ae63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bfc2960 con 0x555c6c007250 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f124ae63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bf76310 con 0x555c6c010750 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f124ae63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bf73840 con 0x555c6c007b70 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f12377fe640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f122c0a1ba0 0x7f122c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 0x555c6c010210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c007630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1245823640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c010750 0x555c6c012b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 0x555c6c010210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:34028/0 (socket says 172.21.15.100:34028) 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 -- 172.21.15.100:0/4170391890 learned_addr learned my addr 172.21.15.100:0/4170391890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c007630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52826/0 (socket says 172.21.15.100:52826) 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246825640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f122c0a1ba0 0x7f122c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f12377fe640 1 -- 172.21.15.100:0/2468863098 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1230014070 con 0x7f12401064e0 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 -- 172.21.15.100:0/4170391890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c010750 msgr2=0x555c6c012b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c010750 0x555c6c012b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 -- 172.21.15.100:0/4170391890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 msgr2=0x555c6c007630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c007630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 -- 172.21.15.100:0/4170391890 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c6bf31060 con 0x555c6c007b70 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246825640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f122c0a1ba0 0x7f122c0a4060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1238033000 tx=0x7f12380095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1246024640 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 0x555c6c010210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f123c009d20 tx=0x7f123c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96349d4857b3f613 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1234bfe640 1 -- 172.21.15.100:0/4170391890 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c019630 con 0x555c6c007b70 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.983+0000 7f1234bfe640 1 -- 172.21.15.100:0/4170391890 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f123c019c70 con 0x555c6c007b70 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f1234bfe640 1 -- 172.21.15.100:0/4170391890 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00ec10 con 0x555c6c007b70 2024-03-12T05:19:28.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 -- 172.21.15.100:0/4170391890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 msgr2=0x555c6c010210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 0x555c6c010210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f123c009d20 tx=0x7f123c011810 comp rx=0 tx=0).stop 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 -- 172.21.15.100:0/4170391890 shutdown_connections 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c010750 0x555c6c012b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007b70 0x555c6c010210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 --2- 172.21.15.100:0/4170391890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c007630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 -- 172.21.15.100:0/4170391890 >> 172.21.15.100:0/4170391890 conn(0x555c6c001980 msgr2=0x555c6c001d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 -- 172.21.15.100:0/4170391890 shutdown_connections 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.984+0000 7f124ae63740 1 -- 172.21.15.100:0/4170391890 wait complete. 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 -- start start 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 Processor -- start 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c00d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c010750 0x555c6c00e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bf6f020 con 0x555c6c007250 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bc42430 con 0x555c6c007b70 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bbfee00 con 0x555c6c010750 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f124ae63740 10 client.0 _ll_register_callbacks cb 0x555c6bf16cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c00d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c00d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52834/0 (socket says 172.21.15.100:52834) 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 -- 172.21.15.100:0/2212651533 learned_addr learned my addr 172.21.15.100:0/2212651533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1245823640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246825640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c010750 0x555c6c00e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 -- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c010750 msgr2=0x555c6c00e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c010750 0x555c6c00e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 -- 172.21.15.100:0/2212651533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 msgr2=0x555c6c00dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:28.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 -- 172.21.15.100:0/2212651533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c009950 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1246024640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c00d7e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f123c004470 tx=0x7f123c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44afdcf0d8e63c12 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.985+0000 7f1229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f1229ffb640 1 -- 172.21.15.100:0/2212651533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00ebb0 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f1229ffb640 1 -- 172.21.15.100:0/2212651533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f123c00ed10 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c6bc6e7e0 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f1229ffb640 1 -- 172.21.15.100:0/2212651533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f123c009ed0 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 10 client.17052 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c6bf6d6a0 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f1229ffb640 1 -- 172.21.15.100:0/2212651533 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c6bc6e7e0 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f1229ffb640 1 -- 172.21.15.100:0/2212651533 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f123c003980 con 0x555c6c007250 2024-03-12T05:19:28.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 10 client.17052 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 10 client.17052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 20 client.17052 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c020ba0 0x555c6c022fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 4 client.17052 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c6bcab720 con 0x555c6c020ba0 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.986+0000 7f1245823640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c020ba0 0x555c6c022fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f1245823640 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c020ba0 0x555c6c022fb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aeb66ac1d95ee27b server_cookie=f479760d5288d54f in_seq=0 out_seq=0 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f1229ffb640 10 client.17052 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f1229ffb640 1 -- 172.21.15.100:0/2212651533 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c6bcab720 con 0x555c6c020ba0 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f1229ffb640 10 client.17052 handle_command_reply: tid=0 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 client.17052 shutdown 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 msgr2=0x555c6c00d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:28.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c00d7e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f123c004470 tx=0x7f123c0044a0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 shutdown_connections 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c010750 0x555c6c00e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c020ba0 0x555c6c022fb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f1229ffb640 0 client.17052 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 --2- 172.21.15.100:0/2212651533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c007250 0x555c6c00d7e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 >> 172.21.15.100:0/2212651533 conn(0x555c6c001980 msgr2=0x555c6c0036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 shutdown_connections 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.987+0000 7f124ae63740 1 -- 172.21.15.100:0/2212651533 wait complete. 2024-03-12T05:19:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.988+0000 7f124ae63740 20 client.17052 trim_cache size 0 max 16384 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 Processor -- start 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 -- start start 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c00ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c00c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bef4130 con 0x555c6c010750 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bffd080 con 0x555c6c007b70 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:28.999+0000 7f124ae63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6bffd2b0 con 0x555c6c007250 2024-03-12T05:19:29.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c00c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c00c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52836/0 (socket says 172.21.15.100:52836) 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 -- 172.21.15.100:0/1974234732 learned_addr learned my addr 172.21.15.100:0/1974234732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1245823640 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246024640 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c00ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 -- 172.21.15.100:0/1974234732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 msgr2=0x555c6c00ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c00ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 -- 172.21.15.100:0/1974234732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 msgr2=0x555c6c00e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 -- 172.21.15.100:0/1974234732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c6bf09c20 con 0x555c6c010750 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f1246825640 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c00c0b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f12380096c0 tx=0x7f1238041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39df10bde00d52f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f122bfff640 1 -- 172.21.15.100:0/1974234732 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238048040 con 0x555c6c010750 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f122bfff640 1 -- 172.21.15.100:0/1974234732 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1238032d90 con 0x555c6c010750 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.000+0000 7f122bfff640 1 -- 172.21.15.100:0/1974234732 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238004040 con 0x555c6c010750 2024-03-12T05:19:29.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- 172.21.15.100:0/1974234732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 msgr2=0x555c6c00c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c00c0b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f12380096c0 tx=0x7f1238041000 comp rx=0 tx=0).stop 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- 172.21.15.100:0/1974234732 shutdown_connections 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c00c0b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c00e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- 172.21.15.100:0/1974234732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c00ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- 172.21.15.100:0/1974234732 >> 172.21.15.100:0/1974234732 conn(0x555c6c001980 msgr2=0x555c6c027950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- 172.21.15.100:0/1974234732 shutdown_connections 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- 172.21.15.100:0/1974234732 wait complete. 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- start start 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 Processor -- start 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c016210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c016750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c016c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6c0771d0 con 0x555c6c010750 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6c077340 con 0x555c6c007b70 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.001+0000 7f124ae63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6c077500 con 0x555c6c007250 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f124ae63740 10 client.0 _ll_register_callbacks cb 0x7f123c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c016c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246024640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c016210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c016c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52838/0 (socket says 172.21.15.100:52838) 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 -- 172.21.15.100:0/1395067316 learned_addr learned my addr 172.21.15.100:0/1395067316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1245823640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c016750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 -- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 msgr2=0x555c6c016210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c016210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 -- 172.21.15.100:0/1395067316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 msgr2=0x555c6c016750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c016750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 -- 172.21.15.100:0/1395067316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124010e8a0 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1245823640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c016750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1246825640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c016c90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1238009cd0 tx=0x7f12380346a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bb47ae25d21486 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1229ffb640 1 -- 172.21.15.100:0/1395067316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238048070 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1229ffb640 1 -- 172.21.15.100:0/1395067316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1238032ac0 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f123c019c70 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1229ffb640 1 -- 172.21.15.100:0/1395067316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1238003970 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f124ae63740 10 client.17058 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c6bf2f8c0 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1229ffb640 1 -- 172.21.15.100:0/1395067316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1238063020 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.002+0000 7f1229ffb640 1 -- 172.21.15.100:0/1395067316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f123805f450 con 0x555c6c010750 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 10 client.17058 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:19:29.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 20 client.17058 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c078ab0 0x555c6c07aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 4 client.17058 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c6bcfa890 con 0x555c6c078ab0 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f1246024640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c078ab0 0x555c6c07aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f1246024640 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c078ab0 0x555c6c07aed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a36f809fcd0f8170 server_cookie=d698c26c33a9257a in_seq=0 out_seq=0 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f1229ffb640 10 client.17058 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f1229ffb640 1 -- 172.21.15.100:0/1395067316 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c6bcfa890 con 0x555c6c078ab0 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f1229ffb640 10 client.17058 handle_command_reply: tid=0 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 client.17058 shutdown 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 msgr2=0x555c6c016c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c016c90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1238009cd0 tx=0x7f12380346a0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 shutdown_connections 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c6c010750 0x555c6c016c90 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x555c6c078ab0 0x555c6c07aed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6c007b70 0x555c6c016750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 --2- 172.21.15.100:0/1395067316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c6c007250 0x555c6c016210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f1229ffb640 0 client.17058 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 >> 172.21.15.100:0/1395067316 conn(0x555c6c001980 msgr2=0x555c6c00bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 shutdown_connections 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.003+0000 7f124ae63740 1 -- 172.21.15.100:0/1395067316 wait complete. 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f124ae63740 20 client.17058 trim_cache size 0 max 16384 2024-03-12T05:19:29.006 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f122c0a1ba0 msgr2=0x7f122c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f122c0a1ba0 0x7f122c0a4060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1238033000 tx=0x7f12380095c0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 msgr2=0x7f124019d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124019d950 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f123000e9f0 tx=0x7f123000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 shutdown_connections 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f122c0a1ba0 0x7f122c0a4060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1240197610 0x7f1240197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12401064e0 0x7f124019d950 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 --2- 172.21.15.100:0/2468863098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1240069aa0 0x7f124019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 >> 172.21.15.100:0/2468863098 conn(0x7f1240077150 msgr2=0x7f12400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 shutdown_connections 2024-03-12T05:19:29.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.004+0000 7f1248288640 1 -- 172.21.15.100:0/2468863098 wait complete. 2024-03-12T05:19:29.009 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:28 smithi100 ceph-mon[32210]: pgmap v226: 97 pgs: 97 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 313 op/s 2024-03-12T05:19:29.009 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:28 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1270157988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:19:29.010 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:19:29.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:19:29.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/1838829712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc41002a0 msgr2=0x7f1bc4100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/1838829712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc41002a0 0x7f1bc4100680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009930 tx=0x7f1bc002bc30 comp rx=0 tx=0).stop 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/1838829712 shutdown_connections 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/1838829712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc4105370 0x7f1bc410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/1838829712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc4100c50 0x7f1bc4104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/1838829712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc41002a0 0x7f1bc4100680 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/1838829712 >> 172.21.15.100:0/1838829712 conn(0x7f1bc4076400 msgr2=0x7f1bc4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/1838829712 shutdown_connections 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.121+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/1838829712 wait complete. 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 Processor -- start 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 -- start start 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc41002a0 0x7f1bc4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc4105370 0x7f1bc4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 0x7f1bc41a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41a3da0 con 0x7f1bc4199120 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41a3f10 con 0x7f1bc41002a0 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bcbdd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41a40d0 con 0x7f1bc4105370 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 0x7f1bc41a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc41002a0 0x7f1bc4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 0x7f1bc41a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52856/0 (socket says 172.21.15.100:52856) 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 -- 172.21.15.100:0/3047463222 learned_addr learned my addr 172.21.15.100:0/3047463222 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bc936b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc4105370 0x7f1bc4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc41002a0 0x7f1bc4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37156/0 (socket says 172.21.15.100:37156) 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 -- 172.21.15.100:0/3047463222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc4105370 msgr2=0x7f1bc4198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc4105370 0x7f1bc4198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 -- 172.21.15.100:0/3047463222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc41002a0 msgr2=0x7f1bc4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc41002a0 0x7f1bc4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.122+0000 7f1bca36d640 1 -- 172.21.15.100:0/3047463222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0009560 con 0x7f1bc4199120 2024-03-12T05:19:29.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.123+0000 7f1bca36d640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 0x7f1bc41a36d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00eb10 tx=0x7f1bbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.123+0000 7f1bbaffd640 1 -- 172.21.15.100:0/3047463222 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004b20 con 0x7f1bc4199120 2024-03-12T05:19:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.123+0000 7f1bbaffd640 1 -- 172.21.15.100:0/3047463222 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc004c80 con 0x7f1bc4199120 2024-03-12T05:19:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.123+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc41a43b0 con 0x7f1bc4199120 2024-03-12T05:19:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.123+0000 7f1bbaffd640 1 -- 172.21.15.100:0/3047463222 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc010650 con 0x7f1bc4199120 2024-03-12T05:19:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.123+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc41a4870 con 0x7f1bc4199120 2024-03-12T05:19:29.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.125+0000 7f1bbaffd640 1 -- 172.21.15.100:0/3047463222 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1bbc005020 con 0x7f1bc4199120 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.125+0000 7f1bce99b740 1 Processor -- start 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 -- start start 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bbaffd640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1bb00a1ba0 0x7f1bb00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 0x55f0b0119630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b0122210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0122750 0x55f0b0124b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bbaffd640 1 -- 172.21.15.100:0/3047463222 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1bbc014070 con 0x7f1bc4199120 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b00d4960 con 0x55f0b0122750 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0088310 con 0x55f0b0119b70 2024-03-12T05:19:29.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bce99b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0085840 con 0x55f0b0119250 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc936b640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1bb00a1ba0 0x7f1bb00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 0x55f0b0119630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bca36d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0122750 0x55f0b0124b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc936b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b0122210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 0x55f0b0119630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:34094/0 (socket says 172.21.15.100:34094) 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/963250528 learned_addr learned my addr 172.21.15.100:0/963250528 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/963250528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 msgr2=0x55f0b0122210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b0122210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/963250528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0122750 msgr2=0x55f0b0124b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0122750 0x55f0b0124b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/963250528 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b0043060 con 0x55f0b0119250 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc936b640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1bb00a1ba0 0x7f1bb00a4060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1bc406a2c0 tx=0x7f1bb4004550 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.126+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 0x55f0b0119630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036040 tx=0x7f1bc002c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9eb714dd940d73 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1baffff640 1 -- 172.21.15.100:0/963250528 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0004880 con 0x55f0b0119250 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1baffff640 1 -- 172.21.15.100:0/963250528 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc00049e0 con 0x55f0b0119250 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1baffff640 1 -- 172.21.15.100:0/963250528 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0005050 con 0x55f0b0119250 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 -- 172.21.15.100:0/963250528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 msgr2=0x55f0b0119630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 0x55f0b0119630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036040 tx=0x7f1bc002c680 comp rx=0 tx=0).stop 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 -- 172.21.15.100:0/963250528 shutdown_connections 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0122750 0x55f0b0124b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b0122210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 --2- 172.21.15.100:0/963250528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119250 0x55f0b0119630 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 -- 172.21.15.100:0/963250528 >> 172.21.15.100:0/963250528 conn(0x55f0b0113980 msgr2=0x55f0b0113d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 -- 172.21.15.100:0/963250528 shutdown_connections 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.127+0000 7f1bce99b740 1 -- 172.21.15.100:0/963250528 wait complete. 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 -- start start 2024-03-12T05:19:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 Processor -- start 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b011f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b011fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b0120150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0081020 con 0x55f0b0119250 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0afd54430 con 0x55f0b0122750 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0afd10e00 con 0x55f0b0119b70 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bce99b740 10 client.0 _ll_register_callbacks cb 0x55f0b0028cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b011f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b011f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52868/0 (socket says 172.21.15.100:52868) 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/2689330992 learned_addr learned my addr 172.21.15.100:0/2689330992 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bca36d640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b0120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.128+0000 7f1bc936b640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b011fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/2689330992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 msgr2=0x55f0b011fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b011fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 msgr2=0x55f0b0120150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b0120150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/2689330992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc410d750 con 0x55f0b0119250 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bca36d640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b0120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b011f6d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005c40 tx=0x7f1bc0009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb145cb7df58b875 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bad7fa640 1 -- 172.21.15.100:0/2689330992 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0004880 con 0x55f0b0119250 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bad7fa640 1 -- 172.21.15.100:0/2689330992 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1bc004d070 con 0x55f0b0119250 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0afd807e0 con 0x55f0b0119250 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bad7fa640 1 -- 172.21.15.100:0/2689330992 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc0005200 con 0x55f0b0119250 2024-03-12T05:19:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bce99b740 10 client.17070 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f0b007f6a0 con 0x55f0b0119250 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bad7fa640 1 -- 172.21.15.100:0/2689330992 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc005d080 con 0x55f0b0119250 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.129+0000 7f1bad7fa640 1 -- 172.21.15.100:0/2689330992 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1bc003c050 con 0x55f0b0119250 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bce99b740 10 client.17070 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bce99b740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bce99b740 20 client.17070 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bce99b740 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b0132650 0x55f0b0134a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bce99b740 4 client.17070 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0afdbd720 con 0x55f0b0132650 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bc936b640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b0132650 0x55f0b0134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bc936b640 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b0132650 0x55f0b0134a70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c47d965713bcc4 server_cookie=4e2647222d784c96 in_seq=0 out_seq=0 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.130+0000 7f1bad7fa640 10 client.17070 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bad7fa640 1 -- 172.21.15.100:0/2689330992 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0afdbd720 con 0x55f0b0132650 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bad7fa640 10 client.17070 handle_command_reply: tid=0 2024-03-12T05:19:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 client.17070 shutdown 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 msgr2=0x55f0b011f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b011f6d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005c40 tx=0x7f1bc0009290 comp rx=0 tx=0).stop 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 shutdown_connections 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b0132650 0x55f0b0134a70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b0120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b011fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 --2- 172.21.15.100:0/2689330992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b011f6d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bad7fa640 0 client.17070 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 >> 172.21.15.100:0/2689330992 conn(0x55f0b0113980 msgr2=0x55f0b01156d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 shutdown_connections 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 1 -- 172.21.15.100:0/2689330992 wait complete. 2024-03-12T05:19:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.131+0000 7f1bce99b740 20 client.17070 trim_cache size 0 max 16384 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 Processor -- start 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 -- start start 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0121e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b011ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0122750 0x55f0b011c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0006130 con 0x55f0b0119250 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b010f080 con 0x55f0b0119b70 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bce99b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b010df90 con 0x55f0b0122750 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0121e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0121e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52878/0 (socket says 172.21.15.100:52878) 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/3076125072 learned_addr learned my addr 172.21.15.100:0/3076125072 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc936b640 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b011ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bca36d640 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0122750 0x55f0b011c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/3076125072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0122750 msgr2=0x55f0b011c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0122750 0x55f0b011c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/3076125072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 msgr2=0x55f0b011ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b011ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/3076125072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b001bc20 con 0x55f0b0119250 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc936b640 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b011ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0121e00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1bc003fec0 tx=0x7f1bc0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cbde177d949461 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1baf7fe640 1 -- 172.21.15.100:0/3076125072 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc002c5d0 con 0x55f0b0119250 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1baf7fe640 1 -- 172.21.15.100:0/3076125072 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f0afd807e0 con 0x55f0b0119250 2024-03-12T05:19:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.143+0000 7f1baf7fe640 1 -- 172.21.15.100:0/3076125072 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00054c0 con 0x55f0b0119250 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 -- 172.21.15.100:0/3076125072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 msgr2=0x55f0b0121e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0121e00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1bc003fec0 tx=0x7f1bc0005fd0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 -- 172.21.15.100:0/3076125072 shutdown_connections 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0122750 0x55f0b011c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0119b70 0x55f0b011ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 --2- 172.21.15.100:0/3076125072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0121e00 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 -- 172.21.15.100:0/3076125072 >> 172.21.15.100:0/3076125072 conn(0x55f0b0113980 msgr2=0x55f0b011a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 -- 172.21.15.100:0/3076125072 shutdown_connections 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.144+0000 7f1bce99b740 1 -- 172.21.15.100:0/3076125072 wait complete. 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 -- start start 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 Processor -- start 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0127f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b0128460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b01289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0129ec0 con 0x55f0b0119250 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0188ba0 con 0x55f0b0122750 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0188ce0 con 0x55f0b0119b70 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bce99b740 10 client.0 _ll_register_callbacks cb 0x7f1bc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0127f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0127f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52888/0 (socket says 172.21.15.100:52888) 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/254172947 learned_addr learned my addr 172.21.15.100:0/254172947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc936b640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b0128460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bca36d640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b01289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/254172947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 msgr2=0x55f0b0128460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b0128460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 msgr2=0x55f0b01289a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.145+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b01289a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bc9b6c640 1 -- 172.21.15.100:0/254172947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0b007f6a0 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bc9b6c640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0127f20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005cd0 tx=0x7f1bc0004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1628bac73f90ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bad7fa640 1 -- 172.21.15.100:0/254172947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc003f850 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bad7fa640 1 -- 172.21.15.100:0/254172947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1bc004d040 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bc00049e0 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bad7fa640 1 -- 172.21.15.100:0/254172947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc0005710 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 10 client.17076 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f0b00418c0 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bad7fa640 1 -- 172.21.15.100:0/254172947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc00049e0 con 0x55f0b0119250 2024-03-12T05:19:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bad7fa640 1 -- 172.21.15.100:0/254172947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1bc0068070 con 0x55f0b0119250 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 10 client.17076 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 20 client.17076 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b018a040 0x55f0b018c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 4 client.17076 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f0afe0c890 con 0x55f0b018a040 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.146+0000 7f1bc936b640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b018a040 0x55f0b018c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bad7fa640 10 client.17076 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bc936b640 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b018a040 0x55f0b018c430 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ebfa89a16c8a79 server_cookie=c075720bbb51ee2 in_seq=0 out_seq=0 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bad7fa640 1 -- 172.21.15.100:0/254172947 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f0afe0c890 con 0x55f0b018a040 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bad7fa640 10 client.17076 handle_command_reply: tid=0 2024-03-12T05:19:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 client.17076 shutdown 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 msgr2=0x55f0b0127f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0127f20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005cd0 tx=0x7f1bc0004480 comp rx=0 tx=0).stop 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 shutdown_connections 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0b018a040 0x55f0b018c430 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0b0122750 0x55f0b01289a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bad7fa640 0 client.17076 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0b0119b70 0x55f0b0128460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 --2- 172.21.15.100:0/254172947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0119250 0x55f0b0127f20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 >> 172.21.15.100:0/254172947 conn(0x55f0b0113980 msgr2=0x55f0b01122f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 shutdown_connections 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 1 -- 172.21.15.100:0/254172947 wait complete. 2024-03-12T05:19:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.147+0000 7f1bce99b740 20 client.17076 trim_cache size 0 max 16384 2024-03-12T05:19:29.151 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1bb00a1ba0 msgr2=0x7f1bb00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1bb00a1ba0 0x7f1bb00a4060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1bc406a2c0 tx=0x7f1bb4004550 comp rx=0 tx=0).stop 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 msgr2=0x7f1bc41a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 0x7f1bc41a36d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00eb10 tx=0x7f1bbc00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 shutdown_connections 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1bb00a1ba0 0x7f1bb00a4060 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc4199120 0x7f1bc41a36d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc4105370 0x7f1bc4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 --2- 172.21.15.100:0/3047463222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bc41002a0 0x7f1bc4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 >> 172.21.15.100:0/3047463222 conn(0x7f1bc4076400 msgr2=0x7f1bc40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 shutdown_connections 2024-03-12T05:19:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:29.148+0000 7f1bcbdd0640 1 -- 172.21.15.100:0/3047463222 wait complete. 2024-03-12T05:19:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:28 smithi156 ceph-mon[36270]: pgmap v226: 97 pgs: 97 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 313 op/s 2024-03-12T05:19:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:28 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1270157988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:19:29.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:28 smithi161 ceph-mon[36043]: pgmap v226: 97 pgs: 97 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 313 op/s 2024-03-12T05:19:29.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:28 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1270157988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:19:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 874334 12.13 MB/sec execute 253 sec latency 15.977 ms 2024-03-12T05:19:30.154 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:19:30.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:19:30.155 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:19:30.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 -- 172.21.15.100:0/2501888921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 msgr2=0x7fba9c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 --2- 172.21.15.100:0/2501888921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c101750 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fba98009930 tx=0x7fba9802bc30 comp rx=0 tx=0).stop 2024-03-12T05:19:30.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 -- 172.21.15.100:0/2501888921 shutdown_connections 2024-03-12T05:19:30.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 --2- 172.21.15.100:0/2501888921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9c106440 0x7fba9c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 --2- 172.21.15.100:0/2501888921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c101d20 0x7fba9c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 --2- 172.21.15.100:0/2501888921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c101750 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 -- 172.21.15.100:0/2501888921 >> 172.21.15.100:0/2501888921 conn(0x7fba9c0fcc10 msgr2=0x7fba9c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 -- 172.21.15.100:0/2501888921 shutdown_connections 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.270+0000 7fbaa413e640 1 -- 172.21.15.100:0/2501888921 wait complete. 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 Processor -- start 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 -- start start 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c106440 0x7fba9c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9c197630 0x7fba9c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c198100 con 0x7fba9c101370 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c198270 con 0x7fba9c106440 2024-03-12T05:19:30.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa413e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c198430 con 0x7fba9c197630 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa16d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c106440 0x7fba9c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52908/0 (socket says 172.21.15.100:52908) 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 -- 172.21.15.100:0/3634263489 learned_addr learned my addr 172.21.15.100:0/3634263489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa26db640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9c197630 0x7fba9c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 -- 172.21.15.100:0/3634263489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9c197630 msgr2=0x7fba9c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9c197630 0x7fba9c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 -- 172.21.15.100:0/3634263489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c106440 msgr2=0x7fba9c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c106440 0x7fba9c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa1eda640 1 -- 172.21.15.100:0/3634263489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba98009560 con 0x7fba9c101370 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.271+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c106440 0x7fba9c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.272+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c19d570 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fba98009230 tx=0x7fba9802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.272+0000 7fba92ffd640 1 -- 172.21.15.100:0/3634263489 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98004350 con 0x7fba9c101370 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.272+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c1a4370 con 0x7fba9c101370 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.272+0000 7fba92ffd640 1 -- 172.21.15.100:0/3634263489 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba980044b0 con 0x7fba9c101370 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.272+0000 7fba92ffd640 1 -- 172.21.15.100:0/3634263489 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98003d00 con 0x7fba9c101370 2024-03-12T05:19:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.272+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c1a48a0 con 0x7fba9c101370 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.274+0000 7fba92ffd640 1 -- 172.21.15.100:0/3634263489 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fba98004df0 con 0x7fba9c101370 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.274+0000 7fbaa6ded740 1 Processor -- start 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 -- start start 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221acb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221acdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221a30f90 con 0x559221acb8b0 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221a2e4c0 con 0x559221ac2380 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221a29ca0 con 0x559221ac2d30 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fba92ffd640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba880a1ba0 0x7fba880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa16d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221acdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa1eda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221acb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa16d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221acdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52916/0 (socket says 172.21.15.100:52916) 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa16d9640 1 -- 172.21.15.100:0/2415599523 learned_addr learned my addr 172.21.15.100:0/2415599523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba880a1ba0 0x7fba880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fba92ffd640 1 -- 172.21.15.100:0/3634263489 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fba980ef710 con 0x7fba9c101370 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 -- 172.21.15.100:0/2415599523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 msgr2=0x559221acb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221acb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 -- 172.21.15.100:0/2415599523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 msgr2=0x559221acdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221acdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.275+0000 7fbaa26db640 1 -- 172.21.15.100:0/2415599523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592219ec0b0 con 0x559221ac2380 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fbaa26db640 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac2760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fba94011cc0 tx=0x7fba9400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4082f169a7640dc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fbaa26db640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba880a1ba0 0x7fba880a4060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba9400fc90 tx=0x7fba94018000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fba87fff640 1 -- 172.21.15.100:0/2415599523 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94004b20 con 0x559221ac2380 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fba87fff640 1 -- 172.21.15.100:0/2415599523 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba94009d20 con 0x559221ac2380 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fba87fff640 1 -- 172.21.15.100:0/2415599523 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940053a0 con 0x559221ac2380 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2415599523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 msgr2=0x559221ac2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.276+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac2760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fba94011cc0 tx=0x7fba9400f790 comp rx=0 tx=0).stop 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2415599523 shutdown_connections 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221acdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221acb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2415599523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac2760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2415599523 >> 172.21.15.100:0/2415599523 conn(0x559221abc840 msgr2=0x559221abcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2415599523 shutdown_connections 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2415599523 wait complete. 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- start start 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 Processor -- start 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ac8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ac9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592219cef40 con 0x559221acb8b0 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592216fd4b0 con 0x559221ac2380 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592219af670 con 0x559221ac2d30 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.277+0000 7fbaa6ded740 10 client.0 _ll_register_callbacks cb 0x559221a1cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ac9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa1eda640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ac9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52918/0 (socket says 172.21.15.100:52918) 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 -- 172.21.15.100:0/1676966600 learned_addr learned my addr 172.21.15.100:0/1676966600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ac8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 -- 172.21.15.100:0/1676966600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 msgr2=0x559221ac8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ac8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 -- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 msgr2=0x559221ac88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 -- 172.21.15.100:0/1676966600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba94019040 con 0x559221acb8b0 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ac8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa26db640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ac9330 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fba94011fd0 tx=0x7fba940094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5d39a7d37505da server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fba857fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fba857fa640 1 -- 172.21.15.100:0/1676966600 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9402c070 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559221a03f40 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fba857fa640 1 -- 172.21.15.100:0/1676966600 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fba94010800 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fba857fa640 1 -- 172.21.15.100:0/1676966600 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba940109c0 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa6ded740 10 client.17088 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.278+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559221a28320 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fba857fa640 1 -- 172.21.15.100:0/1676966600 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559221a03f40 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fba857fa640 1 -- 172.21.15.100:0/1676966600 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fba94024070 con 0x559221acb8b0 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa6ded740 10 client.17088 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa6ded740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa6ded740 20 client.17088 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221adbdb0 0x559221ade1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa6ded740 4 client.17088 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559221a20310 con 0x559221adbdb0 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221adbdb0 0x559221ade1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221adbdb0 0x559221ade1d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20dd86b9dbfa8321 server_cookie=9270cb0a0c8330a5 in_seq=0 out_seq=0 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.279+0000 7fba857fa640 10 client.17088 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fba857fa640 1 -- 172.21.15.100:0/1676966600 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559221a20310 con 0x559221adbdb0 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fba857fa640 10 client.17088 handle_command_reply: tid=0 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 client.17088 shutdown 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 msgr2=0x559221ac9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ac9330 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fba94011fd0 tx=0x7fba940094a0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 shutdown_connections 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221adbdb0 0x559221ade1d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ac9330 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ac8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fba857fa640 0 client.17088 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/1676966600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221ac88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 >> 172.21.15.100:0/1676966600 conn(0x559221abc840 msgr2=0x559221abe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 shutdown_connections 2024-03-12T05:19:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 1 -- 172.21.15.100:0/1676966600 wait complete. 2024-03-12T05:19:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.280+0000 7fbaa6ded740 20 client.17088 trim_cache size 0 max 16384 2024-03-12T05:19:30.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.295+0000 7fbaa6ded740 1 Processor -- start 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.295+0000 7fbaa6ded740 1 -- start start 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.295+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221aca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.295+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ad2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.295+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ad2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221a20310 con 0x559221acb8b0 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221a03f40 con 0x559221ac2380 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221adbb30 con 0x559221ac2d30 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ad2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ad2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52922/0 (socket says 172.21.15.100:52922) 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 -- 172.21.15.100:0/2978142874 learned_addr learned my addr 172.21.15.100:0/2978142874 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221aca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ad2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 -- 172.21.15.100:0/2978142874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 msgr2=0x559221ad2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ad2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 -- 172.21.15.100:0/2978142874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 msgr2=0x559221aca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221aca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 -- 172.21.15.100:0/2978142874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592219c4c00 con 0x559221acb8b0 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fbaa26db640 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ad2dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fba94009b80 tx=0x7fba94005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b50c39a39a4c4e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fba877fe640 1 -- 172.21.15.100:0/2978142874 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9402c070 con 0x559221acb8b0 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fba877fe640 1 -- 172.21.15.100:0/2978142874 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba94009ce0 con 0x559221acb8b0 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.296+0000 7fba877fe640 1 -- 172.21.15.100:0/2978142874 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94010a60 con 0x559221acb8b0 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2978142874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 msgr2=0x559221ad2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ad2dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fba94009b80 tx=0x7fba94005b10 comp rx=0 tx=0).stop 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2978142874 shutdown_connections 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221acb8b0 0x559221ad2dc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2d30 0x559221ad2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/2978142874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221ac2380 0x559221aca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2978142874 >> 172.21.15.100:0/2978142874 conn(0x559221abc840 msgr2=0x559221ac9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2978142874 shutdown_connections 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 -- 172.21.15.100:0/2978142874 wait complete. 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 -- start start 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.297+0000 7fbaa6ded740 1 Processor -- start 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2380 0x559221b2cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 0x559221b2d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221acb8b0 0x559221ad1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221ad2480 con 0x559221ac2d30 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221ad25f0 con 0x559221acb8b0 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559221ad27b0 con 0x559221ac2380 2024-03-12T05:19:30.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 10 client.0 _ll_register_callbacks cb 0x7fba94009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 0x559221b2d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 0x559221b2d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52930/0 (socket says 172.21.15.100:52930) 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 -- 172.21.15.100:0/3013165534 learned_addr learned my addr 172.21.15.100:0/3013165534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2380 0x559221b2cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa26db640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221acb8b0 0x559221ad1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 -- 172.21.15.100:0/3013165534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2380 msgr2=0x559221b2cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2380 0x559221b2cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 -- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221acb8b0 msgr2=0x559221ad1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221acb8b0 0x559221ad1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 -- 172.21.15.100:0/3013165534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559221a28320 con 0x559221ac2d30 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa16d9640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 0x559221b2d010 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fba8c012fc0 tx=0x7fba8c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35779d74aa271cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fba857fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fba857fa640 1 -- 172.21.15.100:0/3013165534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c017070 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fba857fa640 1 -- 172.21.15.100:0/3013165534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fba8c004250 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559221a4f900 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fba857fa640 1 -- 172.21.15.100:0/3013165534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba8c0122e0 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 10 client.17094 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.298+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592219ea910 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fba857fa640 1 -- 172.21.15.100:0/3013165534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559221a4f900 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fba857fa640 1 -- 172.21.15.100:0/3013165534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fba8c01d600 con 0x559221ac2d30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa6ded740 10 client.17094 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa6ded740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa6ded740 20 client.17094 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221b34a30 0x559221b36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa6ded740 4 client.17094 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559221729860 con 0x559221b34a30 2024-03-12T05:19:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221b34a30 0x559221b36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fba857fa640 10 client.17094 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.299+0000 7fbaa1eda640 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221b34a30 0x559221b36e50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e611477e5897e28c server_cookie=4f566bbc281db052 in_seq=0 out_seq=0 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.300+0000 7fba857fa640 1 -- 172.21.15.100:0/3013165534 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559221729860 con 0x559221b34a30 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.300+0000 7fba857fa640 10 client.17094 handle_command_reply: tid=0 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.300+0000 7fbaa6ded740 1 client.17094 shutdown 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.300+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 msgr2=0x559221b2d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.300+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 0x559221b2d010 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fba8c012fc0 tx=0x7fba8c00f790 comp rx=0 tx=0).stop 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 shutdown_connections 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559221b34a30 0x559221b36e50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559221acb8b0 0x559221ad1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559221ac2d30 0x559221b2d010 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fba857fa640 0 client.17094 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 --2- 172.21.15.100:0/3013165534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559221ac2380 0x559221b2cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 >> 172.21.15.100:0/3013165534 conn(0x559221abc840 msgr2=0x559221abb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 shutdown_connections 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 1 -- 172.21.15.100:0/3013165534 wait complete. 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa6ded740 20 client.17094 trim_cache size 0 max 16384 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "fd6464a8-55d2-47d5-8f86-de67d9a9a98d", 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:19:30.303 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:19:30.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba880a1ba0 msgr2=0x7fba880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba880a1ba0 0x7fba880a4060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba9400fc90 tx=0x7fba94018000 comp rx=0 tx=0).stop 2024-03-12T05:19:30.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 msgr2=0x7fba9c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c19d570 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fba98009230 tx=0x7fba9802c650 comp rx=0 tx=0).stop 2024-03-12T05:19:30.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 shutdown_connections 2024-03-12T05:19:30.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fba880a1ba0 0x7fba880a4060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9c197630 0x7fba9c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9c106440 0x7fba9c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 --2- 172.21.15.100:0/3634263489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9c101370 0x7fba9c19d570 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 >> 172.21.15.100:0/3634263489 conn(0x7fba9c0fcc10 msgr2=0x7fba9c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 shutdown_connections 2024-03-12T05:19:30.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.301+0000 7fbaa413e640 1 -- 172.21.15.100:0/3634263489 wait complete. 2024-03-12T05:19:30.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:19:30.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 -- 172.21.15.100:0/1117009461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26081023a0 msgr2=0x7f2608102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 --2- 172.21.15.100:0/1117009461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26081023a0 0x7f2608102780 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009900 tx=0x7f25f802bcd0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 -- 172.21.15.100:0/1117009461 shutdown_connections 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 --2- 172.21.15.100:0/1117009461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260810b6e0 0x7f260810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 --2- 172.21.15.100:0/1117009461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2608102cc0 0x7f260810b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 --2- 172.21.15.100:0/1117009461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26081023a0 0x7f2608102780 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 -- 172.21.15.100:0/1117009461 >> 172.21.15.100:0/1117009461 conn(0x7f2608077230 msgr2=0x7f2608077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 -- 172.21.15.100:0/1117009461 shutdown_connections 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 -- 172.21.15.100:0/1117009461 wait complete. 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 Processor -- start 2024-03-12T05:19:30.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.429+0000 7f260e681640 1 -- start start 2024-03-12T05:19:30.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260e681640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26081023a0 0x7f2608197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260e681640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608102cc0 0x7f2608197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260e681640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 0x7f260819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260e681640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260819c590 con 0x7f260810b6e0 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260e681640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260819c700 con 0x7f26081023a0 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260e681640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260819c8c0 con 0x7f2608102cc0 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 0x7f260819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 0x7f260819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52948/0 (socket says 172.21.15.100:52948) 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 -- 172.21.15.100:0/301020241 learned_addr learned my addr 172.21.15.100:0/301020241 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f2607fff640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26081023a0 0x7f2608197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f26077fe640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608102cc0 0x7f2608197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 -- 172.21.15.100:0/301020241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608102cc0 msgr2=0x7f2608197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608102cc0 0x7f2608197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 -- 172.21.15.100:0/301020241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26081023a0 msgr2=0x7f2608197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26081023a0 0x7f2608197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f260cc1e640 1 -- 172.21.15.100:0/301020241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8009560 con 0x7f260810b6e0 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.430+0000 7f2607fff640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26081023a0 0x7f2608197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.431+0000 7f260cc1e640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 0x7f260819bf90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f260000cce0 tx=0x7f260000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.431+0000 7f26057fa640 1 -- 172.21.15.100:0/301020241 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260000f450 con 0x7f260810b6e0 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.431+0000 7f26057fa640 1 -- 172.21.15.100:0/301020241 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f260000fa90 con 0x7f260810b6e0 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.431+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260819cba0 con 0x7f260810b6e0 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.431+0000 7f26057fa640 1 -- 172.21.15.100:0/301020241 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600018b90 con 0x7f260810b6e0 2024-03-12T05:19:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.431+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26081a4420 con 0x7f260810b6e0 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.433+0000 7f26057fa640 1 -- 172.21.15.100:0/301020241 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f260000fc00 con 0x7f260810b6e0 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.433+0000 7f26057fa640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f25f00a1c20 0x7f25f00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.433+0000 7f2611250740 1 Processor -- start 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.433+0000 7f2607fff640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f25f00a1c20 0x7f25f00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.433+0000 7f26057fa640 1 -- 172.21.15.100:0/301020241 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f26000cde40 con 0x7f260810b6e0 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.433+0000 7f2611250740 1 -- start start 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2611250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2611250740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2611250740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2611250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae625280 con 0x55d7ae6b62b0 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2611250740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae6227b0 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2611250740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae61df90 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2607fff640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f25f00a1c20 0x7f25f00a40e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25f8006010 tx=0x7f25f8005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f2607fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:34202/0 (socket says 172.21.15.100:34202) 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f260cc1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 -- 172.21.15.100:0/3987359831 learned_addr learned my addr 172.21.15.100:0/3987359831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 -- 172.21.15.100:0/3987359831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 msgr2=0x55d7ae6c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 -- 172.21.15.100:0/3987359831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 msgr2=0x55d7ae6b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 -- 172.21.15.100:0/3987359831 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ae5dc570 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f26077fe640 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bf270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009c90 tx=0x7f25fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64ed6d9aabec346 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f25ee7fc640 1 -- 172.21.15.100:0/3987359831 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc019630 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f25ee7fc640 1 -- 172.21.15.100:0/3987359831 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25fc019c70 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.434+0000 7f25ee7fc640 1 -- 172.21.15.100:0/3987359831 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00ebf0 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 -- 172.21.15.100:0/3987359831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 msgr2=0x55d7ae6bf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bf270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009c90 tx=0x7f25fc011810 comp rx=0 tx=0).stop 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 -- 172.21.15.100:0/3987359831 shutdown_connections 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bf270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 --2- 172.21.15.100:0/3987359831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 -- 172.21.15.100:0/3987359831 >> 172.21.15.100:0/3987359831 conn(0x55d7ae6b0990 msgr2=0x55d7ae6b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 -- 172.21.15.100:0/3987359831 shutdown_connections 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 -- 172.21.15.100:0/3987359831 wait complete. 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.435+0000 7f2611250740 1 -- start start 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 Processor -- start 2024-03-12T05:19:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6bf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae2f1430 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae2ade00 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae5a2fd0 con 0x55d7ae6b62b0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2611250740 10 client.0 _ll_register_callbacks cb 0x55d7ae5f8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52964/0 (socket says 172.21.15.100:52964) 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 -- 172.21.15.100:0/1070799123 learned_addr learned my addr 172.21.15.100:0/1070799123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f2607fff640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6bf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f260cc1e640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 -- 172.21.15.100:0/1070799123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 msgr2=0x55d7ae6bf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6bf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 -- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 msgr2=0x55d7ae6bcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 -- 172.21.15.100:0/1070799123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc0098f0 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f26077fe640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bc5f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005bc0 tx=0x7f25fc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f5428ac3f515b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.436+0000 7f25e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f25e77fe640 1 -- 172.21.15.100:0/1070799123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc005450 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f25e77fe640 1 -- 172.21.15.100:0/1070799123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f25fc019de0 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7ae31d7e0 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f25e77fe640 1 -- 172.21.15.100:0/1070799123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25fc0055b0 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 10 client.17106 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d7ae61c610 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f25e77fe640 1 -- 172.21.15.100:0/1070799123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d7ae31d7e0 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f25e77fe640 1 -- 172.21.15.100:0/1070799123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25fc019930 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 10 client.17106 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 20 client.17106 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae6cf920 0x55d7ae6d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 4 client.17106 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7ae35a720 con 0x55d7ae6cf920 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.437+0000 7f2607fff640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae6cf920 0x55d7ae6d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.438+0000 7f2607fff640 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae6cf920 0x55d7ae6d1d40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d75d7ac718628a server_cookie=1169a58d18415545 in_seq=0 out_seq=0 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.438+0000 7f25e77fe640 10 client.17106 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f25e77fe640 1 -- 172.21.15.100:0/1070799123 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7ae35a720 con 0x55d7ae6cf920 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f25e77fe640 10 client.17106 handle_command_reply: tid=0 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 client.17106 shutdown 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 msgr2=0x55d7ae6bc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bc5f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005bc0 tx=0x7f25fc00ef00 comp rx=0 tx=0).stop 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 shutdown_connections 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae6cf920 0x55d7ae6d1d40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6bc5f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f25e77fe640 0 client.17106 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 --2- 172.21.15.100:0/1070799123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6bf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 >> 172.21.15.100:0/1070799123 conn(0x55d7ae6b0990 msgr2=0x55d7ae6b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 shutdown_connections 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 1 -- 172.21.15.100:0/1070799123 wait complete. 2024-03-12T05:19:30.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.439+0000 7f2611250740 20 client.17106 trim_cache size 0 max 16384 2024-03-12T05:19:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 Processor -- start 2024-03-12T05:19:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 -- start start 2024-03-12T05:19:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae6b07f0 con 0x55d7ae6b62b0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae6cb570 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2611250740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae6b19e0 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39844/0 (socket says 172.21.15.100:39844) 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 -- 172.21.15.100:0/674089023 learned_addr learned my addr 172.21.15.100:0/674089023 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f260cc1e640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f26077fe640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 -- 172.21.15.100:0/674089023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6bf7b0 msgr2=0x55d7ae6bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 -- 172.21.15.100:0/674089023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 msgr2=0x55d7ae6c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f2607fff640 1 -- 172.21.15.100:0/674089023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ae5b8ac0 con 0x55d7ae6b62b0 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f260cc1e640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:30.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.455+0000 7f26077fe640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2607fff640 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6660 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f25f802c210 tx=0x7f25f8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06fe31e8193a89c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f25edffb640 1 -- 172.21.15.100:0/674089023 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8053040 con 0x55d7ae6b62b0 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f25edffb640 1 -- 172.21.15.100:0/674089023 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25f8056810 con 0x55d7ae6b62b0 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f25edffb640 1 -- 172.21.15.100:0/674089023 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8003b70 con 0x55d7ae6b62b0 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 -- 172.21.15.100:0/674089023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 msgr2=0x55d7ae6c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6660 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f25f802c210 tx=0x7f25f8041000 comp rx=0 tx=0).stop 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 -- 172.21.15.100:0/674089023 shutdown_connections 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 --2- 172.21.15.100:0/674089023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6660 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 -- 172.21.15.100:0/674089023 >> 172.21.15.100:0/674089023 conn(0x55d7ae6b0990 msgr2=0x55d7ae6b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 -- 172.21.15.100:0/674089023 shutdown_connections 2024-03-12T05:19:30.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.456+0000 7f2611250740 1 -- 172.21.15.100:0/674089023 wait complete. 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 -- start start 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 Processor -- start 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae6c5d50 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae725e30 con 0x55d7ae6b6bd0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ae725f70 con 0x55d7ae6b62b0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2611250740 10 client.0 _ll_register_callbacks cb 0x7f25fc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f260cc1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f260cc1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39858/0 (socket says 172.21.15.100:39858) 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f26077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f2607fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.457+0000 7f260cc1e640 1 -- 172.21.15.100:0/4082021791 learned_addr learned my addr 172.21.15.100:0/4082021791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f260cc1e640 1 -- 172.21.15.100:0/4082021791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 msgr2=0x55d7ae6c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f260cc1e640 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f260cc1e640 1 -- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 msgr2=0x55d7ae6c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f260cc1e640 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f260cc1e640 1 -- 172.21.15.100:0/4082021791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260810e800 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f260cc1e640 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c5650 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f260009a730 tx=0x7f260009a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab96ef89d64e2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f25e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f25e77fe640 1 -- 172.21.15.100:0/4082021791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260009eb80 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f25e77fe640 1 -- 172.21.15.100:0/4082021791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f26000a31d0 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25fc019c70 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f25e77fe640 1 -- 172.21.15.100:0/4082021791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26000a2d40 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 10 client.17112 fetch_fsmap learned FSMap version 25 2024-03-12T05:19:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d7ae5de6a0 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f25e77fe640 1 -- 172.21.15.100:0/4082021791 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25fc019c70 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f25e77fe640 1 -- 172.21.15.100:0/4082021791 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2600016020 con 0x55d7ae6bf7b0 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 10 client.17112 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 20 client.17112 populate_metadata read hostname 'smithi100' 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae727850 0x55d7ae729c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 4 client.17112 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.458+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7ae3a9890 con 0x55d7ae727850 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2607fff640 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae727850 0x55d7ae729c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f25e77fe640 10 client.17112 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2607fff640 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae727850 0x55d7ae729c70 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14e51e8f490fd7a server_cookie=73171fa699e317ed in_seq=0 out_seq=0 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f25e77fe640 1 -- 172.21.15.100:0/4082021791 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d7ae3a9890 con 0x55d7ae727850 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f25e77fe640 10 client.17112 handle_command_reply: tid=0 2024-03-12T05:19:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 client.17112 shutdown 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 msgr2=0x55d7ae6c5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c5650 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f260009a730 tx=0x7f260009a5f0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 shutdown_connections 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7ae727850 0x55d7ae729c70 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7ae6bf7b0 0x55d7ae6c5650 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ae6b6bd0 0x55d7ae6c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 --2- 172.21.15.100:0/4082021791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7ae6b62b0 0x55d7ae6c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f25e77fe640 0 client.17112 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 >> 172.21.15.100:0/4082021791 conn(0x55d7ae6b0990 msgr2=0x55d7ae6bbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 shutdown_connections 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.459+0000 7f2611250740 1 -- 172.21.15.100:0/4082021791 wait complete. 2024-03-12T05:19:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f2611250740 20 client.17112 trim_cache size 0 max 16384 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: "fd6464a8-55d2-47d5-8f86-de67d9a9a98d": { 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "fd6464a8-55d2-47d5-8f86-de67d9a9a98d", 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:19:30.463 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:19:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f25f00a1c20 msgr2=0x7f25f00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f25f00a1c20 0x7f25f00a40e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25f8006010 tx=0x7f25f8005e40 comp rx=0 tx=0).stop 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 msgr2=0x7f260819bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 0x7f260819bf90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f260000cce0 tx=0x7f260000a620 comp rx=0 tx=0).stop 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 shutdown_connections 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f25f00a1c20 0x7f25f00a40e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260810b6e0 0x7f260819bf90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2608102cc0 0x7f2608197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 --2- 172.21.15.100:0/301020241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26081023a0 0x7f2608197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 >> 172.21.15.100:0/301020241 conn(0x7f2608077230 msgr2=0x7f26080fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 shutdown_connections 2024-03-12T05:19:30.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:19:30.460+0000 7f260e681640 1 -- 172.21.15.100:0/301020241 wait complete. 2024-03-12T05:19:30.466 INFO:tasks.cephfs.filesystem:scrub status for tag:fd6464a8-55d2-47d5-8f86-de67d9a9a98d - {'path': '/', 'tag': 'fd6464a8-55d2-47d5-8f86-de67d9a9a98d', 'options': 'recursive,force'} 2024-03-12T05:19:30.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 876873 12.15 MB/sec execute 254 sec latency 21.285 ms 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: pgmap v227: 97 pgs: 97 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 211 op/s 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: pgmap v227: 97 pgs: 97 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 211 op/s 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: pgmap v227: 97 pgs: 97 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 211 op/s 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:19:31.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 878446 12.12 MB/sec execute 255 sec latency 74.112 ms 2024-03-12T05:19:32.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 880921 12.13 MB/sec execute 256 sec latency 16.725 ms 2024-03-12T05:19:33.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:32 smithi156 ceph-mon[36270]: pgmap v228: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 311 op/s 2024-03-12T05:19:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:32 smithi161 ceph-mon[36043]: pgmap v228: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 311 op/s 2024-03-12T05:19:33.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:32 smithi100 ceph-mon[32210]: pgmap v228: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 311 op/s 2024-03-12T05:19:33.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 883035 12.12 MB/sec execute 257 sec latency 38.475 ms 2024-03-12T05:19:34.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 885196 12.12 MB/sec execute 258 sec latency 15.555 ms 2024-03-12T05:19:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:34 smithi156 ceph-mon[36270]: pgmap v229: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-12T05:19:35.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:34 smithi161 ceph-mon[36043]: pgmap v229: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-12T05:19:35.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:34 smithi100 ceph-mon[32210]: pgmap v229: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-12T05:19:35.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 887843 12.14 MB/sec execute 259 sec latency 15.762 ms 2024-03-12T05:19:36.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 889270 12.11 MB/sec execute 260 sec latency 94.470 ms 2024-03-12T05:19:37.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:36 smithi156 ceph-mon[36270]: pgmap v230: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-12T05:19:37.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:36 smithi161 ceph-mon[36043]: pgmap v230: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-12T05:19:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:36 smithi100 ceph-mon[32210]: pgmap v230: 97 pgs: 97 active+clean; 477 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-12T05:19:37.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 891703 12.12 MB/sec execute 261 sec latency 17.662 ms 2024-03-12T05:19:38.796 INFO:tasks.workunit.client.0.smithi100.stdout: 1 894118 12.12 MB/sec execute 262 sec latency 28.979 ms 2024-03-12T05:19:39.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:38 smithi156 ceph-mon[36270]: pgmap v231: 97 pgs: 97 active+clean; 486 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 308 op/s 2024-03-12T05:19:39.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:38 smithi161 ceph-mon[36043]: pgmap v231: 97 pgs: 97 active+clean; 486 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 308 op/s 2024-03-12T05:19:39.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:38 smithi100 ceph-mon[32210]: pgmap v231: 97 pgs: 97 active+clean; 486 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 308 op/s 2024-03-12T05:19:39.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 896057 12.12 MB/sec execute 263 sec latency 14.722 ms 2024-03-12T05:19:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:39 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:19:40.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:39 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:19:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:39 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:19:40.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 898514 12.13 MB/sec execute 264 sec latency 15.238 ms 2024-03-12T05:19:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:40 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:19:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:40 smithi156 ceph-mon[36270]: pgmap v232: 97 pgs: 97 active+clean; 486 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:41.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:40 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:19:41.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:40 smithi161 ceph-mon[36043]: pgmap v232: 97 pgs: 97 active+clean; 486 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:41.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:40 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:19:41.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:40 smithi100 ceph-mon[32210]: pgmap v232: 97 pgs: 97 active+clean; 486 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:41.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 899894 12.10 MB/sec execute 265 sec latency 69.743 ms 2024-03-12T05:19:42.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 902219 12.11 MB/sec execute 266 sec latency 15.801 ms 2024-03-12T05:19:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:42 smithi156 ceph-mon[36270]: pgmap v233: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.3 MiB/s wr, 303 op/s 2024-03-12T05:19:43.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:42 smithi161 ceph-mon[36043]: pgmap v233: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.3 MiB/s wr, 303 op/s 2024-03-12T05:19:43.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:42 smithi100 ceph-mon[32210]: pgmap v233: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.3 MiB/s wr, 303 op/s 2024-03-12T05:19:43.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 904339 12.09 MB/sec execute 267 sec latency 26.620 ms 2024-03-12T05:19:44.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 906541 12.10 MB/sec execute 268 sec latency 15.301 ms 2024-03-12T05:19:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:44 smithi156 ceph-mon[36270]: pgmap v234: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:44 smithi161 ceph-mon[36043]: pgmap v234: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:44 smithi100 ceph-mon[32210]: pgmap v234: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:45.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 909163 12.11 MB/sec execute 269 sec latency 15.601 ms 2024-03-12T05:19:46.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 910534 12.09 MB/sec execute 270 sec latency 75.838 ms 2024-03-12T05:19:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:46 smithi156 ceph-mon[36270]: pgmap v235: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:47.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:46 smithi161 ceph-mon[36043]: pgmap v235: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:47.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:46 smithi100 ceph-mon[32210]: pgmap v235: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-03-12T05:19:47.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 913058 12.10 MB/sec execute 271 sec latency 17.040 ms 2024-03-12T05:19:48.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 915909 12.12 MB/sec execute 272 sec latency 15.027 ms 2024-03-12T05:19:49.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:48 smithi156 ceph-mon[36270]: pgmap v236: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-03-12T05:19:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:48 smithi161 ceph-mon[36043]: pgmap v236: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-03-12T05:19:49.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:48 smithi100 ceph-mon[32210]: pgmap v236: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-03-12T05:19:49.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 917061 12.13 MB/sec execute 273 sec latency 91.663 ms 2024-03-12T05:19:50.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 919165 12.14 MB/sec execute 274 sec latency 31.549 ms 2024-03-12T05:19:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:50 smithi156 ceph-mon[36270]: pgmap v237: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-03-12T05:19:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:50 smithi161 ceph-mon[36043]: pgmap v237: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-03-12T05:19:51.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:50 smithi100 ceph-mon[32210]: pgmap v237: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-03-12T05:19:51.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 921139 12.14 MB/sec execute 275 sec latency 62.608 ms 2024-03-12T05:19:52.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 924000 12.16 MB/sec execute 276 sec latency 13.915 ms 2024-03-12T05:19:53.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:52 smithi156 ceph-mon[36270]: pgmap v238: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2024-03-12T05:19:53.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:52 smithi161 ceph-mon[36043]: pgmap v238: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2024-03-12T05:19:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:52 smithi100 ceph-mon[32210]: pgmap v238: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2024-03-12T05:19:53.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 925932 12.14 MB/sec execute 277 sec latency 54.804 ms 2024-03-12T05:19:54.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 928210 12.14 MB/sec execute 278 sec latency 18.032 ms 2024-03-12T05:19:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:54 smithi156 ceph-mon[36270]: pgmap v239: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-03-12T05:19:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:54 smithi161 ceph-mon[36043]: pgmap v239: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-03-12T05:19:55.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:54 smithi100 ceph-mon[32210]: pgmap v239: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-03-12T05:19:55.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 930715 12.15 MB/sec execute 279 sec latency 22.798 ms 2024-03-12T05:19:56.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 932414 12.14 MB/sec execute 280 sec latency 78.820 ms 2024-03-12T05:19:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:56 smithi156 ceph-mon[36270]: pgmap v240: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-03-12T05:19:57.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:56 smithi161 ceph-mon[36043]: pgmap v240: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-03-12T05:19:57.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:56 smithi100 ceph-mon[32210]: pgmap v240: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-03-12T05:19:57.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 934899 12.15 MB/sec execute 281 sec latency 18.804 ms 2024-03-12T05:19:58.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 937390 12.15 MB/sec execute 282 sec latency 18.579 ms 2024-03-12T05:19:59.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:19:58 smithi156 ceph-mon[36270]: pgmap v241: 97 pgs: 97 active+clean; 493 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-03-12T05:19:59.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:19:58 smithi161 ceph-mon[36043]: pgmap v241: 97 pgs: 97 active+clean; 493 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-03-12T05:19:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:19:58 smithi100 ceph-mon[32210]: pgmap v241: 97 pgs: 97 active+clean; 493 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-03-12T05:19:59.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 939521 12.15 MB/sec execute 283 sec latency 18.973 ms 2024-03-12T05:20:00.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:20:00.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 -- 172.21.15.100:0/4213812473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598069a20 msgr2=0x7ff598069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 --2- 172.21.15.100:0/4213812473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598069a20 0x7ff598069e00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff588009930 tx=0x7ff58802bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 -- 172.21.15.100:0/4213812473 shutdown_connections 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 --2- 172.21.15.100:0/4213812473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598106430 0x7ff59810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 --2- 172.21.15.100:0/4213812473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff59806a3d0 0x7ff598105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 --2- 172.21.15.100:0/4213812473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598069a20 0x7ff598069e00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 -- 172.21.15.100:0/4213812473 >> 172.21.15.100:0/4213812473 conn(0x7ff598076e20 msgr2=0x7ff598077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 -- 172.21.15.100:0/4213812473 shutdown_connections 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.594+0000 7ff5a0064640 1 -- 172.21.15.100:0/4213812473 wait complete. 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 Processor -- start 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 -- start start 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff598069a20 0x7ff59819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598106430 0x7ff59819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 0x7ff598197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598198100 con 0x7ff598197630 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598198270 con 0x7ff598106430 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff5a0064640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598198430 con 0x7ff598069a20 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff59e601640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 0x7ff598197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff59e601640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 0x7ff598197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52220/0 (socket says 172.21.15.100:52220) 2024-03-12T05:20:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff59e601640 1 -- 172.21.15.100:0/661840422 learned_addr learned my addr 172.21.15.100:0/661840422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598106430 0x7ff59819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff59de00640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff598069a20 0x7ff59819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.595+0000 7ff59e601640 1 -- 172.21.15.100:0/661840422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff598069a20 msgr2=0x7ff59819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff59e601640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff598069a20 0x7ff59819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff59e601640 1 -- 172.21.15.100:0/661840422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598106430 msgr2=0x7ff59819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff59e601640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598106430 0x7ff59819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff59e601640 1 -- 172.21.15.100:0/661840422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590009840 con 0x7ff598197630 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff59de00640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff598069a20 0x7ff59819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff59e601640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 0x7ff598197ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff59000efd0 tx=0x7ff59000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff596ffd640 1 -- 172.21.15.100:0/661840422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590009b40 con 0x7ff598197630 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff588009560 con 0x7ff598197630 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff596ffd640 1 -- 172.21.15.100:0/661840422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff590003b70 con 0x7ff598197630 2024-03-12T05:20:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff596ffd640 1 -- 172.21.15.100:0/661840422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5900108f0 con 0x7ff598197630 2024-03-12T05:20:00.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.596+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5981a4720 con 0x7ff598197630 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.598+0000 7ff596ffd640 1 -- 172.21.15.100:0/661840422 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff590003ce0 con 0x7ff598197630 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 Processor -- start 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 -- start start 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff596ffd640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5800a1ba0 0x7ff5800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4bdb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5800a1ba0 0x7ff5800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4b4a280 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4b477b0 con 0x55d7c4be47b0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4b42f90 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff596ffd640 1 -- 172.21.15.100:0/661840422 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff590014070 con 0x7ff598197630 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.599+0000 7ff59de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4bdb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4bdb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52224/0 (socket says 172.21.15.100:52224) 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 -- 172.21.15.100:0/3014688434 learned_addr learned my addr 172.21.15.100:0/3014688434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59e601640 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 -- 172.21.15.100:0/3014688434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdbbd0 msgr2=0x55d7c4be4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 -- 172.21.15.100:0/3014688434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 msgr2=0x55d7c4be6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 -- 172.21.15.100:0/3014688434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7c4b01570 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59e601640 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5800a1ba0 0x7ff5800a4060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff58c004650 tx=0x7ff58c013880 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff59de00640 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4bdb690 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff588006010 tx=0x7ff588005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38aa86d224496dc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff57ffff640 1 -- 172.21.15.100:0/3014688434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588004940 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff57ffff640 1 -- 172.21.15.100:0/3014688434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff588004aa0 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.600+0000 7ff57ffff640 1 -- 172.21.15.100:0/3014688434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880051b0 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/3014688434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 msgr2=0x55d7c4bdb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4bdb690 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff588006010 tx=0x7ff588005e80 comp rx=0 tx=0).stop 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/3014688434 shutdown_connections 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/3014688434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4bdb690 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/3014688434 >> 172.21.15.100:0/3014688434 conn(0x55d7c4bd5990 msgr2=0x55d7c4bd5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/3014688434 shutdown_connections 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/3014688434 wait complete. 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.601+0000 7ff5a2c3a740 1 -- start start 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 Processor -- start 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4be1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4816430 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c47d2e00 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4ac7fd0 con 0x55d7c4be47b0 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff5a2c3a740 10 client.0 _ll_register_callbacks cb 0x55d7c4b1dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59d5ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59d5ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52238/0 (socket says 172.21.15.100:52238) 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41634/0 (socket says 172.21.15.100:41634) 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59d5ff640 1 -- 172.21.15.100:0/2627219039 learned_addr learned my addr 172.21.15.100:0/2627219039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59e601640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4be1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 -- 172.21.15.100:0/2627219039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 msgr2=0x55d7c4be1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4be1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 -- 172.21.15.100:0/2627219039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 msgr2=0x55d7c4be14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59de00640 1 -- 172.21.15.100:0/2627219039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59810e830 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.602+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff59de00640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be4130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff58802c780 tx=0x7ff588005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1366c692b352f50c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff57d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:20:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff57d7fa640 1 -- 172.21.15.100:0/2627219039 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588005300 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff57d7fa640 1 -- 172.21.15.100:0/2627219039 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff588005460 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7c48427e0 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff57d7fa640 1 -- 172.21.15.100:0/2627219039 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff58802cc10 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 10 client.26501 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d7c4b41610 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff57d7fa640 1 -- 172.21.15.100:0/2627219039 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d7c48427e0 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff57d7fa640 1 -- 172.21.15.100:0/2627219039 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff58803c040 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 10 client.26501 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 10 client.26501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 20 client.26501 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4bf44b0 0x55d7c4bf68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 4 client.26501 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.603+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7c487f720 con 0x55d7c4bf44b0 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.604+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4bf44b0 0x55d7c4bf68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.604+0000 7ff57d7fa640 10 client.26501 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.604+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4bf44b0 0x55d7c4bf68a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2702d8d8bbca2365 server_cookie=f0a4e0726e874193 in_seq=0 out_seq=0 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.604+0000 7ff57d7fa640 1 -- 172.21.15.100:0/2627219039 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7c487f720 con 0x55d7c4bf44b0 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.604+0000 7ff57d7fa640 10 client.26501 handle_command_reply: tid=0 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.604+0000 7ff5a2c3a740 1 client.26501 shutdown 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 msgr2=0x55d7c4be4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be4130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff58802c780 tx=0x7ff588005bd0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 shutdown_connections 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4bf44b0 0x55d7c4bf68a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4be1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/2627219039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be4130 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff57d7fa640 0 client.26501 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:00.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 >> 172.21.15.100:0/2627219039 conn(0x55d7c4bd5990 msgr2=0x55d7c4bd7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 shutdown_connections 2024-03-12T05:20:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/2627219039 wait complete. 2024-03-12T05:20:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.605+0000 7ff5a2c3a740 20 client.26501 trim_cache size 0 max 16384 2024-03-12T05:20:00.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 Processor -- start 2024-03-12T05:20:00.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 -- start start 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4bdc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4bd1120 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4bd67c0 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4be2040 con 0x55d7c4be47b0 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff59de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff59d5ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff59de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52240/0 (socket says 172.21.15.100:52240) 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff59de00640 1 -- 172.21.15.100:0/4046862807 learned_addr learned my addr 172.21.15.100:0/4046862807 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.620+0000 7ff59e601640 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4bdc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4046862807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 msgr2=0x55d7c4bdc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4bdc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4046862807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 msgr2=0x55d7c4be3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4046862807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7c4addac0 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59e601640 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4bdc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59de00640 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d7c4bf71a0 tx=0x7ff58c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747e051fa90e5957 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff57f7fe640 1 -- 172.21.15.100:0/4046862807 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c031070 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff57f7fe640 1 -- 172.21.15.100:0/4046862807 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff58c02c8d0 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.621+0000 7ff57f7fe640 1 -- 172.21.15.100:0/4046862807 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c018040 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4046862807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdbbd0 msgr2=0x55d7c4be4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d7c4bf71a0 tx=0x7ff58c020000 comp rx=0 tx=0).stop 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4046862807 shutdown_connections 2024-03-12T05:20:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4be47b0 0x55d7c4bdc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be4130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4046862807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4046862807 >> 172.21.15.100:0/4046862807 conn(0x55d7c4bd5990 msgr2=0x55d7c4bd5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4046862807 shutdown_connections 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4046862807 wait complete. 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 -- start start 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.622+0000 7ff5a2c3a740 1 Processor -- start 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4c4a970 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4c4aab0 con 0x55d7c4be47b0 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7c4c4abf0 con 0x55d7c4bdb2b0 2024-03-12T05:20:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 10 client.0 _ll_register_callbacks cb 0x7ff5900093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52246/0 (socket says 172.21.15.100:52246) 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4070588770 learned_addr learned my addr 172.21.15.100:0/4070588770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59e601640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4070588770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdb2b0 msgr2=0x55d7c4be8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 msgr2=0x55d7c4be8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 -- 172.21.15.100:0/4070588770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7c4b41610 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59de00640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff59d5ff640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be85d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005e00 tx=0x7ff58c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b667b81dcf884c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff57d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff57d7fa640 1 -- 172.21.15.100:0/4070588770 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c031040 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff57d7fa640 1 -- 172.21.15.100:0/4070588770 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff58c009040 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.623+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff588004aa0 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff57d7fa640 1 -- 172.21.15.100:0/4070588770 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff58c018040 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 10 client.17130 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7ff588009560 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff57d7fa640 1 -- 172.21.15.100:0/4070588770 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff588004aa0 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff57d7fa640 1 -- 172.21.15.100:0/4070588770 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff58c005160 con 0x55d7c4bdbbd0 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 10 client.17130 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 20 client.17130 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4c4c500 0x55d7c4c4e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 4 client.17130 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:20:00.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7c48ce890 con 0x55d7c4c4c500 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.624+0000 7ff59de00640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4c4c500 0x55d7c4c4e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff57d7fa640 10 client.17130 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff59de00640 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4c4c500 0x55d7c4c4e920 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8b76c9ea1b0774 server_cookie=ee2fdeba58d397de in_seq=0 out_seq=0 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff57d7fa640 1 -- 172.21.15.100:0/4070588770 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d7c48ce890 con 0x55d7c4c4c500 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff57d7fa640 10 client.17130 handle_command_reply: tid=0 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 client.17130 shutdown 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 msgr2=0x55d7c4be85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be85d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005e00 tx=0x7ff58c009d20 comp rx=0 tx=0).stop 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 shutdown_connections 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7c4c4c500 0x55d7c4c4e920 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7c4be47b0 0x55d7c4be8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff57d7fa640 0 client.17130 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7c4bdbbd0 0x55d7c4be85d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 --2- 172.21.15.100:0/4070588770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7c4bdb2b0 0x55d7c4be8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 >> 172.21.15.100:0/4070588770 conn(0x55d7c4bd5990 msgr2=0x55d7c4bdc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 shutdown_connections 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 1 -- 172.21.15.100:0/4070588770 wait complete. 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.625+0000 7ff5a2c3a740 20 client.17130 trim_cache size 0 max 16384 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:20:00.628 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5800a1ba0 msgr2=0x7ff5800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5800a1ba0 0x7ff5800a4060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff58c004650 tx=0x7ff58c013880 comp rx=0 tx=0).stop 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 msgr2=0x7ff598197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 0x7ff598197ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff59000efd0 tx=0x7ff59000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 shutdown_connections 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5800a1ba0 0x7ff5800a4060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff598197630 0x7ff598197ab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff598106430 0x7ff59819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 --2- 172.21.15.100:0/661840422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff598069a20 0x7ff59819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 >> 172.21.15.100:0/661840422 conn(0x7ff598076e20 msgr2=0x7ff5980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 shutdown_connections 2024-03-12T05:20:00.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.626+0000 7ff5a0064640 1 -- 172.21.15.100:0/661840422 wait complete. 2024-03-12T05:20:00.632 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:20:00.632 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:20:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/805161378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681002f0 msgr2=0x7f5d681006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/805161378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681002f0 0x7f5d681006d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009930 tx=0x7f5d5c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/805161378 shutdown_connections 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/805161378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d681053c0 0x7f5d6810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/805161378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d68100ca0 0x7f5d68104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/805161378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681002f0 0x7f5d681006d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/805161378 >> 172.21.15.100:0/805161378 conn(0x7f5d68077180 msgr2=0x7f5d68077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/805161378 shutdown_connections 2024-03-12T05:20:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.749+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/805161378 wait complete. 2024-03-12T05:20:00.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 Processor -- start 2024-03-12T05:20:00.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 -- start start 2024-03-12T05:20:00.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d681002f0 0x7f5d68198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 0x7f5d68198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d68199160 0x7f5d681a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a3de0 con 0x7f5d681053c0 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a3f50 con 0x7f5d681002f0 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a4110 con 0x7f5d68199160 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 0x7f5d68198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 0x7f5d68198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52268/0 (socket says 172.21.15.100:52268) 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 -- 172.21.15.100:0/1222549568 learned_addr learned my addr 172.21.15.100:0/1222549568 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6659c640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d681002f0 0x7f5d68198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d66d9d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d68199160 0x7f5d681a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 -- 172.21.15.100:0/1222549568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d68199160 msgr2=0x7f5d681a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d68199160 0x7f5d681a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 -- 172.21.15.100:0/1222549568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d681002f0 msgr2=0x7f5d68198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d681002f0 0x7f5d68198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 -- 172.21.15.100:0/1222549568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c009560 con 0x7f5d681053c0 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d65d9b640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 0x7f5d68198bd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005fd0 tx=0x7f5d5800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58009c30 con 0x7f5d681053c0 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d58009d90 con 0x7f5d681053c0 2024-03-12T05:20:00.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d681a43f0 con 0x7f5d681053c0 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.750+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58010460 con 0x7f5d681053c0 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.751+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d681a4900 con 0x7f5d681053c0 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.752+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d68067f70 con 0x7f5d681053c0 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.752+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5d58003f20 con 0x7f5d681053c0 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.753+0000 7f5d577fe640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d500a1ba0 0x7f5d500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.753+0000 7f5d6659c640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d500a1ba0 0x7f5d500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:00.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.753+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5d580cd3f0 con 0x7f5d681053c0 2024-03-12T05:20:00.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.754+0000 7f5d6659c640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d500a1ba0 0x7f5d500a4060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009900 tx=0x7f5d5c005a20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.758+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d58096280 con 0x7f5d681053c0 2024-03-12T05:20:00.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 941930 12.16 MB/sec execute 284 sec latency 20.178 ms 2024-03-12T05:20:00.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.997+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d681085f0 con 0x7f5d681053c0 2024-03-12T05:20:00.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:00.998+0000 7f5d577fe640 1 -- 172.21.15.100:0/1222549568 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f5d58099940 con 0x7f5d681053c0 2024-03-12T05:20:00.993 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:20:00.993 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:20:00.994 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:20:00.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d500a1ba0 msgr2=0x7f5d500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d500a1ba0 0x7f5d500a4060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009900 tx=0x7f5d5c005a20 comp rx=0 tx=0).stop 2024-03-12T05:20:00.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 msgr2=0x7f5d68198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:00.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 0x7f5d68198bd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005fd0 tx=0x7f5d5800dde0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 shutdown_connections 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d500a1ba0 0x7f5d500a4060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d68199160 0x7f5d681a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d681053c0 0x7f5d68198bd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 --2- 172.21.15.100:0/1222549568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d681002f0 0x7f5d68198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 >> 172.21.15.100:0/1222549568 conn(0x7f5d68077180 msgr2=0x7f5d680fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 shutdown_connections 2024-03-12T05:20:00.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.000+0000 7f5d6cd0d640 1 -- 172.21.15.100:0/1222549568 wait complete. 2024-03-12T05:20:01.006 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:20:01.006 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 -- 172.21.15.100:0/188633606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 msgr2=0x7f7ea4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 --2- 172.21.15.100:0/188633606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea4100640 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009930 tx=0x7f7e9802bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 -- 172.21.15.100:0/188633606 shutdown_connections 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 --2- 172.21.15.100:0/188633606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ea4105330 0x7f7ea410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 --2- 172.21.15.100:0/188633606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea4100c10 0x7f7ea4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 --2- 172.21.15.100:0/188633606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea4100640 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 -- 172.21.15.100:0/188633606 >> 172.21.15.100:0/188633606 conn(0x7f7ea4077180 msgr2=0x7f7ea4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 -- 172.21.15.100:0/188633606 shutdown_connections 2024-03-12T05:20:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.121+0000 7f7eaab81640 1 -- 172.21.15.100:0/188633606 wait complete. 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 Processor -- start 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 -- start start 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea4105330 0x7f7ea419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ea4197610 0x7f7ea4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4198000 con 0x7f7ea4100260 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4198170 con 0x7f7ea4197610 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7eaab81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4198330 con 0x7f7ea4105330 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52282/0 (socket says 172.21.15.100:52282) 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 -- 172.21.15.100:0/3476665111 learned_addr learned my addr 172.21.15.100:0/3476665111 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea4105330 0x7f7ea419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea911e640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ea4197610 0x7f7ea4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 -- 172.21.15.100:0/3476665111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea4105330 msgr2=0x7f7ea419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea4105330 0x7f7ea419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 -- 172.21.15.100:0/3476665111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ea4197610 msgr2=0x7f7ea4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ea4197610 0x7f7ea4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 -- 172.21.15.100:0/3476665111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009560 con 0x7f7ea4100260 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.122+0000 7f7ea891d640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea419d460 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009230 tx=0x7f7e9802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.123+0000 7f7ea1ffb640 1 -- 172.21.15.100:0/3476665111 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98004350 con 0x7f7ea4100260 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.123+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea41a4320 con 0x7f7ea4100260 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.123+0000 7f7ea1ffb640 1 -- 172.21.15.100:0/3476665111 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e980044b0 con 0x7f7ea4100260 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.123+0000 7f7ea1ffb640 1 -- 172.21.15.100:0/3476665111 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98003d00 con 0x7f7ea4100260 2024-03-12T05:20:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.123+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea41a47d0 con 0x7f7ea4100260 2024-03-12T05:20:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea1ffb640 1 -- 172.21.15.100:0/3476665111 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7e9803e450 con 0x7f7ea4100260 2024-03-12T05:20:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 Processor -- start 2024-03-12T05:20:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 -- start start 2024-03-12T05:20:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 0x55a9103af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103afb70 0x55a9103b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103bab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91036a960 con 0x55a9103b8750 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91031e310 con 0x55a9103afb70 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ead832740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91031b840 con 0x55a9103af250 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea911e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 0x55a9103af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea911e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 0x55a9103af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51200/0 (socket says 172.21.15.100:51200) 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea911e640 1 -- 172.21.15.100:0/2764983740 learned_addr learned my addr 172.21.15.100:0/2764983740 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea891d640 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103afb70 0x55a9103b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea1ffb640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e940a1ba0 0x7f7e940a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103bab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.126+0000 7f7ea911e640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e940a1ba0 0x7f7e940a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 -- 172.21.15.100:0/2764983740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103afb70 msgr2=0x55a9103b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103afb70 0x55a9103b8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 -- 172.21.15.100:0/2764983740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 msgr2=0x55a9103bab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103bab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 -- 172.21.15.100:0/2764983740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9102d9060 con 0x55a9103af250 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea1ffb640 1 -- 172.21.15.100:0/3476665111 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e980ef6c0 con 0x7f7ea4100260 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 0x55a9103af630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005c40 tx=0x7f7e9c013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e594429abaf40dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7ea911e640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e940a1ba0 0x7f7e940a4060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c01d000 tx=0x7f7e9c01f000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7e8effd640 1 -- 172.21.15.100:0/2764983740 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00fde0 con 0x55a9103af250 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7e8effd640 1 -- 172.21.15.100:0/2764983740 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c012d00 con 0x55a9103af250 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.127+0000 7f7e8effd640 1 -- 172.21.15.100:0/2764983740 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c010780 con 0x55a9103af250 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 -- 172.21.15.100:0/2764983740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 msgr2=0x55a9103af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 0x55a9103af630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005c40 tx=0x7f7e9c013f50 comp rx=0 tx=0).stop 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 -- 172.21.15.100:0/2764983740 shutdown_connections 2024-03-12T05:20:01.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103bab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103afb70 0x55a9103b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 --2- 172.21.15.100:0/2764983740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103af250 0x55a9103af630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 -- 172.21.15.100:0/2764983740 >> 172.21.15.100:0/2764983740 conn(0x55a9103a9980 msgr2=0x55a9103a9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 -- 172.21.15.100:0/2764983740 shutdown_connections 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 -- 172.21.15.100:0/2764983740 wait complete. 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 -- start start 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.128+0000 7f7ead832740 1 Processor -- start 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910317020 con 0x55a9103af250 2024-03-12T05:20:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90ffea430 con 0x55a9103b8750 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90ffa6e00 con 0x55a9103afb70 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ead832740 10 client.0 _ll_register_callbacks cb 0x55a9102becb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52288/0 (socket says 172.21.15.100:52288) 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea891d640 1 -- 172.21.15.100:0/3086987834 learned_addr learned my addr 172.21.15.100:0/3086987834 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea911e640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea3fff640 1 -- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 msgr2=0x55a9103b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea3fff640 1 -- 172.21.15.100:0/3086987834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 msgr2=0x55a9103b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.129+0000 7f7ea3fff640 1 -- 172.21.15.100:0/3086987834 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0136a0 con 0x55a9103afb70 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b5d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4198750 tx=0x7f7e90012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c757a49ff896d337 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7e83fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7e83fff640 1 -- 172.21.15.100:0/3086987834 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90017070 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7e83fff640 1 -- 172.21.15.100:0/3086987834 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e90004020 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9100167e0 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7e83fff640 1 -- 172.21.15.100:0/3086987834 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e900041e0 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 10 client.26542 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a9103156a0 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7e83fff640 1 -- 172.21.15.100:0/3086987834 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e90028080 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7e83fff640 1 -- 172.21.15.100:0/3086987834 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e900039a0 con 0x55a9103afb70 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 10 client.26542 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 10 client.26542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:20:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 20 client.26542 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a9103c8ba0 0x55a9103cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 4 client.26542 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a910053720 con 0x55a9103c8ba0 2024-03-12T05:20:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.130+0000 7f7ea891d640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a9103c8ba0 0x55a9103cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.131+0000 7f7e83fff640 10 client.26542 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.131+0000 7f7ea891d640 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a9103c8ba0 0x55a9103cafc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b396af380554b87 server_cookie=8d1e0a772a0c47b1 in_seq=0 out_seq=0 2024-03-12T05:20:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7e83fff640 1 -- 172.21.15.100:0/3086987834 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a910053720 con 0x55a9103c8ba0 2024-03-12T05:20:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7e83fff640 10 client.26542 handle_command_reply: tid=0 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 client.26542 shutdown 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 msgr2=0x55a9103b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b5d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4198750 tx=0x7f7e90012b60 comp rx=0 tx=0).stop 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 shutdown_connections 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a9103c8ba0 0x55a9103cafc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7e83fff640 0 client.26542 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b5d00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 --2- 172.21.15.100:0/3086987834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 >> 172.21.15.100:0/3086987834 conn(0x55a9103a9980 msgr2=0x55a9103ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 shutdown_connections 2024-03-12T05:20:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 1 -- 172.21.15.100:0/3086987834 wait complete. 2024-03-12T05:20:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.133+0000 7f7ead832740 20 client.26542 trim_cache size 0 max 16384 2024-03-12T05:20:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.146+0000 7f7ead832740 1 Processor -- start 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.146+0000 7f7ead832740 1 -- start start 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.146+0000 7f7ead832740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.146+0000 7f7ead832740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.146+0000 7f7ead832740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.146+0000 7f7ead832740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91029c130 con 0x55a9103af250 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ead832740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9103aeeb0 con 0x55a9103b8750 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ead832740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9103b13e0 con 0x55a9103afb70 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52302/0 (socket says 172.21.15.100:52302) 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51216/0 (socket says 172.21.15.100:51216) 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea891d640 1 -- 172.21.15.100:0/104265936 learned_addr learned my addr 172.21.15.100:0/104265936 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea911e640 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 -- 172.21.15.100:0/104265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 msgr2=0x55a9103cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 -- 172.21.15.100:0/104265936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 msgr2=0x55a9103b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 -- 172.21.15.100:0/104265936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9102b1c20 con 0x55a9103afb70 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b6060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e900059f0 tx=0x7f7e900096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b230fbbb92fc7aab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7e8e7fc640 1 -- 172.21.15.100:0/104265936 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90017070 con 0x55a9103afb70 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7e8e7fc640 1 -- 172.21.15.100:0/104265936 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e90018040 con 0x55a9103afb70 2024-03-12T05:20:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.147+0000 7f7e8e7fc640 1 -- 172.21.15.100:0/104265936 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90005070 con 0x55a9103afb70 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 -- 172.21.15.100:0/104265936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 msgr2=0x55a9103b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b6060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e900059f0 tx=0x7f7e900096d0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 -- 172.21.15.100:0/104265936 shutdown_connections 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103b8750 0x55a9103cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103b6060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 --2- 172.21.15.100:0/104265936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103af250 0x55a9103b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 -- 172.21.15.100:0/104265936 >> 172.21.15.100:0/104265936 conn(0x55a9103a9980 msgr2=0x55a9103a9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 -- 172.21.15.100:0/104265936 shutdown_connections 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 -- 172.21.15.100:0/104265936 wait complete. 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.148+0000 7f7ead832740 1 -- start start 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 Processor -- start 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103af250 0x55a9103be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91041f140 con 0x55a9103b8750 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91041f2b0 con 0x55a9103af250 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91041f470 con 0x55a9103afb70 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ead832740 10 client.0 _ll_register_callbacks cb 0x7f7e9c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103beb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52314/0 (socket says 172.21.15.100:52314) 2024-03-12T05:20:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 -- 172.21.15.100:0/1784917288 learned_addr learned my addr 172.21.15.100:0/1784917288 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea891d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103af250 0x55a9103be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea3fff640 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 -- 172.21.15.100:0/1784917288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 msgr2=0x55a9103be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 -- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103af250 msgr2=0x55a9103be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103af250 0x55a9103be0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 -- 172.21.15.100:0/1784917288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9103156a0 con 0x55a9103b8750 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.149+0000 7f7ea911e640 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103beb60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c012360 tx=0x7f7e9c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55192c707b3ffe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7e83fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7e83fff640 1 -- 172.21.15.100:0/1784917288 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0120d0 con 0x55a9103b8750 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e9c012d00 con 0x55a9103b8750 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7e83fff640 1 -- 172.21.15.100:0/1784917288 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e9c0134e0 con 0x55a9103b8750 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7e83fff640 1 -- 172.21.15.100:0/1784917288 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9c012e70 con 0x55a9103b8750 2024-03-12T05:20:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 10 client.17154 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a9102d78c0 con 0x55a9103b8750 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7e83fff640 1 -- 172.21.15.100:0/1784917288 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c012d00 con 0x55a9103b8750 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7e83fff640 1 -- 172.21.15.100:0/1784917288 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e9c0274b0 con 0x55a9103b8750 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 10 client.17154 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 20 client.17154 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a910420ce0 0x55a910423100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 4 client.17154 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9100a2890 con 0x55a910420ce0 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.150+0000 7f7ea891d640 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a910420ce0 0x55a910423100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7e83fff640 10 client.17154 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ea891d640 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a910420ce0 0x55a910423100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9e0d8b9e9e93355 server_cookie=a5b3b730dcb37cad in_seq=0 out_seq=0 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7e83fff640 1 -- 172.21.15.100:0/1784917288 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9100a2890 con 0x55a910420ce0 2024-03-12T05:20:01.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7e83fff640 10 client.17154 handle_command_reply: tid=0 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 client.17154 shutdown 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 msgr2=0x55a9103beb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103beb60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c012360 tx=0x7f7e9c009480 comp rx=0 tx=0).stop 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 shutdown_connections 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55a910420ce0 0x55a910423100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9103b8750 0x55a9103beb60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9103afb70 0x55a9103be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7e83fff640 0 client.17154 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 --2- 172.21.15.100:0/1784917288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9103af250 0x55a9103be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 >> 172.21.15.100:0/1784917288 conn(0x55a9103a9980 msgr2=0x55a9103cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 shutdown_connections 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 1 -- 172.21.15.100:0/1784917288 wait complete. 2024-03-12T05:20:01.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.151+0000 7f7ead832740 20 client.17154 trim_cache size 0 max 16384 2024-03-12T05:20:01.154 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:20:01.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e940a1ba0 msgr2=0x7f7e940a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e940a1ba0 0x7f7e940a4060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c01d000 tx=0x7f7e9c01f000 comp rx=0 tx=0).stop 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 msgr2=0x7f7ea419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea419d460 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009230 tx=0x7f7e9802c650 comp rx=0 tx=0).stop 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 shutdown_connections 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e940a1ba0 0x7f7e940a4060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ea4197610 0x7f7ea4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea4105330 0x7f7ea419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 --2- 172.21.15.100:0/3476665111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea4100260 0x7f7ea419d460 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 >> 172.21.15.100:0/3476665111 conn(0x7f7ea4077180 msgr2=0x7f7ea40686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 shutdown_connections 2024-03-12T05:20:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.152+0000 7f7eaab81640 1 -- 172.21.15.100:0/3476665111 wait complete. 2024-03-12T05:20:01.158 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:20:01.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:20:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:00 smithi156 ceph-mon[36270]: pgmap v242: 97 pgs: 97 active+clean; 493 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-03-12T05:20:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:00 smithi156 ceph-mon[36270]: overall HEALTH_OK 2024-03-12T05:20:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:00 smithi161 ceph-mon[36043]: pgmap v242: 97 pgs: 97 active+clean; 493 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-03-12T05:20:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:00 smithi161 ceph-mon[36043]: overall HEALTH_OK 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 -- 172.21.15.100:0/3568063091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58100c10 msgr2=0x7f0d58104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/3568063091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58100c10 0x7f0d58104c50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009930 tx=0x7f0d4c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 -- 172.21.15.100:0/3568063091 shutdown_connections 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/3568063091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d58105330 0x7f0d5810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/3568063091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58100c10 0x7f0d58104c50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/3568063091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d58100260 0x7f0d58100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 -- 172.21.15.100:0/3568063091 >> 172.21.15.100:0/3568063091 conn(0x7f0d58077180 msgr2=0x7f0d58077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 -- 172.21.15.100:0/3568063091 shutdown_connections 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.275+0000 7f0d5db8a640 1 -- 172.21.15.100:0/3568063091 wait complete. 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 Processor -- start 2024-03-12T05:20:01.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 -- start start 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d58100260 0x7f0d5819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d58105330 0x7f0d5819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 0x7f0d58197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58198010 con 0x7f0d58197620 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58198180 con 0x7f0d58105330 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d5db8a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58198340 con 0x7f0d58100260 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 0x7f0d58197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d56ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d58105330 0x7f0d5819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 0x7f0d58197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52332/0 (socket says 172.21.15.100:52332) 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 -- 172.21.15.100:0/377103762 learned_addr learned my addr 172.21.15.100:0/377103762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d577fe640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d58100260 0x7f0d5819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 -- 172.21.15.100:0/377103762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d58100260 msgr2=0x7f0d5819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d58100260 0x7f0d5819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 -- 172.21.15.100:0/377103762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d58105330 msgr2=0x7f0d5819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.276+0000 7f0d57fff640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d58105330 0x7f0d5819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d57fff640 1 -- 172.21.15.100:0/377103762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c009560 con 0x7f0d58197620 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d57fff640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 0x7f0d58197ad0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000eb10 tx=0x7f0d5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d54ff9640 1 -- 172.21.15.100:0/377103762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50004b20 con 0x7f0d58197620 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d54ff9640 1 -- 172.21.15.100:0/377103762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d50004c80 con 0x7f0d58197620 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d54ff9640 1 -- 172.21.15.100:0/377103762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50010650 con 0x7f0d58197620 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d581a4330 con 0x7f0d58197620 2024-03-12T05:20:01.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.277+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d581a47c0 con 0x7f0d58197620 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.279+0000 7f0d54ff9640 1 -- 172.21.15.100:0/377103762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0d50005020 con 0x7f0d58197620 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d54ff9640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d400a1ba0 0x7f0d400a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 Processor -- start 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 -- start start 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d54ff9640 1 -- 172.21.15.100:0/377103762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0d50014070 con 0x7f0d58197620 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d577fe640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d400a1ba0 0x7f0d400a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x563984418360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984425620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x563984427f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639843d3960 con 0x563984425b60 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563984387310 con 0x5639844188a0 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d60835740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563984384840 con 0x563984417f80 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x563984427f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x563984427f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52342/0 (socket says 172.21.15.100:52342) 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d56ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984425620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 -- 172.21.15.100:0/2664013624 learned_addr learned my addr 172.21.15.100:0/2664013624 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d577fe640 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x563984418360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d577fe640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d400a1ba0 0x7f0d400a4060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009be0 tx=0x7f0d48011440 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 -- 172.21.15.100:0/2664013624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 msgr2=0x563984418360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x563984418360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 -- 172.21.15.100:0/2664013624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 msgr2=0x563984425620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984425620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.280+0000 7f0d57fff640 1 -- 172.21.15.100:0/2664013624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563984342060 con 0x563984425b60 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.281+0000 7f0d57fff640 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x563984427f50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0d500a2bf0 tx=0x7f0d50009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3439ce2a2d35cf02 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.281+0000 7f0d3dffb640 1 -- 172.21.15.100:0/2664013624 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5009ead0 con 0x563984425b60 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.281+0000 7f0d3dffb640 1 -- 172.21.15.100:0/2664013624 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d500a3120 con 0x563984425b60 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.281+0000 7f0d3dffb640 1 -- 172.21.15.100:0/2664013624 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d500a2c90 con 0x563984425b60 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.281+0000 7f0d60835740 1 -- 172.21.15.100:0/2664013624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 msgr2=0x563984427f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.281+0000 7f0d60835740 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x563984427f50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0d500a2bf0 tx=0x7f0d50009a70 comp rx=0 tx=0).stop 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- 172.21.15.100:0/2664013624 shutdown_connections 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x563984427f50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984425620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 --2- 172.21.15.100:0/2664013624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x563984418360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- 172.21.15.100:0/2664013624 >> 172.21.15.100:0/2664013624 conn(0x563984412980 msgr2=0x563984412d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- 172.21.15.100:0/2664013624 shutdown_connections 2024-03-12T05:20:01.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- 172.21.15.100:0/2664013624 wait complete. 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- start start 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 Processor -- start 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 0x563984422bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 0x563984423100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984425b60 0x563984423640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563984380020 con 0x5639844188a0 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563984053430 con 0x563984417f80 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398400fe00 con 0x563984425b60 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.282+0000 7f0d60835740 10 client.0 _ll_register_callbacks cb 0x563984327cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 0x563984423100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d577fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 0x563984422bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d577fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 0x563984422bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41724/0 (socket says 172.21.15.100:41724) 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 0x563984423100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52352/0 (socket says 172.21.15.100:52352) 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d577fe640 1 -- 172.21.15.100:0/2485692246 learned_addr learned my addr 172.21.15.100:0/2485692246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d57fff640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984425b60 0x563984423640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 -- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984425b60 msgr2=0x563984423640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984425b60 0x563984423640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 -- 172.21.15.100:0/2485692246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 msgr2=0x563984422bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 0x563984422bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 -- 172.21.15.100:0/2485692246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d581a47c0 con 0x5639844188a0 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d577fe640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 0x563984422bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 0x563984423100 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c02c170 tx=0x7f0d4c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d82c6fcf3c3b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d36ffd640 1 -- 172.21.15.100:0/2485692246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c004040 con 0x5639844188a0 2024-03-12T05:20:01.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d36ffd640 1 -- 172.21.15.100:0/2485692246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0d4c0041a0 con 0x5639844188a0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56398407f7e0 con 0x5639844188a0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d36ffd640 1 -- 172.21.15.100:0/2485692246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d4c004360 con 0x5639844188a0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d60835740 10 client.17166 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.283+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56398437e6a0 con 0x5639844188a0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d36ffd640 1 -- 172.21.15.100:0/2485692246 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56398407f7e0 con 0x5639844188a0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d36ffd640 1 -- 172.21.15.100:0/2485692246 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0d4c050070 con 0x5639844188a0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d60835740 10 client.17166 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d60835740 10 client.17166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d60835740 20 client.17166 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d60835740 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563984435f50 0x563984438370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d60835740 4 client.17166 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639840bc720 con 0x563984435f50 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.284+0000 7f0d577fe640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563984435f50 0x563984438370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d577fe640 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563984435f50 0x563984438370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2930d8ad67001834 server_cookie=bb6a1635760a9965 in_seq=0 out_seq=0 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d36ffd640 10 client.17166 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d36ffd640 1 -- 172.21.15.100:0/2485692246 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639840bc720 con 0x563984435f50 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d36ffd640 10 client.17166 handle_command_reply: tid=0 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 client.17166 shutdown 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 msgr2=0x563984423100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 0x563984423100 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c02c170 tx=0x7f0d4c02ca10 comp rx=0 tx=0).stop 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 shutdown_connections 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984425b60 0x563984423640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639844188a0 0x563984423100 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563984435f50 0x563984438370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 --2- 172.21.15.100:0/2485692246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984417f80 0x563984422bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 >> 172.21.15.100:0/2485692246 conn(0x563984412980 msgr2=0x5639844146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d36ffd640 0 client.17166 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.285+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 shutdown_connections 2024-03-12T05:20:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.286+0000 7f0d60835740 1 -- 172.21.15.100:0/2485692246 wait complete. 2024-03-12T05:20:01.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.286+0000 7f0d60835740 20 client.17166 trim_cache size 0 max 16384 2024-03-12T05:20:01.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 Processor -- start 2024-03-12T05:20:01.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 -- start start 2024-03-12T05:20:01.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x5639844252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984423460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x56398443ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563984305130 con 0x563984425b60 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563984418da0 con 0x5639844188a0 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d60835740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398440e5c0 con 0x563984417f80 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x56398443ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.297+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x56398443ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52360/0 (socket says 172.21.15.100:52360) 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 -- 172.21.15.100:0/3947185078 learned_addr learned my addr 172.21.15.100:0/3947185078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984423460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d577fe640 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x5639844252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 -- 172.21.15.100:0/3947185078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 msgr2=0x5639844252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x5639844252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 -- 172.21.15.100:0/3947185078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 msgr2=0x563984423460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984423460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 -- 172.21.15.100:0/3947185078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56398431ac20 con 0x563984425b60 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984423460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d57fff640 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x56398443ccf0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d5009a680 tx=0x7f0d50096cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045e9c77ad909b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d3d7fa640 1 -- 172.21.15.100:0/3947185078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5009a980 con 0x563984425b60 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d3d7fa640 1 -- 172.21.15.100:0/3947185078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d50098c80 con 0x563984425b60 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.298+0000 7f0d3d7fa640 1 -- 172.21.15.100:0/3947185078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d500a2ef0 con 0x563984425b60 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 -- 172.21.15.100:0/3947185078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 msgr2=0x56398443ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x56398443ccf0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d5009a680 tx=0x7f0d50096cb0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 -- 172.21.15.100:0/3947185078 shutdown_connections 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984425b60 0x56398443ccf0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639844188a0 0x563984423460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 --2- 172.21.15.100:0/3947185078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563984417f80 0x5639844252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 -- 172.21.15.100:0/3947185078 >> 172.21.15.100:0/3947185078 conn(0x563984412980 msgr2=0x56398440da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 -- 172.21.15.100:0/3947185078 shutdown_connections 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 -- 172.21.15.100:0/3947185078 wait complete. 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 -- start start 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.299+0000 7f0d60835740 1 Processor -- start 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 0x56398442aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639844188a0 0x56398442af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984425b60 0x56398442b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398448b610 con 0x563984417f80 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398448b780 con 0x563984425b60 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398448b940 con 0x5639844188a0 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d60835740 10 client.0 _ll_register_callbacks cb 0x7f0d50004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 0x56398442aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 0x56398442aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52368/0 (socket says 172.21.15.100:52368) 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 -- 172.21.15.100:0/1247204358 learned_addr learned my addr 172.21.15.100:0/1247204358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d57fff640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984425b60 0x56398442b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639844188a0 0x56398442af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 -- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639844188a0 msgr2=0x56398442af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639844188a0 0x56398442af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 -- 172.21.15.100:0/1247204358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984425b60 msgr2=0x56398442b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984425b60 0x56398442b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 -- 172.21.15.100:0/1247204358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56398437e6a0 con 0x563984417f80 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d577fe640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 0x56398442aa00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x563984438c40 tx=0x7f0d48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d091ab4d54c79355 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.300+0000 7f0d36ffd640 1 -- 172.21.15.100:0/1247204358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48023040 con 0x563984417f80 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d36ffd640 1 -- 172.21.15.100:0/1247204358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0d480285b0 con 0x563984417f80 2024-03-12T05:20:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d500a3120 con 0x563984417f80 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d36ffd640 1 -- 172.21.15.100:0/1247204358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d48028770 con 0x563984417f80 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 10 client.17172 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563984342060 con 0x563984417f80 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d36ffd640 1 -- 172.21.15.100:0/1247204358 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d500a3120 con 0x563984417f80 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d36ffd640 1 -- 172.21.15.100:0/1247204358 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0d48004b80 con 0x563984417f80 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 10 client.17172 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 10 client.17172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 20 client.17172 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56398448d170 0x56398448f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 4 client.17172 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56398410b890 con 0x56398448d170 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.301+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56398448d170 0x56398448f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d36ffd640 10 client.17172 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d56ffd640 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56398448d170 0x56398448f590 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5d146c185b62fb6 server_cookie=2c9d0acdc1ecf8f6 in_seq=0 out_seq=0 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d36ffd640 1 -- 172.21.15.100:0/1247204358 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56398410b890 con 0x56398448d170 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d36ffd640 10 client.17172 handle_command_reply: tid=0 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 client.17172 shutdown 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 msgr2=0x56398442aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 0x56398442aa00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x563984438c40 tx=0x7f0d48004730 comp rx=0 tx=0).stop 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 shutdown_connections 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x56398448d170 0x56398448f590 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563984425b60 0x56398442b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d36ffd640 0 client.17172 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639844188a0 0x56398442af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 --2- 172.21.15.100:0/1247204358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563984417f80 0x56398442aa00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 >> 172.21.15.100:0/1247204358 conn(0x563984412980 msgr2=0x56398441dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 shutdown_connections 2024-03-12T05:20:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 1 -- 172.21.15.100:0/1247204358 wait complete. 2024-03-12T05:20:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.302+0000 7f0d60835740 20 client.17172 trim_cache size 0 max 16384 2024-03-12T05:20:01.307 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d400a1ba0 msgr2=0x7f0d400a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d400a1ba0 0x7f0d400a4060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009be0 tx=0x7f0d48011440 comp rx=0 tx=0).stop 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 msgr2=0x7f0d58197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 0x7f0d58197ad0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000eb10 tx=0x7f0d5000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 shutdown_connections 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0d400a1ba0 0x7f0d400a4060 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d58197620 0x7f0d58197ad0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d58105330 0x7f0d5819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 --2- 172.21.15.100:0/377103762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d58100260 0x7f0d5819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 >> 172.21.15.100:0/377103762 conn(0x7f0d58077180 msgr2=0x7f0d580feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 shutdown_connections 2024-03-12T05:20:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.303+0000 7f0d5db8a640 1 -- 172.21.15.100:0/377103762 wait complete. 2024-03-12T05:20:01.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:00 smithi100 ceph-mon[32210]: pgmap v242: 97 pgs: 97 active+clean; 493 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-03-12T05:20:01.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:00 smithi100 ceph-mon[32210]: overall HEALTH_OK 2024-03-12T05:20:01.310 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:20:01.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 -- 172.21.15.100:0/3900104392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100c10 msgr2=0x7efe3c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 --2- 172.21.15.100:0/3900104392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100c10 0x7efe3c104c50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efe30009930 tx=0x7efe3002bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 -- 172.21.15.100:0/3900104392 shutdown_connections 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 --2- 172.21.15.100:0/3900104392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe3c105330 0x7efe3c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 --2- 172.21.15.100:0/3900104392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100c10 0x7efe3c104c50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 --2- 172.21.15.100:0/3900104392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c100260 0x7efe3c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 -- 172.21.15.100:0/3900104392 >> 172.21.15.100:0/3900104392 conn(0x7efe3c077180 msgr2=0x7efe3c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 -- 172.21.15.100:0/3900104392 shutdown_connections 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 -- 172.21.15.100:0/3900104392 wait complete. 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 Processor -- start 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.429+0000 7efe4246f640 1 -- start start 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe4246f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 0x7efe3c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe4246f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c105330 0x7efe3c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe4246f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe3c199820 0x7efe3c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe4246f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c19a2d0 con 0x7efe3c100260 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe4246f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c19a440 con 0x7efe3c199820 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe4246f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c19a5b0 con 0x7efe3c105330 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 0x7efe3c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 0x7efe3c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52398/0 (socket says 172.21.15.100:52398) 2024-03-12T05:20:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 -- 172.21.15.100:0/906639509 learned_addr learned my addr 172.21.15.100:0/906639509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c105330 0x7efe3c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe40a0c640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe3c199820 0x7efe3c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 -- 172.21.15.100:0/906639509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c105330 msgr2=0x7efe3c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c105330 0x7efe3c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 -- 172.21.15.100:0/906639509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe3c199820 msgr2=0x7efe3c199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe3c199820 0x7efe3c199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 -- 172.21.15.100:0/906639509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30009560 con 0x7efe3c100260 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c105330 0x7efe3c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.430+0000 7efe3bfff640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 0x7efe3c19f6b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00e9f0 tx=0x7efe2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.431+0000 7efe397fa640 1 -- 172.21.15.100:0/906639509 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c009700 con 0x7efe3c100260 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.431+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c1a6560 con 0x7efe3c100260 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.431+0000 7efe397fa640 1 -- 172.21.15.100:0/906639509 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe2c00cd80 con 0x7efe3c100260 2024-03-12T05:20:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.431+0000 7efe397fa640 1 -- 172.21.15.100:0/906639509 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c018920 con 0x7efe3c100260 2024-03-12T05:20:01.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.431+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c1a6a40 con 0x7efe3c100260 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe397fa640 1 -- 172.21.15.100:0/906639509 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7efe2c018a80 con 0x7efe3c100260 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 Processor -- start 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 -- start start 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe397fa640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efe240a1ba0 0x7efe240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f691475630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f691480b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f691430960 con 0x55f691475b70 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6913e4310 con 0x55f69147e750 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe45043740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6913e1840 con 0x55f691475250 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe397fa640 1 -- 172.21.15.100:0/906639509 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efe2c014070 con 0x7efe3c100260 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe40a0c640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efe240a1ba0 0x7efe240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52400/0 (socket says 172.21.15.100:52400) 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe3b7fe640 1 -- 172.21.15.100:0/4249796024 learned_addr learned my addr 172.21.15.100:0/4249796024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.434+0000 7efe40a0c640 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f691480b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3bfff640 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f691475630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3b7fe640 1 -- 172.21.15.100:0/4249796024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 msgr2=0x55f691475630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f691475630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3b7fe640 1 -- 172.21.15.100:0/4249796024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 msgr2=0x55f691480b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f691480b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3b7fe640 1 -- 172.21.15.100:0/4249796024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f69139f060 con 0x55f691475b70 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe40a0c640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efe240a1ba0 0x7efe240a4060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe34005fd0 tx=0x7efe34009e90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147e210 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efe30006010 tx=0x7efe3002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4da013e75df43f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe227fc640 1 -- 172.21.15.100:0/4249796024 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3002c7e0 con 0x55f691475b70 2024-03-12T05:20:01.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe227fc640 1 -- 172.21.15.100:0/4249796024 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe30004510 con 0x55f691475b70 2024-03-12T05:20:01.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.435+0000 7efe227fc640 1 -- 172.21.15.100:0/4249796024 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe300053e0 con 0x55f691475b70 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 -- 172.21.15.100:0/4249796024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 msgr2=0x55f69147e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147e210 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efe30006010 tx=0x7efe3002c660 comp rx=0 tx=0).stop 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 -- 172.21.15.100:0/4249796024 shutdown_connections 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f691480b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147e210 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 --2- 172.21.15.100:0/4249796024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f691475630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 -- 172.21.15.100:0/4249796024 >> 172.21.15.100:0/4249796024 conn(0x55f69146f980 msgr2=0x55f69146fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 -- 172.21.15.100:0/4249796024 shutdown_connections 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 -- 172.21.15.100:0/4249796024 wait complete. 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.436+0000 7efe45043740 1 -- start start 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 Processor -- start 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f69147b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f69147c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6913dd020 con 0x55f691475b70 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6910b0430 con 0x55f69147e750 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69106ce00 con 0x55f691475250 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe45043740 10 client.0 _ll_register_callbacks cb 0x55f691384cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f69147b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f69147b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51324/0 (socket says 172.21.15.100:51324) 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52410/0 (socket says 172.21.15.100:52410) 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3bfff640 1 -- 172.21.15.100:0/3529628465 learned_addr learned my addr 172.21.15.100:0/3529628465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe40a0c640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f69147c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 -- 172.21.15.100:0/3529628465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 msgr2=0x55f69147b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f69147b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 -- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 msgr2=0x55f69147c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f69147c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3b7fe640 1 -- 172.21.15.100:0/3529628465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c0682e0 con 0x55f691475b70 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.437+0000 7efe3bfff640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f69147b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe40a0c640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f69147c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147bce0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efe30009900 tx=0x7efe3002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c191156cb1ea0de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe1b7fe640 1 -- 172.21.15.100:0/3529628465 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30005570 con 0x55f691475b70 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe1b7fe640 1 -- 172.21.15.100:0/3529628465 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efe3004d070 con 0x55f691475b70 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6910dc7e0 con 0x55f691475b70 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe1b7fe640 1 -- 172.21.15.100:0/3529628465 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe30004df0 con 0x55f691475b70 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe45043740 10 client.17184 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f6913db6a0 con 0x55f691475b70 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.438+0000 7efe1b7fe640 1 -- 172.21.15.100:0/3529628465 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe3005d080 con 0x55f691475b70 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe1b7fe640 1 -- 172.21.15.100:0/3529628465 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe3003c050 con 0x55f691475b70 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe45043740 10 client.17184 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe45043740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe45043740 20 client.17184 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe45043740 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f69148e820 0x55f691490c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe45043740 4 client.17184 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f691119720 con 0x55f69148e820 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe3bfff640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f69148e820 0x55f691490c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe3bfff640 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f69148e820 0x55f691490c10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4640331b28b6e576 server_cookie=ff7681de7cba3dfe in_seq=0 out_seq=0 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.439+0000 7efe1b7fe640 10 client.17184 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe1b7fe640 1 -- 172.21.15.100:0/3529628465 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f691119720 con 0x55f69148e820 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe1b7fe640 10 client.17184 handle_command_reply: tid=0 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 client.17184 shutdown 2024-03-12T05:20:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 msgr2=0x55f69147bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147bce0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efe30009900 tx=0x7efe3002ca90 comp rx=0 tx=0).stop 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 shutdown_connections 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f69148e820 0x55f691490c10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f69147e750 0x55f69147c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475b70 0x55f69147bce0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe1b7fe640 0 client.17184 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 --2- 172.21.15.100:0/3529628465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475250 0x55f69147b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 >> 172.21.15.100:0/3529628465 conn(0x55f69146f980 msgr2=0x55f6914716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 shutdown_connections 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 1 -- 172.21.15.100:0/3529628465 wait complete. 2024-03-12T05:20:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.440+0000 7efe45043740 20 client.17184 trim_cache size 0 max 16384 2024-03-12T05:20:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 Processor -- start 2024-03-12T05:20:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 -- start start 2024-03-12T05:20:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475250 0x55f69147ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475b70 0x55f69147c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 0x55f691476a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f691362130 con 0x55f69147e750 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f691474c90 con 0x55f691475250 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe45043740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69146b5c0 con 0x55f691475b70 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe40a0c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 0x55f691476a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe40a0c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 0x55f691476a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52422/0 (socket says 172.21.15.100:52422) 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475250 0x55f69147ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe40a0c640 1 -- 172.21.15.100:0/3001960656 learned_addr learned my addr 172.21.15.100:0/3001960656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.452+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475b70 0x55f69147c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3b7fe640 1 -- 172.21.15.100:0/3001960656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475250 msgr2=0x55f69147ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475250 0x55f69147ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3b7fe640 1 -- 172.21.15.100:0/3001960656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 msgr2=0x55f691476a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 0x55f691476a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3b7fe640 1 -- 172.21.15.100:0/3001960656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f691377c20 con 0x55f691475b70 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe40a0c640 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 0x55f691476a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3bfff640 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475250 0x55f69147ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475b70 0x55f69147c040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe3002c170 tx=0x7efe300050a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa39b2b2bb5f04e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe21ffb640 1 -- 172.21.15.100:0/3001960656 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30004a70 con 0x55f691475b70 2024-03-12T05:20:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe21ffb640 1 -- 172.21.15.100:0/3001960656 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6910dc7e0 con 0x55f691475b70 2024-03-12T05:20:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.453+0000 7efe21ffb640 1 -- 172.21.15.100:0/3001960656 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30003970 con 0x55f691475b70 2024-03-12T05:20:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 -- 172.21.15.100:0/3001960656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475b70 msgr2=0x55f69147c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475b70 0x55f69147c040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe3002c170 tx=0x7efe300050a0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 -- 172.21.15.100:0/3001960656 shutdown_connections 2024-03-12T05:20:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f69147e750 0x55f691476a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f691475b70 0x55f69147c040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 --2- 172.21.15.100:0/3001960656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475250 0x55f69147ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 -- 172.21.15.100:0/3001960656 >> 172.21.15.100:0/3001960656 conn(0x55f69146f980 msgr2=0x55f69146aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 -- 172.21.15.100:0/3001960656 shutdown_connections 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 -- 172.21.15.100:0/3001960656 wait complete. 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 -- start start 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.454+0000 7efe45043740 1 Processor -- start 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 0x55f691483690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475b70 0x55f691483bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f69147e750 0x55f6914e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6914842f0 con 0x55f691475250 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6914e4230 con 0x55f691475b70 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6914e43a0 con 0x55f69147e750 2024-03-12T05:20:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe45043740 10 client.0 _ll_register_callbacks cb 0x7efe2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 0x55f691483690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 0x55f691483690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52436/0 (socket says 172.21.15.100:52436) 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 -- 172.21.15.100:0/772264057 learned_addr learned my addr 172.21.15.100:0/772264057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475b70 0x55f691483bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe40a0c640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f69147e750 0x55f6914e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 -- 172.21.15.100:0/772264057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f69147e750 msgr2=0x55f6914e3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f69147e750 0x55f6914e3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 -- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475b70 msgr2=0x55f691483bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475b70 0x55f691483bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 -- 172.21.15.100:0/772264057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c1a6a40 con 0x55f691475250 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475b70 0x55f691483bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe3bfff640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 0x55f691483690 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000c00 tx=0x7efe2c085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6f4dbb40f5d8c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.455+0000 7efe1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe1b7fe640 1 -- 172.21.15.100:0/772264057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c09e900 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe1b7fe640 1 -- 172.21.15.100:0/772264057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efe2c0a2f50 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe30004510 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe1b7fe640 1 -- 172.21.15.100:0/772264057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe2c096ed0 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 10 client.17190 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7efe34019040 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe1b7fe640 1 -- 172.21.15.100:0/772264057 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe30004510 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe1b7fe640 1 -- 172.21.15.100:0/772264057 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe2c0ba0b0 con 0x55f691475250 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 10 client.17190 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 20 client.17190 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f6914e5ba0 0x55f6914e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 4 client.17190 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f691168890 con 0x55f6914e5ba0 2024-03-12T05:20:01.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.456+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f6914e5ba0 0x55f6914e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe1b7fe640 10 client.17190 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe3b7fe640 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f6914e5ba0 0x55f6914e7fc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4bb60e600d629a server_cookie=45bcc0143edb0fe9 in_seq=0 out_seq=0 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe1b7fe640 1 -- 172.21.15.100:0/772264057 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f691168890 con 0x55f6914e5ba0 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe1b7fe640 10 client.17190 handle_command_reply: tid=0 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 client.17190 shutdown 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 msgr2=0x55f691483690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 0x55f691483690 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000c00 tx=0x7efe2c085040 comp rx=0 tx=0).stop 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 shutdown_connections 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f6914e5ba0 0x55f6914e7fc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f69147e750 0x55f6914e3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f691475b70 0x55f691483bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe1b7fe640 0 client.17190 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:01.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 --2- 172.21.15.100:0/772264057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f691475250 0x55f691483690 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 >> 172.21.15.100:0/772264057 conn(0x55f69146f980 msgr2=0x55f691476a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 shutdown_connections 2024-03-12T05:20:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 1 -- 172.21.15.100:0/772264057 wait complete. 2024-03-12T05:20:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.457+0000 7efe45043740 20 client.17190 trim_cache size 0 max 16384 2024-03-12T05:20:01.459 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efe240a1ba0 msgr2=0x7efe240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efe240a1ba0 0x7efe240a4060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe34005fd0 tx=0x7efe34009e90 comp rx=0 tx=0).stop 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 msgr2=0x7efe3c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 0x7efe3c19f6b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00e9f0 tx=0x7efe2c00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 shutdown_connections 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efe240a1ba0 0x7efe240a4060 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe3c199820 0x7efe3c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3c105330 0x7efe3c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 --2- 172.21.15.100:0/906639509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe3c100260 0x7efe3c19f6b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 >> 172.21.15.100:0/906639509 conn(0x7efe3c077180 msgr2=0x7efe3c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 shutdown_connections 2024-03-12T05:20:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:01.458+0000 7efe4246f640 1 -- 172.21.15.100:0/906639509 wait complete. 2024-03-12T05:20:01.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 943014 12.13 MB/sec execute 285 sec latency 110.063 ms 2024-03-12T05:20:02.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:01 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1222549568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:20:02.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:01 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1222549568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:20:02.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:01 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1222549568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:20:02.468 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:20:02.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:20:02.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 -- 172.21.15.100:0/3283592825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c81064c0 msgr2=0x7f98c810e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/3283592825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c81064c0 0x7f98c810e070 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009cd0 tx=0x7f98c002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 -- 172.21.15.100:0/3283592825 shutdown_connections 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/3283592825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c81064c0 0x7f98c810e070 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/3283592825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8068fa0 0x7f98c8105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/3283592825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c80685f0 0x7f98c80689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.603+0000 7f98cfbe9640 1 -- 172.21.15.100:0/3283592825 >> 172.21.15.100:0/3283592825 conn(0x7f98c8077180 msgr2=0x7f98c8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 -- 172.21.15.100:0/3283592825 shutdown_connections 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 -- 172.21.15.100:0/3283592825 wait complete. 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 Processor -- start 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 -- start start 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 0x7f98c81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c81064c0 0x7f98c8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c819bfb0 0x7f98c819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c819c9a0 con 0x7f98c80685f0 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c819cb10 con 0x7f98c81064c0 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cfbe9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c819cc80 con 0x7f98c819bfb0 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cd985640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 0x7f98c81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cd985640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 0x7f98c81976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52450/0 (socket says 172.21.15.100:52450) 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.604+0000 7f98cd985640 1 -- 172.21.15.100:0/2581677146 learned_addr learned my addr 172.21.15.100:0/2581677146 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd184640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c81064c0 0x7f98c8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98ce186640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c819bfb0 0x7f98c819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd985640 1 -- 172.21.15.100:0/2581677146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c819bfb0 msgr2=0x7f98c819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd985640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c819bfb0 0x7f98c819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd985640 1 -- 172.21.15.100:0/2581677146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c81064c0 msgr2=0x7f98c8197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd985640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c81064c0 0x7f98c8197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd985640 1 -- 172.21.15.100:0/2581677146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009900 con 0x7f98c80685f0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cd985640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 0x7f98c81976f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c920 tx=0x7f98c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98beffd640 1 -- 172.21.15.100:0/2581677146 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4007c70 con 0x7f98c80685f0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98beffd640 1 -- 172.21.15.100:0/2581677146 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98c4007dd0 con 0x7f98c80685f0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c81a4260 con 0x7f98c80685f0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98beffd640 1 -- 172.21.15.100:0/2581677146 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4018860 con 0x7f98c80685f0 2024-03-12T05:20:02.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.605+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c81a4710 con 0x7f98c80685f0 2024-03-12T05:20:02.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.607+0000 7f98beffd640 1 -- 172.21.15.100:0/2581677146 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f98c400fac0 con 0x7f98c80685f0 2024-03-12T05:20:02.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 Processor -- start 2024-03-12T05:20:02.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 -- start start 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98beffd640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f98b40a1ba0 0x7f98b40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 0x55caca6ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca7086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca70b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca669f90 con 0x55caca6ff790 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca6674c0 con 0x55caca700140 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98d27bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca662ca0 con 0x55caca708cc0 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98beffd640 1 -- 172.21.15.100:0/2581677146 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f98c40cd900 con 0x7f98c80685f0 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.608+0000 7f98ce186640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f98b40a1ba0 0x7f98b40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd184640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca7086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 0x55caca6ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 0x55caca6ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52460/0 (socket says 172.21.15.100:52460) 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 -- 172.21.15.100:0/4170619292 learned_addr learned my addr 172.21.15.100:0/4170619292 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd184640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca7086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41810/0 (socket says 172.21.15.100:41810) 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98ce186640 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca70b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 -- 172.21.15.100:0/4170619292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 msgr2=0x55caca70b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca70b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 -- 172.21.15.100:0/4170619292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 msgr2=0x55caca7086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca7086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 -- 172.21.15.100:0/4170619292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caca6250b0 con 0x55caca6ff790 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd184640 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca7086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98ce186640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f98b40a1ba0 0x7f98b40a4060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f98c0005cc0 tx=0x7f98c0034520 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98cd985640 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 0x55caca6ffb70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a2760 tx=0x7f98c4099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aaa018ad3f00730 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98b3fff640 1 -- 172.21.15.100:0/4170619292 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c409e640 con 0x55caca6ff790 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98b3fff640 1 -- 172.21.15.100:0/4170619292 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98c40a2c90 con 0x55caca6ff790 2024-03-12T05:20:02.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.609+0000 7f98b3fff640 1 -- 172.21.15.100:0/4170619292 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4096b60 con 0x55caca6ff790 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 -- 172.21.15.100:0/4170619292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 msgr2=0x55caca6ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 0x55caca6ffb70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a2760 tx=0x7f98c4099f10 comp rx=0 tx=0).stop 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 -- 172.21.15.100:0/4170619292 shutdown_connections 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca70b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca7086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 --2- 172.21.15.100:0/4170619292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca6ff790 0x55caca6ffb70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 -- 172.21.15.100:0/4170619292 >> 172.21.15.100:0/4170619292 conn(0x55caca6f9c50 msgr2=0x55caca6fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 -- 172.21.15.100:0/4170619292 shutdown_connections 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 -- 172.21.15.100:0/4170619292 wait complete. 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.610+0000 7f98d27bf740 1 -- start start 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 Processor -- start 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca6ff790 0x55caca705cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 0x55caca706230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca706770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca607f40 con 0x55caca700140 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca3364b0 con 0x55caca6ff790 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca5e8670 con 0x55caca708cc0 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98d27bf740 10 client.0 _ll_register_callbacks cb 0x55caca655f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 0x55caca706230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 0x55caca706230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52472/0 (socket says 172.21.15.100:52472) 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 -- 172.21.15.100:0/2369813190 learned_addr learned my addr 172.21.15.100:0/2369813190 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd985640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca6ff790 0x55caca705cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98ce186640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca706770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 -- 172.21.15.100:0/2369813190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 msgr2=0x55caca706770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca706770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 -- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca6ff790 msgr2=0x55caca705cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca6ff790 0x55caca705cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 -- 172.21.15.100:0/2369813190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c81a4710 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98cd184640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 0x55caca706230 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f98c8102290 tx=0x7f98b8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214281efa5b7c325 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.611+0000 7f98b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98b17fa640 1 -- 172.21.15.100:0/2369813190 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8017070 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98b17fa640 1 -- 172.21.15.100:0/2369813190 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f98b8004030 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caca63cf40 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98b17fa640 1 -- 172.21.15.100:0/2369813190 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98b80041f0 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 10 client.17202 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55caca661320 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98b17fa640 1 -- 172.21.15.100:0/2369813190 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98b8028080 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98b17fa640 1 -- 172.21.15.100:0/2369813190 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f98b8003960 con 0x55caca700140 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 10 client.17202 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:02.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 20 client.17202 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca719330 0x55caca71b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 4 client.17202 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caca659310 con 0x55caca719330 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.612+0000 7f98cd985640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca719330 0x55caca71b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.613+0000 7f98b17fa640 10 client.17202 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.613+0000 7f98cd985640 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca719330 0x55caca71b750 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cc2ec032f58260 server_cookie=b2793cb821207972 in_seq=0 out_seq=0 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.613+0000 7f98b17fa640 1 -- 172.21.15.100:0/2369813190 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55caca659310 con 0x55caca719330 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.613+0000 7f98b17fa640 10 client.17202 handle_command_reply: tid=0 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.613+0000 7f98d27bf740 1 client.17202 shutdown 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 msgr2=0x55caca706230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 0x55caca706230 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f98c8102290 tx=0x7f98b8012b60 comp rx=0 tx=0).stop 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 shutdown_connections 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca719330 0x55caca71b750 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca708cc0 0x55caca706770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca700140 0x55caca706230 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98b17fa640 0 client.17202 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2369813190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca6ff790 0x55caca705cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 >> 172.21.15.100:0/2369813190 conn(0x55caca6f9c50 msgr2=0x55caca6fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 shutdown_connections 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 1 -- 172.21.15.100:0/2369813190 wait complete. 2024-03-12T05:20:02.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.614+0000 7f98d27bf740 20 client.17202 trim_cache size 0 max 16384 2024-03-12T05:20:02.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 Processor -- start 2024-03-12T05:20:02.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 -- start start 2024-03-12T05:20:02.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca706590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca6f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca6fa820 con 0x55caca708cc0 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca720960 con 0x55caca700140 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98d27bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca6fa9e0 con 0x55caca6ff790 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98ce186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98cd985640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca706590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98ce186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52482/0 (socket says 172.21.15.100:52482) 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98ce186640 1 -- 172.21.15.100:0/1466199278 learned_addr learned my addr 172.21.15.100:0/1466199278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.630+0000 7f98cd184640 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca6f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98ce186640 1 -- 172.21.15.100:0/1466199278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 msgr2=0x55caca706590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98ce186640 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca706590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98ce186640 1 -- 172.21.15.100:0/1466199278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 msgr2=0x55caca6f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98ce186640 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca6f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98ce186640 1 -- 172.21.15.100:0/1466199278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caca5fdc00 con 0x55caca708cc0 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98cd985640 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca706590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:02.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98ce186640 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70fcf0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f98c002c700 tx=0x7f98c0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d45aea8deaae91 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98b37fe640 1 -- 172.21.15.100:0/1466199278 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0032a30 con 0x55caca708cc0 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98b37fe640 1 -- 172.21.15.100:0/1466199278 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98c0003e50 con 0x55caca708cc0 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.631+0000 7f98b37fe640 1 -- 172.21.15.100:0/1466199278 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0057460 con 0x55caca708cc0 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 -- 172.21.15.100:0/1466199278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 msgr2=0x55caca70fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70fcf0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f98c002c700 tx=0x7f98c0046000 comp rx=0 tx=0).stop 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 -- 172.21.15.100:0/1466199278 shutdown_connections 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70fcf0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca6f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 --2- 172.21.15.100:0/1466199278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca706590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 -- 172.21.15.100:0/1466199278 >> 172.21.15.100:0/1466199278 conn(0x55caca6f9c50 msgr2=0x55caca6f4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 -- 172.21.15.100:0/1466199278 shutdown_connections 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 -- 172.21.15.100:0/1466199278 wait complete. 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 -- start start 2024-03-12T05:20:02.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.632+0000 7f98d27bf740 1 Processor -- start 2024-03-12T05:20:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca76a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca76a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca70fad0 con 0x55caca708cc0 2024-03-12T05:20:02.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca70fc40 con 0x55caca700140 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caca70fe00 con 0x55caca6ff790 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 10 client.0 _ll_register_callbacks cb 0x7f98c400a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98cd184640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca76a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98cd985640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca76a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98cd184640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca76a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41836/0 (socket says 172.21.15.100:41836) 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98cd184640 1 -- 172.21.15.100:0/2790143625 learned_addr learned my addr 172.21.15.100:0/2790143625 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52486/0 (socket says 172.21.15.100:52486) 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 -- 172.21.15.100:0/2790143625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 msgr2=0x55caca76a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca76a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 -- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 msgr2=0x55caca76a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca76a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 -- 172.21.15.100:0/2790143625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caca6250b0 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98cd985640 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca76a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98ce186640 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70f510 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f98c002c630 tx=0x7f98c0004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e284b47962d6477 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98b17fa640 1 -- 172.21.15.100:0/2790143625 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00042f0 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98b17fa640 1 -- 172.21.15.100:0/2790143625 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f98c0034730 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caca688900 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.633+0000 7f98b17fa640 1 -- 172.21.15.100:0/2790143625 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98c0003970 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 10 client.17208 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f98c810e880 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98b17fa640 1 -- 172.21.15.100:0/2790143625 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55caca688900 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98b17fa640 1 -- 172.21.15.100:0/2790143625 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f98c004c070 con 0x55caca708cc0 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 10 client.17208 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 20 client.17208 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca771de0 0x55caca774200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 4 client.17208 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55caca362860 con 0x55caca771de0 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98cd985640 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca771de0 0x55caca774200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98cd985640 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca771de0 0x55caca774200 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177bdf892d3253fd server_cookie=6ed1d33ce7c1008d in_seq=0 out_seq=0 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.634+0000 7f98b17fa640 10 client.17208 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.635+0000 7f98b17fa640 1 -- 172.21.15.100:0/2790143625 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55caca362860 con 0x55caca771de0 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.635+0000 7f98b17fa640 10 client.17208 handle_command_reply: tid=0 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.635+0000 7f98d27bf740 1 client.17208 shutdown 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.635+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 msgr2=0x55caca70f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70f510 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f98c002c630 tx=0x7f98c0004480 comp rx=0 tx=0).stop 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 shutdown_connections 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55caca771de0 0x55caca774200 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caca708cc0 0x55caca70f510 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caca700140 0x55caca76a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 --2- 172.21.15.100:0/2790143625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caca6ff790 0x55caca76a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98b17fa640 0 client.17208 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 >> 172.21.15.100:0/2790143625 conn(0x55caca6f9c50 msgr2=0x55caca6f48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 shutdown_connections 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 1 -- 172.21.15.100:0/2790143625 wait complete. 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98d27bf740 20 client.17208 trim_cache size 0 max 16384 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "0390055e-6b05-4129-b5e4-0921d139e90c", 2024-03-12T05:20:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:20:02.638 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:20:02.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f98b40a1ba0 msgr2=0x7f98b40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f98b40a1ba0 0x7f98b40a4060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f98c0005cc0 tx=0x7f98c0034520 comp rx=0 tx=0).stop 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 msgr2=0x7f98c81976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 0x7f98c81976f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c920 tx=0x7f98c400cde0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 shutdown_connections 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f98b40a1ba0 0x7f98b40a4060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c819bfb0 0x7f98c819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c81064c0 0x7f98c8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 --2- 172.21.15.100:0/2581677146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c80685f0 0x7f98c81976f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 >> 172.21.15.100:0/2581677146 conn(0x7f98c8077180 msgr2=0x7f98c80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 shutdown_connections 2024-03-12T05:20:02.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.636+0000 7f98cfbe9640 1 -- 172.21.15.100:0/2581677146 wait complete. 2024-03-12T05:20:02.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:20:02.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:20:02.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.759+0000 7ff036a1c640 1 -- 172.21.15.100:0/3163712252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030100270 msgr2=0x7ff030100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.759+0000 7ff036a1c640 1 --2- 172.21.15.100:0/3163712252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030100270 0x7ff030100650 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff020009930 tx=0x7ff02002bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:02.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 -- 172.21.15.100:0/3163712252 shutdown_connections 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 --2- 172.21.15.100:0/3163712252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff030105340 0x7ff03010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 --2- 172.21.15.100:0/3163712252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff030100c20 0x7ff030104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 --2- 172.21.15.100:0/3163712252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030100270 0x7ff030100650 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 -- 172.21.15.100:0/3163712252 >> 172.21.15.100:0/3163712252 conn(0x7ff030077230 msgr2=0x7ff030077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 -- 172.21.15.100:0/3163712252 shutdown_connections 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 -- 172.21.15.100:0/3163712252 wait complete. 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 Processor -- start 2024-03-12T05:20:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.760+0000 7ff036a1c640 1 -- start start 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff030100270 0x7ff03019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 0x7ff03019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff030197640 0x7ff030197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030198110 con 0x7ff030105340 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030198280 con 0x7ff030100270 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030198440 con 0x7ff030197640 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 0x7ff03019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff030100270 0x7ff03019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 0x7ff03019dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52508/0 (socket says 172.21.15.100:52508) 2024-03-12T05:20:02.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff034fb9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff030197640 0x7ff030197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 -- 172.21.15.100:0/891525693 learned_addr learned my addr 172.21.15.100:0/891525693 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 -- 172.21.15.100:0/891525693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff030197640 msgr2=0x7ff030197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff030197640 0x7ff030197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 -- 172.21.15.100:0/891525693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff030100270 msgr2=0x7ff03019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff030100270 0x7ff03019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 -- 172.21.15.100:0/891525693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff020009560 con 0x7ff030105340 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 0x7ff03019dac0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff02400e9c0 tx=0x7ff02400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02d7fa640 1 -- 172.21.15.100:0/891525693 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024009700 con 0x7ff030105340 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02d7fa640 1 -- 172.21.15.100:0/891525693 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02400cd80 con 0x7ff030105340 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff02d7fa640 1 -- 172.21.15.100:0/891525693 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024010430 con 0x7ff030105340 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0301a43e0 con 0x7ff030105340 2024-03-12T05:20:02.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.761+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0301a4870 con 0x7ff030105340 2024-03-12T05:20:02.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02d7fa640 1 -- 172.21.15.100:0/891525693 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff024005000 con 0x7ff030105340 2024-03-12T05:20:02.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 Processor -- start 2024-03-12T05:20:02.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 -- start start 2024-03-12T05:20:02.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f5f1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f1bd0 0x56035f5fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 0x56035f5fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02d7fa640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0180a1ba0 0x7ff0180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f560280 con 0x56035f5f12b0 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f55d7b0 con 0x56035f5fa7b0 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff0395f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f558f90 con 0x56035f5f1bd0 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff034fb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f5f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 0x56035f5fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f1bd0 0x56035f5fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 0x56035f5fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41864/0 (socket says 172.21.15.100:41864) 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2333812733 learned_addr learned my addr 172.21.15.100:0/2333812733 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff034fb9640 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f5f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52520/0 (socket says 172.21.15.100:52520) 2024-03-12T05:20:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.764+0000 7ff02d7fa640 1 -- 172.21.15.100:0/891525693 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff024014070 con 0x7ff030105340 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff034fb9640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0180a1ba0 0x7ff0180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2333812733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f1bd0 msgr2=0x56035f5fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f1bd0 0x56035f5fa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2333812733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 msgr2=0x56035f5f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f5f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2333812733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56035f517570 con 0x56035f5fa7b0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff034fb9640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0180a1ba0 0x7ff0180a4060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff028005fd0 tx=0x7ff028009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 0x56035f5fcba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff024004fd0 tx=0x7ff0240964d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f82ac8e9a148751 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff0167fc640 1 -- 172.21.15.100:0/2333812733 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02401a0c0 con 0x56035f5fa7b0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff0167fc640 1 -- 172.21.15.100:0/2333812733 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02409af30 con 0x56035f5fa7b0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.765+0000 7ff0167fc640 1 -- 172.21.15.100:0/2333812733 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024096ea0 con 0x56035f5fa7b0 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 -- 172.21.15.100:0/2333812733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 msgr2=0x56035f5fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 0x56035f5fcba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff024004fd0 tx=0x7ff0240964d0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 -- 172.21.15.100:0/2333812733 shutdown_connections 2024-03-12T05:20:02.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5fa7b0 0x56035f5fcba0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f1bd0 0x56035f5fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2333812733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f5f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 -- 172.21.15.100:0/2333812733 >> 172.21.15.100:0/2333812733 conn(0x56035f5eb990 msgr2=0x56035f5ebd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 -- 172.21.15.100:0/2333812733 shutdown_connections 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 -- 172.21.15.100:0/2333812733 wait complete. 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.766+0000 7ff0395f3740 1 -- start start 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 Processor -- start 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f12b0 0x56035f5fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f5f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 0x56035f5f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f22c430 con 0x56035f5fa7b0 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f1e8e00 con 0x56035f5f1bd0 2024-03-12T05:20:02.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f4ddfd0 con 0x56035f5f12b0 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff0395f3740 10 client.0 _ll_register_callbacks cb 0x56035f533fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff034fb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 0x56035f5f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f5f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f5f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41868/0 (socket says 172.21.15.100:41868) 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 -- 172.21.15.100:0/3853702555 learned_addr learned my addr 172.21.15.100:0/3853702555 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff034fb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 0x56035f5f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52526/0 (socket says 172.21.15.100:52526) 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02ffff640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f12b0 0x56035f5fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 -- 172.21.15.100:0/3853702555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f12b0 msgr2=0x56035f5fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f12b0 0x56035f5fa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 -- 172.21.15.100:0/3853702555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 msgr2=0x56035f5f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 0x56035f5f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 -- 172.21.15.100:0/3853702555 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0301a4870 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff034fb9640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 0x56035f5f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f5f7540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff02400c6a0 tx=0x7ff02401a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18fce445a1ea2985 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.767+0000 7ff00f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff00f7fe640 1 -- 172.21.15.100:0/3853702555 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a2fd0 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56035f2587e0 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff00f7fe640 1 -- 172.21.15.100:0/3853702555 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff0240b3220 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff00f7fe640 1 -- 172.21.15.100:0/3853702555 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0240052b0 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 10 client.26597 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56035f557610 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff00f7fe640 1 -- 172.21.15.100:0/3853702555 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56035f2587e0 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff00f7fe640 1 -- 172.21.15.100:0/3853702555 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff02401f060 con 0x56035f5f1bd0 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 10 client.26597 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 20 client.26597 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f60a5b0 0x56035f60c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 4 client.26597 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56035f295720 con 0x56035f60a5b0 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.768+0000 7ff02ffff640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f60a5b0 0x56035f60c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.769+0000 7ff00f7fe640 10 client.26597 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.769+0000 7ff02ffff640 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f60a5b0 0x56035f60c9d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837172190c31e43 server_cookie=ec40f748895558ee in_seq=0 out_seq=0 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.769+0000 7ff00f7fe640 1 -- 172.21.15.100:0/3853702555 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56035f295720 con 0x56035f60a5b0 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.769+0000 7ff00f7fe640 10 client.26597 handle_command_reply: tid=0 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.769+0000 7ff0395f3740 1 client.26597 shutdown 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 msgr2=0x56035f5f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f5f7540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff02400c6a0 tx=0x7ff02401a160 comp rx=0 tx=0).stop 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 shutdown_connections 2024-03-12T05:20:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f60a5b0 0x56035f60c9d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5fa7b0 0x56035f5f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f5f7540 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3853702555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5f12b0 0x56035f5fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff00f7fe640 0 client.26597 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 >> 172.21.15.100:0/3853702555 conn(0x56035f5eb990 msgr2=0x56035f5ed730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 shutdown_connections 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 1 -- 172.21.15.100:0/3853702555 wait complete. 2024-03-12T05:20:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.770+0000 7ff0395f3740 20 client.26597 trim_cache size 0 max 16384 2024-03-12T05:20:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.785+0000 7ff0395f3740 1 Processor -- start 2024-03-12T05:20:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.785+0000 7ff0395f3740 1 -- start start 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f60c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f611330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff0395f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f5f3160 con 0x56035f5f12b0 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff0395f3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f5fcf20 con 0x56035f5f1bd0 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff0395f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f5e5e20 con 0x56035f5fa7b0 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f60c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f60c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52530/0 (socket says 172.21.15.100:52530) 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 -- 172.21.15.100:0/3012789022 learned_addr learned my addr 172.21.15.100:0/3012789022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f611330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff034fb9640 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 -- 172.21.15.100:0/3012789022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 msgr2=0x56035f5f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 -- 172.21.15.100:0/3012789022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 msgr2=0x56035f611330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f611330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 -- 172.21.15.100:0/3012789022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56035f4f3ac0 con 0x56035f5f12b0 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff034fb9640 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff02ffff640 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f60c9a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff020009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1955843e95649134 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff015ffb640 1 -- 172.21.15.100:0/3012789022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020004a00 con 0x56035f5f12b0 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff015ffb640 1 -- 172.21.15.100:0/3012789022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff020048be0 con 0x56035f5f12b0 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.786+0000 7ff015ffb640 1 -- 172.21.15.100:0/3012789022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020048600 con 0x56035f5f12b0 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 -- 172.21.15.100:0/3012789022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 msgr2=0x56035f60c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f60c9a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff020009260 comp rx=0 tx=0).stop 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 -- 172.21.15.100:0/3012789022 shutdown_connections 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f1bd0 0x56035f611330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 --2- 172.21.15.100:0/3012789022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f12b0 0x56035f60c9a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 -- 172.21.15.100:0/3012789022 >> 172.21.15.100:0/3012789022 conn(0x56035f5eb990 msgr2=0x56035f5ebd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 -- 172.21.15.100:0/3012789022 shutdown_connections 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 -- 172.21.15.100:0/3012789022 wait complete. 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.787+0000 7ff0395f3740 1 -- start start 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 Processor -- start 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f12b0 0x56035f5fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 0x56035f5fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f660c20 con 0x56035f5f1bd0 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f660d90 con 0x56035f5f12b0 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035f660f50 con 0x56035f5fa7b0 2024-03-12T05:20:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 10 client.0 _ll_register_callbacks cb 0x7ff024004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 0x56035f5fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 0x56035f5fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52534/0 (socket says 172.21.15.100:52534) 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2973688899 learned_addr learned my addr 172.21.15.100:0/2973688899 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02ffff640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f12b0 0x56035f5fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff034fb9640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2973688899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 msgr2=0x56035f5fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f12b0 msgr2=0x56035f5fe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f12b0 0x56035f5fe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 -- 172.21.15.100:0/2973688899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03010d720 con 0x56035f5f1bd0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff02f7fe640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 0x56035f5fe880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff02409a7c0 tx=0x7ff02409b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fd92a107577e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff00f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff00f7fe640 1 -- 172.21.15.100:0/2973688899 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240054b0 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff00f7fe640 1 -- 172.21.15.100:0/2973688899 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff0240a2fd0 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.788+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff02409af30 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff00f7fe640 1 -- 172.21.15.100:0/2973688899 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02409a830 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 10 client.17226 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56035f5196a0 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff00f7fe640 1 -- 172.21.15.100:0/2973688899 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02409af30 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff00f7fe640 1 -- 172.21.15.100:0/2973688899 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff02401f070 con 0x56035f5f1bd0 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 10 client.17226 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 20 client.17226 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f662250 0x56035f664640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 4 client.17226 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:20:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56035f2e4890 con 0x56035f662250 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff02ffff640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f662250 0x56035f664640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff00f7fe640 10 client.17226 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.789+0000 7ff02ffff640 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f662250 0x56035f664640 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd3ba4e578b5c9f server_cookie=a47fc67def849059 in_seq=0 out_seq=0 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff00f7fe640 1 -- 172.21.15.100:0/2973688899 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56035f2e4890 con 0x56035f662250 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff00f7fe640 10 client.17226 handle_command_reply: tid=0 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 client.17226 shutdown 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 msgr2=0x56035f5fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 0x56035f5fe880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff02409a7c0 tx=0x7ff02409b0a0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 shutdown_connections 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56035f662250 0x56035f664640 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f5fa7b0 0x56035f5fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56035f5f1bd0 0x56035f5fe880 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff00f7fe640 0 client.17226 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 --2- 172.21.15.100:0/2973688899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56035f5f12b0 0x56035f5fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 >> 172.21.15.100:0/2973688899 conn(0x56035f5eb990 msgr2=0x56035f611330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 shutdown_connections 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 1 -- 172.21.15.100:0/2973688899 wait complete. 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.790+0000 7ff0395f3740 20 client.17226 trim_cache size 0 max 16384 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stdout: "0390055e-6b05-4129-b5e4-0921d139e90c": { 2024-03-12T05:20:02.793 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:20:02.794 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "0390055e-6b05-4129-b5e4-0921d139e90c", 2024-03-12T05:20:02.794 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:20:02.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:20:02.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:20:02.794 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0180a1ba0 msgr2=0x7ff0180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0180a1ba0 0x7ff0180a4060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff028005fd0 tx=0x7ff028009500 comp rx=0 tx=0).stop 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 msgr2=0x7ff03019dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 0x7ff03019dac0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff02400e9c0 tx=0x7ff02400ee80 comp rx=0 tx=0).stop 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 shutdown_connections 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0180a1ba0 0x7ff0180a4060 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff030197640 0x7ff030197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff030105340 0x7ff03019dac0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 --2- 172.21.15.100:0/891525693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff030100270 0x7ff03019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 >> 172.21.15.100:0/891525693 conn(0x7ff030077230 msgr2=0x7ff030068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 shutdown_connections 2024-03-12T05:20:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:02.791+0000 7ff036a1c640 1 -- 172.21.15.100:0/891525693 wait complete. 2024-03-12T05:20:02.797 INFO:tasks.cephfs.filesystem:scrub status for tag:0390055e-6b05-4129-b5e4-0921d139e90c - {'path': '/', 'tag': '0390055e-6b05-4129-b5e4-0921d139e90c', 'options': 'recursive,force'} 2024-03-12T05:20:02.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 945432 12.14 MB/sec execute 286 sec latency 18.920 ms 2024-03-12T05:20:03.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:02 smithi100 ceph-mon[32210]: pgmap v243: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-03-12T05:20:03.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:02 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:20:03.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:02 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:03.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:02 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:20:03.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:02 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:02 smithi156 ceph-mon[36270]: pgmap v243: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-03-12T05:20:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:02 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:20:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:02 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:02 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:20:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:02 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:02 smithi161 ceph-mon[36043]: pgmap v243: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-03-12T05:20:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:02 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:20:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:02 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:02 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:20:03.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:02 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:03.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 947596 12.12 MB/sec execute 287 sec latency 15.923 ms 2024-03-12T05:20:04.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:03 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:20:04.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:03 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:04.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:03 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:20:04.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:03 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:04.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:03 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:20:04.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:03 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:04.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1 949721 12.13 MB/sec execute 288 sec latency 14.898 ms 2024-03-12T05:20:05.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:04 smithi161 ceph-mon[36043]: pgmap v244: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 214 op/s 2024-03-12T05:20:05.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:04 smithi100 ceph-mon[32210]: pgmap v244: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 214 op/s 2024-03-12T05:20:05.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:04 smithi156 ceph-mon[36270]: pgmap v244: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 214 op/s 2024-03-12T05:20:05.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 952436 12.14 MB/sec execute 289 sec latency 21.787 ms 2024-03-12T05:20:06.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:05 smithi161 ceph-mon[36043]: osdmap e92: 12 total, 12 up, 12 in 2024-03-12T05:20:06.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:05 smithi161 ceph-mon[36043]: pgmap v246: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 257 op/s 2024-03-12T05:20:06.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:05 smithi100 ceph-mon[32210]: osdmap e92: 12 total, 12 up, 12 in 2024-03-12T05:20:06.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:05 smithi100 ceph-mon[32210]: pgmap v246: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 257 op/s 2024-03-12T05:20:06.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:05 smithi156 ceph-mon[36270]: osdmap e92: 12 total, 12 up, 12 in 2024-03-12T05:20:06.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:05 smithi156 ceph-mon[36270]: pgmap v246: 97 pgs: 97 active+clean; 502 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 257 op/s 2024-03-12T05:20:06.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 953864 12.12 MB/sec execute 290 sec latency 116.266 ms 2024-03-12T05:20:07.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 956141 12.13 MB/sec execute 291 sec latency 40.251 ms 2024-03-12T05:20:08.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 958502 12.12 MB/sec execute 292 sec latency 16.408 ms 2024-03-12T05:20:09.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:08 smithi156 ceph-mon[36270]: pgmap v247: 97 pgs: 97 active+clean; 509 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-03-12T05:20:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:08 smithi161 ceph-mon[36043]: pgmap v247: 97 pgs: 97 active+clean; 509 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-03-12T05:20:09.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:08 smithi100 ceph-mon[32210]: pgmap v247: 97 pgs: 97 active+clean; 509 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-03-12T05:20:09.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 960695 12.13 MB/sec execute 293 sec latency 17.816 ms 2024-03-12T05:20:10.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:09 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:20:10.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:09 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:20:10.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:09 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:20:10.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 963224 12.13 MB/sec execute 294 sec latency 14.060 ms 2024-03-12T05:20:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:10 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:20:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:10 smithi156 ceph-mon[36270]: pgmap v248: 97 pgs: 97 active+clean; 509 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-03-12T05:20:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:10 smithi156 ceph-mon[36270]: osdmap e93: 12 total, 12 up, 12 in 2024-03-12T05:20:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:10 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:20:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:10 smithi161 ceph-mon[36043]: pgmap v248: 97 pgs: 97 active+clean; 509 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-03-12T05:20:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:10 smithi161 ceph-mon[36043]: osdmap e93: 12 total, 12 up, 12 in 2024-03-12T05:20:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:10 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:20:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:10 smithi100 ceph-mon[32210]: pgmap v248: 97 pgs: 97 active+clean; 509 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-03-12T05:20:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:10 smithi100 ceph-mon[32210]: osdmap e93: 12 total, 12 up, 12 in 2024-03-12T05:20:11.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 964926 12.11 MB/sec execute 295 sec latency 63.137 ms 2024-03-12T05:20:12.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 967055 12.12 MB/sec execute 296 sec latency 120.977 ms 2024-03-12T05:20:13.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:12 smithi156 ceph-mon[36270]: pgmap v250: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 300 op/s 2024-03-12T05:20:13.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:12 smithi161 ceph-mon[36043]: pgmap v250: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 300 op/s 2024-03-12T05:20:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:12 smithi100 ceph-mon[32210]: pgmap v250: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 300 op/s 2024-03-12T05:20:13.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 969727 12.12 MB/sec execute 297 sec latency 28.402 ms 2024-03-12T05:20:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 971586 12.12 MB/sec execute 298 sec latency 19.780 ms 2024-03-12T05:20:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:14 smithi156 ceph-mon[36270]: pgmap v251: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2024-03-12T05:20:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:14 smithi161 ceph-mon[36043]: pgmap v251: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2024-03-12T05:20:15.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:14 smithi100 ceph-mon[32210]: pgmap v251: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2024-03-12T05:20:15.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 974038 12.12 MB/sec execute 299 sec latency 12.823 ms 2024-03-12T05:20:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 976761 12.13 MB/sec execute 300 sec latency 14.424 ms 2024-03-12T05:20:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:16 smithi156 ceph-mon[36270]: pgmap v252: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2024-03-12T05:20:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:16 smithi161 ceph-mon[36043]: pgmap v252: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2024-03-12T05:20:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:16 smithi100 ceph-mon[32210]: pgmap v252: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2024-03-12T05:20:17.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 978158 12.11 MB/sec execute 301 sec latency 105.289 ms 2024-03-12T05:20:18.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:17 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:20:18.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:17 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:20:18.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:17 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:20:18.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 980787 12.13 MB/sec execute 302 sec latency 17.381 ms 2024-03-12T05:20:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:18 smithi156 ceph-mon[36270]: pgmap v253: 97 pgs: 97 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-12T05:20:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:18 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:20:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:18 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:20:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:18 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:20:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:18 smithi161 ceph-mon[36043]: pgmap v253: 97 pgs: 97 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-12T05:20:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:18 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:20:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:18 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:20:19.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:18 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:20:19.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:18 smithi100 ceph-mon[32210]: pgmap v253: 97 pgs: 97 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-12T05:20:19.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:18 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:20:19.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:18 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:20:19.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:18 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:20:19.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 982758 12.11 MB/sec execute 303 sec latency 16.844 ms 2024-03-12T05:20:20.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 984911 12.11 MB/sec execute 304 sec latency 14.598 ms 2024-03-12T05:20:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:20 smithi156 ceph-mon[36270]: pgmap v254: 97 pgs: 97 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-12T05:20:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:20 smithi161 ceph-mon[36043]: pgmap v254: 97 pgs: 97 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-12T05:20:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:20 smithi100 ceph-mon[32210]: pgmap v254: 97 pgs: 97 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-12T05:20:21.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 987739 12.13 MB/sec execute 305 sec latency 13.015 ms 2024-03-12T05:20:22.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 988882 12.10 MB/sec execute 306 sec latency 102.806 ms 2024-03-12T05:20:23.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:22 smithi156 ceph-mon[36270]: pgmap v255: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-03-12T05:20:23.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:22 smithi161 ceph-mon[36043]: pgmap v255: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-03-12T05:20:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:22 smithi100 ceph-mon[32210]: pgmap v255: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-03-12T05:20:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 991524 12.12 MB/sec execute 307 sec latency 15.717 ms 2024-03-12T05:20:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 993591 12.10 MB/sec execute 308 sec latency 17.513 ms 2024-03-12T05:20:25.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:24 smithi156 ceph-mon[36270]: pgmap v256: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-03-12T05:20:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:24 smithi161 ceph-mon[36043]: pgmap v256: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-03-12T05:20:25.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:24 smithi100 ceph-mon[32210]: pgmap v256: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-03-12T05:20:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 995658 12.10 MB/sec execute 309 sec latency 14.709 ms 2024-03-12T05:20:26.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 998283 12.12 MB/sec execute 310 sec latency 19.394 ms 2024-03-12T05:20:27.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:26 smithi156 ceph-mon[36270]: pgmap v257: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-03-12T05:20:27.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:26 smithi161 ceph-mon[36043]: pgmap v257: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-03-12T05:20:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:26 smithi100 ceph-mon[32210]: pgmap v257: 97 pgs: 97 active+clean; 498 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-03-12T05:20:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 999708 12.10 MB/sec execute 311 sec latency 107.205 ms 2024-03-12T05:20:28.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1002169 12.11 MB/sec execute 312 sec latency 13.100 ms 2024-03-12T05:20:29.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:28 smithi156 ceph-mon[36270]: pgmap v258: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 291 op/s 2024-03-12T05:20:29.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:28 smithi161 ceph-mon[36043]: pgmap v258: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 291 op/s 2024-03-12T05:20:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:28 smithi100 ceph-mon[32210]: pgmap v258: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 291 op/s 2024-03-12T05:20:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1004300 12.09 MB/sec execute 313 sec latency 24.042 ms 2024-03-12T05:20:30.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1006522 12.10 MB/sec execute 314 sec latency 15.426 ms 2024-03-12T05:20:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:30 smithi156 ceph-mon[36270]: pgmap v259: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 201 op/s 2024-03-12T05:20:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:30 smithi161 ceph-mon[36043]: pgmap v259: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 201 op/s 2024-03-12T05:20:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:30 smithi100 ceph-mon[32210]: pgmap v259: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 201 op/s 2024-03-12T05:20:31.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1009115 12.11 MB/sec execute 315 sec latency 19.772 ms 2024-03-12T05:20:32.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:20:32.797 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:20:32.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1010463 12.09 MB/sec execute 316 sec latency 106.380 ms 2024-03-12T05:20:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 -- 172.21.15.100:0/2196734284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be010b6e0 msgr2=0x7f7be010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 --2- 172.21.15.100:0/2196734284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be010b6e0 0x7f7be010db80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009ca0 tx=0x7f7bd802c180 comp rx=0 tx=0).stop 2024-03-12T05:20:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 -- 172.21.15.100:0/2196734284 shutdown_connections 2024-03-12T05:20:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 --2- 172.21.15.100:0/2196734284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be010b6e0 0x7f7be010db80 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 --2- 172.21.15.100:0/2196734284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be0102cc0 0x7f7be010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 --2- 172.21.15.100:0/2196734284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7be01023a0 0x7f7be0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 -- 172.21.15.100:0/2196734284 >> 172.21.15.100:0/2196734284 conn(0x7f7be0077230 msgr2=0x7f7be0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 -- 172.21.15.100:0/2196734284 shutdown_connections 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.914+0000 7f7be598c640 1 -- 172.21.15.100:0/2196734284 wait complete. 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 Processor -- start 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 -- start start 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 0x7f7be01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7be0102cc0 0x7f7be0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010b6e0 0x7f7be019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019c5a0 con 0x7f7be01023a0 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019c710 con 0x7f7be0102cc0 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7be598c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019c8d0 con 0x7f7be010b6e0 2024-03-12T05:20:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 0x7f7be01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 0x7f7be01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36168/0 (socket says 172.21.15.100:36168) 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3916518423 learned_addr learned my addr 172.21.15.100:0/3916518423 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010b6e0 0x7f7be019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.915+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7be0102cc0 0x7f7be0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3916518423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010b6e0 msgr2=0x7f7be019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010b6e0 0x7f7be019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3916518423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7be0102cc0 msgr2=0x7f7be0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7be0102cc0 0x7f7be0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3916518423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd8009900 con 0x7f7be01023a0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 0x7f7be01976e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000e9f0 tx=0x7f7bd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7be498a640 1 -- 172.21.15.100:0/3916518423 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0009700 con 0x7f7be01023a0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7be498a640 1 -- 172.21.15.100:0/3916518423 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bd000cd80 con 0x7f7be01023a0 2024-03-12T05:20:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be019cbb0 con 0x7f7be01023a0 2024-03-12T05:20:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7be498a640 1 -- 172.21.15.100:0/3916518423 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0018920 con 0x7f7be01023a0 2024-03-12T05:20:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.916+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be01a4430 con 0x7f7be01023a0 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.918+0000 7f7be498a640 1 -- 172.21.15.100:0/3916518423 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7bd0018a80 con 0x7f7be01023a0 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 Processor -- start 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 -- start start 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b6823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b691fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b5f1280 con 0x56501b682900 2024-03-12T05:20:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b5ee7b0 con 0x56501b68fbc0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be81fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b5e9f90 con 0x56501b681fe0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be498a640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7bc80a1ba0 0x7f7bc80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b6823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7bc80a1ba0 0x7f7bc80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b691fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36170/0 (socket says 172.21.15.100:36170) 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3302948720 learned_addr learned my addr 172.21.15.100:0/3302948720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7be498a640 1 -- 172.21.15.100:0/3916518423 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7bd0014070 con 0x7f7be01023a0 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3302948720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 msgr2=0x56501b6823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b6823c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3302948720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 msgr2=0x56501b691fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b691fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.919+0000 7f7bdeffd640 1 -- 172.21.15.100:0/3302948720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56501b5a8570 con 0x56501b682900 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b6823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b691fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68f680 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009a540 tx=0x7f7bd0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8deee2624867b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7bc80a1ba0 0x7f7bc80a4060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8004af0 tx=0x7f7bd8005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bc5ffb640 1 -- 172.21.15.100:0/3302948720 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00a2f30 con 0x56501b682900 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bc5ffb640 1 -- 172.21.15.100:0/3302948720 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bd009a840 con 0x56501b682900 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7bc5ffb640 1 -- 172.21.15.100:0/3302948720 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00cb750 con 0x56501b682900 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7be81fd740 1 -- 172.21.15.100:0/3302948720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 msgr2=0x56501b68f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.920+0000 7f7be81fd740 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68f680 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009a540 tx=0x7f7bd0010f50 comp rx=0 tx=0).stop 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- 172.21.15.100:0/3302948720 shutdown_connections 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b691fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68f680 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 --2- 172.21.15.100:0/3302948720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b6823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- 172.21.15.100:0/3302948720 >> 172.21.15.100:0/3302948720 conn(0x56501b67c990 msgr2=0x56501b67cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- 172.21.15.100:0/3302948720 shutdown_connections 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- 172.21.15.100:0/3302948720 wait complete. 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- start start 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 Processor -- start 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b68d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b2bd430 con 0x56501b682900 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b279e00 con 0x56501b68fbc0 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b56efd0 con 0x56501b681fe0 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7be81fd740 10 client.0 _ll_register_callbacks cb 0x56501b5c4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.921+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b68d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35706/0 (socket says 172.21.15.100:35706) 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36178/0 (socket says 172.21.15.100:36178) 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 -- 172.21.15.100:0/839746471 learned_addr learned my addr 172.21.15.100:0/839746471 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 -- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 msgr2=0x56501b68d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b68d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 -- 172.21.15.100:0/839746471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 msgr2=0x56501b68c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 -- 172.21.15.100:0/839746471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be01a4430 con 0x56501b681fe0 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009a7d0 tx=0x7f7bd00cb2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880ba3bc4828df6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:32.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bbeffd640 1 -- 172.21.15.100:0/839746471 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0096b20 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56501b2e97e0 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bbeffd640 1 -- 172.21.15.100:0/839746471 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7bd0096eb0 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bbeffd640 1 -- 172.21.15.100:0/839746471 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bd00a2f30 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7be81fd740 10 client.26629 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56501b5e8610 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.922+0000 7f7bbeffd640 1 -- 172.21.15.100:0/839746471 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56501b2e97e0 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7bbeffd640 1 -- 172.21.15.100:0/839746471 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7bd0099d90 con 0x56501b681fe0 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7be81fd740 10 client.26629 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7be81fd740 10 client.26629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7be81fd740 20 client.26629 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7be81fd740 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b69f9f0 0x56501b6a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7be81fd740 4 client.26629 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56501b326720 con 0x56501b69f9f0 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b69f9f0 0x56501b6a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7bbeffd640 10 client.26629 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.923+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b69f9f0 0x56501b6a1e20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432084fb0b742d02 server_cookie=7e4f8235b29f0f6b in_seq=0 out_seq=0 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7bbeffd640 1 -- 172.21.15.100:0/839746471 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56501b326720 con 0x56501b69f9f0 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7bbeffd640 10 client.26629 handle_command_reply: tid=0 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 client.26629 shutdown 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 msgr2=0x56501b68f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009a7d0 tx=0x7f7bd00cb2c0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 shutdown_connections 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b69f9f0 0x56501b6a1e20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b68fbc0 0x56501b68d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7bbeffd640 0 client.26629 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b682900 0x56501b68c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 --2- 172.21.15.100:0/839746471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 >> 172.21.15.100:0/839746471 conn(0x56501b67c990 msgr2=0x56501b67e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 shutdown_connections 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 1 -- 172.21.15.100:0/839746471 wait complete. 2024-03-12T05:20:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.924+0000 7f7be81fd740 20 client.26629 trim_cache size 0 max 16384 2024-03-12T05:20:32.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 Processor -- start 2024-03-12T05:20:32.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 -- start start 2024-03-12T05:20:32.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b682900 0x56501b68f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b68c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b678120 con 0x56501b68fbc0 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b6a7040 con 0x56501b682900 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7be81fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b69c1c0 con 0x56501b681fe0 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b68c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b68c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36182/0 (socket says 172.21.15.100:36182) 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 -- 172.21.15.100:0/2059926316 learned_addr learned my addr 172.21.15.100:0/2059926316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b682900 0x56501b68f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 -- 172.21.15.100:0/2059926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 msgr2=0x56501b68f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 -- 172.21.15.100:0/2059926316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b682900 msgr2=0x56501b68f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b682900 0x56501b68f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 -- 172.21.15.100:0/2059926316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56501b584ac0 con 0x56501b68fbc0 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b68c240 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009ca0 tx=0x7f7bd8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29e2ae131ace028 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bc57fa640 1 -- 172.21.15.100:0/2059926316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8003cf0 con 0x56501b68fbc0 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.940+0000 7f7bc57fa640 1 -- 172.21.15.100:0/2059926316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bd8049040 con 0x56501b68fbc0 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7bc57fa640 1 -- 172.21.15.100:0/2059926316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80045d0 con 0x56501b68fbc0 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 -- 172.21.15.100:0/2059926316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 msgr2=0x56501b68c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b68c240 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009ca0 tx=0x7f7bd8009820 comp rx=0 tx=0).stop 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 -- 172.21.15.100:0/2059926316 shutdown_connections 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b68c240 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b682900 0x56501b68f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 --2- 172.21.15.100:0/2059926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b681fe0 0x56501b68f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 -- 172.21.15.100:0/2059926316 >> 172.21.15.100:0/2059926316 conn(0x56501b67c990 msgr2=0x56501b687d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 -- 172.21.15.100:0/2059926316 shutdown_connections 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 -- 172.21.15.100:0/2059926316 wait complete. 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.941+0000 7f7be81fd740 1 -- start start 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 Processor -- start 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 0x56501b693af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 0x56501b694030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b6f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b694750 con 0x56501b68fbc0 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b6f6450 con 0x56501b681fe0 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501b6f65c0 con 0x56501b682900 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7be81fd740 10 client.0 _ll_register_callbacks cb 0x7f7bd009a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:32.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b6f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 0x56501b693af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 0x56501b694030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b6f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36186/0 (socket says 172.21.15.100:36186) 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 0x56501b694030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35730/0 (socket says 172.21.15.100:35730) 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 0x56501b693af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60264/0 (socket says 172.21.15.100:60264) 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdf7fe640 1 -- 172.21.15.100:0/355571531 learned_addr learned my addr 172.21.15.100:0/355571531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 -- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 msgr2=0x56501b693af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 0x56501b693af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 -- 172.21.15.100:0/355571531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 msgr2=0x56501b6f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b6f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bde7fc640 1 -- 172.21.15.100:0/355571531 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd8030070 con 0x56501b682900 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 0x56501b693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:32.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.942+0000 7f7bdf7fe640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b6f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bde7fc640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 0x56501b694030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x56501b6a2740 tx=0x7f7bd4012d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ada311b6a5db00 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bbeffd640 1 -- 172.21.15.100:0/355571531 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4017040 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bd009a840 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bbeffd640 1 -- 172.21.15.100:0/355571531 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7bd401fa40 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bbeffd640 1 -- 172.21.15.100:0/355571531 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bd401fc00 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 10 client.26635 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56501b5aa6a0 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bbeffd640 1 -- 172.21.15.100:0/355571531 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bd009a840 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7bbeffd640 1 -- 172.21.15.100:0/355571531 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7bd4030070 con 0x56501b682900 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 10 client.26635 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:32.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 20 client.26635 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b6f7e50 0x56501b6fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 4 client.26635 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.943+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56501b375890 con 0x56501b6f7e50 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b6f7e50 0x56501b6fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7bbeffd640 10 client.26635 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7bdeffd640 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b6f7e50 0x56501b6fa270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4ccdf50f9990cb server_cookie=a2b7b9c78d2f7e44 in_seq=0 out_seq=0 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7bbeffd640 1 -- 172.21.15.100:0/355571531 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56501b375890 con 0x56501b6f7e50 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7bbeffd640 10 client.26635 handle_command_reply: tid=0 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 client.26635 shutdown 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 msgr2=0x56501b694030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 0x56501b694030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x56501b6a2740 tx=0x7f7bd4012d10 comp rx=0 tx=0).stop 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 shutdown_connections 2024-03-12T05:20:32.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56501b6f7e50 0x56501b6fa270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56501b68fbc0 0x56501b6f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56501b682900 0x56501b694030 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 --2- 172.21.15.100:0/355571531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56501b681fe0 0x56501b693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7bbeffd640 0 client.26635 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 >> 172.21.15.100:0/355571531 conn(0x56501b67c990 msgr2=0x56501b68c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 shutdown_connections 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.944+0000 7f7be81fd740 1 -- 172.21.15.100:0/355571531 wait complete. 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be81fd740 20 client.26635 trim_cache size 0 max 16384 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:20:32.947 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:20:32.948 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:20:32.948 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7bc80a1ba0 msgr2=0x7f7bc80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7bc80a1ba0 0x7f7bc80a4060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8004af0 tx=0x7f7bd8005c50 comp rx=0 tx=0).stop 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 msgr2=0x7f7be01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 0x7f7be01976e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000e9f0 tx=0x7f7bd000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 shutdown_connections 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7bc80a1ba0 0x7f7bc80a4060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010b6e0 0x7f7be019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7be0102cc0 0x7f7be0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 --2- 172.21.15.100:0/3916518423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7be01023a0 0x7f7be01976e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 >> 172.21.15.100:0/3916518423 conn(0x7f7be0077230 msgr2=0x7f7be00fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 shutdown_connections 2024-03-12T05:20:32.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:32.945+0000 7f7be598c640 1 -- 172.21.15.100:0/3916518423 wait complete. 2024-03-12T05:20:32.951 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:20:32.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 -- 172.21.15.100:0/1633981657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9010100ca0 msgr2=0x7f9010104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 --2- 172.21.15.100:0/1633981657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9010100ca0 0x7f9010104ce0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009930 tx=0x7f8ffc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 -- 172.21.15.100:0/1633981657 shutdown_connections 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 --2- 172.21.15.100:0/1633981657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 0x7f901010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 --2- 172.21.15.100:0/1633981657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9010100ca0 0x7f9010104ce0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 --2- 172.21.15.100:0/1633981657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90101002f0 0x7f90101006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.060+0000 7f9016beb640 1 -- 172.21.15.100:0/1633981657 >> 172.21.15.100:0/1633981657 conn(0x7f9010077180 msgr2=0x7f9010077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 -- 172.21.15.100:0/1633981657 shutdown_connections 2024-03-12T05:20:33.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 -- 172.21.15.100:0/1633981657 wait complete. 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 Processor -- start 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 -- start start 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 0x7f901019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 0x7f901019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9010197690 0x7f9010197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010198160 con 0x7f90101002f0 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101982d0 con 0x7f9010197690 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9016beb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010198490 con 0x7f90101053c0 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9014987640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 0x7f901019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9014987640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 0x7f901019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36196/0 (socket says 172.21.15.100:36196) 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9014987640 1 -- 172.21.15.100:0/1235640804 learned_addr learned my addr 172.21.15.100:0/1235640804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f900ffff640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 0x7f901019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.061+0000 7f9015188640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9010197690 0x7f9010197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9014987640 1 -- 172.21.15.100:0/1235640804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 msgr2=0x7f901019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9014987640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 0x7f901019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9014987640 1 -- 172.21.15.100:0/1235640804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9010197690 msgr2=0x7f9010197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9014987640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9010197690 0x7f9010197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9014987640 1 -- 172.21.15.100:0/1235640804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc009560 con 0x7f90101002f0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f900ffff640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 0x7f901019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9014987640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 0x7f901019d5d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f900400e9f0 tx=0x7f900400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004009700 con 0x7f90101002f0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f900400cd80 con 0x7f90101002f0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004018920 con 0x7f90101002f0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90101a4430 con 0x7f90101002f0 2024-03-12T05:20:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.062+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90101a4910 con 0x7f90101002f0 2024-03-12T05:20:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.063+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90101017c0 con 0x7f90101002f0 2024-03-12T05:20:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.064+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9004018a80 con 0x7f90101002f0 2024-03-12T05:20:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.065+0000 7f900dffb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f90000a1ba0 0x7f90000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.065+0000 7f900ffff640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f90000a1ba0 0x7f90000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.065+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9004014070 con 0x7f90101002f0 2024-03-12T05:20:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.066+0000 7f900ffff640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f90000a1ba0 0x7f90000a4060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.070+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9004052d20 con 0x7f90101002f0 2024-03-12T05:20:33.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:32 smithi156 ceph-mon[36270]: pgmap v260: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 305 op/s 2024-03-12T05:20:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:32 smithi161 ceph-mon[36043]: pgmap v260: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 305 op/s 2024-03-12T05:20:33.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.290+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9010100ca0 con 0x7f90101002f0 2024-03-12T05:20:33.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.291+0000 7f900dffb640 1 -- 172.21.15.100:0/1235640804 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f9004096ae0 con 0x7f90101002f0 2024-03-12T05:20:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:20:33.287 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:20:33.287 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:20:33.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f90000a1ba0 msgr2=0x7f90000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f90000a1ba0 0x7f90000a4060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc005fd0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 msgr2=0x7f901019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 0x7f901019d5d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f900400e9f0 tx=0x7f900400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 shutdown_connections 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f90000a1ba0 0x7f90000a4060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9010197690 0x7f9010197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90101053c0 0x7f901019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 --2- 172.21.15.100:0/1235640804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90101002f0 0x7f901019d5d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 >> 172.21.15.100:0/1235640804 conn(0x7f9010077180 msgr2=0x7f9010104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 shutdown_connections 2024-03-12T05:20:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.293+0000 7f9016beb640 1 -- 172.21.15.100:0/1235640804 wait complete. 2024-03-12T05:20:33.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:32 smithi100 ceph-mon[32210]: pgmap v260: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 305 op/s 2024-03-12T05:20:33.299 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:20:33.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:20:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 -- 172.21.15.100:0/146801717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c068fa0 msgr2=0x7fc39c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 --2- 172.21.15.100:0/146801717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c068fa0 0x7fc39c105de0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc390009930 tx=0x7fc39002bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 -- 172.21.15.100:0/146801717 shutdown_connections 2024-03-12T05:20:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 --2- 172.21.15.100:0/146801717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c1064c0 0x7fc39c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 --2- 172.21.15.100:0/146801717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c068fa0 0x7fc39c105de0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 --2- 172.21.15.100:0/146801717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c0685f0 0x7fc39c0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 -- 172.21.15.100:0/146801717 >> 172.21.15.100:0/146801717 conn(0x7fc39c077180 msgr2=0x7fc39c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 -- 172.21.15.100:0/146801717 shutdown_connections 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 -- 172.21.15.100:0/146801717 wait complete. 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 Processor -- start 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.411+0000 7fc3a133e640 1 -- start start 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0685f0 0x7fc39c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 0x7fc39c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c197600 0x7fc39c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c197ff0 con 0x7fc39c1064c0 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c198160 con 0x7fc39c197600 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c198320 con 0x7fc39c0685f0 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 0x7fc39c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 0x7fc39c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36206/0 (socket says 172.21.15.100:36206) 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0685f0 0x7fc39c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1921015353 learned_addr learned my addr 172.21.15.100:0/1921015353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c197600 0x7fc39c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1921015353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0685f0 msgr2=0x7fc39c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0685f0 0x7fc39c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1921015353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c197600 msgr2=0x7fc39c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c197600 0x7fc39c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1921015353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390009560 con 0x7fc39c1064c0 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39affd640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0685f0 0x7fc39c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c197600 0x7fc39c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 0x7fc39c19d990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc390000c00 tx=0x7fc39002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc383fff640 1 -- 172.21.15.100:0/1921015353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900042d0 con 0x7fc39c1064c0 2024-03-12T05:20:33.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc383fff640 1 -- 172.21.15.100:0/1921015353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc390004430 con 0x7fc39c1064c0 2024-03-12T05:20:33.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c1a4310 con 0x7fc39c1064c0 2024-03-12T05:20:33.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc383fff640 1 -- 172.21.15.100:0/1921015353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390003c90 con 0x7fc39c1064c0 2024-03-12T05:20:33.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.412+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c1a47c0 con 0x7fc39c1064c0 2024-03-12T05:20:33.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.414+0000 7fc383fff640 1 -- 172.21.15.100:0/1921015353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc390004df0 con 0x7fc39c1064c0 2024-03-12T05:20:33.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 Processor -- start 2024-03-12T05:20:33.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 -- start start 2024-03-12T05:20:33.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc383fff640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc3840a1ba0 0x7fc3840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 0x562544fe0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544febb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544f9b960 con 0x562544fe0250 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544f4f310 con 0x562544fe9750 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc3840a1ba0 0x7fc3840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544f4c840 con 0x562544fe0b70 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc383fff640 1 -- 172.21.15.100:0/1921015353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc3900ef650 con 0x7fc39c1064c0 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc39affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 0x562544fe0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc39affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 0x562544fe0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36218/0 (socket says 172.21.15.100:36218) 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc39affd640 1 -- 172.21.15.100:0/1080545774 learned_addr learned my addr 172.21.15.100:0/1080545774 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.415+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544febb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39affd640 1 -- 172.21.15.100:0/1080545774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 msgr2=0x562544fe9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39affd640 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39affd640 1 -- 172.21.15.100:0/1080545774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 msgr2=0x562544febb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39affd640 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544febb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39affd640 1 -- 172.21.15.100:0/1080545774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562544f0a060 con 0x562544fe0250 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc3840a1ba0 0x7fc3840a4060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc394009fd0 tx=0x7fc394011c10 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc39affd640 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 0x562544fe0630 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc38c005f40 tx=0x7fc38c011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ab440aff550905 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc3817fa640 1 -- 172.21.15.100:0/1080545774 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c011330 con 0x562544fe0250 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc3817fa640 1 -- 172.21.15.100:0/1080545774 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc38c00fe80 con 0x562544fe0250 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.416+0000 7fc3817fa640 1 -- 172.21.15.100:0/1080545774 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0193e0 con 0x562544fe0250 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1080545774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 msgr2=0x562544fe0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 0x562544fe0630 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc38c005f40 tx=0x7fc38c011d30 comp rx=0 tx=0).stop 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1080545774 shutdown_connections 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544febb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1080545774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0250 0x562544fe0630 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1080545774 >> 172.21.15.100:0/1080545774 conn(0x562544fda980 msgr2=0x562544fdad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1080545774 shutdown_connections 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1080545774 wait complete. 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- start start 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 Processor -- start 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fe6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x562544fe71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544f48020 con 0x562544fe9750 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544c1b430 con 0x562544fe0250 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544bd7e00 con 0x562544fe0b70 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc3a3fef740 10 client.0 _ll_register_callbacks cb 0x562544eefcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:33.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.417+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x562544fe71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x562544fe71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36226/0 (socket says 172.21.15.100:36226) 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 -- 172.21.15.100:0/806748891 learned_addr learned my addr 172.21.15.100:0/806748891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39affd640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fe6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 -- 172.21.15.100:0/806748891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 msgr2=0x562544fe6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 -- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 msgr2=0x562544fe6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fe6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 -- 172.21.15.100:0/806748891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c011500 con 0x562544fe9750 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x562544fe71b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc39400ff90 tx=0x7fc394004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1391f390d7d3f0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc37eff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc37eff5640 1 -- 172.21.15.100:0/806748891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394003cf0 con 0x562544fe9750 2024-03-12T05:20:33.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc37eff5640 1 -- 172.21.15.100:0/806748891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc394020420 con 0x562544fe9750 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562544c477e0 con 0x562544fe9750 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc37eff5640 1 -- 172.21.15.100:0/806748891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc394003e50 con 0x562544fe9750 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc3a3fef740 10 client.17262 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562544f466a0 con 0x562544fe9750 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.418+0000 7fc37eff5640 1 -- 172.21.15.100:0/806748891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562544c477e0 con 0x562544fe9750 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc37eff5640 1 -- 172.21.15.100:0/806748891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc39402e070 con 0x562544fe9750 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc3a3fef740 10 client.17262 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc3a3fef740 10 client.17262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc3a3fef740 20 client.17262 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562544ff9ba0 0x562544ffbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc3a3fef740 4 client.17262 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562544c84720 con 0x562544ff9ba0 2024-03-12T05:20:33.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc39affd640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562544ff9ba0 0x562544ffbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc37eff5640 10 client.17262 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.419+0000 7fc39affd640 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562544ff9ba0 0x562544ffbf90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=438ff5e3b271ca29 server_cookie=788de5d575a2741e in_seq=0 out_seq=0 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc37eff5640 1 -- 172.21.15.100:0/806748891 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562544c84720 con 0x562544ff9ba0 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc37eff5640 10 client.17262 handle_command_reply: tid=0 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 client.17262 shutdown 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 msgr2=0x562544fe71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x562544fe71b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc39400ff90 tx=0x7fc394004730 comp rx=0 tx=0).stop 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 shutdown_connections 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562544ff9ba0 0x562544ffbf90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x562544fe71b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fe6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/806748891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fe6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc37eff5640 0 client.17262 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:33.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 >> 172.21.15.100:0/806748891 conn(0x562544fda980 msgr2=0x562544fdc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 shutdown_connections 2024-03-12T05:20:33.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 1 -- 172.21.15.100:0/806748891 wait complete. 2024-03-12T05:20:33.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.420+0000 7fc3a3fef740 20 client.17262 trim_cache size 0 max 16384 2024-03-12T05:20:33.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 Processor -- start 2024-03-12T05:20:33.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 -- start start 2024-03-12T05:20:33.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0250 0x562544fd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 0x562544ff0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544ff0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544ecd130 con 0x562544fe0b70 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544fdbbb0 con 0x562544fe9750 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562545001240 con 0x562544fe0250 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 0x562544ff0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 0x562544ff0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36230/0 (socket says 172.21.15.100:36230) 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1603736087 learned_addr learned my addr 172.21.15.100:0/1603736087 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39affd640 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0250 0x562544fd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544ff0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1603736087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0250 msgr2=0x562544fd9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0250 0x562544fd9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1603736087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 msgr2=0x562544ff0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544ff0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.433+0000 7fc39a7fc640 1 -- 172.21.15.100:0/1603736087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562544ee2c20 con 0x562544fe0b70 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 0x562544ff0600 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc39002cde0 tx=0x7fc3900dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688a2375681513f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc380ff9640 1 -- 172.21.15.100:0/1603736087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900bca50 con 0x562544fe0b70 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc380ff9640 1 -- 172.21.15.100:0/1603736087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3900bc350 con 0x562544fe0b70 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc380ff9640 1 -- 172.21.15.100:0/1603736087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900ecfe0 con 0x562544fe0b70 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1603736087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 msgr2=0x562544ff0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 0x562544ff0600 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc39002cde0 tx=0x7fc3900dc040 comp rx=0 tx=0).stop 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1603736087 shutdown_connections 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe9750 0x562544ff0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe0b70 0x562544ff0600 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/1603736087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0250 0x562544fd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1603736087 >> 172.21.15.100:0/1603736087 conn(0x562544fda980 msgr2=0x562544fd92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1603736087 shutdown_connections 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.434+0000 7fc3a3fef740 1 -- 172.21.15.100:0/1603736087 wait complete. 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 -- start start 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 Processor -- start 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x5625450500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544ff0090 con 0x562544fe9750 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562544ff0200 con 0x562544fe0250 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562545050610 con 0x562544fe0b70 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc3a3fef740 10 client.0 _ll_register_callbacks cb 0x7fc38c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x5625450500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x5625450500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36244/0 (socket says 172.21.15.100:36244) 2024-03-12T05:20:33.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 -- 172.21.15.100:0/3473949322 learned_addr learned my addr 172.21.15.100:0/3473949322 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39a7fc640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 -- 172.21.15.100:0/3473949322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 msgr2=0x562544fef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 -- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 msgr2=0x562544fef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 -- 172.21.15.100:0/3473949322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c1a47c0 con 0x562544fe9750 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39affd640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc39b7fe640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x5625450500a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc394009f00 tx=0x7fc3940058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c956059d3b1ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.435+0000 7fc37eff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc37eff5640 1 -- 172.21.15.100:0/3473949322 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394003dd0 con 0x562544fe9750 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc37eff5640 1 -- 172.21.15.100:0/3473949322 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc3940289c0 con 0x562544fe9750 2024-03-12T05:20:33.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc38c00fe80 con 0x562544fe9750 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc37eff5640 1 -- 172.21.15.100:0/3473949322 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc394028b80 con 0x562544fe9750 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 10 client.17268 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562544f088c0 con 0x562544fe9750 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc37eff5640 1 -- 172.21.15.100:0/3473949322 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc38c00fe80 con 0x562544fe9750 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc37eff5640 1 -- 172.21.15.100:0/3473949322 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc39402e040 con 0x562544fe9750 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 10 client.17268 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 10 client.17268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 20 client.17268 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562545051dc0 0x5625450541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 4 client.17268 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562544cd3890 con 0x562545051dc0 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.436+0000 7fc39affd640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562545051dc0 0x5625450541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc37eff5640 10 client.17268 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:33.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc39affd640 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562545051dc0 0x5625450541e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c9013035b619ea5 server_cookie=6f345e20f14fd2da in_seq=0 out_seq=0 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc37eff5640 1 -- 172.21.15.100:0/3473949322 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562544cd3890 con 0x562545051dc0 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc37eff5640 10 client.17268 handle_command_reply: tid=0 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 client.17268 shutdown 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 msgr2=0x5625450500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x5625450500a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc394009f00 tx=0x7fc3940058c0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 shutdown_connections 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562545051dc0 0x5625450541e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562544fe9750 0x5625450500a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc37eff5640 0 client.17268 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562544fe0b70 0x562544fef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 --2- 172.21.15.100:0/3473949322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562544fe0250 0x562544fef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 >> 172.21.15.100:0/3473949322 conn(0x562544fda980 msgr2=0x562544fd9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 shutdown_connections 2024-03-12T05:20:33.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 1 -- 172.21.15.100:0/3473949322 wait complete. 2024-03-12T05:20:33.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.437+0000 7fc3a3fef740 20 client.17268 trim_cache size 0 max 16384 2024-03-12T05:20:33.441 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc3840a1ba0 msgr2=0x7fc3840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc3840a1ba0 0x7fc3840a4060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc394009fd0 tx=0x7fc394011c10 comp rx=0 tx=0).stop 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 msgr2=0x7fc39c19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 0x7fc39c19d990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc390000c00 tx=0x7fc39002c5d0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 shutdown_connections 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc3840a1ba0 0x7fc3840a4060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c197600 0x7fc39c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc39c1064c0 0x7fc39c19d990 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 --2- 172.21.15.100:0/1921015353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0685f0 0x7fc39c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 >> 172.21.15.100:0/1921015353 conn(0x7fc39c077180 msgr2=0x7fc39c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 shutdown_connections 2024-03-12T05:20:33.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.438+0000 7fc3a133e640 1 -- 172.21.15.100:0/1921015353 wait complete. 2024-03-12T05:20:33.444 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:20:33.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/642812870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24106240 msgr2=0x7f7e24106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/642812870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24106240 0x7f7e24106620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d960 tx=0x7f7e2002db00 comp rx=0 tx=0).stop 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/642812870 shutdown_connections 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/642812870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e24100d20 0x7f7e2410ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/642812870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24100360 0x7f7e241007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/642812870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24106240 0x7f7e24106620 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/642812870 >> 172.21.15.100:0/642812870 conn(0x7f7e24077060 msgr2=0x7f7e24077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.559+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/642812870 shutdown_connections 2024-03-12T05:20:33.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.560+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/642812870 wait complete. 2024-03-12T05:20:33.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.560+0000 7f7e2ba2c640 1 Processor -- start 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.560+0000 7f7e2ba2c640 1 -- start start 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e2ba2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100360 0x7f7e2419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e2ba2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e24100d20 0x7f7e2419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e2ba2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24106240 0x7f7e24199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e2ba2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e24199ed0 con 0x7f7e24100d20 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e2ba2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419a040 con 0x7f7e24106240 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e2ba2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419a200 con 0x7f7e24100360 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100360 0x7f7e2419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100360 0x7f7e2419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35816/0 (socket says 172.21.15.100:35816) 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e297c8640 1 -- 172.21.15.100:0/2558535187 learned_addr learned my addr 172.21.15.100:0/2558535187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24106240 0x7f7e24199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e29fc9640 1 -- 172.21.15.100:0/2558535187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100360 msgr2=0x7f7e2419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100360 0x7f7e2419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e29fc9640 1 -- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e24100d20 msgr2=0x7f7e2419fbb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e24100d20 0x7f7e2419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.561+0000 7f7e29fc9640 1 -- 172.21.15.100:0/2558535187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2000d590 con 0x7f7e24106240 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.562+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24106240 0x7f7e24199830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00ef00 tx=0x7f7e1c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.562+0000 7f7e1a7fc640 1 -- 172.21.15.100:0/2558535187 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c004b20 con 0x7f7e24106240 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.562+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2419a4e0 con 0x7f7e24106240 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.562+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e241a66a0 con 0x7f7e24106240 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.562+0000 7f7e1a7fc640 1 -- 172.21.15.100:0/2558535187 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e1c003e50 con 0x7f7e24106240 2024-03-12T05:20:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.562+0000 7f7e1a7fc640 1 -- 172.21.15.100:0/2558535187 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00b570 con 0x7f7e24106240 2024-03-12T05:20:33.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.564+0000 7f7e1a7fc640 1 -- 172.21.15.100:0/2558535187 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7e1c00a040 con 0x7f7e24106240 2024-03-12T05:20:33.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 Processor -- start 2024-03-12T05:20:33.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 -- start start 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e1a7fc640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e100a1c20 0x7f7e100a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 0x55ae09ba1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1b70 0x55ae09baa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e100a1c20 0x7f7e100a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09bacb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09b5c960 con 0x55ae09ba1b70 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09b10310 con 0x55ae09ba1250 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09b0d840 con 0x55ae09baa750 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e1a7fc640 1 -- 172.21.15.100:0/2558535187 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e1c0cd650 con 0x7f7e24106240 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 0x55ae09ba1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 0x55ae09ba1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60356/0 (socket says 172.21.15.100:60356) 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e297c8640 1 -- 172.21.15.100:0/3433886819 learned_addr learned my addr 172.21.15.100:0/3433886819 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1b70 0x55ae09baa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.565+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09bacb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e297c8640 1 -- 172.21.15.100:0/3433886819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 msgr2=0x55ae09bacb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e297c8640 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09bacb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e297c8640 1 -- 172.21.15.100:0/3433886819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1b70 msgr2=0x55ae09baa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e297c8640 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1b70 0x55ae09baa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e297c8640 1 -- 172.21.15.100:0/3433886819 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae09acb060 con 0x55ae09ba1250 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09bacb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e100a1c20 0x7f7e100a40e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e140058c0 tx=0x7f7e140127c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e297c8640 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 0x55ae09ba1630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e20039040 tx=0x7f7e20005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1bed521083d48a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e0f7fe640 1 -- 172.21.15.100:0/3433886819 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20037d00 con 0x55ae09ba1250 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e0f7fe640 1 -- 172.21.15.100:0/3433886819 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e20037e60 con 0x55ae09ba1250 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.566+0000 7f7e0f7fe640 1 -- 172.21.15.100:0/3433886819 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20005460 con 0x55ae09ba1250 2024-03-12T05:20:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3433886819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 msgr2=0x55ae09ba1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 0x55ae09ba1630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e20039040 tx=0x7f7e20005b20 comp rx=0 tx=0).stop 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3433886819 shutdown_connections 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09bacb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1b70 0x55ae09baa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3433886819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1250 0x55ae09ba1630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3433886819 >> 172.21.15.100:0/3433886819 conn(0x55ae09b9b980 msgr2=0x55ae09b9bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3433886819 shutdown_connections 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3433886819 wait complete. 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 -- start start 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.567+0000 7f7e2e5fd740 1 Processor -- start 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09ba7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09b09020 con 0x55ae09ba1250 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae097dc430 con 0x55ae09ba1b70 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09798e00 con 0x55ae09baa750 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 10 client.0 _ll_register_callbacks cb 0x55ae09ab0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36272/0 (socket says 172.21.15.100:36272) 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 -- 172.21.15.100:0/1895269558 learned_addr learned my addr 172.21.15.100:0/1895269558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09ba7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 -- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 msgr2=0x55ae09ba8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 -- 172.21.15.100:0/1895269558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 msgr2=0x55ae09ba7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09ba7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 -- 172.21.15.100:0/1895269558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2410d7a0 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e297c8640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba7690 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7e20037a30 tx=0x7f7e20037650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf63202c2c2135ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1895269558 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2004f070 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae098087e0 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1895269558 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e20033070 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1895269558 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e200051e0 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 10 client.17277 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.568+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae09b076a0 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1895269558 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2005f020 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1895269558 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e2003f040 con 0x55ae09ba1250 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e2e5fd740 10 client.17277 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e2e5fd740 10 client.17277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e2e5fd740 20 client.17277 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09bba860 0x55ae09bbcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e2e5fd740 4 client.17277 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae09845720 con 0x55ae09bba860 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09bba860 0x55ae09bbcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e0cff9640 10 client.17277 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.569+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09bba860 0x55ae09bbcc80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7dd8f64a66c2a45 server_cookie=1fb433e8e0a79e86 in_seq=0 out_seq=0 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1895269558 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae09845720 con 0x55ae09bba860 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e0cff9640 10 client.17277 handle_command_reply: tid=0 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 client.17277 shutdown 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 msgr2=0x55ae09ba7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba7690 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7e20037a30 tx=0x7f7e20037650 comp rx=0 tx=0).stop 2024-03-12T05:20:33.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 shutdown_connections 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09bba860 0x55ae09bbcc80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09ba7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1895269558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba7690 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e0cff9640 0 client.17277 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 >> 172.21.15.100:0/1895269558 conn(0x55ae09b9b980 msgr2=0x55ae09b9d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 shutdown_connections 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1895269558 wait complete. 2024-03-12T05:20:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.570+0000 7f7e2e5fd740 20 client.17277 trim_cache size 0 max 16384 2024-03-12T05:20:33.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 Processor -- start 2024-03-12T05:20:33.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 -- start start 2024-03-12T05:20:33.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09b9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09a8e130 con 0x55ae09ba1250 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09bc1f50 con 0x55ae09ba1b70 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09ba6db0 con 0x55ae09baa750 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36284/0 (socket says 172.21.15.100:36284) 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e297c8640 1 -- 172.21.15.100:0/3666548456 learned_addr learned my addr 172.21.15.100:0/3666548456 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.582+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09b9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e297c8640 1 -- 172.21.15.100:0/3666548456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 msgr2=0x55ae09ba7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e297c8640 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e297c8640 1 -- 172.21.15.100:0/3666548456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 msgr2=0x55ae09b9aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e297c8640 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09b9aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e297c8640 1 -- 172.21.15.100:0/3666548456 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae09aa3c20 con 0x55ae09ba1250 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e297c8640 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba67f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7e20037e30 tx=0x7f7e20030e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3969aa6faac35291 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e0effd640 1 -- 172.21.15.100:0/3666548456 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2004f020 con 0x55ae09ba1250 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e0effd640 1 -- 172.21.15.100:0/3666548456 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2005f070 con 0x55ae09ba1250 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e0effd640 1 -- 172.21.15.100:0/3666548456 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20005460 con 0x55ae09ba1250 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3666548456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 msgr2=0x55ae09ba67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba67f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7e20037e30 tx=0x7f7e20030e70 comp rx=0 tx=0).stop 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3666548456 shutdown_connections 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09baa750 0x55ae09ba7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09b9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.583+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/3666548456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09ba1250 0x55ae09ba67f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3666548456 >> 172.21.15.100:0/3666548456 conn(0x55ae09b9b980 msgr2=0x55ae09b9a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3666548456 shutdown_connections 2024-03-12T05:20:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/3666548456 wait complete. 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- start start 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 Processor -- start 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09ba1250 0x55ae09baffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09bb0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 0x55ae09bb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09bb1f30 con 0x55ae09baa750 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09c10de0 con 0x55ae09ba1b70 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae09c10fa0 con 0x55ae09ba1250 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e2e5fd740 10 client.0 _ll_register_callbacks cb 0x7f7e14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e29fc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 0x55ae09bb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e28fc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09bb0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09ba1250 0x55ae09baffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e28fc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09bb0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60376/0 (socket says 172.21.15.100:60376) 2024-03-12T05:20:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e297c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09ba1250 0x55ae09baffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35850/0 (socket says 172.21.15.100:35850) 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e29fc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 0x55ae09bb0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36290/0 (socket says 172.21.15.100:36290) 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.584+0000 7f7e28fc7640 1 -- 172.21.15.100:0/1897362776 learned_addr learned my addr 172.21.15.100:0/1897362776 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e29fc9640 1 -- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09ba1250 msgr2=0x55ae09baffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09ba1250 0x55ae09baffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e29fc9640 1 -- 172.21.15.100:0/1897362776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 msgr2=0x55ae09bb0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09bb0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e29fc9640 1 -- 172.21.15.100:0/1897362776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae09b076a0 con 0x55ae09baa750 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e28fc7640 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09bb0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e29fc9640 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 0x55ae09bb0a60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0cad60 tx=0x7f7e1c09a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c747eb336b6129 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1897362776 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c004200 con 0x55ae09baa750 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1897362776 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e1c09a240 con 0x55ae09baa750 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e20037e60 con 0x55ae09baa750 2024-03-12T05:20:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1897362776 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e1c09e390 con 0x55ae09baa750 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 10 client.17283 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae09ac98c0 con 0x55ae09baa750 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1897362776 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e20037e60 con 0x55ae09baa750 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1897362776 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e1c095f90 con 0x55ae09baa750 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 10 client.17283 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 10 client.17283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 20 client.17283 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09c12820 0x55ae09c14c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 4 client.17283 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.585+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae09894890 con 0x55ae09c12820 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e297c8640 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09c12820 0x55ae09c14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e0cff9640 10 client.17283 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e297c8640 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09c12820 0x55ae09c14c40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fce9929ea18f9486 server_cookie=f68008474542ca27 in_seq=0 out_seq=0 2024-03-12T05:20:33.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e0cff9640 1 -- 172.21.15.100:0/1897362776 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae09894890 con 0x55ae09c12820 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e0cff9640 10 client.17283 handle_command_reply: tid=0 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 client.17283 shutdown 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 msgr2=0x55ae09bb0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 0x55ae09bb0a60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0cad60 tx=0x7f7e1c09a0d0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 shutdown_connections 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae09baa750 0x55ae09bb0a60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ae09c12820 0x55ae09c14c40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae09ba1b70 0x55ae09bb0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e0cff9640 0 client.17283 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 --2- 172.21.15.100:0/1897362776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae09ba1250 0x55ae09baffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 >> 172.21.15.100:0/1897362776 conn(0x55ae09b9b980 msgr2=0x55ae09b9ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 shutdown_connections 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.586+0000 7f7e2e5fd740 1 -- 172.21.15.100:0/1897362776 wait complete. 2024-03-12T05:20:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2e5fd740 20 client.17283 trim_cache size 0 max 16384 2024-03-12T05:20:33.590 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e100a1c20 msgr2=0x7f7e100a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e100a1c20 0x7f7e100a40e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e140058c0 tx=0x7f7e140127c0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24106240 msgr2=0x7f7e24199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24106240 0x7f7e24199830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00ef00 tx=0x7f7e1c00c740 comp rx=0 tx=0).stop 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 shutdown_connections 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e100a1c20 0x7f7e100a40e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e24106240 0x7f7e24199830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e24100d20 0x7f7e2419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 --2- 172.21.15.100:0/2558535187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100360 0x7f7e2419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 >> 172.21.15.100:0/2558535187 conn(0x7f7e24077060 msgr2=0x7f7e240feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 shutdown_connections 2024-03-12T05:20:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.587+0000 7f7e2ba2c640 1 -- 172.21.15.100:0/2558535187 wait complete. 2024-03-12T05:20:33.593 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:20:33.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 -- 172.21.15.100:0/4103548335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718100c70 msgr2=0x7ff718104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 --2- 172.21.15.100:0/4103548335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718100c70 0x7ff718104cb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009930 tx=0x7ff70c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 -- 172.21.15.100:0/4103548335 shutdown_connections 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 --2- 172.21.15.100:0/4103548335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718105390 0x7ff71810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 --2- 172.21.15.100:0/4103548335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718100c70 0x7ff718104cb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 --2- 172.21.15.100:0/4103548335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7181002c0 0x7ff7181006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 -- 172.21.15.100:0/4103548335 >> 172.21.15.100:0/4103548335 conn(0x7ff718076400 msgr2=0x7ff718076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 -- 172.21.15.100:0/4103548335 shutdown_connections 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.703+0000 7ff71cd48640 1 -- 172.21.15.100:0/4103548335 wait complete. 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 Processor -- start 2024-03-12T05:20:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 -- start start 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7181002c0 0x7ff718198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718105390 0x7ff718198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 0x7ff7181a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7181a3d70 con 0x7ff718199120 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7181a3ee0 con 0x7ff718105390 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71cd48640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7181a40a0 con 0x7ff7181002c0 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff716d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 0x7ff7181a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff716d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 0x7ff7181a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36304/0 (socket says 172.21.15.100:36304) 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff71659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7181002c0 0x7ff718198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff716d9d640 1 -- 172.21.15.100:0/1119547417 learned_addr learned my addr 172.21.15.100:0/1119547417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.704+0000 7ff715d9b640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718105390 0x7ff718198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff716d9d640 1 -- 172.21.15.100:0/1119547417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7181002c0 msgr2=0x7ff718198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff716d9d640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7181002c0 0x7ff718198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff716d9d640 1 -- 172.21.15.100:0/1119547417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718105390 msgr2=0x7ff718198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff716d9d640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718105390 0x7ff718198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff716d9d640 1 -- 172.21.15.100:0/1119547417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c009560 con 0x7ff718199120 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff716d9d640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 0x7ff7181a36d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff71000eb10 tx=0x7ff71000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff7077fe640 1 -- 172.21.15.100:0/1119547417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710004b20 con 0x7ff718199120 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7181a4380 con 0x7ff718199120 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff7077fe640 1 -- 172.21.15.100:0/1119547417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff710004c80 con 0x7ff718199120 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff7077fe640 1 -- 172.21.15.100:0/1119547417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710010650 con 0x7ff718199120 2024-03-12T05:20:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.705+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7181a4810 con 0x7ff718199120 2024-03-12T05:20:33.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.707+0000 7ff7077fe640 1 -- 172.21.15.100:0/1119547417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff710005020 con 0x7ff718199120 2024-03-12T05:20:33.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 Processor -- start 2024-03-12T05:20:33.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 -- start start 2024-03-12T05:20:33.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff7077fe640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff7000a1ba0 0x7ff7000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd20c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd215210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd217b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff715d9b640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff7000a1ba0 0x7ff7000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd1c7960 con 0x55ffcd20cb70 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd17b310 con 0x55ffcd20c250 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd178840 con 0x55ffcd215750 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff7077fe640 1 -- 172.21.15.100:0/1119547417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff710014070 con 0x7ff718199120 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd20c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd215210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd20c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60394/0 (socket says 172.21.15.100:60394) 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 -- 172.21.15.100:0/2432289699 learned_addr learned my addr 172.21.15.100:0/2432289699 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff716d9d640 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd217b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 -- 172.21.15.100:0/2432289699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 msgr2=0x55ffcd217b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd217b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 -- 172.21.15.100:0/2432289699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 msgr2=0x55ffcd215210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd215210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff71659c640 1 -- 172.21.15.100:0/2432289699 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffcd136060 con 0x55ffcd20c250 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff715d9b640 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd215210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.708+0000 7ff716d9d640 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd217b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.709+0000 7ff715d9b640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff7000a1ba0 0x7ff7000a4060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff70c000c00 tx=0x7ff70c0047f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.709+0000 7ff71659c640 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd20c630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff70800ac10 tx=0x7ff70800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520dd4374b99194f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.709+0000 7ff704ff9640 1 -- 172.21.15.100:0/2432289699 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800a6e0 con 0x55ffcd20c250 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.709+0000 7ff704ff9640 1 -- 172.21.15.100:0/2432289699 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff708018b20 con 0x55ffcd20c250 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.709+0000 7ff704ff9640 1 -- 172.21.15.100:0/2432289699 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708010be0 con 0x55ffcd20c250 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2432289699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 msgr2=0x55ffcd20c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd20c630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff70800ac10 tx=0x7ff70800f790 comp rx=0 tx=0).stop 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2432289699 shutdown_connections 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd217b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd215210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2432289699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd20c630 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2432289699 >> 172.21.15.100:0/2432289699 conn(0x55ffcd206980 msgr2=0x55ffcd206d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2432289699 shutdown_connections 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2432289699 wait complete. 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- start start 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 Processor -- start 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd2127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd212d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd174020 con 0x55ffcd20cb70 2024-03-12T05:20:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcce47430 con 0x55ffcd215750 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcce03e00 con 0x55ffcd20c250 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.710+0000 7ff71f9fc740 10 client.0 _ll_register_callbacks cb 0x55ffcd11bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd212d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd212d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36328/0 (socket says 172.21.15.100:36328) 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 -- 172.21.15.100:0/1702816989 learned_addr learned my addr 172.21.15.100:0/1702816989 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff716d9d640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff71659c640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd2127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 -- 172.21.15.100:0/1702816989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 msgr2=0x55ffcd2127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd2127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 -- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 msgr2=0x55ffcd213240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 -- 172.21.15.100:0/1702816989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70800a840 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff715d9b640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd212d00 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff70c02c780 tx=0x7ff70c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40097199581e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff6fdffb640 1 -- 172.21.15.100:0/1702816989 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c048070 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff6fdffb640 1 -- 172.21.15.100:0/1702816989 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff70c005000 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffcce737e0 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff6fdffb640 1 -- 172.21.15.100:0/1702816989 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70c0051c0 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff71f9fc740 10 client.17295 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ffcd1726a0 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.711+0000 7ff6fdffb640 1 -- 172.21.15.100:0/1702816989 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ffcce737e0 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff6fdffb640 1 -- 172.21.15.100:0/1702816989 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff70c034440 con 0x55ffcd20cb70 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71f9fc740 10 client.17295 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71f9fc740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71f9fc740 20 client.17295 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd225af0 0x55ffcd227f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71f9fc740 4 client.17295 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffcceb0720 con 0x55ffcd225af0 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71659c640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd225af0 0x55ffcd227f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff6fdffb640 10 client.17295 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.712+0000 7ff71659c640 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd225af0 0x55ffcd227f10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6a24753d5a1c05 server_cookie=f9fd0137bac5d4c0 in_seq=0 out_seq=0 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff6fdffb640 1 -- 172.21.15.100:0/1702816989 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffcceb0720 con 0x55ffcd225af0 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff6fdffb640 10 client.17295 handle_command_reply: tid=0 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 client.17295 shutdown 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 msgr2=0x55ffcd212d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd212d00 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff70c02c780 tx=0x7ff70c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 shutdown_connections 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd225af0 0x55ffcd227f10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff6fdffb640 0 client.17295 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd212d00 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/1702816989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd2127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 >> 172.21.15.100:0/1702816989 conn(0x55ffcd206980 msgr2=0x55ffcd2086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 shutdown_connections 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 1 -- 172.21.15.100:0/1702816989 wait complete. 2024-03-12T05:20:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.713+0000 7ff71f9fc740 20 client.17295 trim_cache size 0 max 16384 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 Processor -- start 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 -- start start 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd214ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd21c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd0f9130 con 0x55ffcd20cb70 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd207980 con 0x55ffcd215750 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd22d160 con 0x55ffcd20c250 2024-03-12T05:20:33.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd21c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.725+0000 7ff71659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd214ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd21c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36332/0 (socket says 172.21.15.100:36332) 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 -- 172.21.15.100:0/2365046742 learned_addr learned my addr 172.21.15.100:0/2365046742 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff716d9d640 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 -- 172.21.15.100:0/2365046742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 msgr2=0x55ffcd214ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd214ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 -- 172.21.15.100:0/2365046742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 msgr2=0x55ffcd213060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 -- 172.21.15.100:0/2365046742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffcd10ec20 con 0x55ffcd20cb70 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff715d9b640 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd21c9e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff70c004730 tx=0x7ff70c005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a52cc247a8dcc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff6fffff640 1 -- 172.21.15.100:0/2365046742 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c048020 con 0x55ffcd20cb70 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff6fffff640 1 -- 172.21.15.100:0/2365046742 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff70c004410 con 0x55ffcd20cb70 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.726+0000 7ff6fffff640 1 -- 172.21.15.100:0/2365046742 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0054e0 con 0x55ffcd20cb70 2024-03-12T05:20:33.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2365046742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 msgr2=0x55ffcd21c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd21c9e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff70c004730 tx=0x7ff70c005d70 comp rx=0 tx=0).stop 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2365046742 shutdown_connections 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd215750 0x55ffcd213060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd21c9e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/2365046742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd20c250 0x55ffcd214ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2365046742 >> 172.21.15.100:0/2365046742 conn(0x55ffcd206980 msgr2=0x55ffcd20daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2365046742 shutdown_connections 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- 172.21.15.100:0/2365046742 wait complete. 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- start start 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 Processor -- start 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd27a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd27a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd274560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd274b20 con 0x55ffcd20cb70 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd274c90 con 0x55ffcd20c250 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcd274e50 con 0x55ffcd215750 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71f9fc740 10 client.0 _ll_register_callbacks cb 0x7ff70800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd27a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd27a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36336/0 (socket says 172.21.15.100:36336) 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff716d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd274560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff71659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd27a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.727+0000 7ff715d9b640 1 -- 172.21.15.100:0/4164336735 learned_addr learned my addr 172.21.15.100:0/4164336735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff715d9b640 1 -- 172.21.15.100:0/4164336735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 msgr2=0x55ffcd274560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff715d9b640 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd274560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff715d9b640 1 -- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 msgr2=0x55ffcd27a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff715d9b640 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd27a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff715d9b640 1 -- 172.21.15.100:0/4164336735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffcd1726a0 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff715d9b640 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd27a690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0347d0 tx=0x7ff70c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2693bbffbfe8c139 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff6fdffb640 1 -- 172.21.15.100:0/4164336735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0345f0 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff6fdffb640 1 -- 172.21.15.100:0/4164336735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff70c0349c0 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffcceff890 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff6fdffb640 1 -- 172.21.15.100:0/4164336735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70c034ba0 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 10 client.17301 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ffcd1348c0 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff6fdffb640 1 -- 172.21.15.100:0/4164336735 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ffcceff890 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff6fdffb640 1 -- 172.21.15.100:0/4164336735 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff70c069070 con 0x55ffcd20cb70 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 10 client.17301 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:33.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 20 client.17301 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd281b50 0x55ffcd283f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 4 client.17301 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.728+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffcce1f700 con 0x55ffcd281b50 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71659c640 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd281b50 0x55ffcd283f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff6fdffb640 10 client.17301 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71659c640 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd281b50 0x55ffcd283f40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cdb69771e695e6 server_cookie=d4265955f2d03ef6 in_seq=0 out_seq=0 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff6fdffb640 1 -- 172.21.15.100:0/4164336735 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffcce1f700 con 0x55ffcd281b50 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff6fdffb640 10 client.17301 handle_command_reply: tid=0 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 client.17301 shutdown 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 msgr2=0x55ffcd27a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd27a690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0347d0 tx=0x7ff70c005b50 comp rx=0 tx=0).stop 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 shutdown_connections 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ffcd281b50 0x55ffcd283f40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffcd215750 0x55ffcd274560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff6fdffb640 0 client.17301 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffcd20cb70 0x55ffcd27a690 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 --2- 172.21.15.100:0/4164336735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffcd20c250 0x55ffcd27a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 >> 172.21.15.100:0/4164336735 conn(0x55ffcd206980 msgr2=0x55ffcd207db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 shutdown_connections 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.729+0000 7ff71f9fc740 1 -- 172.21.15.100:0/4164336735 wait complete. 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71f9fc740 20 client.17301 trim_cache size 0 max 16384 2024-03-12T05:20:33.732 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:20:33.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff7000a1ba0 msgr2=0x7ff7000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff7000a1ba0 0x7ff7000a4060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff70c000c00 tx=0x7ff70c0047f0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 msgr2=0x7ff7181a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:33.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 0x7ff7181a36d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff71000eb10 tx=0x7ff71000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 shutdown_connections 2024-03-12T05:20:33.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff7000a1ba0 0x7ff7000a4060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff718199120 0x7ff7181a36d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff718105390 0x7ff718198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 --2- 172.21.15.100:0/1119547417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7181002c0 0x7ff718198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:33.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 >> 172.21.15.100:0/1119547417 conn(0x7ff718076400 msgr2=0x7ff7180fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:33.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 shutdown_connections 2024-03-12T05:20:33.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:33.730+0000 7ff71cd48640 1 -- 172.21.15.100:0/1119547417 wait complete. 2024-03-12T05:20:33.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1012890 12.10 MB/sec execute 317 sec latency 27.270 ms 2024-03-12T05:20:34.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:33 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1235640804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:20:34.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:33 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1235640804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:20:34.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:33 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1235640804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:20:34.737 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:20:34.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:20:34.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:20:34.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1014906 12.08 MB/sec execute 318 sec latency 17.369 ms 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 -- 172.21.15.100:0/2790761730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494100260 msgr2=0x7f6494100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 --2- 172.21.15.100:0/2790761730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494100260 0x7f6494100640 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6484009930 tx=0x7f648402bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 -- 172.21.15.100:0/2790761730 shutdown_connections 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 --2- 172.21.15.100:0/2790761730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6494105330 0x7f649410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 --2- 172.21.15.100:0/2790761730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6494100c10 0x7f6494104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 --2- 172.21.15.100:0/2790761730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494100260 0x7f6494100640 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 -- 172.21.15.100:0/2790761730 >> 172.21.15.100:0/2790761730 conn(0x7f6494077180 msgr2=0x7f6494077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 -- 172.21.15.100:0/2790761730 shutdown_connections 2024-03-12T05:20:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.853+0000 7f6498e76640 1 -- 172.21.15.100:0/2790761730 wait complete. 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 Processor -- start 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 -- start start 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6494100260 0x7f6494198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 0x7f6494198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64941990e0 0x7f64941a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a3d30 con 0x7f6494105330 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a3ea0 con 0x7f64941990e0 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6498e76640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a4060 con 0x7f6494100260 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 0x7f6494198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 0x7f6494198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36350/0 (socket says 172.21.15.100:36350) 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f648bfff640 1 -- 172.21.15.100:0/1803958697 learned_addr learned my addr 172.21.15.100:0/1803958697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f6492d9d640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64941990e0 0x7f64941a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.854+0000 7f649259c640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6494100260 0x7f6494198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6492d9d640 1 -- 172.21.15.100:0/1803958697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6494100260 msgr2=0x7f6494198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6492d9d640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6494100260 0x7f6494198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6492d9d640 1 -- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 msgr2=0x7f6494198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6492d9d640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 0x7f6494198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6492d9d640 1 -- 172.21.15.100:0/1803958697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484009560 con 0x7f64941990e0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f648bfff640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 0x7f6494198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6492d9d640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64941990e0 0x7f64941a3690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f648c009fd0 tx=0x7f648c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f648b7fe640 1 -- 172.21.15.100:0/1803958697 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00ce20 con 0x7f64941990e0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64941a4340 con 0x7f64941990e0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64941a4850 con 0x7f64941990e0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f648b7fe640 1 -- 172.21.15.100:0/1803958697 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f648c010430 con 0x7f64941990e0 2024-03-12T05:20:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.855+0000 7f648b7fe640 1 -- 172.21.15.100:0/1803958697 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00f620 con 0x7f64941990e0 2024-03-12T05:20:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f648b7fe640 1 -- 172.21.15.100:0/1803958697 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f648c0105a0 con 0x7f64941990e0 2024-03-12T05:20:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 Processor -- start 2024-03-12T05:20:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 -- start start 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11bdcc0 0x5601a11c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a111ef90 con 0x5601a11b0320 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a111c4c0 con 0x5601a11bdcc0 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.858+0000 7f649ba46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a1117ca0 con 0x5601a11b0cd0 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f649259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36352/0 (socket says 172.21.15.100:36352) 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f649259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35950/0 (socket says 172.21.15.100:35950) 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 -- 172.21.15.100:0/382333056 learned_addr learned my addr 172.21.15.100:0/382333056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11bdcc0 0x5601a11c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648b7fe640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f64740a1ba0 0x7f64740a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 -- 172.21.15.100:0/382333056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 msgr2=0x5601a11bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f6492d9d640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f64740a1ba0 0x7f64740a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 -- 172.21.15.100:0/382333056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11bdcc0 msgr2=0x5601a11c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11bdcc0 0x5601a11c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 -- 172.21.15.100:0/382333056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601a10da0b0 con 0x5601a11b0320 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648b7fe640 1 -- 172.21.15.100:0/1803958697 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f648c014070 con 0x7f64941990e0 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f649259c640 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f648bfff640 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11b0700 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f647c0058c0 tx=0x7f647c00abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3662900ba834828e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f6488ff9640 1 -- 172.21.15.100:0/382333056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c00fc50 con 0x5601a11b0320 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f6492d9d640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f64740a1ba0 0x7f64740a4060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f648c0cb2a0 tx=0x7f648c07b040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:34.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f6488ff9640 1 -- 172.21.15.100:0/382333056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f647c00fdb0 con 0x5601a11b0320 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.859+0000 7f6488ff9640 1 -- 172.21.15.100:0/382333056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c0047a0 con 0x5601a11b0320 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 -- 172.21.15.100:0/382333056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 msgr2=0x5601a11b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11b0700 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f647c0058c0 tx=0x7f647c00abd0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 -- 172.21.15.100:0/382333056 shutdown_connections 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11bdcc0 0x5601a11c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 --2- 172.21.15.100:0/382333056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11b0700 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 -- 172.21.15.100:0/382333056 >> 172.21.15.100:0/382333056 conn(0x5601a11aa840 msgr2=0x5601a11aac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 -- 172.21.15.100:0/382333056 shutdown_connections 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.860+0000 7f649ba46740 1 -- 172.21.15.100:0/382333056 wait complete. 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 -- start start 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 Processor -- start 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0320 0x5601a11babb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a10bcf40 con 0x5601a11bdcc0 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0deb4b0 con 0x5601a11b0cd0 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a109d670 con 0x5601a11b0320 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649ba46740 10 client.0 _ll_register_callbacks cb 0x5601a110af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11bb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36358/0 (socket says 172.21.15.100:36358) 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f648bfff640 1 -- 172.21.15.100:0/449009545 learned_addr learned my addr 172.21.15.100:0/449009545 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649259c640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f648bfff640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0320 0x5601a11babb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649259c640 1 -- 172.21.15.100:0/449009545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0320 msgr2=0x5601a11babb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649259c640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0320 0x5601a11babb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649259c640 1 -- 172.21.15.100:0/449009545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 msgr2=0x5601a11bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649259c640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f649259c640 1 -- 172.21.15.100:0/449009545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c019040 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.861+0000 7f648bfff640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649259c640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11bb0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6484009230 tx=0x7f648402ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de29ae770c57096b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f6481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f6481ffb640 1 -- 172.21.15.100:0/449009545 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648402cbd0 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f6481ffb640 1 -- 172.21.15.100:0/449009545 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6484004250 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601a10f1f40 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f6481ffb640 1 -- 172.21.15.100:0/449009545 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6484004450 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 10 client.26693 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5601a1116320 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f6481ffb640 1 -- 172.21.15.100:0/449009545 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6484056080 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f6481ffb640 1 -- 172.21.15.100:0/449009545 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f648403c040 con 0x5601a11b0cd0 2024-03-12T05:20:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 10 client.26693 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 20 client.26693 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a11ce1b0 0x5601a11d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 4 client.26693 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.862+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601a110e310 con 0x5601a11ce1b0 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.863+0000 7f648bfff640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a11ce1b0 0x5601a11d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.863+0000 7f6481ffb640 10 client.26693 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.863+0000 7f648bfff640 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a11ce1b0 0x5601a11d05d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c306949d3df6a4eb server_cookie=b8e51271aaaa75cd in_seq=0 out_seq=0 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f6481ffb640 1 -- 172.21.15.100:0/449009545 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601a110e310 con 0x5601a11ce1b0 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f6481ffb640 10 client.26693 handle_command_reply: tid=0 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 client.26693 shutdown 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 msgr2=0x5601a11bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11bb0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6484009230 tx=0x7f648402ca60 comp rx=0 tx=0).stop 2024-03-12T05:20:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 shutdown_connections 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a11ce1b0 0x5601a11d05d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f6481ffb640 0 client.26693 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11bb0f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 --2- 172.21.15.100:0/449009545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0320 0x5601a11babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 >> 172.21.15.100:0/449009545 conn(0x5601a11aa840 msgr2=0x5601a11ac7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 shutdown_connections 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 1 -- 172.21.15.100:0/449009545 wait complete. 2024-03-12T05:20:34.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.864+0000 7f649ba46740 20 client.26693 trim_cache size 0 max 16384 2024-03-12T05:20:34.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 Processor -- start 2024-03-12T05:20:34.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 -- start start 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0320 0x5601a11bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11b9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a11b1130 con 0x5601a11bdcc0 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a11d57f0 con 0x5601a11b0320 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649ba46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a11b0150 con 0x5601a11b0cd0 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11b9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36362/0 (socket says 172.21.15.100:36362) 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 -- 172.21.15.100:0/3498536515 learned_addr learned my addr 172.21.15.100:0/3498536515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0320 0x5601a11bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649259c640 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 -- 172.21.15.100:0/3498536515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 msgr2=0x5601a11bb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 -- 172.21.15.100:0/3498536515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0320 msgr2=0x5601a11bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0320 0x5601a11bd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f648bfff640 1 -- 172.21.15.100:0/3498536515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601a10b2c00 con 0x5601a11bdcc0 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.879+0000 7f649259c640 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.880+0000 7f648bfff640 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11b9480 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f647c0058c0 tx=0x7f647c005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864396c065b028c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.880+0000 7f6483fff640 1 -- 172.21.15.100:0/3498536515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c024040 con 0x5601a11bdcc0 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.880+0000 7f6483fff640 1 -- 172.21.15.100:0/3498536515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f647c015070 con 0x5601a11bdcc0 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.880+0000 7f6483fff640 1 -- 172.21.15.100:0/3498536515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c02e3c0 con 0x5601a11bdcc0 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.880+0000 7f649ba46740 1 -- 172.21.15.100:0/3498536515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 msgr2=0x5601a11b9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.880+0000 7f649ba46740 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11b9480 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f647c0058c0 tx=0x7f647c005390 comp rx=0 tx=0).stop 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- 172.21.15.100:0/3498536515 shutdown_connections 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11bdcc0 0x5601a11b9480 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11b0cd0 0x5601a11bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 --2- 172.21.15.100:0/3498536515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0320 0x5601a11bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- 172.21.15.100:0/3498536515 >> 172.21.15.100:0/3498536515 conn(0x5601a11aa840 msgr2=0x5601a11aac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- 172.21.15.100:0/3498536515 shutdown_connections 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- 172.21.15.100:0/3498536515 wait complete. 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- start start 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 Processor -- start 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11bdcc0 0x5601a11c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a11c3f80 con 0x5601a11b0320 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a11c40f0 con 0x5601a11b0cd0 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a12246c0 con 0x5601a11bdcc0 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649ba46740 10 client.0 _ll_register_callbacks cb 0x7f647c00af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36368/0 (socket says 172.21.15.100:36368) 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 -- 172.21.15.100:0/2349415329 learned_addr learned my addr 172.21.15.100:0/2349415329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f649259c640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11bdcc0 0x5601a11c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 -- 172.21.15.100:0/2349415329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11bdcc0 msgr2=0x5601a11c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11bdcc0 0x5601a11c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 -- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 msgr2=0x5601a11c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:34.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.881+0000 7f648bfff640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f648bfff640 1 -- 172.21.15.100:0/2349415329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c027040 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f648bfff640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11c4fd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f647c0054e0 tx=0x7f647c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2b3d5b3ae75589 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f6481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f6481ffb640 1 -- 172.21.15.100:0/2349415329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c024070 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f6481ffb640 1 -- 172.21.15.100:0/2349415329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f647c000e40 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601a113d900 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f6481ffb640 1 -- 172.21.15.100:0/2349415329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f647c02e480 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 10 client.17319 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5601a10d8910 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f6481ffb640 1 -- 172.21.15.100:0/2349415329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f647c040080 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f6481ffb640 1 -- 172.21.15.100:0/2349415329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f647c02fd20 con 0x5601a11b0320 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 10 client.17319 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:34.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 20 client.17319 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a1225ae0 0x5601a1227ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 4 client.17319 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.882+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5601a0e17860 con 0x5601a1225ae0 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.883+0000 7f649259c640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a1225ae0 0x5601a1227ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.883+0000 7f649259c640 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a1225ae0 0x5601a1227ed0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b65a1c8cd35b1b server_cookie=a0a30d085ecf8e46 in_seq=0 out_seq=0 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.883+0000 7f6481ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f6481ffb640 1 -- 172.21.15.100:0/2349415329 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5601a0e17860 con 0x5601a1225ae0 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f6481ffb640 10 client.17319 handle_command_reply: tid=0 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 client.17319 shutdown 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 msgr2=0x5601a11c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11c4fd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f647c0054e0 tx=0x7f647c021040 comp rx=0 tx=0).stop 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 shutdown_connections 2024-03-12T05:20:34.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5601a1225ae0 0x5601a1227ed0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601a11bdcc0 0x5601a11c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f6481ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601a11b0cd0 0x5601a11c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 --2- 172.21.15.100:0/2349415329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601a11b0320 0x5601a11c4fd0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 >> 172.21.15.100:0/2349415329 conn(0x5601a11aa840 msgr2=0x5601a11b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.884+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 shutdown_connections 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f649ba46740 1 -- 172.21.15.100:0/2349415329 wait complete. 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f649ba46740 20 client.17319 trim_cache size 0 max 16384 2024-03-12T05:20:34.887 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:20:34.888 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:20:34.888 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "365c6efb-2b38-4e05-9875-2d40d98434eb", 2024-03-12T05:20:34.888 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:20:34.888 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f64740a1ba0 msgr2=0x7f64740a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f64740a1ba0 0x7f64740a4060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f648c0cb2a0 tx=0x7f648c07b040 comp rx=0 tx=0).stop 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64941990e0 msgr2=0x7f64941a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64941990e0 0x7f64941a3690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f648c009fd0 tx=0x7f648c00ef10 comp rx=0 tx=0).stop 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 shutdown_connections 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f64740a1ba0 0x7f64740a4060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64941990e0 0x7f64941a3690 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6494105330 0x7f6494198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 --2- 172.21.15.100:0/1803958697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6494100260 0x7f6494198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 >> 172.21.15.100:0/1803958697 conn(0x7f6494077180 msgr2=0x7f64940fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 shutdown_connections 2024-03-12T05:20:34.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:34.885+0000 7f6498e76640 1 -- 172.21.15.100:0/1803958697 wait complete. 2024-03-12T05:20:34.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:20:34.891 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 -- 172.21.15.100:0/115672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef41023a0 msgr2=0x7f5ef4102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 --2- 172.21.15.100:0/115672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef41023a0 0x7f5ef4102780 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009900 tx=0x7f5ee402bc30 comp rx=0 tx=0).stop 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 -- 172.21.15.100:0/115672832 shutdown_connections 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 --2- 172.21.15.100:0/115672832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef410b770 0x7f5ef410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 --2- 172.21.15.100:0/115672832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef4102cc0 0x7f5ef410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 --2- 172.21.15.100:0/115672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef41023a0 0x7f5ef4102780 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 -- 172.21.15.100:0/115672832 >> 172.21.15.100:0/115672832 conn(0x7f5ef4077210 msgr2=0x7f5ef4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 -- 172.21.15.100:0/115672832 shutdown_connections 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 -- 172.21.15.100:0/115672832 wait complete. 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.007+0000 7f5ef995e640 1 Processor -- start 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 -- start start 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef41023a0 0x7f5ef40ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 0x7f5ef4100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef410b770 0x7f5ef4100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41a5b20 con 0x7f5ef4102cc0 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41a5c90 con 0x7f5ef410b770 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41a5e50 con 0x7f5ef41023a0 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 0x7f5ef4100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 0x7f5ef4100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36396/0 (socket says 172.21.15.100:36396) 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 -- 172.21.15.100:0/2011158663 learned_addr learned my addr 172.21.15.100:0/2011158663 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef410b770 0x7f5ef4100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef41023a0 0x7f5ef40ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 -- 172.21.15.100:0/2011158663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef41023a0 msgr2=0x7f5ef40ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef41023a0 0x7f5ef40ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 -- 172.21.15.100:0/2011158663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef410b770 msgr2=0x7f5ef4100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef410b770 0x7f5ef4100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 -- 172.21.15.100:0/2011158663 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009560 con 0x7f5ef4102cc0 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5eebfff640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 0x7f5ef4100530 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00e9f0 tx=0x7f5edc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef895c640 1 -- 172.21.15.100:0/2011158663 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc009700 con 0x7f5ef4102cc0 2024-03-12T05:20:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef895c640 1 -- 172.21.15.100:0/2011158663 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5edc00cd80 con 0x7f5ef4102cc0 2024-03-12T05:20:35.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef41a6130 con 0x7f5ef4102cc0 2024-03-12T05:20:35.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.008+0000 7f5ef895c640 1 -- 172.21.15.100:0/2011158663 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc018920 con 0x7f5ef4102cc0 2024-03-12T05:20:35.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.009+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef41a6640 con 0x7f5ef4102cc0 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef895c640 1 -- 172.21.15.100:0/2011158663 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5edc018a80 con 0x7f5ef4102cc0 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 Processor -- start 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef895c640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5ee00a1bd0 0x7f5ee00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 -- start start 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 0x55d7f9b91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5ee00a1bd0 0x7f5ee00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef895c640 1 -- 172.21.15.100:0/2011158663 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5edc014070 con 0x7f5ef4102cc0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9b00280 con 0x55d7f9b912b0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9afd7b0 con 0x55d7f9b9a7b0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5efc60d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9af8f90 con 0x55d7f9b91bd0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 0x55d7f9b91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 0x55d7f9b91690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36398/0 (socket says 172.21.15.100:36398) 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.011+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/1771273143 learned_addr learned my addr 172.21.15.100:0/1771273143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/1771273143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b91bd0 msgr2=0x55d7f9b9a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/1771273143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 msgr2=0x55d7f9b9cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/1771273143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f9ab7570 con 0x55d7f9b912b0 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5ee00a1bd0 0x7f5ee00a4090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5eec005fd0 tx=0x7f5eec013ae0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 0x55d7f9b91690 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009900 tx=0x7f5ee402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e69a64a3acd132 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ee9ffb640 1 -- 172.21.15.100:0/1771273143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4004a00 con 0x55d7f9b912b0 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ee9ffb640 1 -- 172.21.15.100:0/1771273143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ee4004030 con 0x55d7f9b912b0 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.012+0000 7f5ee9ffb640 1 -- 172.21.15.100:0/1771273143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4005130 con 0x55d7f9b912b0 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 -- 172.21.15.100:0/1771273143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 msgr2=0x55d7f9b91690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 0x55d7f9b91690 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009900 tx=0x7f5ee402c5d0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 -- 172.21.15.100:0/1771273143 shutdown_connections 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1771273143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b912b0 0x55d7f9b91690 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 -- 172.21.15.100:0/1771273143 >> 172.21.15.100:0/1771273143 conn(0x55d7f9b8b990 msgr2=0x55d7f9b8bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 -- 172.21.15.100:0/1771273143 shutdown_connections 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 -- 172.21.15.100:0/1771273143 wait complete. 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 -- start start 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.013+0000 7f5efc60d740 1 Processor -- start 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b912b0 0x55d7f9b9a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f97cc430 con 0x55d7f9b91bd0 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9788e00 con 0x55d7f9b912b0 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9a7dfd0 con 0x55d7f9b9a7b0 2024-03-12T05:20:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 10 client.0 _ll_register_callbacks cb 0x55d7f9ad3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36400/0 (socket says 172.21.15.100:36400) 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 -- 172.21.15.100:0/1385359317 learned_addr learned my addr 172.21.15.100:0/1385359317 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b912b0 0x55d7f9b9a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 -- 172.21.15.100:0/1385359317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b9a7b0 msgr2=0x55d7f9b97a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 -- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b912b0 msgr2=0x55d7f9b9a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b912b0 0x55d7f9b9a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 -- 172.21.15.100:0/1385359317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef410e890 con 0x55d7f9b91bd0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b974c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5edc09a7d0 tx=0x7f5edc09b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897f6b6bf2cfa9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5ed2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/1385359317 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0a2fe0 con 0x55d7f9b91bd0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/1385359317 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5edc09bb70 con 0x55d7f9b91bd0 2024-03-12T05:20:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.014+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7f97f87e0 con 0x55d7f9b91bd0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/1385359317 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc0b32f0 con 0x55d7f9b91bd0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 10 client.17331 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d7f9af7610 con 0x55d7f9b91bd0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/1385359317 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5edc0b7070 con 0x55d7f9b91bd0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/1385359317 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5edc0cbab0 con 0x55d7f9b91bd0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 10 client.17331 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 20 client.17331 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9baa8e0 0x55d7f9baccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 4 client.17331 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7f9835720 con 0x55d7f9baa8e0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.015+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9baa8e0 0x55d7f9baccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9baa8e0 0x55d7f9baccf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339124632d93e5cf server_cookie=c3eeeaf7e54feb82 in_seq=0 out_seq=0 2024-03-12T05:20:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5ed2ffd640 10 client.17331 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/1385359317 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7f9835720 con 0x55d7f9baa8e0 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5ed2ffd640 10 client.17331 handle_command_reply: tid=0 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 client.17331 shutdown 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 msgr2=0x55d7f9b974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b974c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5edc09a7d0 tx=0x7f5edc09b000 comp rx=0 tx=0).stop 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 shutdown_connections 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9baa8e0 0x55d7f9baccf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b97a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b974c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1385359317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b912b0 0x55d7f9b9a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 >> 172.21.15.100:0/1385359317 conn(0x55d7f9b8b990 msgr2=0x55d7f9b8d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5ed2ffd640 0 client.17331 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 shutdown_connections 2024-03-12T05:20:35.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.016+0000 7f5efc60d740 1 -- 172.21.15.100:0/1385359317 wait complete. 2024-03-12T05:20:35.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.017+0000 7f5efc60d740 20 client.17331 trim_cache size 0 max 16384 2024-03-12T05:20:35.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.032+0000 7f5efc60d740 1 Processor -- start 2024-03-12T05:20:35.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.032+0000 7f5efc60d740 1 -- start start 2024-03-12T05:20:35.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9b99bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5efc60d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9b90aa0 con 0x55d7f9b91bd0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5efc60d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9bb1f20 con 0x55d7f9b9a7b0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5efc60d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9b95a10 con 0x55d7f9b912b0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36416/0 (socket says 172.21.15.100:36416) 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ef2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9b99bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 -- 172.21.15.100:0/1072027412 learned_addr learned my addr 172.21.15.100:0/1072027412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 -- 172.21.15.100:0/1072027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 msgr2=0x55d7f9b99bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9b99bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 -- 172.21.15.100:0/1072027412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 msgr2=0x55d7f9b927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 -- 172.21.15.100:0/1072027412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f9a93ac0 con 0x55d7f9b91bd0 2024-03-12T05:20:35.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9b99bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5eebfff640 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a130 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f5edc09a540 tx=0x7f5edc09eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c11e95547b66b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ee97fa640 1 -- 172.21.15.100:0/1072027412 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc09a610 con 0x55d7f9b91bd0 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ee97fa640 1 -- 172.21.15.100:0/1072027412 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5edc0b7070 con 0x55d7f9b91bd0 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.033+0000 7f5ee97fa640 1 -- 172.21.15.100:0/1072027412 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc09bb70 con 0x55d7f9b91bd0 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 -- 172.21.15.100:0/1072027412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 msgr2=0x55d7f9b9a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a130 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f5edc09a540 tx=0x7f5edc09eb10 comp rx=0 tx=0).stop 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 -- 172.21.15.100:0/1072027412 shutdown_connections 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b91bd0 0x55d7f9b9a130 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 --2- 172.21.15.100:0/1072027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9b99bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 -- 172.21.15.100:0/1072027412 >> 172.21.15.100:0/1072027412 conn(0x55d7f9b8b990 msgr2=0x55d7f9b8bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 -- 172.21.15.100:0/1072027412 shutdown_connections 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 -- 172.21.15.100:0/1072027412 wait complete. 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.034+0000 7f5efc60d740 1 -- start start 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 Processor -- start 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9bfa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 0x55d7f9bfa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9b9dd90 con 0x55d7f9b9a7b0 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9b9df00 con 0x55d7f9b91bd0 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f9b9e0c0 con 0x55d7f9b912b0 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5efc60d740 10 client.0 _ll_register_callbacks cb 0x7f5edc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9bfa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 0x55d7f9bfa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9bfa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:36036/0 (socket says 172.21.15.100:36036) 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 0x55d7f9bfa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60550/0 (socket says 172.21.15.100:60550) 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/2130275195 learned_addr learned my addr 172.21.15.100:0/2130275195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36432/0 (socket says 172.21.15.100:36432) 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 msgr2=0x55d7f9bfa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 0x55d7f9bfa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/2130275195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 msgr2=0x55d7f9b9d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef2ffd640 1 -- 172.21.15.100:0/2130275195 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7f9ab7570 con 0x55d7f9b912b0 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5eebfff640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 0x55d7f9bfa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.035+0000 7f5ef37fe640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:20:35.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ef2ffd640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9bfa180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005c40 tx=0x7f5ee4004df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9e5f335c7f8e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ed2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/2130275195 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4004430 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/2130275195 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ee404e050 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee4004030 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/2130275195 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee4050b70 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 10 client.26719 fetch_fsmap learned FSMap version 25 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f5eec012050 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/2130275195 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ee4004030 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/2130275195 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ee405e070 con 0x55d7f9b912b0 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 10 client.26719 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 10 client.26719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 20 client.26719 populate_metadata read hostname 'smithi100' 2024-03-12T05:20:35.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9c01cc0 0x55d7f9c040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 4 client.26719 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.036+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7f9884890 con 0x55d7f9c01cc0 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5eebfff640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9c01cc0 0x55d7f9c040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5ed2ffd640 10 client.26719 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5eebfff640 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9c01cc0 0x55d7f9c040e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e9f5a7c4575843 server_cookie=99f2ae6b2f69d708 in_seq=0 out_seq=0 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5ed2ffd640 1 -- 172.21.15.100:0/2130275195 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d7f9884890 con 0x55d7f9c01cc0 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5ed2ffd640 10 client.26719 handle_command_reply: tid=0 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 client.26719 shutdown 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 msgr2=0x55d7f9bfa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9bfa180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005c40 tx=0x7f5ee4004df0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 shutdown_connections 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d7f9c01cc0 0x55d7f9c040e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f9b9a7b0 0x55d7f9b9d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5ed2ffd640 0 client.26719 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7f9b91bd0 0x55d7f9bfa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 --2- 172.21.15.100:0/2130275195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7f9b912b0 0x55d7f9bfa180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 >> 172.21.15.100:0/2130275195 conn(0x55d7f9b8b990 msgr2=0x55d7f9b927b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 shutdown_connections 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.037+0000 7f5efc60d740 1 -- 172.21.15.100:0/2130275195 wait complete. 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5efc60d740 20 client.26719 trim_cache size 0 max 16384 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:20:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "365c6efb-2b38-4e05-9875-2d40d98434eb": { 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "365c6efb-2b38-4e05-9875-2d40d98434eb", 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:20:35.041 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:20:35.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5ee00a1bd0 msgr2=0x7f5ee00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5ee00a1bd0 0x7f5ee00a4090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5eec005fd0 tx=0x7f5eec013ae0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 msgr2=0x7f5ef4100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:20:35.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 0x7f5ef4100530 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00e9f0 tx=0x7f5edc00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 shutdown_connections 2024-03-12T05:20:35.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5ee00a1bd0 0x7f5ee00a4090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef410b770 0x7f5ef4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ef4102cc0 0x7f5ef4100530 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 --2- 172.21.15.100:0/2011158663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef41023a0 0x7f5ef40ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:20:35.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 >> 172.21.15.100:0/2011158663 conn(0x7f5ef4077210 msgr2=0x7f5ef40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:20:35.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 shutdown_connections 2024-03-12T05:20:35.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:20:35.038+0000 7f5ef995e640 1 -- 172.21.15.100:0/2011158663 wait complete. 2024-03-12T05:20:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:34 smithi100 ceph-mon[32210]: pgmap v261: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 207 op/s 2024-03-12T05:20:35.044 INFO:tasks.cephfs.filesystem:scrub status for tag:365c6efb-2b38-4e05-9875-2d40d98434eb - {'path': '/', 'tag': '365c6efb-2b38-4e05-9875-2d40d98434eb', 'options': 'recursive,force'} 2024-03-12T05:20:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:34 smithi156 ceph-mon[36270]: pgmap v261: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 207 op/s 2024-03-12T05:20:35.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:34 smithi161 ceph-mon[36043]: pgmap v261: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 207 op/s 2024-03-12T05:20:35.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1016973 12.08 MB/sec execute 319 sec latency 15.004 ms 2024-03-12T05:20:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:35 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:20:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:35 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:35 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:20:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:35 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:20:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:35 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:35 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:35 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:20:36.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:35 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:35 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:20:36.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:35 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:20:36.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:35 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:35 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:35 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:20:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:35 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:35 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:20:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:35 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:20:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:35 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:35 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:20:36.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1019738 12.10 MB/sec execute 320 sec latency 14.623 ms 2024-03-12T05:20:37.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:36 smithi156 ceph-mon[36270]: pgmap v262: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 207 op/s 2024-03-12T05:20:37.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:36 smithi161 ceph-mon[36043]: pgmap v262: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 207 op/s 2024-03-12T05:20:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:36 smithi100 ceph-mon[32210]: pgmap v262: 97 pgs: 97 active+clean; 537 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 207 op/s 2024-03-12T05:20:37.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1021157 12.08 MB/sec execute 321 sec latency 102.815 ms 2024-03-12T05:20:38.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1023570 12.09 MB/sec execute 322 sec latency 18.488 ms 2024-03-12T05:20:39.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:38 smithi156 ceph-mon[36270]: pgmap v263: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.9 MiB/s wr, 307 op/s 2024-03-12T05:20:39.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:38 smithi161 ceph-mon[36043]: pgmap v263: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.9 MiB/s wr, 307 op/s 2024-03-12T05:20:39.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:38 smithi100 ceph-mon[32210]: pgmap v263: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.9 MiB/s wr, 307 op/s 2024-03-12T05:20:39.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1025568 12.07 MB/sec execute 323 sec latency 30.566 ms 2024-03-12T05:20:40.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1027596 12.07 MB/sec execute 324 sec latency 16.523 ms 2024-03-12T05:20:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:40 smithi156 ceph-mon[36270]: pgmap v264: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-03-12T05:20:41.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:40 smithi161 ceph-mon[36043]: pgmap v264: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-03-12T05:20:41.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:40 smithi100 ceph-mon[32210]: pgmap v264: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-03-12T05:20:41.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1030337 12.09 MB/sec execute 325 sec latency 14.580 ms 2024-03-12T05:20:42.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1031750 12.07 MB/sec execute 326 sec latency 57.857 ms 2024-03-12T05:20:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:42 smithi156 ceph-mon[36270]: pgmap v265: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-03-12T05:20:43.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:42 smithi161 ceph-mon[36043]: pgmap v265: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-03-12T05:20:43.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:42 smithi100 ceph-mon[32210]: pgmap v265: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-03-12T05:20:43.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1034177 12.08 MB/sec execute 327 sec latency 15.237 ms 2024-03-12T05:20:44.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1036386 12.07 MB/sec execute 328 sec latency 15.115 ms 2024-03-12T05:20:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:44 smithi156 ceph-mon[36270]: pgmap v266: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-03-12T05:20:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:44 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:20:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:44 smithi161 ceph-mon[36043]: pgmap v266: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-03-12T05:20:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:44 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:20:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:44 smithi100 ceph-mon[32210]: pgmap v266: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-03-12T05:20:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:44 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:20:45.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1038429 12.07 MB/sec execute 329 sec latency 16.862 ms 2024-03-12T05:20:46.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:45 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:20:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:45 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:20:46.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:45 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:20:46.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1041088 12.08 MB/sec execute 330 sec latency 17.441 ms 2024-03-12T05:20:47.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:46 smithi161 ceph-mon[36043]: pgmap v267: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-03-12T05:20:47.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:46 smithi100 ceph-mon[32210]: pgmap v267: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-03-12T05:20:47.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:46 smithi156 ceph-mon[36270]: pgmap v267: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-03-12T05:20:47.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1042305 12.06 MB/sec execute 331 sec latency 206.745 ms 2024-03-12T05:20:48.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:48 smithi100 ceph-mon[32210]: pgmap v268: 97 pgs: 97 active+clean; 569 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 301 op/s 2024-03-12T05:20:48.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:48 smithi156 ceph-mon[36270]: pgmap v268: 97 pgs: 97 active+clean; 569 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 301 op/s 2024-03-12T05:20:48.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:48 smithi161 ceph-mon[36043]: pgmap v268: 97 pgs: 97 active+clean; 569 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 301 op/s 2024-03-12T05:20:48.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1044671 12.07 MB/sec execute 332 sec latency 49.330 ms 2024-03-12T05:20:49.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1046332 12.05 MB/sec execute 333 sec latency 33.141 ms 2024-03-12T05:20:50.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1048754 12.05 MB/sec execute 334 sec latency 14.737 ms 2024-03-12T05:20:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:50 smithi156 ceph-mon[36270]: pgmap v269: 97 pgs: 97 active+clean; 569 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-03-12T05:20:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:50 smithi161 ceph-mon[36043]: pgmap v269: 97 pgs: 97 active+clean; 569 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-03-12T05:20:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:50 smithi100 ceph-mon[32210]: pgmap v269: 97 pgs: 97 active+clean; 569 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-03-12T05:20:51.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1051570 12.07 MB/sec execute 335 sec latency 13.935 ms 2024-03-12T05:20:52.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1053039 12.05 MB/sec execute 336 sec latency 90.294 ms 2024-03-12T05:20:53.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:52 smithi156 ceph-mon[36270]: pgmap v270: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 303 op/s 2024-03-12T05:20:53.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:52 smithi161 ceph-mon[36043]: pgmap v270: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 303 op/s 2024-03-12T05:20:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:52 smithi100 ceph-mon[32210]: pgmap v270: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 303 op/s 2024-03-12T05:20:53.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1055441 12.06 MB/sec execute 337 sec latency 39.561 ms 2024-03-12T05:20:54.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1057485 12.04 MB/sec execute 338 sec latency 17.735 ms 2024-03-12T05:20:55.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:54 smithi156 ceph-mon[36270]: pgmap v271: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-03-12T05:20:55.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:54 smithi161 ceph-mon[36043]: pgmap v271: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-03-12T05:20:55.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:54 smithi100 ceph-mon[32210]: pgmap v271: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-03-12T05:20:55.799 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1059350 12.05 MB/sec execute 339 sec latency 21.629 ms 2024-03-12T05:20:56.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1061936 12.05 MB/sec execute 340 sec latency 17.185 ms 2024-03-12T05:20:57.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:56 smithi156 ceph-mon[36270]: pgmap v272: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-03-12T05:20:57.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:56 smithi161 ceph-mon[36043]: pgmap v272: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-03-12T05:20:57.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:56 smithi100 ceph-mon[32210]: pgmap v272: 97 pgs: 97 active+clean; 580 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-03-12T05:20:57.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1063677 12.05 MB/sec execute 341 sec latency 19.809 ms 2024-03-12T05:20:58.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1065930 12.05 MB/sec execute 342 sec latency 128.562 ms 2024-03-12T05:20:59.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:20:58 smithi156 ceph-mon[36270]: pgmap v273: 97 pgs: 97 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 306 op/s 2024-03-12T05:20:59.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:20:58 smithi161 ceph-mon[36043]: pgmap v273: 97 pgs: 97 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 306 op/s 2024-03-12T05:20:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:20:58 smithi100 ceph-mon[32210]: pgmap v273: 97 pgs: 97 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 306 op/s 2024-03-12T05:20:59.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1068024 12.04 MB/sec execute 343 sec latency 14.940 ms 2024-03-12T05:21:00.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1069960 12.04 MB/sec execute 344 sec latency 17.358 ms 2024-03-12T05:21:01.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:00 smithi156 ceph-mon[36270]: pgmap v274: 97 pgs: 97 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 203 op/s 2024-03-12T05:21:01.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:00 smithi161 ceph-mon[36043]: pgmap v274: 97 pgs: 97 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 203 op/s 2024-03-12T05:21:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:00 smithi100 ceph-mon[32210]: pgmap v274: 97 pgs: 97 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 203 op/s 2024-03-12T05:21:01.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1071592 12.02 MB/sec execute 345 sec latency 23.371 ms 2024-03-12T05:21:02.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1073565 12.02 MB/sec execute 346 sec latency 19.170 ms 2024-03-12T05:21:03.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:02 smithi156 ceph-mon[36270]: pgmap v275: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 333 op/s 2024-03-12T05:21:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:02 smithi161 ceph-mon[36043]: pgmap v275: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 333 op/s 2024-03-12T05:21:03.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:02 smithi100 ceph-mon[32210]: pgmap v275: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 333 op/s 2024-03-12T05:21:03.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1075700 12.02 MB/sec execute 347 sec latency 62.023 ms 2024-03-12T05:21:04.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1077737 12.02 MB/sec execute 348 sec latency 16.918 ms 2024-03-12T05:21:05.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:21:05.045 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/1210200458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101d20 msgr2=0x7fa6c0105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/1210200458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101d20 0x7fa6c0105d60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009930 tx=0x7fa6b402bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/1210200458 shutdown_connections 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/1210200458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0106440 0x7fa6c010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/1210200458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101d20 0x7fa6c0105d60 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/1210200458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6c0101370 0x7fa6c0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/1210200458 >> 172.21.15.100:0/1210200458 conn(0x7fa6c00fcc30 msgr2=0x7fa6c00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/1210200458 shutdown_connections 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.163+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/1210200458 wait complete. 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 Processor -- start 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 -- start start 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 0x7fa6c019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6c0106440 0x7fa6c019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0197600 0x7fa6c0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0197ff0 con 0x7fa6c0101370 2024-03-12T05:21:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0198160 con 0x7fa6c0197600 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6c4cc9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0198320 con 0x7fa6c0106440 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 0x7fa6c019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 0x7fa6c019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35540/0 (socket says 172.21.15.100:35540) 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 -- 172.21.15.100:0/3026217353 learned_addr learned my addr 172.21.15.100:0/3026217353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0197600 0x7fa6c0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6c0106440 0x7fa6c019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 -- 172.21.15.100:0/3026217353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6c0106440 msgr2=0x7fa6c019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6c0106440 0x7fa6c019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 -- 172.21.15.100:0/3026217353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0197600 msgr2=0x7fa6c0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0197600 0x7fa6c0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6be59c640 1 -- 172.21.15.100:0/3026217353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4009560 con 0x7fa6c0101370 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.164+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0197600 0x7fa6c0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.165+0000 7fa6be59c640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 0x7fa6c019d450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000db80 tx=0x7fa6b000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.165+0000 7fa6af7fe640 1 -- 172.21.15.100:0/3026217353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0003b70 con 0x7fa6c0101370 2024-03-12T05:21:05.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.165+0000 7fa6af7fe640 1 -- 172.21.15.100:0/3026217353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b000de80 con 0x7fa6c0101370 2024-03-12T05:21:05.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.165+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c01a4310 con 0x7fa6c0101370 2024-03-12T05:21:05.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.165+0000 7fa6af7fe640 1 -- 172.21.15.100:0/3026217353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00106c0 con 0x7fa6c0101370 2024-03-12T05:21:05.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.165+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c01a47f0 con 0x7fa6c0101370 2024-03-12T05:21:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.167+0000 7fa6af7fe640 1 -- 172.21.15.100:0/3026217353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fa6b0004020 con 0x7fa6c0101370 2024-03-12T05:21:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.167+0000 7fa6c7979740 1 Processor -- start 2024-03-12T05:21:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.167+0000 7fa6c7979740 1 -- start start 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c22690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c2b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6c7979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141b91280 con 0x558141c2b7b0 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6c7979740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141b8e7b0 con 0x558141c222b0 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6c7979740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141b89f90 con 0x558141c22bd0 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6af7fe640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa6a80a1ba0 0x7fa6a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c22690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35546/0 (socket says 172.21.15.100:35546) 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c22690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37262/0 (socket says 172.21.15.100:37262) 2024-03-12T05:21:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 -- 172.21.15.100:0/2633691450 learned_addr learned my addr 172.21.15.100:0/2633691450 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c2b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa6a80a1ba0 0x7fa6a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 -- 172.21.15.100:0/2633691450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 msgr2=0x558141c2b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c2b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 -- 172.21.15.100:0/2633691450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 msgr2=0x558141c22690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c22690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 -- 172.21.15.100:0/2633691450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558141b48570 con 0x558141c2b7b0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6af7fe640 1 -- 172.21.15.100:0/3026217353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa6b00cd820 con 0x7fa6c0101370 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2dba0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4005ab0 tx=0x7fa6b402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e27da65bc48cb13 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa6a80a1ba0 0x7fa6a80a4060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b8004730 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6acff9640 1 -- 172.21.15.100:0/2633691450 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4004880 con 0x558141c2b7b0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6acff9640 1 -- 172.21.15.100:0/2633691450 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b40049e0 con 0x558141c2b7b0 2024-03-12T05:21:05.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.168+0000 7fa6acff9640 1 -- 172.21.15.100:0/2633691450 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4005150 con 0x558141c2b7b0 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 -- 172.21.15.100:0/2633691450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 msgr2=0x558141c2dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2dba0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4005ab0 tx=0x7fa6b402c680 comp rx=0 tx=0).stop 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 -- 172.21.15.100:0/2633691450 shutdown_connections 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2dba0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c2b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2633691450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c22690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 -- 172.21.15.100:0/2633691450 >> 172.21.15.100:0/2633691450 conn(0x558141c1c990 msgr2=0x558141c1cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 -- 172.21.15.100:0/2633691450 shutdown_connections 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.169+0000 7fa6c7979740 1 -- 172.21.15.100:0/2633691450 wait complete. 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 -- start start 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 Processor -- start 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 0x558141c2b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c2b7b0 0x558141c28b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814185d430 con 0x558141c222b0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141819e00 con 0x558141c2b7b0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141b0efd0 con 0x558141c22bd0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6c7979740 10 client.0 _ll_register_callbacks cb 0x558141b64fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 0x558141c2b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 0x558141c2b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35552/0 (socket says 172.21.15.100:35552) 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 -- 172.21.15.100:0/2899783664 learned_addr learned my addr 172.21.15.100:0/2899783664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c2b7b0 0x558141c28b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 -- 172.21.15.100:0/2899783664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 msgr2=0x558141c28550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c28550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 -- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c2b7b0 msgr2=0x558141c28b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c2b7b0 0x558141c28b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.170+0000 7fa6be59c640 1 -- 172.21.15.100:0/2899783664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c010e820 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 0x558141c2b1c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0000f80 tx=0x7fa6b006a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69366c404321beaa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2899783664 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0099d70 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581418897e0 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2899783664 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa6b009ab10 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2899783664 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6b000be60 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 10 client.17349 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558141b88610 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2899783664 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5581418897e0 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2899783664 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa6b0096a80 con 0x558141c222b0 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 10 client.17349 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:05.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 20 client.17349 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c3b940 0x558141c3dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 4 client.17349 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581418c6720 con 0x558141c3b940 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.171+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c3b940 0x558141c3dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.172+0000 7fa6a5ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.172+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c3b940 0x558141c3dd30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55adcf2a35420045 server_cookie=9931a5a0b01dac39 in_seq=0 out_seq=0 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.172+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2899783664 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581418c6720 con 0x558141c3b940 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.172+0000 7fa6a5ffb640 10 client.17349 handle_command_reply: tid=0 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 client.17349 shutdown 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 msgr2=0x558141c2b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 0x558141c2b1c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0000f80 tx=0x7fa6b006a4b0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 shutdown_connections 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c3b940 0x558141c3dd30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c2b7b0 0x558141c28b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c22bd0 0x558141c28550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2899783664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c222b0 0x558141c2b1c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 >> 172.21.15.100:0/2899783664 conn(0x558141c1c990 msgr2=0x558141c1e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6a5ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 shutdown_connections 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 1 -- 172.21.15.100:0/2899783664 wait complete. 2024-03-12T05:21:05.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.173+0000 7fa6c7979740 20 client.17349 trim_cache size 0 max 16384 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 Processor -- start 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 -- start start 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c2ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 0x558141c2b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c2b7b0 0x558141c27de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141c21cf0 con 0x558141c22bd0 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141c21f10 con 0x558141c222b0 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6c7979740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141c3b6f0 con 0x558141c2b7b0 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.189+0000 7fa6bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 0x558141c2b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c2ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 0x558141c2b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35568/0 (socket says 172.21.15.100:35568) 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6bdd9b640 1 -- 172.21.15.100:0/855845553 learned_addr learned my addr 172.21.15.100:0/855845553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c2b7b0 0x558141c27de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 -- 172.21.15.100:0/855845553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c2b7b0 msgr2=0x558141c27de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c2b7b0 0x558141c27de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 -- 172.21.15.100:0/855845553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 msgr2=0x558141c2b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 0x558141c2b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 -- 172.21.15.100:0/855845553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558141b24ac0 con 0x558141c222b0 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 0x558141c2b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6be59c640 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c2ac80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6b009a3a0 tx=0x7fa6b00960b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14dbefd8938b76e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6a7fff640 1 -- 172.21.15.100:0/855845553 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000be60 con 0x558141c222b0 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6a7fff640 1 -- 172.21.15.100:0/855845553 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b0099d70 con 0x558141c222b0 2024-03-12T05:21:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.190+0000 7fa6a7fff640 1 -- 172.21.15.100:0/855845553 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b009e5f0 con 0x558141c222b0 2024-03-12T05:21:05.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 -- 172.21.15.100:0/855845553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 msgr2=0x558141c2ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c2ac80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6b009a3a0 tx=0x7fa6b00960b0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 -- 172.21.15.100:0/855845553 shutdown_connections 2024-03-12T05:21:05.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c2b7b0 0x558141c27de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c22bd0 0x558141c2b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 --2- 172.21.15.100:0/855845553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c222b0 0x558141c2ac80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 -- 172.21.15.100:0/855845553 >> 172.21.15.100:0/855845553 conn(0x558141c1c990 msgr2=0x558141c26e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 -- 172.21.15.100:0/855845553 shutdown_connections 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.191+0000 7fa6c7979740 1 -- 172.21.15.100:0/855845553 wait complete. 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 -- start start 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 Processor -- start 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c222b0 0x558141c32b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c22bd0 0x558141c2eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141c2fd20 con 0x558141c2b7b0 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141c91e00 con 0x558141c22bd0 2024-03-12T05:21:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558141c91f40 con 0x558141c222b0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6c7979740 10 client.0 _ll_register_callbacks cb 0x7fa6b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35576/0 (socket says 172.21.15.100:35576) 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 -- 172.21.15.100:0/2818659791 learned_addr learned my addr 172.21.15.100:0/2818659791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bdd9b640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c22bd0 0x558141c2eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c222b0 0x558141c32b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 -- 172.21.15.100:0/2818659791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c222b0 msgr2=0x558141c32b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c222b0 0x558141c32b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 -- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c22bd0 msgr2=0x558141c2eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c22bd0 0x558141c2eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.192+0000 7fa6bed9d640 1 -- 172.21.15.100:0/2818659791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558141b48570 con 0x558141c2b7b0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6bed9d640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2f620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800fc60 tx=0x7fa6b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaca12a76621312 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2818659791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8028070 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2818659791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa6b8005020 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6b40049e0 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2818659791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6b80051e0 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 10 client.17355 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558141b88610 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2818659791 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b8038020 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2818659791 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa6b8010430 con 0x558141c2b7b0 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 10 client.17355 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 20 client.17355 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c937d0 0x558141c95bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 4 client.17355 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.193+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558141915890 con 0x558141c937d0 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c937d0 0x558141c95bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6be59c640 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c937d0 0x558141c95bf0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a59a43c1cf5e2ad server_cookie=2b87e85b2132788 in_seq=0 out_seq=0 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6a5ffb640 10 client.17355 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6a5ffb640 1 -- 172.21.15.100:0/2818659791 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558141915890 con 0x558141c937d0 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6a5ffb640 10 client.17355 handle_command_reply: tid=0 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 client.17355 shutdown 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 msgr2=0x558141c2f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2f620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800fc60 tx=0x7fa6b80046c0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 shutdown_connections 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558141c937d0 0x558141c95bf0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558141c2b7b0 0x558141c2f620 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558141c22bd0 0x558141c2eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6a5ffb640 0 client.17355 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 --2- 172.21.15.100:0/2818659791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558141c222b0 0x558141c32b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 >> 172.21.15.100:0/2818659791 conn(0x558141c1c990 msgr2=0x558141c27db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 shutdown_connections 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.194+0000 7fa6c7979740 1 -- 172.21.15.100:0/2818659791 wait complete. 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c7979740 20 client.17355 trim_cache size 0 max 16384 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:21:05.197 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:21:05.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa6a80a1ba0 msgr2=0x7fa6a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa6a80a1ba0 0x7fa6a80a4060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b8004730 comp rx=0 tx=0).stop 2024-03-12T05:21:05.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 msgr2=0x7fa6c019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 0x7fa6c019d450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000db80 tx=0x7fa6b000b630 comp rx=0 tx=0).stop 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 shutdown_connections 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa6a80a1ba0 0x7fa6a80a4060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0197600 0x7fa6c0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6c0106440 0x7fa6c019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 --2- 172.21.15.100:0/3026217353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c0101370 0x7fa6c019d450 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 >> 172.21.15.100:0/3026217353 conn(0x7fa6c00fcc30 msgr2=0x7fa6c00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 shutdown_connections 2024-03-12T05:21:05.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.195+0000 7fa6c4cc9640 1 -- 172.21.15.100:0/3026217353 wait complete. 2024-03-12T05:21:05.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:04 smithi100 ceph-mon[32210]: pgmap v276: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-03-12T05:21:05.201 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:21:05.202 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:21:05.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:04 smithi156 ceph-mon[36270]: pgmap v276: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-03-12T05:21:05.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:04 smithi161 ceph-mon[36043]: pgmap v276: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-03-12T05:21:05.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 -- 172.21.15.100:0/1507019010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2914101380 msgr2=0x7f2914101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 --2- 172.21.15.100:0/1507019010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2914101380 0x7f2914101760 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2904009930 tx=0x7f290402bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:05.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 -- 172.21.15.100:0/1507019010 shutdown_connections 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 --2- 172.21.15.100:0/1507019010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914106450 0x7f291410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 --2- 172.21.15.100:0/1507019010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914101d30 0x7f2914105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 --2- 172.21.15.100:0/1507019010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2914101380 0x7f2914101760 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 -- 172.21.15.100:0/1507019010 >> 172.21.15.100:0/1507019010 conn(0x7f29140fcc20 msgr2=0x7f29140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 -- 172.21.15.100:0/1507019010 shutdown_connections 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 -- 172.21.15.100:0/1507019010 wait complete. 2024-03-12T05:21:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.318+0000 7f291970d640 1 Processor -- start 2024-03-12T05:21:05.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 -- start start 2024-03-12T05:21:05.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914101380 0x7f291419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914106450 0x7f291419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29141976b0 0x7f2914197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914198180 con 0x7f29141976b0 2024-03-12T05:21:05.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141982f0 con 0x7f2914101380 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f291970d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141984b0 con 0x7f2914106450 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f29137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29141976b0 0x7f2914197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f2912ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914101380 0x7f291419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f290a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914106450 0x7f291419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f29137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29141976b0 0x7f2914197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35584/0 (socket says 172.21.15.100:35584) 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f29137fe640 1 -- 172.21.15.100:0/2777622916 learned_addr learned my addr 172.21.15.100:0/2777622916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f290a7fc640 1 -- 172.21.15.100:0/2777622916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914101380 msgr2=0x7f291419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f290a7fc640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914101380 0x7f291419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f290a7fc640 1 -- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29141976b0 msgr2=0x7f2914197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f290a7fc640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29141976b0 0x7f2914197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f290a7fc640 1 -- 172.21.15.100:0/2777622916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009560 con 0x7f2914106450 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.319+0000 7f2912ffd640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914101380 0x7f291419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.320+0000 7f290a7fc640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914106450 0x7f291419dae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00e9c0 tx=0x7f28fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.320+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc00ccc0 con 0x7f2914106450 2024-03-12T05:21:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.320+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28fc00ce20 con 0x7f2914106450 2024-03-12T05:21:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.320+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc004a00 con 0x7f2914106450 2024-03-12T05:21:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.320+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29141a4400 con 0x7f2914106450 2024-03-12T05:21:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.320+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29141a48e0 con 0x7f2914106450 2024-03-12T05:21:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.321+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2914102850 con 0x7f2914106450 2024-03-12T05:21:05.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.322+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f28fc005020 con 0x7f2914106450 2024-03-12T05:21:05.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.322+0000 7f2910ff9640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f29000a1c20 0x7f29000a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.323+0000 7f2912ffd640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f29000a1c20 0x7f29000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.323+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f28fc014070 con 0x7f2914106450 2024-03-12T05:21:05.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.323+0000 7f2912ffd640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f29000a1c20 0x7f29000a40e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2904005ab0 tx=0x7f2904005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.327+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28fc096540 con 0x7f2914106450 2024-03-12T05:21:05.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.564+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2914109680 con 0x7f2914106450 2024-03-12T05:21:05.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.565+0000 7f2910ff9640 1 -- 172.21.15.100:0/2777622916 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f28fc01c090 con 0x7f2914106450 2024-03-12T05:21:05.560 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:21:05.561 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:21:05.561 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f29000a1c20 msgr2=0x7f29000a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f29000a1c20 0x7f29000a40e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2904005ab0 tx=0x7f2904005fd0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914106450 msgr2=0x7f291419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914106450 0x7f291419dae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00e9c0 tx=0x7f28fc00ee80 comp rx=0 tx=0).stop 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 shutdown_connections 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f29000a1c20 0x7f29000a40e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29141976b0 0x7f2914197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2914106450 0x7f291419dae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 --2- 172.21.15.100:0/2777622916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2914101380 0x7f291419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.567+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 >> 172.21.15.100:0/2777622916 conn(0x7f29140fcc20 msgr2=0x7f29140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.568+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 shutdown_connections 2024-03-12T05:21:05.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.568+0000 7f291970d640 1 -- 172.21.15.100:0/2777622916 wait complete. 2024-03-12T05:21:05.575 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:21:05.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 -- 172.21.15.100:0/414764044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70105220 msgr2=0x7f3f7010cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 --2- 172.21.15.100:0/414764044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70105220 0x7f3f7010cdf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009cd0 tx=0x7f3f6802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 -- 172.21.15.100:0/414764044 shutdown_connections 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 --2- 172.21.15.100:0/414764044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f70105220 0x7f3f7010cdf0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 --2- 172.21.15.100:0/414764044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100b00 0x7f3f70104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 --2- 172.21.15.100:0/414764044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70100150 0x7f3f70100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.693+0000 7f3f771a0640 1 -- 172.21.15.100:0/414764044 >> 172.21.15.100:0/414764044 conn(0x7f3f700fb9d0 msgr2=0x7f3f700fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 -- 172.21.15.100:0/414764044 shutdown_connections 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 -- 172.21.15.100:0/414764044 wait complete. 2024-03-12T05:21:05.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 Processor -- start 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 -- start start 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100150 0x7f3f700fc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70105220 0x7f3f700fc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.694+0000 7f3f771a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 0x7f3f701a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f771a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701a5e20 con 0x7f3f700fcdd0 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f771a0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701a5f90 con 0x7f3f70100150 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f771a0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701a6150 con 0x7f3f70105220 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 0x7f3f701a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 0x7f3f701a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35600/0 (socket says 172.21.15.100:35600) 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70105220 0x7f3f700fc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7619e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100150 0x7f3f700fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 -- 172.21.15.100:0/2745913789 learned_addr learned my addr 172.21.15.100:0/2745913789 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 -- 172.21.15.100:0/2745913789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70105220 msgr2=0x7f3f700fc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70105220 0x7f3f700fc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 -- 172.21.15.100:0/2745913789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100150 msgr2=0x7f3f700fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100150 0x7f3f700fc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 -- 172.21.15.100:0/2745913789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60009490 con 0x7f3f700fcdd0 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7599d640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70105220 0x7f3f700fc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7619e640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100150 0x7f3f700fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 0x7f3f701a5850 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3f68004af0 tx=0x7f3f68032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f677fe640 1 -- 172.21.15.100:0/2745913789 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680328b0 con 0x7f3f700fcdd0 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68009900 con 0x7f3f700fcdd0 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f677fe640 1 -- 172.21.15.100:0/2745913789 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f68003e50 con 0x7f3f700fcdd0 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f677fe640 1 -- 172.21.15.100:0/2745913789 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68004740 con 0x7f3f700fcdd0 2024-03-12T05:21:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.695+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f701a66f0 con 0x7f3f700fcdd0 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.698+0000 7f3f677fe640 1 -- 172.21.15.100:0/2745913789 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3f6800a040 con 0x7f3f700fcdd0 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.698+0000 7f3f7afd0740 1 Processor -- start 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.698+0000 7f3f7afd0740 1 -- start start 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.698+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598148820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.698+0000 7f3f677fe640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3f5c0a1bd0 0x7f3f5c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.698+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c598153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c598103960 con 0x55c598148d60 2024-03-12T05:21:05.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5980b7310 con 0x55c598151940 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5980b4840 con 0x55c598148440 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7619e640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3f5c0a1bd0 0x7f3f5c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f677fe640 1 -- 172.21.15.100:0/2745913789 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3f680f0250 con 0x7f3f700fcdd0 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35606/0 (socket says 172.21.15.100:35606) 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7599d640 1 -- 172.21.15.100:0/2640338582 learned_addr learned my addr 172.21.15.100:0/2640338582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c598153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7599d640 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 -- 172.21.15.100:0/2640338582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 msgr2=0x55c598148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 -- 172.21.15.100:0/2640338582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 msgr2=0x55c598151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 -- 172.21.15.100:0/2640338582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c598072060 con 0x55c598151940 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7599d640 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7619e640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3f5c0a1bd0 0x7f3f5c0a4090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009fc0 tx=0x7f3f6c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c598153d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f680eda70 tx=0x7f3f680b9320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8bc53a6655a9ff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f64bfe640 1 -- 172.21.15.100:0/2640338582 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680bd650 con 0x55c598151940 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f64bfe640 1 -- 172.21.15.100:0/2640338582 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f680bcf50 con 0x55c598151940 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.699+0000 7f3f64bfe640 1 -- 172.21.15.100:0/2640338582 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68032a10 con 0x55c598151940 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2640338582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 msgr2=0x55c598153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c598153d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f680eda70 tx=0x7f3f680b9320 comp rx=0 tx=0).stop 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2640338582 shutdown_connections 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c598153d30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2640338582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2640338582 >> 172.21.15.100:0/2640338582 conn(0x55c598142b70 msgr2=0x55c598142f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2640338582 shutdown_connections 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.700+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2640338582 wait complete. 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 -- start start 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 Processor -- start 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148440 0x55c59814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148d60 0x55c59814ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c59814f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5980b0020 con 0x55c598148440 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c597d83430 con 0x55c598151940 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c597d3fe00 con 0x55c598148d60 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7afd0740 10 client.0 _ll_register_callbacks cb 0x55c598057cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7619e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148440 0x55c59814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7699f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c59814f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148d60 0x55c59814ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7699f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c59814f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37336/0 (socket says 172.21.15.100:37336) 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7699f640 1 -- 172.21.15.100:0/3627682276 learned_addr learned my addr 172.21.15.100:0/3627682276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7619e640 1 -- 172.21.15.100:0/3627682276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148d60 msgr2=0x55c59814ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7619e640 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148d60 0x55c59814ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7619e640 1 -- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 msgr2=0x55c59814f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7619e640 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c59814f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.701+0000 7f3f7619e640 1 -- 172.21.15.100:0/3627682276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f701a66f0 con 0x55c598148440 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f7699f640 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c59814f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f7619e640 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148440 0x55c59814e8d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c011ca0 tx=0x7f3f6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb857145fda01cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f59ffb640 1 -- 172.21.15.100:0/3627682276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00f740 con 0x55c598148440 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f59ffb640 1 -- 172.21.15.100:0/3627682276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3f6c00f8a0 con 0x55c598148440 2024-03-12T05:21:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c597daf7e0 con 0x55c598148440 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f59ffb640 1 -- 172.21.15.100:0/3627682276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f6c00fa60 con 0x55c598148440 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f7afd0740 10 client.17373 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c5980ae6a0 con 0x55c598148440 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.702+0000 7f3f59ffb640 1 -- 172.21.15.100:0/3627682276 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c597daf7e0 con 0x55c598148440 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f59ffb640 1 -- 172.21.15.100:0/3627682276 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3f6c01d3f0 con 0x55c598148440 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7afd0740 10 client.17373 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7afd0740 10 client.17373 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7afd0740 20 client.17373 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c598161bb0 0x55c598163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7afd0740 4 client.17373 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c597dec720 con 0x55c598161bb0 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7599d640 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c598161bb0 0x55c598163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f7599d640 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c598161bb0 0x55c598163fd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c92cc43e0f6369e server_cookie=220affbaf80b4346 in_seq=0 out_seq=0 2024-03-12T05:21:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.703+0000 7f3f59ffb640 10 client.17373 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:05.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.705+0000 7f3f59ffb640 1 -- 172.21.15.100:0/3627682276 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c597dec720 con 0x55c598161bb0 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.705+0000 7f3f59ffb640 10 client.17373 handle_command_reply: tid=0 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.705+0000 7f3f7afd0740 1 client.17373 shutdown 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148440 msgr2=0x55c59814e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148440 0x55c59814e8d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c011ca0 tx=0x7f3f6c004730 comp rx=0 tx=0).stop 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 shutdown_connections 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c598161bb0 0x55c598163fd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598151940 0x55c59814f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148d60 0x55c59814ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f59ffb640 0 client.17373 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/3627682276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148440 0x55c59814e8d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 >> 172.21.15.100:0/3627682276 conn(0x55c598142b70 msgr2=0x55c5981448c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 shutdown_connections 2024-03-12T05:21:05.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 1 -- 172.21.15.100:0/3627682276 wait complete. 2024-03-12T05:21:05.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.706+0000 7f3f7afd0740 20 client.17373 trim_cache size 0 max 16384 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 Processor -- start 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 -- start start 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598151000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148d60 0x55c59814f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 0x55c59814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c598035130 con 0x55c598151940 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c598153d30 con 0x55c598148d60 2024-03-12T05:21:05.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59813e4b0 con 0x55c598148440 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 0x55c59814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148d60 0x55c59814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 0x55c59814bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35632/0 (socket says 172.21.15.100:35632) 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 -- 172.21.15.100:0/1866881825 learned_addr learned my addr 172.21.15.100:0/1866881825 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7619e640 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 -- 172.21.15.100:0/1866881825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 msgr2=0x55c598151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 -- 172.21.15.100:0/1866881825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148d60 msgr2=0x55c59814f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148d60 0x55c59814f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 -- 172.21.15.100:0/1866881825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c59804ac20 con 0x55c598151940 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.719+0000 7f3f7699f640 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 0x55c59814bb30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3f680348f0 tx=0x7f3f680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f7ff1ea9406e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f5bfff640 1 -- 172.21.15.100:0/1866881825 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680baff0 con 0x55c598151940 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f5bfff640 1 -- 172.21.15.100:0/1866881825 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f680bd650 con 0x55c598151940 2024-03-12T05:21:05.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f5bfff640 1 -- 172.21.15.100:0/1866881825 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680bd1b0 con 0x55c598151940 2024-03-12T05:21:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 -- 172.21.15.100:0/1866881825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 msgr2=0x55c59814bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 0x55c59814bb30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3f680348f0 tx=0x7f3f680095c0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 -- 172.21.15.100:0/1866881825 shutdown_connections 2024-03-12T05:21:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598151940 0x55c59814bb30 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148d60 0x55c59814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/1866881825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598148440 0x55c598151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 -- 172.21.15.100:0/1866881825 >> 172.21.15.100:0/1866881825 conn(0x55c598142b70 msgr2=0x55c59814d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 -- 172.21.15.100:0/1866881825 shutdown_connections 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.720+0000 7f3f7afd0740 1 -- 172.21.15.100:0/1866881825 wait complete. 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 -- start start 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 Processor -- start 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148440 0x55c598158570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598151940 0x55c598156b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5981571e0 con 0x55c598148d60 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c598157350 con 0x55c598148440 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c598157510 con 0x55c598151940 2024-03-12T05:21:05.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7afd0740 10 client.0 _ll_register_callbacks cb 0x7f3f6c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35636/0 (socket says 172.21.15.100:35636) 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 -- 172.21.15.100:0/2841385913 learned_addr learned my addr 172.21.15.100:0/2841385913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7619e640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148440 0x55c598158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7699f640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598151940 0x55c598156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 -- 172.21.15.100:0/2841385913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598151940 msgr2=0x55c598156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598151940 0x55c598156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 -- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148440 msgr2=0x55c598158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148440 0x55c598158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.721+0000 7f3f7599d640 1 -- 172.21.15.100:0/2841385913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c598072060 con 0x55c598148d60 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7599d640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598158ab0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3f60012d00 tx=0x7f3f600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ca3222c41dd2e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f59ffb640 1 -- 172.21.15.100:0/2841385913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60012ea0 con 0x55c598148d60 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f59ffb640 1 -- 172.21.15.100:0/2841385913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3f60004430 con 0x55c598148d60 2024-03-12T05:21:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f680bcf50 con 0x55c598148d60 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f59ffb640 1 -- 172.21.15.100:0/2841385913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f60020420 con 0x55c598148d60 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 10 client.17379 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c5980708c0 con 0x55c598148d60 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f59ffb640 1 -- 172.21.15.100:0/2841385913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f60036080 con 0x55c598148d60 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f59ffb640 1 -- 172.21.15.100:0/2841385913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3f6002ab00 con 0x55c598148d60 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 10 client.17379 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 10 client.17379 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 20 client.17379 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c5981b9460 0x55c5981bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 4 client.17379 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c597e3b890 con 0x55c5981b9460 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.722+0000 7f3f7619e640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c5981b9460 0x55c5981bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f59ffb640 10 client.17379 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7619e640 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c5981b9460 0x55c5981bb880 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24bebc4ab97d3606 server_cookie=68422fd8019f0fa0 in_seq=0 out_seq=0 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f59ffb640 1 -- 172.21.15.100:0/2841385913 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c597e3b890 con 0x55c5981b9460 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f59ffb640 10 client.17379 handle_command_reply: tid=0 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 client.17379 shutdown 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 msgr2=0x55c598158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598158ab0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3f60012d00 tx=0x7f3f600039c0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 shutdown_connections 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c5981b9460 0x55c5981bb880 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c598151940 0x55c598156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c598148d60 0x55c598158ab0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f59ffb640 0 client.17379 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 --2- 172.21.15.100:0/2841385913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c598148440 0x55c598158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 >> 172.21.15.100:0/2841385913 conn(0x55c598142b70 msgr2=0x55c598141440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 shutdown_connections 2024-03-12T05:21:05.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.723+0000 7f3f7afd0740 1 -- 172.21.15.100:0/2841385913 wait complete. 2024-03-12T05:21:05.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f7afd0740 20 client.17379 trim_cache size 0 max 16384 2024-03-12T05:21:05.726 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3f5c0a1bd0 msgr2=0x7f3f5c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3f5c0a1bd0 0x7f3f5c0a4090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009fc0 tx=0x7f3f6c005f90 comp rx=0 tx=0).stop 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 msgr2=0x7f3f701a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 0x7f3f701a5850 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3f68004af0 tx=0x7f3f68032570 comp rx=0 tx=0).stop 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 shutdown_connections 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3f5c0a1bd0 0x7f3f5c0a4090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f700fcdd0 0x7f3f701a5850 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f70105220 0x7f3f700fc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 --2- 172.21.15.100:0/2745913789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f70100150 0x7f3f700fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 >> 172.21.15.100:0/2745913789 conn(0x7f3f700fb9d0 msgr2=0x7f3f70068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 shutdown_connections 2024-03-12T05:21:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.724+0000 7f3f771a0640 1 -- 172.21.15.100:0/2745913789 wait complete. 2024-03-12T05:21:05.730 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:21:05.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:21:05.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1080175 12.03 MB/sec execute 349 sec latency 60.644 ms 2024-03-12T05:21:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 -- 172.21.15.100:0/2359685182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105330 msgr2=0x7fb4c010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/2359685182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105330 0x7fb4c010cf00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009cd0 tx=0x7fb4b802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 -- 172.21.15.100:0/2359685182 shutdown_connections 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/2359685182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0105330 0x7fb4c010cf00 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/2359685182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c0100c10 0x7fb4c0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/2359685182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c0100260 0x7fb4c0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 -- 172.21.15.100:0/2359685182 >> 172.21.15.100:0/2359685182 conn(0x7fb4c0077180 msgr2=0x7fb4c0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 -- 172.21.15.100:0/2359685182 shutdown_connections 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.846+0000 7fb4c83c2640 1 -- 172.21.15.100:0/2359685182 wait complete. 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 Processor -- start 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 -- start start 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 0x7fb4c019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c0105330 0x7fb4c019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c01975f0 0x7fb4c0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0197fe0 con 0x7fb4c0100260 2024-03-12T05:21:05.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0198150 con 0x7fb4c01975f0 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c83c2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0198310 con 0x7fb4c0105330 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 0x7fb4c019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 0x7fb4c019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35654/0 (socket says 172.21.15.100:35654) 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 -- 172.21.15.100:0/1434309480 learned_addr learned my addr 172.21.15.100:0/1434309480 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c595d640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c0105330 0x7fb4c019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c695f640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c01975f0 0x7fb4c0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 -- 172.21.15.100:0/1434309480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c0105330 msgr2=0x7fb4c019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c0105330 0x7fb4c019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 -- 172.21.15.100:0/1434309480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c01975f0 msgr2=0x7fb4c0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c01975f0 0x7fb4c0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 -- 172.21.15.100:0/1434309480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8009900 con 0x7fb4c0100260 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c695f640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c01975f0 0x7fb4c0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.847+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 0x7fb4c019d3f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d950 tx=0x7fb4bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.848+0000 7fb4b77fe640 1 -- 172.21.15.100:0/1434309480 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc009a30 con 0x7fb4c0100260 2024-03-12T05:21:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.848+0000 7fb4b77fe640 1 -- 172.21.15.100:0/1434309480 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc009b90 con 0x7fb4c0100260 2024-03-12T05:21:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.848+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c01a42b0 con 0x7fb4c0100260 2024-03-12T05:21:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.848+0000 7fb4b77fe640 1 -- 172.21.15.100:0/1434309480 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc004a00 con 0x7fb4c0100260 2024-03-12T05:21:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.848+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c01a47c0 con 0x7fb4c0100260 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.850+0000 7fb4b77fe640 1 -- 172.21.15.100:0/1434309480 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb4bc009d00 con 0x7fb4c0100260 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.850+0000 7fb4caf8b740 1 Processor -- start 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 -- start start 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6761960 con 0x55d5b67a6820 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6715310 con 0x55d5b67a5f00 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6712840 con 0x55d5b67b3b60 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c695f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35660/0 (socket says 172.21.15.100:35660) 2024-03-12T05:21:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 -- 172.21.15.100:0/2291989622 learned_addr learned my addr 172.21.15.100:0/2291989622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c695f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37366/0 (socket says 172.21.15.100:37366) 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c595d640 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4b77fe640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4ac0a1ba0 0x7fb4ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 -- 172.21.15.100:0/2291989622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 msgr2=0x55d5b67b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 -- 172.21.15.100:0/2291989622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 msgr2=0x55d5b67a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c615e640 1 -- 172.21.15.100:0/2291989622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5b66d0060 con 0x55d5b67a6820 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c695f640 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4c695f640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4ac0a1ba0 0x7fb4ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.851+0000 7fb4b77fe640 1 -- 172.21.15.100:0/1434309480 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4bc0cd130 con 0x7fb4c0100260 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.852+0000 7fb4c615e640 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67b3620 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ca850 tx=0x7fb4bc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1624fb53538002c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.852+0000 7fb4c695f640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4ac0a1ba0 0x7fb4ac0a4060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8005cc0 tx=0x7fb4b80095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.852+0000 7fb4b4bfe640 1 -- 172.21.15.100:0/2291989622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc005680 con 0x55d5b67a6820 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.852+0000 7fb4b4bfe640 1 -- 172.21.15.100:0/2291989622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc099b90 con 0x55d5b67a6820 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.852+0000 7fb4b4bfe640 1 -- 172.21.15.100:0/2291989622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc099dc0 con 0x55d5b67a6820 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2291989622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 msgr2=0x55d5b67b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67b3620 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ca850 tx=0x7fb4bc021000 comp rx=0 tx=0).stop 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2291989622 shutdown_connections 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67b3620 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2291989622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2291989622 >> 172.21.15.100:0/2291989622 conn(0x55d5b67a0980 msgr2=0x55d5b67a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2291989622 shutdown_connections 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2291989622 wait complete. 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- start start 2024-03-12T05:21:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 Processor -- start 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b670e020 con 0x55d5b67a5f00 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b63e1430 con 0x55d5b67a6820 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b639de00 con 0x55d5b67b3b60 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4caf8b740 10 client.0 _ll_register_callbacks cb 0x55d5b66b5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.853+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35676/0 (socket says 172.21.15.100:35676) 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 -- 172.21.15.100:0/1919203061 learned_addr learned my addr 172.21.15.100:0/1919203061 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c595d640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c695f640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 -- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 msgr2=0x55d5b67b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 -- 172.21.15.100:0/1919203061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 msgr2=0x55d5b67b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 -- 172.21.15.100:0/1919203061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c01a47c0 con 0x55d5b67a5f00 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67b0ab0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005380 tx=0x7fb4bc0d0b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6689e51834bd10 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/1919203061 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc005680 con 0x55d5b67a5f00 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/1919203061 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4bc09df10 con 0x55d5b67a5f00 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5b640d7e0 con 0x55d5b67a5f00 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/1919203061 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4bc005cd0 con 0x55d5b67a5f00 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4caf8b740 10 client.17391 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d5b670c6a0 con 0x55d5b67a5f00 2024-03-12T05:21:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.854+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/1919203061 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5b640d7e0 con 0x55d5b67a5f00 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/1919203061 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4bc095b10 con 0x55d5b67a5f00 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4caf8b740 10 client.17391 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4caf8b740 10 client.17391 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4caf8b740 20 client.17391 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b67c3b90 0x55d5b67c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4caf8b740 4 client.17391 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5b644a720 con 0x55d5b67c3b90 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4c595d640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b67c3b90 0x55d5b67c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4a9ffb640 10 client.17391 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.855+0000 7fb4c595d640 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b67c3b90 0x55d5b67c5ff0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fa2a8d00167f049 server_cookie=c1678363e02b3c22 in_seq=0 out_seq=0 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/1919203061 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5b644a720 con 0x55d5b67c3b90 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4a9ffb640 10 client.17391 handle_command_reply: tid=0 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 client.17391 shutdown 2024-03-12T05:21:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 msgr2=0x55d5b67b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67b0ab0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005380 tx=0x7fb4bc0d0b20 comp rx=0 tx=0).stop 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 shutdown_connections 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b67b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b67c3b90 0x55d5b67c5ff0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1919203061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67b0ab0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4a9ffb640 0 client.17391 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 >> 172.21.15.100:0/1919203061 conn(0x55d5b67a0980 msgr2=0x55d5b67a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 shutdown_connections 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1919203061 wait complete. 2024-03-12T05:21:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.856+0000 7fb4caf8b740 20 client.17391 trim_cache size 0 max 16384 2024-03-12T05:21:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 Processor -- start 2024-03-12T05:21:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 -- start start 2024-03-12T05:21:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b679fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6693130 con 0x55d5b67a5f00 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b67b0150 con 0x55d5b67a6820 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.869+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b67a59d0 con 0x55d5b67b3b60 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35680/0 (socket says 172.21.15.100:35680) 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 -- 172.21.15.100:0/1569739847 learned_addr learned my addr 172.21.15.100:0/1569739847 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c595d640 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c695f640 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b679fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 -- 172.21.15.100:0/1569739847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 msgr2=0x55d5b679fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b679fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 -- 172.21.15.100:0/1569739847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 msgr2=0x55d5b67b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 -- 172.21.15.100:0/1569739847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5b66a8c20 con 0x55d5b67a5f00 2024-03-12T05:21:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4c615e640 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67afc10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0a2030 tx=0x7fb4bc005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6de743ab1cb996b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4abfff640 1 -- 172.21.15.100:0/1569739847 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc005cd0 con 0x55d5b67a5f00 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4abfff640 1 -- 172.21.15.100:0/1569739847 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc09d060 con 0x55d5b67a5f00 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.870+0000 7fb4abfff640 1 -- 172.21.15.100:0/1569739847 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc099df0 con 0x55d5b67a5f00 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1569739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 msgr2=0x55d5b67afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67afc10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0a2030 tx=0x7fb4bc005240 comp rx=0 tx=0).stop 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1569739847 shutdown_connections 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b679fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a6820 0x55d5b67b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/1569739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a5f00 0x55d5b67afc10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1569739847 >> 172.21.15.100:0/1569739847 conn(0x55d5b67a0980 msgr2=0x55d5b679f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1569739847 shutdown_connections 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 -- 172.21.15.100:0/1569739847 wait complete. 2024-03-12T05:21:05.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 -- start start 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.871+0000 7fb4caf8b740 1 Processor -- start 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b68140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6814630 con 0x55d5b67a6820 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6814770 con 0x55d5b67a5f00 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5b6814930 con 0x55d5b67b3b60 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4caf8b740 10 client.0 _ll_register_callbacks cb 0x7fb4bc0d0ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c595d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b68140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c615e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c595d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67ba1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35696/0 (socket says 172.21.15.100:35696) 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c595d640 1 -- 172.21.15.100:0/2837223677 learned_addr learned my addr 172.21.15.100:0/2837223677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 -- 172.21.15.100:0/2837223677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 msgr2=0x55d5b67b9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67b9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 -- 172.21.15.100:0/2837223677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 msgr2=0x55d5b67ba1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67ba1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 -- 172.21.15.100:0/2837223677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5b670c6a0 con 0x55d5b67b3b60 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c595d640 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4c695f640 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b68140f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802c630 tx=0x7fb4b802c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30af3275b9776bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.872+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/2837223677 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8003d80 con 0x55d5b67b3b60 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4bc099b90 con 0x55d5b67b3b60 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/2837223677 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4b8004040 con 0x55d5b67b3b60 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/2837223677 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4b8004200 con 0x55d5b67b3b60 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 10 client.26779 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d5b66ce8c0 con 0x55d5b67b3b60 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/2837223677 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc099b90 con 0x55d5b67b3b60 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/2837223677 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4b805d070 con 0x55d5b67b3b60 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 10 client.26779 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 10 client.26779 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 20 client.26779 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b681c1f0 0x55d5b681e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 4 client.26779 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5b6499890 con 0x55d5b681c1f0 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.873+0000 7fb4c615e640 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b681c1f0 0x55d5b681e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4c615e640 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b681c1f0 0x55d5b681e610 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5aa133deb0814e server_cookie=2edf434b4bf9c9b4 in_seq=0 out_seq=0 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4a9ffb640 10 client.26779 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4a9ffb640 1 -- 172.21.15.100:0/2837223677 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5b6499890 con 0x55d5b681c1f0 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4a9ffb640 10 client.26779 handle_command_reply: tid=0 2024-03-12T05:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 client.26779 shutdown 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 msgr2=0x55d5b68140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b68140f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802c630 tx=0x7fb4b802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 shutdown_connections 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5b67b3b60 0x55d5b68140f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5b67a6820 0x55d5b67ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5b681c1f0 0x55d5b681e610 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 --2- 172.21.15.100:0/2837223677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5b67a5f00 0x55d5b67b9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 >> 172.21.15.100:0/2837223677 conn(0x55d5b67a0980 msgr2=0x55d5b679fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4a9ffb640 0 client.26779 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 shutdown_connections 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 1 -- 172.21.15.100:0/2837223677 wait complete. 2024-03-12T05:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4caf8b740 20 client.26779 trim_cache size 0 max 16384 2024-03-12T05:21:05.877 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4ac0a1ba0 msgr2=0x7fb4ac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4ac0a1ba0 0x7fb4ac0a4060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8005cc0 tx=0x7fb4b80095c0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 msgr2=0x7fb4c019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.874+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 0x7fb4c019d3f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d950 tx=0x7fb4bc00de10 comp rx=0 tx=0).stop 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 shutdown_connections 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb4ac0a1ba0 0x7fb4ac0a4060 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4c01975f0 0x7fb4c0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c0105330 0x7fb4c019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 --2- 172.21.15.100:0/1434309480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c0100260 0x7fb4c019d3f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 >> 172.21.15.100:0/1434309480 conn(0x7fb4c0077180 msgr2=0x7fb4c00fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 shutdown_connections 2024-03-12T05:21:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.875+0000 7fb4c83c2640 1 -- 172.21.15.100:0/1434309480 wait complete. 2024-03-12T05:21:05.881 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:21:05.881 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:21:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 -- 172.21.15.100:0/1834765978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270101d20 msgr2=0x7f6270105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 --2- 172.21.15.100:0/1834765978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270101d20 0x7f6270105d60 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6260009930 tx=0x7f626002bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 -- 172.21.15.100:0/1834765978 shutdown_connections 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 --2- 172.21.15.100:0/1834765978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6270106440 0x7f627010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 --2- 172.21.15.100:0/1834765978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270101d20 0x7f6270105d60 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 --2- 172.21.15.100:0/1834765978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6270101370 0x7f6270101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 -- 172.21.15.100:0/1834765978 >> 172.21.15.100:0/1834765978 conn(0x7f62700fcc10 msgr2=0x7f62700ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 -- 172.21.15.100:0/1834765978 shutdown_connections 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.997+0000 7f6277793640 1 -- 172.21.15.100:0/1834765978 wait complete. 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 Processor -- start 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 -- start start 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6270101370 0x7f627019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 0x7f627019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62701976c0 0x7f6270197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270198190 con 0x7f6270106440 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270198300 con 0x7f62701976c0 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6277793640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62701984c0 con 0x7f6270101370 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 0x7f627019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 0x7f627019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35722/0 (socket says 172.21.15.100:35722) 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6274d2e640 1 -- 172.21.15.100:0/1390531530 learned_addr learned my addr 172.21.15.100:0/1390531530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f6275d30640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62701976c0 0x7f6270197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.998+0000 7f627552f640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6270101370 0x7f627019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6274d2e640 1 -- 172.21.15.100:0/1390531530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6270101370 msgr2=0x7f627019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6274d2e640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6270101370 0x7f627019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6274d2e640 1 -- 172.21.15.100:0/1390531530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62701976c0 msgr2=0x7f6270197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6274d2e640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62701976c0 0x7f6270197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6274d2e640 1 -- 172.21.15.100:0/1390531530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260009560 con 0x7f6270106440 2024-03-12T05:21:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6274d2e640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 0x7f627019daf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6260009230 tx=0x7f6260004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f62667fc640 1 -- 172.21.15.100:0/1390531530 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600043d0 con 0x7f6270106440 2024-03-12T05:21:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f62667fc640 1 -- 172.21.15.100:0/1390531530 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62600052d0 con 0x7f6270106440 2024-03-12T05:21:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62701a43b0 con 0x7f6270106440 2024-03-12T05:21:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f62667fc640 1 -- 172.21.15.100:0/1390531530 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260003d80 con 0x7f6270106440 2024-03-12T05:21:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:05.999+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62701a4890 con 0x7f6270106440 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.001+0000 7f62667fc640 1 -- 172.21.15.100:0/1390531530 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6260005440 con 0x7f6270106440 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.001+0000 7f627a36a740 1 Processor -- start 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.001+0000 7f627a36a740 1 -- start start 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627a36a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd29630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627a36a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29b70 0x56114bd32210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627a36a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd34b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f62667fc640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f625c0a1ba0 0x7f625c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627a36a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bce4960 con 0x56114bd32750 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627a36a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bc98310 con 0x56114bd29b70 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627a36a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bc95840 con 0x56114bd29250 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd34b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29b70 0x56114bd32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6275d30640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f625c0a1ba0 0x7f625c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd34b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35726/0 (socket says 172.21.15.100:35726) 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6274d2e640 1 -- 172.21.15.100:0/2432287709 learned_addr learned my addr 172.21.15.100:0/2432287709 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f62667fc640 1 -- 172.21.15.100:0/1390531530 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62600ef880 con 0x7f6270106440 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6275d30640 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd29630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 -- 172.21.15.100:0/2432287709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 msgr2=0x56114bd29630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd29630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 -- 172.21.15.100:0/2432287709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 msgr2=0x56114bd34b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd34b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 -- 172.21.15.100:0/2432287709 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114bc53060 con 0x56114bd29b70 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6274d2e640 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd34b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f6275d30640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f625c0a1ba0 0x7f625c0a4060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6268005fd0 tx=0x7f6268009e80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.002+0000 7f627552f640 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29b70 0x56114bd32210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f626c004630 tx=0x7f626c00ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23719cbe620d3093 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f625b7fe640 1 -- 172.21.15.100:0/2432287709 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c003a90 con 0x56114bd29b70 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f625b7fe640 1 -- 172.21.15.100:0/2432287709 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f626c005500 con 0x56114bd29b70 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f625b7fe640 1 -- 172.21.15.100:0/2432287709 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c004d40 con 0x56114bd29b70 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 -- 172.21.15.100:0/2432287709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29b70 msgr2=0x56114bd32210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29b70 0x56114bd32210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f626c004630 tx=0x7f626c00ba60 comp rx=0 tx=0).stop 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 -- 172.21.15.100:0/2432287709 shutdown_connections 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd34b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29b70 0x56114bd32210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 --2- 172.21.15.100:0/2432287709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 -- 172.21.15.100:0/2432287709 >> 172.21.15.100:0/2432287709 conn(0x56114bd23980 msgr2=0x56114bd23d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 -- 172.21.15.100:0/2432287709 shutdown_connections 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.003+0000 7f627a36a740 1 -- 172.21.15.100:0/2432287709 wait complete. 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 -- start start 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 Processor -- start 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd32750 0x56114bd30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bc91020 con 0x56114bd29b70 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114b964430 con 0x56114bd32750 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114b920e00 con 0x56114bd29250 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f627a36a740 10 client.0 _ll_register_callbacks cb 0x56114bc38cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd2fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35734/0 (socket says 172.21.15.100:35734) 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6275d30640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd32750 0x56114bd30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 -- 172.21.15.100:0/3738182700 learned_addr learned my addr 172.21.15.100:0/3738182700 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 -- 172.21.15.100:0/3738182700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 msgr2=0x56114bd2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 -- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd32750 msgr2=0x56114bd30240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd32750 0x56114bd30240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.004+0000 7f6274d2e640 1 -- 172.21.15.100:0/3738182700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c004290 con 0x56114bd29b70 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6274d2e640 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd2fd00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6260000c00 tx=0x7f626008c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4272da08fac6ec5c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6258ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6258ff9640 1 -- 172.21.15.100:0/3738182700 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600bbee0 con 0x56114bd29b70 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6258ff9640 1 -- 172.21.15.100:0/3738182700 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62600bcc80 con 0x56114bd29b70 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56114b9907e0 con 0x56114bd29b70 2024-03-12T05:21:06.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6258ff9640 1 -- 172.21.15.100:0/3738182700 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62600bc580 con 0x56114bd29b70 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 10 client.17409 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56114bc8f6a0 con 0x56114bd29b70 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6258ff9640 1 -- 172.21.15.100:0/3738182700 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56114b9907e0 con 0x56114bd29b70 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f6258ff9640 1 -- 172.21.15.100:0/3738182700 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f626003c070 con 0x56114bd29b70 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 10 client.17409 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 20 client.17409 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd42940 0x56114bd44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 4 client.17409 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56114b9cd720 con 0x56114bd42940 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.005+0000 7f627552f640 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd42940 0x56114bd44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.006+0000 7f627552f640 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd42940 0x56114bd44d70 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ef43bdfad581c7 server_cookie=373d8dea776ac13f in_seq=0 out_seq=0 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.006+0000 7f6258ff9640 10 client.17409 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:06.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.006+0000 7f6258ff9640 1 -- 172.21.15.100:0/3738182700 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56114b9cd720 con 0x56114bd42940 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.006+0000 7f6258ff9640 10 client.17409 handle_command_reply: tid=0 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.006+0000 7f627a36a740 1 client.17409 shutdown 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 msgr2=0x56114bd2fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd2fd00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6260000c00 tx=0x7f626008c620 comp rx=0 tx=0).stop 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 shutdown_connections 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd42940 0x56114bd44d70 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd32750 0x56114bd30240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd2fd00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f6258ff9640 0 client.17409 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 --2- 172.21.15.100:0/3738182700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29250 0x56114bd2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 >> 172.21.15.100:0/3738182700 conn(0x56114bd23980 msgr2=0x56114bd256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 shutdown_connections 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 1 -- 172.21.15.100:0/3738182700 wait complete. 2024-03-12T05:21:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.007+0000 7f627a36a740 20 client.17409 trim_cache size 0 max 16384 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.019+0000 7f627a36a740 1 Processor -- start 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.019+0000 7f627a36a740 1 -- start start 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627a36a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd31ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627a36a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627a36a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd32750 0x56114bd2ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627a36a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bc16130 con 0x56114bd29b70 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627a36a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd234b0 con 0x56114bd29250 2024-03-12T05:21:06.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627a36a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd2dd10 con 0x56114bd32750 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd30060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd30060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35748/0 (socket says 172.21.15.100:35748) 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 -- 172.21.15.100:0/4173936735 learned_addr learned my addr 172.21.15.100:0/4173936735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6275d30640 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd32750 0x56114bd2ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f627552f640 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd31ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 -- 172.21.15.100:0/4173936735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd32750 msgr2=0x56114bd2ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd32750 0x56114bd2ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 -- 172.21.15.100:0/4173936735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 msgr2=0x56114bd31ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd31ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 -- 172.21.15.100:0/4173936735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114bc2bc20 con 0x56114bd29b70 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f6274d2e640 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd30060 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f62600ba830 tx=0x7f6260005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9636653b41891b53 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:06.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f625affd640 1 -- 172.21.15.100:0/4173936735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600bbee0 con 0x56114bd29b70 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f625affd640 1 -- 172.21.15.100:0/4173936735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62600bcc80 con 0x56114bd29b70 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.020+0000 7f625affd640 1 -- 172.21.15.100:0/4173936735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600dc0b0 con 0x56114bd29b70 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 -- 172.21.15.100:0/4173936735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 msgr2=0x56114bd30060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd30060 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f62600ba830 tx=0x7f6260005fd0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 -- 172.21.15.100:0/4173936735 shutdown_connections 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd32750 0x56114bd2ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd29b70 0x56114bd30060 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 --2- 172.21.15.100:0/4173936735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd31ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 -- 172.21.15.100:0/4173936735 >> 172.21.15.100:0/4173936735 conn(0x56114bd23980 msgr2=0x56114bd2aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 -- 172.21.15.100:0/4173936735 shutdown_connections 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 -- 172.21.15.100:0/4173936735 wait complete. 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 -- start start 2024-03-12T05:21:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.021+0000 7f627a36a740 1 Processor -- start 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd37fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29b70 0x56114bd38510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd38a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd9d2b0 con 0x56114bd32750 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd9d3f0 con 0x56114bd29250 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd9d530 con 0x56114bd29b70 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 10 client.0 _ll_register_callbacks cb 0x7f626c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd38a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd38a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35750/0 (socket says 172.21.15.100:35750) 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 -- 172.21.15.100:0/294806969 learned_addr learned my addr 172.21.15.100:0/294806969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627552f640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd37fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6274d2e640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29b70 0x56114bd38510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 -- 172.21.15.100:0/294806969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29b70 msgr2=0x56114bd38510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29b70 0x56114bd38510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 -- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 msgr2=0x56114bd37fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd37fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 -- 172.21.15.100:0/294806969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114bc8f6a0 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6275d30640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd38a50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f626800ffd0 tx=0x7f6268009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943be1db7de4ff8b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6258ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6258ff9640 1 -- 172.21.15.100:0/294806969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268016040 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f6258ff9640 1 -- 172.21.15.100:0/294806969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62680045b0 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.022+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f626c005500 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f6258ff9640 1 -- 172.21.15.100:0/294806969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6268004770 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 10 client.17415 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56114bc518c0 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f6258ff9640 1 -- 172.21.15.100:0/294806969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f626c005500 con 0x56114bd32750 2024-03-12T05:21:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f6258ff9640 1 -- 172.21.15.100:0/294806969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f626802a3f0 con 0x56114bd32750 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 10 client.17415 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 20 client.17415 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd9ed20 0x56114bda1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 4 client.17415 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56114ba1c890 con 0x56114bd9ed20 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627552f640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd9ed20 0x56114bda1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f6258ff9640 10 client.17415 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.023+0000 7f627552f640 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd9ed20 0x56114bda1140 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6262cafefe70d7e0 server_cookie=5b9d75a70eb9dbd8 in_seq=0 out_seq=0 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6258ff9640 1 -- 172.21.15.100:0/294806969 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56114ba1c890 con 0x56114bd9ed20 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6258ff9640 10 client.17415 handle_command_reply: tid=0 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 client.17415 shutdown 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 msgr2=0x56114bd38a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd38a50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f626800ffd0 tx=0x7f6268009500 comp rx=0 tx=0).stop 2024-03-12T05:21:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 shutdown_connections 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56114bd9ed20 0x56114bda1140 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd32750 0x56114bd38a50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56114bd29b70 0x56114bd38510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6258ff9640 0 client.17415 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 --2- 172.21.15.100:0/294806969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56114bd29250 0x56114bd37fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 >> 172.21.15.100:0/294806969 conn(0x56114bd23980 msgr2=0x56114bd222f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 shutdown_connections 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 1 -- 172.21.15.100:0/294806969 wait complete. 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f627a36a740 20 client.17415 trim_cache size 0 max 16384 2024-03-12T05:21:06.026 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:21:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f625c0a1ba0 msgr2=0x7f625c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f625c0a1ba0 0x7f625c0a4060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6268005fd0 tx=0x7f6268009e80 comp rx=0 tx=0).stop 2024-03-12T05:21:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 msgr2=0x7f627019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 0x7f627019daf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6260009230 tx=0x7f6260004c00 comp rx=0 tx=0).stop 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 shutdown_connections 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f625c0a1ba0 0x7f625c0a4060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62701976c0 0x7f6270197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6270106440 0x7f627019daf0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.024+0000 7f6277793640 1 --2- 172.21.15.100:0/1390531530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6270101370 0x7f627019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.025+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 >> 172.21.15.100:0/1390531530 conn(0x7f62700fcc10 msgr2=0x7f62700fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.025+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 shutdown_connections 2024-03-12T05:21:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:06.025+0000 7f6277793640 1 -- 172.21.15.100:0/1390531530 wait complete. 2024-03-12T05:21:06.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:05 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2777622916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:21:06.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:05 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2777622916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:21:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:05 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2777622916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:21:06.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1082225 12.01 MB/sec execute 350 sec latency 28.281 ms 2024-03-12T05:21:07.031 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:21:07.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:21:07.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:21:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 -- 172.21.15.100:0/2888393101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 msgr2=0x7f60bc100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2888393101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc100640 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009930 tx=0x7f60b802bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 -- 172.21.15.100:0/2888393101 shutdown_connections 2024-03-12T05:21:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2888393101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60bc105330 0x7f60bc10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2888393101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60bc100c10 0x7f60bc104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2888393101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc100640 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 -- 172.21.15.100:0/2888393101 >> 172.21.15.100:0/2888393101 conn(0x7f60bc077180 msgr2=0x7f60bc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 -- 172.21.15.100:0/2888393101 shutdown_connections 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 -- 172.21.15.100:0/2888393101 wait complete. 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 Processor -- start 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.150+0000 7f60c39a1640 1 -- start start 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c39a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c39a1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60bc105330 0x7f60bc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c39a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60bc199870 0x7f60bc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c39a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc19a320 con 0x7f60bc100260 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c39a1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc19a490 con 0x7f60bc105330 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c39a1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc19a600 con 0x7f60bc199870 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35768/0 (socket says 172.21.15.100:35768) 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 -- 172.21.15.100:0/2592968865 learned_addr learned my addr 172.21.15.100:0/2592968865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60bc105330 0x7f60bc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60bc199870 0x7f60bc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 -- 172.21.15.100:0/2592968865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60bc199870 msgr2=0x7f60bc199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60bc199870 0x7f60bc199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 -- 172.21.15.100:0/2592968865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60bc105330 msgr2=0x7f60bc19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60bc105330 0x7f60bc19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 -- 172.21.15.100:0/2592968865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009560 con 0x7f60bc100260 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60c173d640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc19f6b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009230 tx=0x7f60b802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.151+0000 7f60b27fc640 1 -- 172.21.15.100:0/2592968865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8004350 con 0x7f60bc100260 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.152+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc1a6500 con 0x7f60bc100260 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.152+0000 7f60b27fc640 1 -- 172.21.15.100:0/2592968865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b80044b0 con 0x7f60bc100260 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.152+0000 7f60b27fc640 1 -- 172.21.15.100:0/2592968865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8003d70 con 0x7f60bc100260 2024-03-12T05:21:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.152+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc1a69e0 con 0x7f60bc100260 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.154+0000 7f60b27fc640 1 -- 172.21.15.100:0/2592968865 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f60b8004df0 con 0x7f60bc100260 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.154+0000 7f60c657a740 1 Processor -- start 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.154+0000 7f60c657a740 1 -- start start 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.154+0000 7f60c657a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 0x55af9590c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.154+0000 7f60c657a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590cd30 0x55af959152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c657a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95917d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60b27fc640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f60a80a1ba0 0x7f60a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c657a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9587af90 con 0x55af959158b0 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c657a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af958784c0 con 0x55af9590cd30 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c657a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af95873ca0 con 0x55af9590c380 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 0x55af9590c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 0x55af9590c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51494/0 (socket says 172.21.15.100:51494) 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2596228404 learned_addr learned my addr 172.21.15.100:0/2596228404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c173d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590cd30 0x55af959152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f60a80a1ba0 0x7f60a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95917d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60b27fc640 1 -- 172.21.15.100:0/2592968865 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f60b80ef7c0 con 0x7f60bc100260 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2596228404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590cd30 msgr2=0x55af959152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590cd30 0x55af959152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2596228404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 msgr2=0x55af95917d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95917d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2596228404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af958360b0 con 0x55af9590c380 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 0x55af9590c760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f60b4012cf0 tx=0x7f60b4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fecac271682020cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f60a80a1ba0 0x7f60a80a4060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60b4010c40 tx=0x7f60b4017000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.155+0000 7f60a77fe640 1 -- 172.21.15.100:0/2596228404 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4012de0 con 0x55af9590c380 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60a77fe640 1 -- 172.21.15.100:0/2596228404 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b4009cd0 con 0x55af9590c380 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60a77fe640 1 -- 172.21.15.100:0/2596228404 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4003970 con 0x55af9590c380 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 -- 172.21.15.100:0/2596228404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 msgr2=0x55af9590c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 0x55af9590c760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f60b4012cf0 tx=0x7f60b4010740 comp rx=0 tx=0).stop 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 -- 172.21.15.100:0/2596228404 shutdown_connections 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95917d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590cd30 0x55af959152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 --2- 172.21.15.100:0/2596228404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590c380 0x55af9590c760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 -- 172.21.15.100:0/2596228404 >> 172.21.15.100:0/2596228404 conn(0x55af95906840 msgr2=0x55af95906c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 -- 172.21.15.100:0/2596228404 shutdown_connections 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.156+0000 7f60c657a740 1 -- 172.21.15.100:0/2596228404 wait complete. 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 -- start start 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 Processor -- start 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590c380 0x55af959128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95912e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95913350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af95818f40 con 0x55af959158b0 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af955474b0 con 0x55af9590c380 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af957f9670 con 0x55af9590cd30 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c657a740 10 client.0 _ll_register_callbacks cb 0x55af95866f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:07.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95913350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95913350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35782/0 (socket says 172.21.15.100:35782) 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2544996293 learned_addr learned my addr 172.21.15.100:0/2544996293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c173d640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590c380 0x55af959128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95912e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2544996293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 msgr2=0x55af95912e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95912e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590c380 msgr2=0x55af959128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590c380 0x55af959128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c1f3e640 1 -- 172.21.15.100:0/2544996293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b401a040 con 0x55af959158b0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.157+0000 7f60c173d640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590c380 0x55af959128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95913350 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f60b40126b0 tx=0x7f60b40129a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c295427ca02e4592 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60a4ff9640 1 -- 172.21.15.100:0/2544996293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4012100 con 0x55af959158b0 2024-03-12T05:21:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60a4ff9640 1 -- 172.21.15.100:0/2544996293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f60b400dcc0 con 0x55af959158b0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af9584df40 con 0x55af959158b0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60a4ff9640 1 -- 172.21.15.100:0/2544996293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b400de80 con 0x55af959158b0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 10 client.17427 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55af95872320 con 0x55af959158b0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60a4ff9640 1 -- 172.21.15.100:0/2544996293 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55af9584df40 con 0x55af959158b0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60a4ff9640 1 -- 172.21.15.100:0/2544996293 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60b401e520 con 0x55af959158b0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 10 client.17427 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 20 client.17427 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af95925ef0 0x55af95928310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 4 client.17427 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.158+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af9586a310 con 0x55af95925ef0 2024-03-12T05:21:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.159+0000 7f60c173d640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af95925ef0 0x55af95928310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.159+0000 7f60c173d640 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af95925ef0 0x55af95928310 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923436df7b2e9c9e server_cookie=b48ed74fc97af386 in_seq=0 out_seq=0 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.159+0000 7f60a4ff9640 10 client.17427 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.159+0000 7f60a4ff9640 1 -- 172.21.15.100:0/2544996293 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af9586a310 con 0x55af95925ef0 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.159+0000 7f60a4ff9640 10 client.17427 handle_command_reply: tid=0 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.159+0000 7f60c657a740 1 client.17427 shutdown 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 msgr2=0x55af95913350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95913350 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f60b40126b0 tx=0x7f60b40129a0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 shutdown_connections 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af95925ef0 0x55af95928310 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af959158b0 0x55af95913350 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95912e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 --2- 172.21.15.100:0/2544996293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af9590c380 0x55af959128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 >> 172.21.15.100:0/2544996293 conn(0x55af95906840 msgr2=0x55af959087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60a4ff9640 0 client.17427 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 shutdown_connections 2024-03-12T05:21:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 1 -- 172.21.15.100:0/2544996293 wait complete. 2024-03-12T05:21:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.160+0000 7f60c657a740 20 client.17427 trim_cache size 0 max 16384 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 Processor -- start 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 -- start start 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af95915000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95913190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9590fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9586a340 con 0x55af9590c380 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9584df70 con 0x55af959158b0 2024-03-12T05:21:07.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.175+0000 7f60c657a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af95907640 con 0x55af9590cd30 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af95915000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af95915000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35786/0 (socket says 172.21.15.100:35786) 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c0f3c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95913190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 -- 172.21.15.100:0/1265312799 learned_addr learned my addr 172.21.15.100:0/1265312799 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9590fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 -- 172.21.15.100:0/1265312799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 msgr2=0x55af95913190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95913190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 -- 172.21.15.100:0/1265312799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 msgr2=0x55af9590fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9590fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 -- 172.21.15.100:0/1265312799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af9580ec00 con 0x55af9590c380 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95913190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60c173d640 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af95915000 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009430 tx=0x7f60b808e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdd6faac82a4be2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60a6ffd640 1 -- 172.21.15.100:0/1265312799 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80bcbc0 con 0x55af9590c380 2024-03-12T05:21:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60a6ffd640 1 -- 172.21.15.100:0/1265312799 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b80dc0b0 con 0x55af9590c380 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.176+0000 7f60a6ffd640 1 -- 172.21.15.100:0/1265312799 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80c4c60 con 0x55af9590c380 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 -- 172.21.15.100:0/1265312799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 msgr2=0x55af95915000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af95915000 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009430 tx=0x7f60b808e6c0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 -- 172.21.15.100:0/1265312799 shutdown_connections 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9590fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af95913190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 --2- 172.21.15.100:0/1265312799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af95915000 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 -- 172.21.15.100:0/1265312799 >> 172.21.15.100:0/1265312799 conn(0x55af95906840 msgr2=0x55af95911200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 -- 172.21.15.100:0/1265312799 shutdown_connections 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 -- 172.21.15.100:0/1265312799 wait complete. 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 -- start start 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.177+0000 7f60c657a740 1 Processor -- start 2024-03-12T05:21:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af9591cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af9591b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9591b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9591bd50 con 0x55af9590c380 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9591bec0 con 0x55af959158b0 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9597c400 con 0x55af9590cd30 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 10 client.0 _ll_register_callbacks cb 0x7f60b4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af9591cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af9591cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35792/0 (socket says 172.21.15.100:35792) 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 -- 172.21.15.100:0/1405922486 learned_addr learned my addr 172.21.15.100:0/1405922486 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c1f3e640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9591b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af9591b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 -- 172.21.15.100:0/1405922486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 msgr2=0x55af9591b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af9591b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 -- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 msgr2=0x55af9591b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9591b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 -- 172.21.15.100:0/1405922486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac0099e0 con 0x55af9590c380 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c173d640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af9591cc90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f60b80b8920 tx=0x7f60b80c3d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0184123d05c08d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60a4ff9640 1 -- 172.21.15.100:0/1405922486 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80bcbc0 con 0x55af9590c380 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60a4ff9640 1 -- 172.21.15.100:0/1405922486 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f60b80c4c60 con 0x55af9590c380 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.178+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af95899900 con 0x55af9590c380 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60a4ff9640 1 -- 172.21.15.100:0/1405922486 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b80c0610 con 0x55af9590c380 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 10 client.17433 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f60bc1a69e0 con 0x55af9590c380 2024-03-12T05:21:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60a4ff9640 1 -- 172.21.15.100:0/1405922486 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b80b0020 con 0x55af9590c380 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60a4ff9640 1 -- 172.21.15.100:0/1405922486 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60b803c070 con 0x55af9590c380 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 10 client.17433 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 20 client.17433 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af9597d840 0x55af9597fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 4 client.17433 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55af95573860 con 0x55af9597d840 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af9597d840 0x55af9597fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60c0f3c640 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af9597d840 0x55af9597fc30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece5ede623373b9e server_cookie=fb3b2fb6688a0217 in_seq=0 out_seq=0 2024-03-12T05:21:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.179+0000 7f60a4ff9640 10 client.17433 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60a4ff9640 1 -- 172.21.15.100:0/1405922486 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55af95573860 con 0x55af9597d840 2024-03-12T05:21:07.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60a4ff9640 10 client.17433 handle_command_reply: tid=0 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 client.17433 shutdown 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 msgr2=0x55af9591cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af9591cc90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f60b80b8920 tx=0x7f60b80c3d90 comp rx=0 tx=0).stop 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 shutdown_connections 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55af9597d840 0x55af9597fc30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af959158b0 0x55af9591b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af9590cd30 0x55af9591b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60a4ff9640 0 client.17433 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 --2- 172.21.15.100:0/1405922486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af9590c380 0x55af9591cc90 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 >> 172.21.15.100:0/1405922486 conn(0x55af95906840 msgr2=0x55af95907e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 shutdown_connections 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 1 -- 172.21.15.100:0/1405922486 wait complete. 2024-03-12T05:21:07.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c657a740 20 client.17433 trim_cache size 0 max 16384 2024-03-12T05:21:07.183 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:21:07.183 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:21:07.183 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "77a69349-a8c0-463b-9863-794a6552b8ee", 2024-03-12T05:21:07.183 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:21:07.183 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f60a80a1ba0 msgr2=0x7f60a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f60a80a1ba0 0x7f60a80a4060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60b4010c40 tx=0x7f60b4017000 comp rx=0 tx=0).stop 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 msgr2=0x7f60bc19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.181+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc19f6b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009230 tx=0x7f60b802c6c0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 shutdown_connections 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f60a80a1ba0 0x7f60a80a4060 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60bc199870 0x7f60bc199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60bc105330 0x7f60bc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 --2- 172.21.15.100:0/2592968865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60bc100260 0x7f60bc19f6b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 >> 172.21.15.100:0/2592968865 conn(0x7f60bc077180 msgr2=0x7f60bc068380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 shutdown_connections 2024-03-12T05:21:07.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.182+0000 7f60c39a1640 1 -- 172.21.15.100:0/2592968865 wait complete. 2024-03-12T05:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:06 smithi100 ceph-mon[32210]: pgmap v277: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-03-12T05:21:07.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:21:07.188 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:21:07.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:06 smithi156 ceph-mon[36270]: pgmap v277: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-03-12T05:21:07.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:06 smithi161 ceph-mon[36043]: pgmap v277: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 -- 172.21.15.100:0/113477089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c106430 msgr2=0x7fc87c10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 --2- 172.21.15.100:0/113477089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c106430 0x7fc87c10e000 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc874009930 tx=0x7fc87402bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 -- 172.21.15.100:0/113477089 shutdown_connections 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 --2- 172.21.15.100:0/113477089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c106430 0x7fc87c10e000 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 --2- 172.21.15.100:0/113477089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc87c101d10 0x7fc87c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 --2- 172.21.15.100:0/113477089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc87c101360 0x7fc87c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 -- 172.21.15.100:0/113477089 >> 172.21.15.100:0/113477089 conn(0x7fc87c0fcc00 msgr2=0x7fc87c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 -- 172.21.15.100:0/113477089 shutdown_connections 2024-03-12T05:21:07.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 -- 172.21.15.100:0/113477089 wait complete. 2024-03-12T05:21:07.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 Processor -- start 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.302+0000 7fc880adf640 1 -- start start 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 0x7fc87c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc87c106430 0x7fc87c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc87c1975e0 0x7fc87c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c197fd0 con 0x7fc87c101360 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c198140 con 0x7fc87c106430 2024-03-12T05:21:07.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c198300 con 0x7fc87c1975e0 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 0x7fc87c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 0x7fc87c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35810/0 (socket says 172.21.15.100:35810) 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 -- 172.21.15.100:0/1899210527 learned_addr learned my addr 172.21.15.100:0/1899210527 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc87c1975e0 0x7fc87c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc879d9b640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc87c106430 0x7fc87c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 -- 172.21.15.100:0/1899210527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc87c1975e0 msgr2=0x7fc87c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc87c1975e0 0x7fc87c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 -- 172.21.15.100:0/1899210527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc87c106430 msgr2=0x7fc87c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc87c106430 0x7fc87c19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 -- 172.21.15.100:0/1899210527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874009560 con 0x7fc87c101360 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc87a59c640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 0x7fc87c19d430 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ebc0 tx=0x7fc86c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc86b7fe640 1 -- 172.21.15.100:0/1899210527 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0099b0 con 0x7fc87c101360 2024-03-12T05:21:07.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc86b7fe640 1 -- 172.21.15.100:0/1899210527 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc86c00eec0 con 0x7fc87c101360 2024-03-12T05:21:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c1a42f0 con 0x7fc87c101360 2024-03-12T05:21:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.303+0000 7fc86b7fe640 1 -- 172.21.15.100:0/1899210527 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c018b60 con 0x7fc87c101360 2024-03-12T05:21:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.304+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c1a4800 con 0x7fc87c101360 2024-03-12T05:21:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc86b7fe640 1 -- 172.21.15.100:0/1899210527 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc86c018cc0 con 0x7fc87c101360 2024-03-12T05:21:07.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 Processor -- start 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 -- start start 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef180690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef189270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef18bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef0ef280 con 0x562bef1897b0 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef0ec7b0 con 0x562bef180bd0 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc883794740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef0e7f90 con 0x562bef1802b0 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc86b7fe640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8640a1ba0 0x7fc8640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef189270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc879d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef18bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef189270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37526/0 (socket says 172.21.15.100:37526) 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc87a59c640 1 -- 172.21.15.100:0/2913010277 learned_addr learned my addr 172.21.15.100:0/2913010277 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.306+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef180690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8640a1ba0 0x7fc8640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc86b7fe640 1 -- 172.21.15.100:0/1899210527 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc86c014070 con 0x7fc87c101360 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87a59c640 1 -- 172.21.15.100:0/2913010277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 msgr2=0x562bef180690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87a59c640 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef180690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87a59c640 1 -- 172.21.15.100:0/2913010277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 msgr2=0x562bef18bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87a59c640 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef18bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87a59c640 1 -- 172.21.15.100:0/2913010277 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bef0a6570 con 0x562bef180bd0 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc879d9b640 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef18bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8640a1ba0 0x7fc8640a4060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc874005940 tx=0x7fc8740058d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc87a59c640 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef189270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00eb90 tx=0x7fc86c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b0842002a33c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc868ff9640 1 -- 172.21.15.100:0/2913010277 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c09aaf0 con 0x562bef180bd0 2024-03-12T05:21:07.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc868ff9640 1 -- 172.21.15.100:0/2913010277 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc86c09eb90 con 0x562bef180bd0 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.307+0000 7fc868ff9640 1 -- 172.21.15.100:0/2913010277 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0a2fb0 con 0x562bef180bd0 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 -- 172.21.15.100:0/2913010277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 msgr2=0x562bef189270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef189270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00eb90 tx=0x7fc86c00cf90 comp rx=0 tx=0).stop 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 -- 172.21.15.100:0/2913010277 shutdown_connections 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef18bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef189270 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 --2- 172.21.15.100:0/2913010277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef180690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 -- 172.21.15.100:0/2913010277 >> 172.21.15.100:0/2913010277 conn(0x562bef17a990 msgr2=0x562bef17ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 -- 172.21.15.100:0/2913010277 shutdown_connections 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 -- 172.21.15.100:0/2913010277 wait complete. 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 -- start start 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.308+0000 7fc883794740 1 Processor -- start 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef189210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef180bd0 0x562bef1865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef186cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562beedbb430 con 0x562bef180bd0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562beed77e00 con 0x562bef1897b0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef06cfd0 con 0x562bef1802b0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 10 client.0 _ll_register_callbacks cb 0x562bef0c2fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef189210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef180bd0 0x562bef1865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef189210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51542/0 (socket says 172.21.15.100:51542) 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc87a59c640 1 -- 172.21.15.100:0/3957305827 learned_addr learned my addr 172.21.15.100:0/3957305827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef186cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 -- 172.21.15.100:0/3957305827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 msgr2=0x562bef189210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef189210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 -- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 msgr2=0x562bef186cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef186cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 -- 172.21.15.100:0/3957305827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c1a4800 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc87a59c640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef189210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc879d9b640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef180bd0 0x562bef1865a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc87c100310 tx=0x7fc87000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6d444d8d8bd0d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc861ffb640 1 -- 172.21.15.100:0/3957305827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870016070 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc861ffb640 1 -- 172.21.15.100:0/3957305827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc870005020 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.309+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562beede77e0 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc861ffb640 1 -- 172.21.15.100:0/3957305827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc870003eb0 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 10 client.17445 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562bef0e6610 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc861ffb640 1 -- 172.21.15.100:0/3957305827 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562beede77e0 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc861ffb640 1 -- 172.21.15.100:0/3957305827 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc870021070 con 0x562bef180bd0 2024-03-12T05:21:07.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 10 client.17445 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 20 client.17445 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1998f0 0x562bef19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 4 client.17445 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562beee24720 con 0x562bef1998f0 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.310+0000 7fc87a59c640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1998f0 0x562bef19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.311+0000 7fc87a59c640 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1998f0 0x562bef19bd10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f991154e61db2652 server_cookie=3c02a7a93490f45a in_seq=0 out_seq=0 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.311+0000 7fc861ffb640 10 client.17445 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.311+0000 7fc861ffb640 1 -- 172.21.15.100:0/3957305827 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562beee24720 con 0x562bef1998f0 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.311+0000 7fc861ffb640 10 client.17445 handle_command_reply: tid=0 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.311+0000 7fc883794740 1 client.17445 shutdown 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef180bd0 msgr2=0x562bef1865a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef180bd0 0x562bef1865a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc87c100310 tx=0x7fc87000bda0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 shutdown_connections 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1998f0 0x562bef19bd10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef186cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef180bd0 0x562bef1865a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc861ffb640 0 client.17445 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 --2- 172.21.15.100:0/3957305827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef189210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 >> 172.21.15.100:0/3957305827 conn(0x562bef17a990 msgr2=0x562bef17c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 shutdown_connections 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 1 -- 172.21.15.100:0/3957305827 wait complete. 2024-03-12T05:21:07.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.312+0000 7fc883794740 20 client.17445 trim_cache size 0 max 16384 2024-03-12T05:21:07.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 Processor -- start 2024-03-12T05:21:07.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 -- start start 2024-03-12T05:21:07.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef185580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef1905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef190ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef17b2d0 con 0x562bef1897b0 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef17b560 con 0x562bef180bd0 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc883794740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef1762b0 con 0x562bef1802b0 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef190ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef190ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35830/0 (socket says 172.21.15.100:35830) 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 -- 172.21.15.100:0/1683051136 learned_addr learned my addr 172.21.15.100:0/1683051136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc879d9b640 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef1905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87a59c640 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef185580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 -- 172.21.15.100:0/1683051136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 msgr2=0x562bef185580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef185580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 -- 172.21.15.100:0/1683051136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 msgr2=0x562bef1905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef1905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.327+0000 7fc87ad9d640 1 -- 172.21.15.100:0/1683051136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bef082ac0 con 0x562bef1897b0 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef190ae0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc874005c40 tx=0x7fc874005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f9d65048e43079 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc863fff640 1 -- 172.21.15.100:0/1683051136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874048040 con 0x562bef1897b0 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc863fff640 1 -- 172.21.15.100:0/1683051136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc874049040 con 0x562bef1897b0 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc863fff640 1 -- 172.21.15.100:0/1683051136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874003a10 con 0x562bef1897b0 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc883794740 1 -- 172.21.15.100:0/1683051136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 msgr2=0x562bef190ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc883794740 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef190ae0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc874005c40 tx=0x7fc874005e50 comp rx=0 tx=0).stop 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc883794740 1 -- 172.21.15.100:0/1683051136 shutdown_connections 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc883794740 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1897b0 0x562bef190ae0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc883794740 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef180bd0 0x562bef1905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.328+0000 7fc883794740 1 --2- 172.21.15.100:0/1683051136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef1802b0 0x562bef185580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- 172.21.15.100:0/1683051136 >> 172.21.15.100:0/1683051136 conn(0x562bef17a990 msgr2=0x562bef184e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- 172.21.15.100:0/1683051136 shutdown_connections 2024-03-12T05:21:07.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- 172.21.15.100:0/1683051136 wait complete. 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- start start 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 Processor -- start 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 0x562bef1ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef180bd0 0x562bef1ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef18f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef18fe50 con 0x562bef1802b0 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef18ffc0 con 0x562bef1897b0 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bef190180 con 0x562bef180bd0 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc883794740 10 client.0 _ll_register_callbacks cb 0x7fc86c0966f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 0x562bef1ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc87a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 0x562bef1ea2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35836/0 (socket says 172.21.15.100:35836) 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc87a59c640 1 -- 172.21.15.100:0/2956625514 learned_addr learned my addr 172.21.15.100:0/2956625514 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:07.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc87ad9d640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef18f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.329+0000 7fc879d9b640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef180bd0 0x562bef1ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc87a59c640 1 -- 172.21.15.100:0/2956625514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef180bd0 msgr2=0x562bef1ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc87a59c640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef180bd0 0x562bef1ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc87a59c640 1 -- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 msgr2=0x562bef18f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc87a59c640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef18f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc87a59c640 1 -- 172.21.15.100:0/2956625514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c10e810 con 0x562bef1802b0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc87a59c640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 0x562bef1ea2b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc86c098da0 tx=0x7fc86c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d426a0df97804935 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc861ffb640 1 -- 172.21.15.100:0/2956625514 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0a2f50 con 0x562bef1802b0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc861ffb640 1 -- 172.21.15.100:0/2956625514 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc86c097160 con 0x562bef1802b0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc86c09eb90 con 0x562bef1802b0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc861ffb640 1 -- 172.21.15.100:0/2956625514 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc86c09aaf0 con 0x562bef1802b0 2024-03-12T05:21:07.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 10 client.17451 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562bef0a86a0 con 0x562bef1802b0 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc861ffb640 1 -- 172.21.15.100:0/2956625514 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc86c09eb90 con 0x562bef1802b0 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc861ffb640 1 -- 172.21.15.100:0/2956625514 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc86c099b30 con 0x562bef1802b0 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 10 client.17451 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 20 client.17451 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1f1ef0 0x562bef1f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 4 client.17451 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.330+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562beee73890 con 0x562bef1f1ef0 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc879d9b640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1f1ef0 0x562bef1f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc879d9b640 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1f1ef0 0x562bef1f4310 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ed8e8d7e9e53ea server_cookie=df204814843b0a6f in_seq=0 out_seq=0 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc861ffb640 10 client.17451 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc861ffb640 1 -- 172.21.15.100:0/2956625514 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562beee73890 con 0x562bef1f1ef0 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc861ffb640 10 client.17451 handle_command_reply: tid=0 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 client.17451 shutdown 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 msgr2=0x562bef1ea2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 0x562bef1ea2b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc86c098da0 tx=0x7fc86c000f40 comp rx=0 tx=0).stop 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 shutdown_connections 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562bef1f1ef0 0x562bef1f4310 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bef1897b0 0x562bef18f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562bef180bd0 0x562bef1ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc861ffb640 0 client.17451 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 --2- 172.21.15.100:0/2956625514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562bef1802b0 0x562bef1ea2b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 >> 172.21.15.100:0/2956625514 conn(0x562bef17a990 msgr2=0x562bef1791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 shutdown_connections 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.331+0000 7fc883794740 1 -- 172.21.15.100:0/2956625514 wait complete. 2024-03-12T05:21:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc883794740 20 client.17451 trim_cache size 0 max 16384 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: "77a69349-a8c0-463b-9863-794a6552b8ee": { 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "77a69349-a8c0-463b-9863-794a6552b8ee", 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:21:07.334 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:21:07.335 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:21:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8640a1ba0 msgr2=0x7fc8640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8640a1ba0 0x7fc8640a4060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc874005940 tx=0x7fc8740058d0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 msgr2=0x7fc87c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 0x7fc87c19d430 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ebc0 tx=0x7fc86c00c6a0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 shutdown_connections 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8640a1ba0 0x7fc8640a4060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc87c1975e0 0x7fc87c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc87c106430 0x7fc87c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 --2- 172.21.15.100:0/1899210527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc87c101360 0x7fc87c19d430 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 >> 172.21.15.100:0/1899210527 conn(0x7fc87c0fcc00 msgr2=0x7fc87c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 shutdown_connections 2024-03-12T05:21:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:07.332+0000 7fc880adf640 1 -- 172.21.15.100:0/1899210527 wait complete. 2024-03-12T05:21:07.337 INFO:tasks.cephfs.filesystem:scrub status for tag:77a69349-a8c0-463b-9863-794a6552b8ee - {'path': '/', 'tag': '77a69349-a8c0-463b-9863-794a6552b8ee', 'options': 'recursive,force'} 2024-03-12T05:21:07.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1084137 12.01 MB/sec execute 351 sec latency 20.735 ms 2024-03-12T05:21:08.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:07 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:21:08.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:07 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:07 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:21:08.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:07 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:21:08.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:07 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:07 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:07 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:21:08.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:07 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:07 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:21:08.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:07 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:21:08.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:07 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:07 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:07 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:21:08.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:07 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:07 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:21:08.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:07 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:21:08.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:07 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:07 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:08.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1086016 12.00 MB/sec execute 352 sec latency 48.906 ms 2024-03-12T05:21:09.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:08 smithi156 ceph-mon[36270]: pgmap v278: 97 pgs: 97 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 331 op/s 2024-03-12T05:21:09.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:08 smithi161 ceph-mon[36043]: pgmap v278: 97 pgs: 97 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 331 op/s 2024-03-12T05:21:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:08 smithi100 ceph-mon[32210]: pgmap v278: 97 pgs: 97 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 331 op/s 2024-03-12T05:21:09.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1088144 12.01 MB/sec execute 353 sec latency 16.852 ms 2024-03-12T05:21:10.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1090743 12.02 MB/sec execute 354 sec latency 29.201 ms 2024-03-12T05:21:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:10 smithi156 ceph-mon[36270]: pgmap v279: 97 pgs: 97 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 229 op/s 2024-03-12T05:21:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:10 smithi156 ceph-mon[36270]: osdmap e94: 12 total, 12 up, 12 in 2024-03-12T05:21:11.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:10 smithi161 ceph-mon[36043]: pgmap v279: 97 pgs: 97 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 229 op/s 2024-03-12T05:21:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:10 smithi161 ceph-mon[36043]: osdmap e94: 12 total, 12 up, 12 in 2024-03-12T05:21:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:10 smithi100 ceph-mon[32210]: pgmap v279: 97 pgs: 97 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 229 op/s 2024-03-12T05:21:11.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:10 smithi100 ceph-mon[32210]: osdmap e94: 12 total, 12 up, 12 in 2024-03-12T05:21:11.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1092489 12.00 MB/sec execute 355 sec latency 30.167 ms 2024-03-12T05:21:12.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1094854 12.01 MB/sec execute 356 sec latency 14.878 ms 2024-03-12T05:21:13.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:12 smithi156 ceph-mon[36270]: pgmap v281: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 239 op/s 2024-03-12T05:21:13.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:12 smithi161 ceph-mon[36043]: pgmap v281: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 239 op/s 2024-03-12T05:21:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:12 smithi100 ceph-mon[32210]: pgmap v281: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 239 op/s 2024-03-12T05:21:13.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1096946 12.01 MB/sec execute 357 sec latency 61.225 ms 2024-03-12T05:21:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1098985 12.01 MB/sec execute 358 sec latency 15.124 ms 2024-03-12T05:21:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:14 smithi156 ceph-mon[36270]: pgmap v282: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 239 op/s 2024-03-12T05:21:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:14 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:21:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:14 smithi161 ceph-mon[36043]: pgmap v282: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 239 op/s 2024-03-12T05:21:15.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:14 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:21:15.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:14 smithi100 ceph-mon[32210]: pgmap v282: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 239 op/s 2024-03-12T05:21:15.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:14 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:21:15.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1101438 12.01 MB/sec execute 359 sec latency 13.766 ms 2024-03-12T05:21:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:15 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:21:16.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:15 smithi156 ceph-mon[36270]: osdmap e95: 12 total, 12 up, 12 in 2024-03-12T05:21:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:15 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:21:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:15 smithi161 ceph-mon[36043]: osdmap e95: 12 total, 12 up, 12 in 2024-03-12T05:21:16.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:15 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:21:16.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:15 smithi100 ceph-mon[32210]: osdmap e95: 12 total, 12 up, 12 in 2024-03-12T05:21:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1103344 12.00 MB/sec execute 360 sec latency 26.073 ms 2024-03-12T05:21:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:16 smithi156 ceph-mon[36270]: pgmap v284: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 150 op/s 2024-03-12T05:21:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:16 smithi161 ceph-mon[36043]: pgmap v284: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 150 op/s 2024-03-12T05:21:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:16 smithi100 ceph-mon[32210]: pgmap v284: 97 pgs: 97 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 150 op/s 2024-03-12T05:21:17.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1105290 12.00 MB/sec execute 361 sec latency 172.238 ms 2024-03-12T05:21:18.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1107276 11.99 MB/sec execute 362 sec latency 117.709 ms 2024-03-12T05:21:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:18 smithi156 ceph-mon[36270]: pgmap v285: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.7 MiB/s wr, 302 op/s 2024-03-12T05:21:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:18 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:21:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:18 smithi161 ceph-mon[36043]: pgmap v285: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.7 MiB/s wr, 302 op/s 2024-03-12T05:21:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:18 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:21:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:18 smithi100 ceph-mon[32210]: pgmap v285: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.7 MiB/s wr, 302 op/s 2024-03-12T05:21:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:18 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:21:19.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1109015 11.99 MB/sec execute 363 sec latency 32.113 ms 2024-03-12T05:21:20.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1111422 11.99 MB/sec execute 364 sec latency 20.196 ms 2024-03-12T05:21:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:20 smithi161 ceph-mon[36043]: pgmap v286: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 244 op/s 2024-03-12T05:21:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:20 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:21:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:20 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:21:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:20 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:21:21.287 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:20 smithi156 ceph-mon[36270]: pgmap v286: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 244 op/s 2024-03-12T05:21:21.287 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:20 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:21:21.287 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:20 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:21:21.287 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:20 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:21:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:20 smithi100 ceph-mon[32210]: pgmap v286: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 244 op/s 2024-03-12T05:21:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:20 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:21:21.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:20 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:21:21.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:20 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:21:21.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1112611 11.97 MB/sec execute 365 sec latency 243.891 ms 2024-03-12T05:21:22.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1115316 11.99 MB/sec execute 366 sec latency 22.941 ms 2024-03-12T05:21:23.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:22 smithi161 ceph-mon[36043]: pgmap v287: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2024-03-12T05:21:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:22 smithi100 ceph-mon[32210]: pgmap v287: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2024-03-12T05:21:23.671 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:22 smithi156 ceph-mon[36270]: pgmap v287: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2024-03-12T05:21:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1116712 11.97 MB/sec execute 367 sec latency 111.614 ms 2024-03-12T05:21:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1119116 11.98 MB/sec execute 368 sec latency 32.482 ms 2024-03-12T05:21:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:24 smithi161 ceph-mon[36043]: pgmap v288: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2024-03-12T05:21:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:24 smithi100 ceph-mon[32210]: pgmap v288: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2024-03-12T05:21:25.551 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:24 smithi156 ceph-mon[36270]: pgmap v288: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2024-03-12T05:21:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1120789 11.96 MB/sec execute 369 sec latency 45.768 ms 2024-03-12T05:21:26.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1122987 11.97 MB/sec execute 370 sec latency 165.587 ms 2024-03-12T05:21:27.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:26 smithi161 ceph-mon[36043]: pgmap v289: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-03-12T05:21:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:26 smithi100 ceph-mon[32210]: pgmap v289: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-03-12T05:21:27.465 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:26 smithi156 ceph-mon[36270]: pgmap v289: 97 pgs: 97 active+clean; 596 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-03-12T05:21:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1125391 11.97 MB/sec execute 371 sec latency 21.201 ms 2024-03-12T05:21:28.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1126903 11.95 MB/sec execute 372 sec latency 89.364 ms 2024-03-12T05:21:29.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:28 smithi161 ceph-mon[36043]: pgmap v290: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.7 MiB/s wr, 294 op/s 2024-03-12T05:21:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:28 smithi100 ceph-mon[32210]: pgmap v290: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.7 MiB/s wr, 294 op/s 2024-03-12T05:21:29.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:28 smithi156 ceph-mon[36270]: pgmap v290: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.7 MiB/s wr, 294 op/s 2024-03-12T05:21:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1129321 11.96 MB/sec execute 373 sec latency 19.494 ms 2024-03-12T05:21:30.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1131067 11.95 MB/sec execute 374 sec latency 18.098 ms 2024-03-12T05:21:31.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:30 smithi161 ceph-mon[36043]: pgmap v291: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 192 op/s 2024-03-12T05:21:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:30 smithi100 ceph-mon[32210]: pgmap v291: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 192 op/s 2024-03-12T05:21:31.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:30 smithi156 ceph-mon[36270]: pgmap v291: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 192 op/s 2024-03-12T05:21:31.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1133542 11.96 MB/sec execute 375 sec latency 17.599 ms 2024-03-12T05:21:32.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1135669 11.95 MB/sec execute 376 sec latency 18.144 ms 2024-03-12T05:21:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:32 smithi161 ceph-mon[36043]: pgmap v292: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 291 op/s 2024-03-12T05:21:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:32 smithi100 ceph-mon[32210]: pgmap v292: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 291 op/s 2024-03-12T05:21:33.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:32 smithi156 ceph-mon[36270]: pgmap v292: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 291 op/s 2024-03-12T05:21:33.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1137434 11.95 MB/sec execute 377 sec latency 61.567 ms 2024-03-12T05:21:34.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1140009 11.95 MB/sec execute 378 sec latency 29.050 ms 2024-03-12T05:21:35.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:34 smithi161 ceph-mon[36043]: pgmap v293: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:21:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:34 smithi100 ceph-mon[32210]: pgmap v293: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:21:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:34 smithi156 ceph-mon[36270]: pgmap v293: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:21:35.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1141860 11.95 MB/sec execute 379 sec latency 14.931 ms 2024-03-12T05:21:36.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:36 smithi100 ceph-mon[32210]: pgmap v294: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:21:36.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:36 smithi156 ceph-mon[36270]: pgmap v294: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:21:36.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:36 smithi161 ceph-mon[36043]: pgmap v294: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-03-12T05:21:36.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1144314 11.95 MB/sec execute 380 sec latency 13.322 ms 2024-03-12T05:21:37.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:21:37.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:21:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 -- 172.21.15.100:0/906340131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 msgr2=0x7f6fe8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 --2- 172.21.15.100:0/906340131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe8100650 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009930 tx=0x7f6fd802bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 -- 172.21.15.100:0/906340131 shutdown_connections 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 --2- 172.21.15.100:0/906340131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe8105340 0x7f6fe810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 --2- 172.21.15.100:0/906340131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe8100c20 0x7f6fe8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 --2- 172.21.15.100:0/906340131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe8100650 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 -- 172.21.15.100:0/906340131 >> 172.21.15.100:0/906340131 conn(0x7f6fe8077230 msgr2=0x7f6fe8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 -- 172.21.15.100:0/906340131 shutdown_connections 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 -- 172.21.15.100:0/906340131 wait complete. 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.452+0000 7f6fed464640 1 Processor -- start 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 -- start start 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe8105340 0x7f6fe819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe8197640 0x7f6fe8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8198110 con 0x7f6fe8100270 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8198280 con 0x7f6fe8197640 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8198440 con 0x7f6fe8105340 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59428/0 (socket says 172.21.15.100:59428) 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1937235827 learned_addr learned my addr 172.21.15.100:0/1937235827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe8197640 0x7f6fe8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe8105340 0x7f6fe819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1937235827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe8105340 msgr2=0x7f6fe819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe8105340 0x7f6fe819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1937235827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe8197640 msgr2=0x7f6fe8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe8197640 0x7f6fe8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1937235827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8009560 con 0x7f6fe8100270 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe819d580 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009230 tx=0x7f6fd802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe4ff9640 1 -- 172.21.15.100:0/1937235827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8004350 con 0x7f6fe8100270 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe81a4380 con 0x7f6fe8100270 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe4ff9640 1 -- 172.21.15.100:0/1937235827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd80044b0 con 0x7f6fe8100270 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.453+0000 7f6fe4ff9640 1 -- 172.21.15.100:0/1937235827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003d00 con 0x7f6fe8100270 2024-03-12T05:21:37.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.454+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe81a48b0 con 0x7f6fe8100270 2024-03-12T05:21:37.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6fe4ff9640 1 -- 172.21.15.100:0/1937235827 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6fd8004df0 con 0x7f6fe8100270 2024-03-12T05:21:37.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6ff0039740 1 Processor -- start 2024-03-12T05:21:37.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6ff0039740 1 -- start start 2024-03-12T05:21:37.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6fe4ff9640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fd40a1ba0 0x7f6fd40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fd40a1ba0 0x7f6fd40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6fe4ff9640 1 -- 172.21.15.100:0/1937235827 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6fd80ef6f0 con 0x7f6fe8100270 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4cd43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.456+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6ff0039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4c43280 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6ff0039740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4c407b0 con 0x55f4d4cd4900 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6ff0039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4c3bf90 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4cd43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48176/0 (socket says 172.21.15.100:48176) 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 -- 172.21.15.100:0/222238929 learned_addr learned my addr 172.21.15.100:0/222238929 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4cd43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43088/0 (socket says 172.21.15.100:43088) 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fd40a1ba0 0x7f6fd40a4060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0005fd0 tx=0x7f6fe0011810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 -- 172.21.15.100:0/222238929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 msgr2=0x55f4d4cd43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4cd43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 -- 172.21.15.100:0/222238929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 msgr2=0x55f4d4ce3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 -- 172.21.15.100:0/222238929 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4d4bfa570 con 0x55f4d4cd4900 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdffff640 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce1680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009c60 tx=0x7f6fd0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a295deebb1da35 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdd7fa640 1 -- 172.21.15.100:0/222238929 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0004230 con 0x55f4d4cd4900 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdd7fa640 1 -- 172.21.15.100:0/222238929 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0004390 con 0x55f4d4cd4900 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.457+0000 7f6fdd7fa640 1 -- 172.21.15.100:0/222238929 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00196d0 con 0x55f4d4cd4900 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 -- 172.21.15.100:0/222238929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 msgr2=0x55f4d4ce1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce1680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009c60 tx=0x7f6fd0011810 comp rx=0 tx=0).stop 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 -- 172.21.15.100:0/222238929 shutdown_connections 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce1680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 --2- 172.21.15.100:0/222238929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4cd43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 -- 172.21.15.100:0/222238929 >> 172.21.15.100:0/222238929 conn(0x55f4d4cce990 msgr2=0x55f4d4cced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 -- 172.21.15.100:0/222238929 shutdown_connections 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 -- 172.21.15.100:0/222238929 wait complete. 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.458+0000 7f6ff0039740 1 -- start start 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 Processor -- start 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4cde950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d490f430 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d48cbe00 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4bc0fd0 con 0x55f4d4cd4900 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6ff0039740 10 client.0 _ll_register_callbacks cb 0x55f4d4c16fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4cde950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59448/0 (socket says 172.21.15.100:59448) 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1612494399 learned_addr learned my addr 172.21.15.100:0/1612494399 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1612494399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 msgr2=0x55f4d4cde950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4cde950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 msgr2=0x55f4d4cdef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1612494399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd0009890 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce15c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005c40 tx=0x7f6fd80037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bb662323702881 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.459+0000 7f6fce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1612494399 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80035f0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1612494399 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6fd80bcaf0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4d493b7e0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1612494399 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd80bc3f0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 10 client.17463 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f4d4c3a610 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1612494399 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f4d493b7e0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1612494399 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fd803c070 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 20 client.17463 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4cf1b70 0x55f4d4cf3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 4 client.17463 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4d4978720 con 0x55f4d4cf1b70 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.460+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4cf1b70 0x55f4d4cf3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6fce7fc640 10 client.17463 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4cf1b70 0x55f4d4cf3f60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1228f7083644ad6 server_cookie=85fcf98f2ed6a551 in_seq=0 out_seq=0 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1612494399 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4d4978720 con 0x55f4d4cf1b70 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6fce7fc640 10 client.17463 handle_command_reply: tid=0 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 client.17463 shutdown 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 msgr2=0x55f4d4ce15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce15c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005c40 tx=0x7f6fd80037d0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 shutdown_connections 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4cf1b70 0x55f4d4cf3f60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4cde950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6fce7fc640 0 client.17463 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1612494399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce15c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 >> 172.21.15.100:0/1612494399 conn(0x55f4d4cce990 msgr2=0x55f4d4cd0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 shutdown_connections 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 1 -- 172.21.15.100:0/1612494399 wait complete. 2024-03-12T05:21:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.461+0000 7f6ff0039740 20 client.17463 trim_cache size 0 max 16384 2024-03-12T05:21:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 Processor -- start 2024-03-12T05:21:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 -- start start 2024-03-12T05:21:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4cd5050 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4cd4e00 con 0x55f4d4cd4900 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6ff0039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4cca0f0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59456/0 (socket says 172.21.15.100:59456) 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.477+0000 7f6fe77fe640 1 -- 172.21.15.100:0/2348560212 learned_addr learned my addr 172.21.15.100:0/2348560212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe77fe640 1 -- 172.21.15.100:0/2348560212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 msgr2=0x55f4d4ce1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe77fe640 1 -- 172.21.15.100:0/2348560212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 msgr2=0x55f4d4ce15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe77fe640 1 -- 172.21.15.100:0/2348560212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4d4bd6ac0 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fdffff640 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdd220 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0003f80 tx=0x7f6fe001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bd8db1e051174 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fdcff9640 1 -- 172.21.15.100:0/2348560212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0023040 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fdcff9640 1 -- 172.21.15.100:0/2348560212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fe001ebe0 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.478+0000 7f6fdcff9640 1 -- 172.21.15.100:0/2348560212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0028d20 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- 172.21.15.100:0/2348560212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 msgr2=0x55f4d4cdd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdd220 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0003f80 tx=0x7f6fe001d000 comp rx=0 tx=0).stop 2024-03-12T05:21:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- 172.21.15.100:0/2348560212 shutdown_connections 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4cdd220 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- 172.21.15.100:0/2348560212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- 172.21.15.100:0/2348560212 >> 172.21.15.100:0/2348560212 conn(0x55f4d4cce990 msgr2=0x55f4d4cd9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- 172.21.15.100:0/2348560212 shutdown_connections 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- 172.21.15.100:0/2348560212 wait complete. 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- start start 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 Processor -- start 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4ce70a0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4ce7210 con 0x55f4d4ce1bc0 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d4ce73d0 con 0x55f4d4cd4900 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6ff0039740 10 client.0 _ll_register_callbacks cb 0x7f6fd0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59462/0 (socket says 172.21.15.100:59462) 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1114764514 learned_addr learned my addr 172.21.15.100:0/1114764514 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.479+0000 7f6fe77fe640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1114764514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 msgr2=0x55f4d4ce8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 msgr2=0x55f4d4ce6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fe6ffd640 1 -- 172.21.15.100:0/1114764514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4d4c3a610 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fe6ffd640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce8a80 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80bc0f0 tx=0x7f6fd802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933b1187c90e0be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1114764514 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80bbd50 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1114764514 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6fd80035f0 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fd0004390 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1114764514 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd80b8690 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 10 client.17469 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f4d4bfa570 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1114764514 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0004390 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1114764514 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fd803c030 con 0x55f4d4cd3fe0 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 10 client.17469 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 20 client.17469 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4d49c50 0x55f4d4d4c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 4 client.17469 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.480+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4d49c7890 con 0x55f4d4d49c50 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4d49c50 0x55f4d4d4c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6fce7fc640 10 client.17469 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6fdffff640 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4d49c50 0x55f4d4d4c040 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8369a9f60e865573 server_cookie=48157af6d422ff4a in_seq=0 out_seq=0 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6fce7fc640 1 -- 172.21.15.100:0/1114764514 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f4d49c7890 con 0x55f4d4d49c50 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6fce7fc640 10 client.17469 handle_command_reply: tid=0 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 client.17469 shutdown 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 msgr2=0x55f4d4ce8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce8a80 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80bc0f0 tx=0x7f6fd802bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 shutdown_connections 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f4d4d49c50 0x55f4d4d4c040 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4d4ce1bc0 0x55f4d4ce6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6fce7fc640 0 client.17469 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4d4cd4900 0x55f4d4ce8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 --2- 172.21.15.100:0/1114764514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4d4cd3fe0 0x55f4d4ce8a80 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 >> 172.21.15.100:0/1114764514 conn(0x55f4d4cce990 msgr2=0x55f4d4cdd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 shutdown_connections 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 1 -- 172.21.15.100:0/1114764514 wait complete. 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.481+0000 7f6ff0039740 20 client.17469 trim_cache size 0 max 16384 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:21:37.486 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:21:37.487 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:21:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fd40a1ba0 msgr2=0x7f6fd40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fd40a1ba0 0x7f6fd40a4060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0005fd0 tx=0x7f6fe0011810 comp rx=0 tx=0).stop 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 msgr2=0x7f6fe819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe819d580 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009230 tx=0x7f6fd802c650 comp rx=0 tx=0).stop 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 shutdown_connections 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6fd40a1ba0 0x7f6fd40a4060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe8197640 0x7f6fe8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe8105340 0x7f6fe819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 --2- 172.21.15.100:0/1937235827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8100270 0x7f6fe819d580 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 >> 172.21.15.100:0/1937235827 conn(0x7f6fe8077230 msgr2=0x7f6fe8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 shutdown_connections 2024-03-12T05:21:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.482+0000 7f6fed464640 1 -- 172.21.15.100:0/1937235827 wait complete. 2024-03-12T05:21:37.489 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:21:37.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:21:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 -- 172.21.15.100:0/3726048195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e301023f0 msgr2=0x7f7e301027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 --2- 172.21.15.100:0/3726048195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e301023f0 0x7f7e301027d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009900 tx=0x7f7e2002bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 -- 172.21.15.100:0/3726048195 shutdown_connections 2024-03-12T05:21:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 --2- 172.21.15.100:0/3726048195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e3010b7c0 0x7f7e3010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 --2- 172.21.15.100:0/3726048195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e30102d10 0x7f7e3010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 --2- 172.21.15.100:0/3726048195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e301023f0 0x7f7e301027d0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 -- 172.21.15.100:0/3726048195 >> 172.21.15.100:0/3726048195 conn(0x7f7e30077180 msgr2=0x7f7e30077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 -- 172.21.15.100:0/3726048195 shutdown_connections 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 -- 172.21.15.100:0/3726048195 wait complete. 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.602+0000 7f7e3609a640 1 Processor -- start 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 -- start start 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e301023f0 0x7f7e3019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 0x7f7e3019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e3010b7c0 0x7f7e30197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30197c90 con 0x7f7e30102d10 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30197e00 con 0x7f7e301023f0 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e3609a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30197fc0 con 0x7f7e3010b7c0 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 0x7f7e3019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 0x7f7e3019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59474/0 (socket says 172.21.15.100:59474) 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 -- 172.21.15.100:0/1353732951 learned_addr learned my addr 172.21.15.100:0/1353732951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2f7fe640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e301023f0 0x7f7e3019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 -- 172.21.15.100:0/1353732951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e3010b7c0 msgr2=0x7f7e30197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2ffff640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e3010b7c0 0x7f7e30197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e3010b7c0 0x7f7e30197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 -- 172.21.15.100:0/1353732951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e301023f0 msgr2=0x7f7e3019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e301023f0 0x7f7e3019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 -- 172.21.15.100:0/1353732951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20009560 con 0x7f7e30102d10 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.603+0000 7f7e2effd640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 0x7f7e3019d980 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400e9f0 tx=0x7f7e2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.604+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24009700 con 0x7f7e30102d10 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.604+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2400cd80 con 0x7f7e30102d10 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.604+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e301982a0 con 0x7f7e30102d10 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.604+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24018920 con 0x7f7e30102d10 2024-03-12T05:21:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.604+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e301a44e0 con 0x7f7e30102d10 2024-03-12T05:21:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.605+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e30067f70 con 0x7f7e30102d10 2024-03-12T05:21:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.606+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7e24018a80 con 0x7f7e30102d10 2024-03-12T05:21:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.606+0000 7f7e2cff9640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e180a1ba0 0x7f7e180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.606+0000 7f7e2f7fe640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e180a1ba0 0x7f7e180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.607+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e24014070 con 0x7f7e30102d10 2024-03-12T05:21:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.607+0000 7f7e2f7fe640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e180a1ba0 0x7f7e180a4060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e20006010 tx=0x7f7e20005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.611+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e24052d20 con 0x7f7e30102d10 2024-03-12T05:21:37.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1146658 11.95 MB/sec execute 381 sec latency 17.431 ms 2024-03-12T05:21:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.840+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e301068b0 con 0x7f7e30102d10 2024-03-12T05:21:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.840+0000 7f7e2cff9640 1 -- 172.21.15.100:0/1353732951 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f7e24096ae0 con 0x7f7e30102d10 2024-03-12T05:21:37.837 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:21:37.837 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:21:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:21:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e180a1ba0 msgr2=0x7f7e180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e180a1ba0 0x7f7e180a4060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e20006010 tx=0x7f7e20005e40 comp rx=0 tx=0).stop 2024-03-12T05:21:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 msgr2=0x7f7e3019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 0x7f7e3019d980 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400e9f0 tx=0x7f7e2400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 shutdown_connections 2024-03-12T05:21:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7e180a1ba0 0x7f7e180a4060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e3010b7c0 0x7f7e30197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e30102d10 0x7f7e3019d980 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 --2- 172.21.15.100:0/1353732951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e301023f0 0x7f7e3019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.842+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 >> 172.21.15.100:0/1353732951 conn(0x7f7e30077180 msgr2=0x7f7e300fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.843+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 shutdown_connections 2024-03-12T05:21:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.843+0000 7f7e3609a640 1 -- 172.21.15.100:0/1353732951 wait complete. 2024-03-12T05:21:37.851 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:21:37.851 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:21:37.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 -- 172.21.15.100:0/662748070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764102250 msgr2=0x7fd764102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 --2- 172.21.15.100:0/662748070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764102250 0x7fd764102630 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd754009900 tx=0x7fd75402bcd0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 -- 172.21.15.100:0/662748070 shutdown_connections 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 --2- 172.21.15.100:0/662748070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76410b670 0x7fd76410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 --2- 172.21.15.100:0/662748070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd764102b70 0x7fd76410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 --2- 172.21.15.100:0/662748070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764102250 0x7fd764102630 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 -- 172.21.15.100:0/662748070 >> 172.21.15.100:0/662748070 conn(0x7fd7640fba70 msgr2=0x7fd7640fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 -- 172.21.15.100:0/662748070 shutdown_connections 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 -- 172.21.15.100:0/662748070 wait complete. 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 Processor -- start 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.970+0000 7fd769402640 1 -- start start 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd764102250 0x7fd764197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd764102b70 0x7fd764197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 0x7fd76419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76419c5d0 con 0x7fd76410b670 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76419c740 con 0x7fd764102250 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76419c900 con 0x7fd764102b70 2024-03-12T05:21:37.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 0x7fd76419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd764102250 0x7fd764197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 0x7fd76419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59492/0 (socket says 172.21.15.100:59492) 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 -- 172.21.15.100:0/3125180969 learned_addr learned my addr 172.21.15.100:0/3125180969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7627fc640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd764102b70 0x7fd764197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 -- 172.21.15.100:0/3125180969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd764102b70 msgr2=0x7fd764197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd764102b70 0x7fd764197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 -- 172.21.15.100:0/3125180969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd764102250 msgr2=0x7fd764197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd764102250 0x7fd764197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 -- 172.21.15.100:0/3125180969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd754009560 con 0x7fd76410b670 2024-03-12T05:21:37.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd7637fe640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 0x7fd76419bfd0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009fd0 tx=0x7fd75c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd74bfff640 1 -- 172.21.15.100:0/3125180969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00bea0 con 0x7fd76410b670 2024-03-12T05:21:37.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76419cbe0 con 0x7fd76410b670 2024-03-12T05:21:37.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd74bfff640 1 -- 172.21.15.100:0/3125180969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd75c009940 con 0x7fd76410b670 2024-03-12T05:21:37.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd74bfff640 1 -- 172.21.15.100:0/3125180969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0106c0 con 0x7fd76410b670 2024-03-12T05:21:37.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.971+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7641a4460 con 0x7fd76410b670 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 Processor -- start 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd74bfff640 1 -- 172.21.15.100:0/3125180969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd75c004240 con 0x7fd76410b670 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 -- start start 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c56550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 0x564372c63810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c63d50 0x564372c66140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372c11960 con 0x564372c56a90 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372bc5310 con 0x564372c63d50 2024-03-12T05:21:37.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd76bfce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372bc2840 con 0x564372c56170 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 0x564372c63810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 0x564372c63810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59500/0 (socket says 172.21.15.100:59500) 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd762ffd640 1 -- 172.21.15.100:0/4084315293 learned_addr learned my addr 172.21.15.100:0/4084315293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd74bfff640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd74c0a1ba0 0x7fd74c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd7627fc640 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c63d50 0x564372c66140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd7627fc640 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c56550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.974+0000 7fd7637fe640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd74c0a1ba0 0x7fd74c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd74bfff640 1 -- 172.21.15.100:0/3125180969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd75c0cd510 con 0x7fd76410b670 2024-03-12T05:21:37.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd762ffd640 1 -- 172.21.15.100:0/4084315293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 msgr2=0x564372c56550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd762ffd640 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c56550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd762ffd640 1 -- 172.21.15.100:0/4084315293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c63d50 msgr2=0x564372c66140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd762ffd640 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c63d50 0x564372c66140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd762ffd640 1 -- 172.21.15.100:0/4084315293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564372b80060 con 0x564372c56a90 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd7627fc640 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c63d50 0x564372c66140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd762ffd640 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 0x564372c63810 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd754009900 tx=0x7fd75402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2adf81d2c445a72 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd7637fe640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd74c0a1ba0 0x7fd74c0a4060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0cb220 tx=0x7fd75c004640 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd7497fa640 1 -- 172.21.15.100:0/4084315293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754004980 con 0x564372c56a90 2024-03-12T05:21:37.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd7497fa640 1 -- 172.21.15.100:0/4084315293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd754004ae0 con 0x564372c56a90 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.975+0000 7fd7497fa640 1 -- 172.21.15.100:0/4084315293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754005280 con 0x564372c56a90 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- 172.21.15.100:0/4084315293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 msgr2=0x564372c63810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 0x564372c63810 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd754009900 tx=0x7fd75402c7e0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- 172.21.15.100:0/4084315293 shutdown_connections 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c63d50 0x564372c66140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56a90 0x564372c63810 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- 172.21.15.100:0/4084315293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c56550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- 172.21.15.100:0/4084315293 >> 172.21.15.100:0/4084315293 conn(0x564372c50b70 msgr2=0x564372c50f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- 172.21.15.100:0/4084315293 shutdown_connections 2024-03-12T05:21:37.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- 172.21.15.100:0/4084315293 wait complete. 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- start start 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 Processor -- start 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c60d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c61290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372bbe020 con 0x564372c63d50 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372891430 con 0x564372c56a90 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437284de00 con 0x564372c56170 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.976+0000 7fd76bfce740 10 client.0 _ll_register_callbacks cb 0x564372b65cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd7627fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c61290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c61290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48242/0 (socket says 172.21.15.100:48242) 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd7627fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59508/0 (socket says 172.21.15.100:59508) 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 -- 172.21.15.100:0/517147452 learned_addr learned my addr 172.21.15.100:0/517147452 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd7627fc640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c60d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 -- 172.21.15.100:0/517147452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 msgr2=0x564372c60d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c60d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 -- 172.21.15.100:0/517147452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 msgr2=0x564372c617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 -- 172.21.15.100:0/517147452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76410e790 con 0x564372c56a90 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd7627fc640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd762ffd640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c61290 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd75402ce20 tx=0x7fd75402c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929492ec6623dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd746ff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd746ff5640 1 -- 172.21.15.100:0/517147452 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754004980 con 0x564372c56a90 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643728bd7e0 con 0x564372c56a90 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd746ff5640 1 -- 172.21.15.100:0/517147452 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd754005410 con 0x564372c56a90 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd746ff5640 1 -- 172.21.15.100:0/517147452 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7540055d0 con 0x564372c56a90 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd76bfce740 10 client.26867 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.977+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564372bbc6a0 con 0x564372c56a90 2024-03-12T05:21:37.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd746ff5640 1 -- 172.21.15.100:0/517147452 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5643728bd7e0 con 0x564372c56a90 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd746ff5640 1 -- 172.21.15.100:0/517147452 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd75403c050 con 0x564372c56a90 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd76bfce740 10 client.26867 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd76bfce740 10 client.26867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd76bfce740 20 client.26867 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd76bfce740 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372c73ce0 0x564372c76100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd76bfce740 4 client.26867 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643728fa720 con 0x564372c73ce0 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd7627fc640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372c73ce0 0x564372c76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd7627fc640 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372c73ce0 0x564372c76100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c63a7ab7b9b6bb5d server_cookie=8c3a1d80c5e7a406 in_seq=0 out_seq=0 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.978+0000 7fd746ff5640 10 client.26867 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd746ff5640 1 -- 172.21.15.100:0/517147452 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643728fa720 con 0x564372c73ce0 2024-03-12T05:21:37.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd746ff5640 10 client.26867 handle_command_reply: tid=0 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 client.26867 shutdown 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 msgr2=0x564372c61290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c61290 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd75402ce20 tx=0x7fd75402c3d0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 shutdown_connections 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372c73ce0 0x564372c76100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c61290 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 --2- 172.21.15.100:0/517147452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c60d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd746ff5640 0 client.26867 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 >> 172.21.15.100:0/517147452 conn(0x564372c50b70 msgr2=0x564372c528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:37.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 shutdown_connections 2024-03-12T05:21:37.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 1 -- 172.21.15.100:0/517147452 wait complete. 2024-03-12T05:21:37.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.980+0000 7fd76bfce740 20 client.26867 trim_cache size 0 max 16384 2024-03-12T05:21:37.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 Processor -- start 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 -- start start 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372b43130 con 0x564372c63d50 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372c506a0 con 0x564372c56a90 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd76bfce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372c55530 con 0x564372c56170 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd7627fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c4fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c4fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48258/0 (socket says 172.21.15.100:48258) 2024-03-12T05:21:37.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 -- 172.21.15.100:0/2722861722 learned_addr learned my addr 172.21.15.100:0/2722861722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd7627fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c6aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59510/0 (socket says 172.21.15.100:59510) 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd7627fc640 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 -- 172.21.15.100:0/2722861722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 msgr2=0x564372c615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 -- 172.21.15.100:0/2722861722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 msgr2=0x564372c6aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c6aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd762ffd640 1 -- 172.21.15.100:0/2722861722 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564372b58c20 con 0x564372c56a90 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.993+0000 7fd7627fc640 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd762ffd640 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c4fb50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd754004610 tx=0x7fd754005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edaeb84b1cd36c9e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd748ff9640 1 -- 172.21.15.100:0/2722861722 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754043070 con 0x564372c56a90 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd748ff9640 1 -- 172.21.15.100:0/2722861722 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd75402c6b0 con 0x564372c56a90 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd748ff9640 1 -- 172.21.15.100:0/2722861722 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75404a690 con 0x564372c56a90 2024-03-12T05:21:37.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 -- 172.21.15.100:0/2722861722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 msgr2=0x564372c4fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c4fb50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd754004610 tx=0x7fd754005c90 comp rx=0 tx=0).stop 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 -- 172.21.15.100:0/2722861722 shutdown_connections 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c63d50 0x564372c6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c4fb50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 --2- 172.21.15.100:0/2722861722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c56170 0x564372c615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 -- 172.21.15.100:0/2722861722 >> 172.21.15.100:0/2722861722 conn(0x564372c50b70 msgr2=0x564372c4f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 -- 172.21.15.100:0/2722861722 shutdown_connections 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.994+0000 7fd76bfce740 1 -- 172.21.15.100:0/2722861722 wait complete. 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 -- start start 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 Processor -- start 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 0x564372c69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c69bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c63d50 0x564372c6a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372cca310 con 0x564372c56170 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372cca480 con 0x564372c56a90 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564372cca640 con 0x564372c63d50 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd76bfce740 10 client.0 _ll_register_callbacks cb 0x7fd754005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 0x564372c69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 0x564372c69680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59518/0 (socket says 172.21.15.100:59518) 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 -- 172.21.15.100:0/977310656 learned_addr learned my addr 172.21.15.100:0/977310656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd762ffd640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c69bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c63d50 0x564372c6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 -- 172.21.15.100:0/977310656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c63d50 msgr2=0x564372c6a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c63d50 0x564372c6a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 -- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 msgr2=0x564372c69bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c69bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.995+0000 7fd7627fc640 1 -- 172.21.15.100:0/977310656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758013090 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd7627fc640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 0x564372c69680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd758015010 tx=0x7fd758005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1270692dda5145dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd746ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd746ff5640 1 -- 172.21.15.100:0/977310656 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75800ecd0 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd746ff5640 1 -- 172.21.15.100:0/977310656 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd75800ee30 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd754004ae0 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd746ff5640 1 -- 172.21.15.100:0/977310656 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd758005020 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 10 client.17493 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564372bbc6a0 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd746ff5640 1 -- 172.21.15.100:0/977310656 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd758036020 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd746ff5640 1 -- 172.21.15.100:0/977310656 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd758004cf0 con 0x564372c56170 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 10 client.17493 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:38.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 10 client.17493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 20 client.17493 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372ccbe40 0x564372cce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 4 client.17493 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.996+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564372949890 con 0x564372ccbe40 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd762ffd640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372ccbe40 0x564372cce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd746ff5640 10 client.17493 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd762ffd640 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372ccbe40 0x564372cce260 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d627ea7454860a7 server_cookie=ec688372517d5b3b in_seq=0 out_seq=0 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd746ff5640 1 -- 172.21.15.100:0/977310656 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564372949890 con 0x564372ccbe40 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd746ff5640 10 client.17493 handle_command_reply: tid=0 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 client.17493 shutdown 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 msgr2=0x564372c69680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 0x564372c69680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd758015010 tx=0x7fd758005d00 comp rx=0 tx=0).stop 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 shutdown_connections 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564372ccbe40 0x564372cce260 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564372c63d50 0x564372c6a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564372c56a90 0x564372c69bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd746ff5640 0 client.17493 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 --2- 172.21.15.100:0/977310656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564372c56170 0x564372c69680 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 >> 172.21.15.100:0/977310656 conn(0x564372c50b70 msgr2=0x564372c4fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 shutdown_connections 2024-03-12T05:21:38.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 1 -- 172.21.15.100:0/977310656 wait complete. 2024-03-12T05:21:38.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.997+0000 7fd76bfce740 20 client.17493 trim_cache size 0 max 16384 2024-03-12T05:21:38.005 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd74c0a1ba0 msgr2=0x7fd74c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd74c0a1ba0 0x7fd74c0a4060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0cb220 tx=0x7fd75c004640 comp rx=0 tx=0).stop 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 msgr2=0x7fd76419bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 0x7fd76419bfd0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009fd0 tx=0x7fd75c00b260 comp rx=0 tx=0).stop 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 shutdown_connections 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd74c0a1ba0 0x7fd74c0a4060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd76410b670 0x7fd76419bfd0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd764102b70 0x7fd764197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 --2- 172.21.15.100:0/3125180969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd764102250 0x7fd764197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 >> 172.21.15.100:0/3125180969 conn(0x7fd7640fba70 msgr2=0x7fd7640fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 shutdown_connections 2024-03-12T05:21:38.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:37.998+0000 7fd769402640 1 -- 172.21.15.100:0/3125180969 wait complete. 2024-03-12T05:21:38.008 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:21:38.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:21:38.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 -- 172.21.15.100:0/2861852802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0800685f0 msgr2=0x7fc0800689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2861852802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0800685f0 0x7fc0800689d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009930 tx=0x7fc07c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:38.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 -- 172.21.15.100:0/2861852802 shutdown_connections 2024-03-12T05:21:38.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2861852802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0801064c0 0x7fc08010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2861852802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080068fa0 0x7fc080105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2861852802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0800685f0 0x7fc0800689d0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 -- 172.21.15.100:0/2861852802 >> 172.21.15.100:0/2861852802 conn(0x7fc080077180 msgr2=0x7fc080077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 -- 172.21.15.100:0/2861852802 shutdown_connections 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.121+0000 7fc0872d4640 1 -- 172.21.15.100:0/2861852802 wait complete. 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 Processor -- start 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 -- start start 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0800685f0 0x7fc08019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 0x7fc08019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc080197610 0x7fc080197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080198000 con 0x7fc0801064c0 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080198170 con 0x7fc0800685f0 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc0872d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080198330 con 0x7fc080197610 2024-03-12T05:21:38.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 0x7fc08019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 0x7fc08019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59530/0 (socket says 172.21.15.100:59530) 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 -- 172.21.15.100:0/2616300993 learned_addr learned my addr 172.21.15.100:0/2616300993 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc085070640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0800685f0 0x7fc08019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc085871640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc080197610 0x7fc080197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 -- 172.21.15.100:0/2616300993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc080197610 msgr2=0x7fc080197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc080197610 0x7fc080197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 -- 172.21.15.100:0/2616300993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0800685f0 msgr2=0x7fc08019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0800685f0 0x7fc08019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 -- 172.21.15.100:0/2616300993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c009560 con 0x7fc0801064c0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc085871640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc080197610 0x7fc080197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.122+0000 7fc08486f640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 0x7fc08019d9a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc07400e9f0 tx=0x7fc07400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.123+0000 7fc0727fc640 1 -- 172.21.15.100:0/2616300993 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0740096a0 con 0x7fc0801064c0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.123+0000 7fc0727fc640 1 -- 172.21.15.100:0/2616300993 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc07400ccc0 con 0x7fc0801064c0 2024-03-12T05:21:38.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.123+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0801a4320 con 0x7fc0801064c0 2024-03-12T05:21:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.123+0000 7fc0727fc640 1 -- 172.21.15.100:0/2616300993 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074018860 con 0x7fc0801064c0 2024-03-12T05:21:38.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.123+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0801a4830 con 0x7fc0801064c0 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.125+0000 7fc0727fc640 1 -- 172.21.15.100:0/2616300993 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc074010aa0 con 0x7fc0801064c0 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.125+0000 7fc089f85740 1 Processor -- start 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.125+0000 7fc089f85740 1 -- start start 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc089f85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc0727fc640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc06c0a1ba0 0x7fc06c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc089f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc089f85740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc089f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c056c960 con 0x55c6c05b1b70 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc089f85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0520310 con 0x55c6c05b1250 2024-03-12T05:21:38.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc089f85740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c051d840 con 0x55c6c05ba750 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085070640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc06c0a1ba0 0x7fc06c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48290/0 (socket says 172.21.15.100:48290) 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 -- 172.21.15.100:0/3716905840 learned_addr learned my addr 172.21.15.100:0/3716905840 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc0727fc640 1 -- 172.21.15.100:0/2616300993 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc074014070 con 0x7fc0801064c0 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 -- 172.21.15.100:0/3716905840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 msgr2=0x55c6c05bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 -- 172.21.15.100:0/3716905840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 msgr2=0x55c6c05ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 -- 172.21.15.100:0/3716905840 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c04db060 con 0x55c6c05b1250 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085871640 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b1630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc078011f00 tx=0x7fc078009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca1a9d480a43a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc085070640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc06c0a1ba0 0x7fc06c0a4060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005ab0 tx=0x7fc07c005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc06b7fe640 1 -- 172.21.15.100:0/3716905840 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800fd10 con 0x55c6c05b1250 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc06b7fe640 1 -- 172.21.15.100:0/3716905840 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc078003e50 con 0x55c6c05b1250 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.126+0000 7fc06b7fe640 1 -- 172.21.15.100:0/3716905840 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800e3b0 con 0x55c6c05b1250 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 -- 172.21.15.100:0/3716905840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 msgr2=0x55c6c05b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b1630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc078011f00 tx=0x7fc078009f30 comp rx=0 tx=0).stop 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 -- 172.21.15.100:0/3716905840 shutdown_connections 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 --2- 172.21.15.100:0/3716905840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b1630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 -- 172.21.15.100:0/3716905840 >> 172.21.15.100:0/3716905840 conn(0x55c6c05ab980 msgr2=0x55c6c05abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 -- 172.21.15.100:0/3716905840 shutdown_connections 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 -- 172.21.15.100:0/3716905840 wait complete. 2024-03-12T05:21:38.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.127+0000 7fc089f85740 1 -- start start 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 Processor -- start 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0519020 con 0x55c6c05b1b70 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c01ec430 con 0x55c6c05b1250 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c01a8e00 con 0x55c6c05ba750 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc089f85740 10 client.0 _ll_register_callbacks cb 0x55c6c04c0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc085070640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc085871640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05b7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59544/0 (socket says 172.21.15.100:59544) 2024-03-12T05:21:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 -- 172.21.15.100:0/1625555026 learned_addr learned my addr 172.21.15.100:0/1625555026 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc085070640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48300/0 (socket says 172.21.15.100:48300) 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 -- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 msgr2=0x55c6c05b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 -- 172.21.15.100:0/1625555026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 msgr2=0x55c6c05b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc08486f640 1 -- 172.21.15.100:0/1625555026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc078019040 con 0x55c6c05b1b70 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc085070640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.128+0000 7fc085871640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc08486f640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05b7cc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc0740cb2f0 tx=0x7fc07401c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f72b2a02890da57 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc068ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc068ff9640 1 -- 172.21.15.100:0/1625555026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074014430 con 0x55c6c05b1b70 2024-03-12T05:21:38.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc068ff9640 1 -- 172.21.15.100:0/1625555026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc07409a820 con 0x55c6c05b1b70 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c02187e0 con 0x55c6c05b1b70 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc068ff9640 1 -- 172.21.15.100:0/1625555026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07409a180 con 0x55c6c05b1b70 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 10 client.17505 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c6c05176a0 con 0x55c6c05b1b70 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc068ff9640 1 -- 172.21.15.100:0/1625555026 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0740b2070 con 0x55c6c05b1b70 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc068ff9640 1 -- 172.21.15.100:0/1625555026 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc0740ba0b0 con 0x55c6c05b1b70 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 10 client.17505 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 10 client.17505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 20 client.17505 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c05cabf0 0x55c6c05ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 4 client.17505 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.129+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6c0255720 con 0x55c6c05cabf0 2024-03-12T05:21:38.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.130+0000 7fc085070640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c05cabf0 0x55c6c05ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.130+0000 7fc068ff9640 10 client.17505 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.130+0000 7fc085070640 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c05cabf0 0x55c6c05ccfe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a49ecb0957cf01a6 server_cookie=2b91a6cea076932e in_seq=0 out_seq=0 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc068ff9640 1 -- 172.21.15.100:0/1625555026 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6c0255720 con 0x55c6c05cabf0 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc068ff9640 10 client.17505 handle_command_reply: tid=0 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 client.17505 shutdown 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 msgr2=0x55c6c05b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05b7cc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc0740cb2f0 tx=0x7fc07401c830 comp rx=0 tx=0).stop 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 shutdown_connections 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c05b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05b7cc0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c05cabf0 0x55c6c05ccfe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 --2- 172.21.15.100:0/1625555026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc068ff9640 0 client.17505 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:38.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 >> 172.21.15.100:0/1625555026 conn(0x55c6c05ab980 msgr2=0x55c6c05ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 shutdown_connections 2024-03-12T05:21:38.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 1 -- 172.21.15.100:0/1625555026 wait complete. 2024-03-12T05:21:38.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.131+0000 7fc089f85740 20 client.17505 trim_cache size 0 max 16384 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 Processor -- start 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 -- start start 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 0x55c6c05b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1b70 0x55c6c05b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05ba750 0x55c6c05d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c049e130 con 0x55c6c05ba750 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c05ab4b0 con 0x55c6c05b1b70 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc089f85740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c05b0ee0 con 0x55c6c05b1250 2024-03-12T05:21:38.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085871640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05ba750 0x55c6c05d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 0x55c6c05b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc08486f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1b70 0x55c6c05b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 0x55c6c05b9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43214/0 (socket says 172.21.15.100:43214) 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085871640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05ba750 0x55c6c05d1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59550/0 (socket says 172.21.15.100:59550) 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 -- 172.21.15.100:0/573127923 learned_addr learned my addr 172.21.15.100:0/573127923 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc08486f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1b70 0x55c6c05b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48312/0 (socket says 172.21.15.100:48312) 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 -- 172.21.15.100:0/573127923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1b70 msgr2=0x55c6c05b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1b70 0x55c6c05b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 -- 172.21.15.100:0/573127923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05ba750 msgr2=0x55c6c05d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05ba750 0x55c6c05d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.144+0000 7fc085070640 1 -- 172.21.15.100:0/573127923 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c04b3c20 con 0x55c6c05b1250 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc085070640 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 0x55c6c05b9eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc07c036040 tx=0x7fc07c043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27514162f1f33b3f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc06affd640 1 -- 172.21.15.100:0/573127923 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004040 con 0x55c6c05b1250 2024-03-12T05:21:38.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc06affd640 1 -- 172.21.15.100:0/573127923 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc07c004df0 con 0x55c6c05b1250 2024-03-12T05:21:38.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc06affd640 1 -- 172.21.15.100:0/573127923 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c003830 con 0x55c6c05b1250 2024-03-12T05:21:38.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 -- 172.21.15.100:0/573127923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 msgr2=0x55c6c05b9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 0x55c6c05b9eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc07c036040 tx=0x7fc07c043000 comp rx=0 tx=0).stop 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 -- 172.21.15.100:0/573127923 shutdown_connections 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05ba750 0x55c6c05d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1b70 0x55c6c05b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 --2- 172.21.15.100:0/573127923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05b1250 0x55c6c05b9eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 -- 172.21.15.100:0/573127923 >> 172.21.15.100:0/573127923 conn(0x55c6c05ab980 msgr2=0x55c6c05a6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 -- 172.21.15.100:0/573127923 shutdown_connections 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.145+0000 7fc089f85740 1 -- 172.21.15.100:0/573127923 wait complete. 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 -- start start 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 Processor -- start 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c06210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c05c11d0 con 0x55c6c05b1b70 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0621610 con 0x55c6c05b1250 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0621750 con 0x55c6c05ba750 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc089f85740 10 client.0 _ll_register_callbacks cb 0x7fc0780116a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc08486f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05c0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59564/0 (socket says 172.21.15.100:59564) 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc08486f640 1 -- 172.21.15.100:0/872253217 learned_addr learned my addr 172.21.15.100:0/872253217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc085070640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.146+0000 7fc085871640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c06210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc08486f640 1 -- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 msgr2=0x55c6c06210d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc08486f640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c06210d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc08486f640 1 -- 172.21.15.100:0/872253217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 msgr2=0x55c6c05c03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc08486f640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05c03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc08486f640 1 -- 172.21.15.100:0/872253217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08010e880 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc085070640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05c03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc08486f640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05c0900 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc07400c6a0 tx=0x7fc0740ba480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6e664b9445eba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc068ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc068ff9640 1 -- 172.21.15.100:0/872253217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074096b00 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc068ff9640 1 -- 172.21.15.100:0/872253217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc07409a820 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc078003e50 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc068ff9640 1 -- 172.21.15.100:0/872253217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07409a180 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 10 client.17511 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c6c04d98c0 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc068ff9640 1 -- 172.21.15.100:0/872253217 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc07407f080 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc068ff9640 1 -- 172.21.15.100:0/872253217 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc074099c70 con 0x55c6c05b1b70 2024-03-12T05:21:38.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 10 client.17511 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 10 client.17511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 20 client.17511 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c0622be0 0x55c6c0625000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 4 client.17511 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.147+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6c02a4890 con 0x55c6c0622be0 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc085070640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c0622be0 0x55c6c0625000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc085070640 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c0622be0 0x55c6c0625000 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f105793b78c296d server_cookie=f9ca12099ba82740 in_seq=0 out_seq=0 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc068ff9640 10 client.17511 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc068ff9640 1 -- 172.21.15.100:0/872253217 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6c02a4890 con 0x55c6c0622be0 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc068ff9640 10 client.17511 handle_command_reply: tid=0 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 client.17511 shutdown 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 msgr2=0x55c6c05c0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05c0900 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc07400c6a0 tx=0x7fc0740ba480 comp rx=0 tx=0).stop 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 shutdown_connections 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6c05ba750 0x55c6c06210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c05b1b70 0x55c6c05c0900 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6c0622be0 0x55c6c0625000 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 --2- 172.21.15.100:0/872253217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6c05b1250 0x55c6c05c03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 >> 172.21.15.100:0/872253217 conn(0x55c6c05ab980 msgr2=0x55c6c05b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc068ff9640 0 client.17511 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 shutdown_connections 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 1 -- 172.21.15.100:0/872253217 wait complete. 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.148+0000 7fc089f85740 20 client.17511 trim_cache size 0 max 16384 2024-03-12T05:21:38.152 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:21:38.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc06c0a1ba0 msgr2=0x7fc06c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc06c0a1ba0 0x7fc06c0a4060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005ab0 tx=0x7fc07c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 msgr2=0x7fc08019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 0x7fc08019d9a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc07400e9f0 tx=0x7fc07400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 shutdown_connections 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc06c0a1ba0 0x7fc06c0a4060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc080197610 0x7fc080197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0801064c0 0x7fc08019d9a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 --2- 172.21.15.100:0/2616300993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0800685f0 0x7fc08019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 >> 172.21.15.100:0/2616300993 conn(0x7fc080077180 msgr2=0x7fc0800fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 shutdown_connections 2024-03-12T05:21:38.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.149+0000 7fc0872d4640 1 -- 172.21.15.100:0/2616300993 wait complete. 2024-03-12T05:21:38.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:37 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1353732951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:21:38.156 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:21:38.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:21:38.208 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:37 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1353732951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:21:38.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:37 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1353732951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:21:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/1246335171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec81002c0 msgr2=0x7f5ec8105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/1246335171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec81002c0 0x7f5ec8105df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009900 tx=0x7f5ebc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/1246335171 shutdown_connections 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/1246335171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec8106520 0x7f5ec810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/1246335171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec81002c0 0x7f5ec8105df0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/1246335171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec80ff910 0x7f5ec80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/1246335171 >> 172.21.15.100:0/1246335171 conn(0x7f5ec8076640 msgr2=0x7f5ec80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/1246335171 shutdown_connections 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.266+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/1246335171 wait complete. 2024-03-12T05:21:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 Processor -- start 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 -- start start 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 0x7f5ec8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec8106520 0x7f5ec8197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec819c000 0x7f5ec819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec819c9f0 con 0x7f5ec80ff910 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec819cb60 con 0x7f5ec819c000 2024-03-12T05:21:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ecdc2e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec819ccd0 con 0x7f5ec8106520 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 0x7f5ec8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 0x7f5ec8197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59578/0 (socket says 172.21.15.100:59578) 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 -- 172.21.15.100:0/4155909553 learned_addr learned my addr 172.21.15.100:0/4155909553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec8106520 0x7f5ec8197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec7fff640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec819c000 0x7f5ec819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 -- 172.21.15.100:0/4155909553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec8106520 msgr2=0x7f5ec8197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec8106520 0x7f5ec8197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 -- 172.21.15.100:0/4155909553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec819c000 msgr2=0x7f5ec819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec819c000 0x7f5ec819c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.267+0000 7f5ec77fe640 1 -- 172.21.15.100:0/4155909553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc009560 con 0x7f5ec80ff910 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.268+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 0x7f5ec8197790 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800d950 tx=0x7f5eb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.268+0000 7f5ec4ff9640 1 -- 172.21.15.100:0/4155909553 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8009c30 con 0x7f5ec80ff910 2024-03-12T05:21:38.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.268+0000 7f5ec4ff9640 1 -- 172.21.15.100:0/4155909553 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb8009d90 con 0x7f5ec80ff910 2024-03-12T05:21:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.268+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec81a42b0 con 0x7f5ec80ff910 2024-03-12T05:21:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.268+0000 7f5ec4ff9640 1 -- 172.21.15.100:0/4155909553 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800f7e0 con 0x7f5ec80ff910 2024-03-12T05:21:38.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.268+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81a4730 con 0x7f5ec80ff910 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.270+0000 7f5ec4ff9640 1 -- 172.21.15.100:0/4155909553 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5eb8010ad0 con 0x7f5ec80ff910 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.270+0000 7f5ec4ff9640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5eb00a1ba0 0x7f5eb00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 Processor -- start 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 -- start start 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5eb00a1ba0 0x7f5eb00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec4ff9640 1 -- 172.21.15.100:0/4155909553 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5eb80cd430 con 0x7f5ec80ff910 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb20e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb217210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb219b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb1c9960 con 0x5632eb20eb70 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb17d310 con 0x5632eb20e250 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ed0803740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb17a840 con 0x5632eb217750 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb217210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb20e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb20e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48344/0 (socket says 172.21.15.100:48344) 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 -- 172.21.15.100:0/52402322 learned_addr learned my addr 172.21.15.100:0/52402322 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec7fff640 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb219b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 -- 172.21.15.100:0/52402322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 msgr2=0x5632eb219b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb219b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 -- 172.21.15.100:0/52402322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 msgr2=0x5632eb217210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb217210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 -- 172.21.15.100:0/52402322 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632eb138060 con 0x5632eb20e250 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5eb00a1ba0 0x7f5eb00a4060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009230 tx=0x7f5ebc004b30 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.271+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb20e630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8096530 tx=0x7f5eb8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd3a6d9d9923466 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5eadffb640 1 -- 172.21.15.100:0/52402322 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80a27c0 con 0x5632eb20e250 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5eadffb640 1 -- 172.21.15.100:0/52402322 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb80cadc0 con 0x5632eb20e250 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5eadffb640 1 -- 172.21.15.100:0/52402322 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8098f90 con 0x5632eb20e250 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 -- 172.21.15.100:0/52402322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 msgr2=0x5632eb20e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb20e630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8096530 tx=0x7f5eb8005880 comp rx=0 tx=0).stop 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 -- 172.21.15.100:0/52402322 shutdown_connections 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb219b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb217210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 --2- 172.21.15.100:0/52402322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb20e630 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 -- 172.21.15.100:0/52402322 >> 172.21.15.100:0/52402322 conn(0x5632eb208980 msgr2=0x5632eb208d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 -- 172.21.15.100:0/52402322 shutdown_connections 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.272+0000 7f5ed0803740 1 -- 172.21.15.100:0/52402322 wait complete. 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 -- start start 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 Processor -- start 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20e250 0x5632eb2147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb214cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb217750 0x5632eb215230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb176020 con 0x5632eb20eb70 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eae49430 con 0x5632eb217750 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eae05e00 con 0x5632eb20e250 2024-03-12T05:21:38.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ed0803740 10 client.0 _ll_register_callbacks cb 0x5632eb11dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ec6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb214cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ec6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb214cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59596/0 (socket says 172.21.15.100:59596) 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ec6ffd640 1 -- 172.21.15.100:0/4053090273 learned_addr learned my addr 172.21.15.100:0/4053090273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb217750 0x5632eb215230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.273+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20e250 0x5632eb2147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec6ffd640 1 -- 172.21.15.100:0/4053090273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20e250 msgr2=0x5632eb2147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20e250 0x5632eb2147b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec6ffd640 1 -- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb217750 msgr2=0x5632eb215230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb217750 0x5632eb215230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec6ffd640 1 -- 172.21.15.100:0/4053090273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81a4730 con 0x5632eb20eb70 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec7fff640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb217750 0x5632eb215230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb214cf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80690c0 tx=0x7f5ebc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cab643c3dd7b71d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ea6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/4053090273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc004df0 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/4053090273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ebc004f50 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632eae757e0 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/4053090273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ebc005110 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ed0803740 10 client.17523 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5632eb1746a0 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/4053090273 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5632eae757e0 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/4053090273 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ebc03c040 con 0x5632eb20eb70 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ed0803740 10 client.17523 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ed0803740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.274+0000 7f5ed0803740 20 client.17523 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.275+0000 7f5ed0803740 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb227b80 0x5632eb229fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.275+0000 7f5ed0803740 4 client.17523 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.275+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632eaeb2720 con 0x5632eb227b80 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.275+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb227b80 0x5632eb229fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.275+0000 7f5ea6ffd640 10 client.17523 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.275+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb227b80 0x5632eb229fa0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec2a0cebc8d2c83 server_cookie=56ef3ee98e5b0bc4 in_seq=0 out_seq=0 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/4053090273 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632eaeb2720 con 0x5632eb227b80 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ea6ffd640 10 client.17523 handle_command_reply: tid=0 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 client.17523 shutdown 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 msgr2=0x5632eb214cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb214cf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80690c0 tx=0x7f5ebc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 shutdown_connections 2024-03-12T05:21:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb227b80 0x5632eb229fa0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb217750 0x5632eb215230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ea6ffd640 0 client.17523 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb214cf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 --2- 172.21.15.100:0/4053090273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20e250 0x5632eb2147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 >> 172.21.15.100:0/4053090273 conn(0x5632eb208980 msgr2=0x5632eb20a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 shutdown_connections 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 1 -- 172.21.15.100:0/4053090273 wait complete. 2024-03-12T05:21:38.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.276+0000 7f5ed0803740 20 client.17523 trim_cache size 0 max 16384 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.289+0000 7f5ed0803740 1 Processor -- start 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.289+0000 7f5ed0803740 1 -- start start 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb216ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20eb70 0x5632eb215050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 0x5632eb211a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ed0803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb0fb130 con 0x5632eb217750 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ed0803740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb20d340 con 0x5632eb20e250 2024-03-12T05:21:38.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ed0803740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb223550 con 0x5632eb20eb70 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 0x5632eb211a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb216ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb216ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48366/0 (socket says 172.21.15.100:48366) 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 -- 172.21.15.100:0/749266104 learned_addr learned my addr 172.21.15.100:0/749266104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 0x5632eb211a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59610/0 (socket says 172.21.15.100:59610) 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20eb70 0x5632eb215050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 -- 172.21.15.100:0/749266104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20eb70 msgr2=0x5632eb215050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20eb70 0x5632eb215050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 -- 172.21.15.100:0/749266104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 msgr2=0x5632eb211a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 0x5632eb211a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 -- 172.21.15.100:0/749266104 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632eb110c20 con 0x5632eb20e250 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec7fff640 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 0x5632eb211a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb216ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800bfd0 tx=0x7f5eb8010e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d79cb20c026b091f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ead7fa640 1 -- 172.21.15.100:0/749266104 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb807a030 con 0x5632eb20e250 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ead7fa640 1 -- 172.21.15.100:0/749266104 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb80caf30 con 0x5632eb20e250 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.290+0000 7f5ead7fa640 1 -- 172.21.15.100:0/749266104 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb807ea90 con 0x5632eb20e250 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 -- 172.21.15.100:0/749266104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 msgr2=0x5632eb216ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb216ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800bfd0 tx=0x7f5eb8010e70 comp rx=0 tx=0).stop 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 -- 172.21.15.100:0/749266104 shutdown_connections 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb217750 0x5632eb211a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb20eb70 0x5632eb215050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 --2- 172.21.15.100:0/749266104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb216ee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 -- 172.21.15.100:0/749266104 >> 172.21.15.100:0/749266104 conn(0x5632eb208980 msgr2=0x5632eb20fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 -- 172.21.15.100:0/749266104 shutdown_connections 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 -- 172.21.15.100:0/749266104 wait complete. 2024-03-12T05:21:38.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.291+0000 7f5ed0803740 1 -- start start 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 Processor -- start 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb21e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb21cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb21d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb21da80 con 0x5632eb20eb70 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb21dbf0 con 0x5632eb20e250 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb27e0c0 con 0x5632eb217750 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ed0803740 10 client.0 _ll_register_callbacks cb 0x7f5eb8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb21e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb21e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48374/0 (socket says 172.21.15.100:48374) 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb21cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 -- 172.21.15.100:0/1360430940 learned_addr learned my addr 172.21.15.100:0/1360430940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec7fff640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb21d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 -- 172.21.15.100:0/1360430940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 msgr2=0x5632eb21d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb21d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 -- 172.21.15.100:0/1360430940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 msgr2=0x5632eb21cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb21cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 -- 172.21.15.100:0/1360430940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec0011750 con 0x5632eb20e250 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb21cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ec77fe640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb21e9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80cabf0 tx=0x7f5eb8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a96a191c451a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.292+0000 7f5ea6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/1360430940 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80a6070 con 0x5632eb20e250 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/1360430940 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5eb807b070 con 0x5632eb20e250 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5eb80cadc0 con 0x5632eb20e250 2024-03-12T05:21:38.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/1360430940 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb807a050 con 0x5632eb20e250 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 10 client.26909 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5632eb138060 con 0x5632eb20e250 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/1360430940 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb80cadc0 con 0x5632eb20e250 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/1360430940 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5eb80a9030 con 0x5632eb20e250 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 10 client.26909 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 10 client.26909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 20 client.26909 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb27f3b0 0x5632eb2817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 4 client.26909 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5632eaf01890 con 0x5632eb27f3b0 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.293+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb27f3b0 0x5632eb2817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ec6ffd640 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb27f3b0 0x5632eb2817a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dac6d634fd3f68a server_cookie=fb1b3e3a06bb7e79 in_seq=0 out_seq=0 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ea6ffd640 10 client.26909 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:38.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ea6ffd640 1 -- 172.21.15.100:0/1360430940 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5632eaf01890 con 0x5632eb27f3b0 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ea6ffd640 10 client.26909 handle_command_reply: tid=0 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 client.26909 shutdown 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 msgr2=0x5632eb21e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb21e9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80cabf0 tx=0x7f5eb8000bc0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 shutdown_connections 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5632eb27f3b0 0x5632eb2817a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632eb217750 0x5632eb21d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632eb20eb70 0x5632eb21cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ea6ffd640 0 client.26909 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 --2- 172.21.15.100:0/1360430940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632eb20e250 0x5632eb21e9e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 >> 172.21.15.100:0/1360430940 conn(0x5632eb208980 msgr2=0x5632eb2072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 shutdown_connections 2024-03-12T05:21:38.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 1 -- 172.21.15.100:0/1360430940 wait complete. 2024-03-12T05:21:38.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.294+0000 7f5ed0803740 20 client.26909 trim_cache size 0 max 16384 2024-03-12T05:21:38.299 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5eb00a1ba0 msgr2=0x7f5eb00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5eb00a1ba0 0x7f5eb00a4060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009230 tx=0x7f5ebc004b30 comp rx=0 tx=0).stop 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 msgr2=0x7f5ec8197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 0x7f5ec8197790 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800d950 tx=0x7f5eb800de10 comp rx=0 tx=0).stop 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 shutdown_connections 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5eb00a1ba0 0x7f5eb00a4060 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec819c000 0x7f5ec819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec8106520 0x7f5ec8197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 --2- 172.21.15.100:0/4155909553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80ff910 0x7f5ec8197790 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:38.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 >> 172.21.15.100:0/4155909553 conn(0x7f5ec8076640 msgr2=0x7f5ec8104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:38.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 shutdown_connections 2024-03-12T05:21:38.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:38.295+0000 7f5ecdc2e640 1 -- 172.21.15.100:0/4155909553 wait complete. 2024-03-12T05:21:38.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1148543 11.95 MB/sec execute 382 sec latency 23.956 ms 2024-03-12T05:21:39.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:38 smithi156 ceph-mon[36270]: pgmap v295: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 301 op/s 2024-03-12T05:21:39.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:38 smithi161 ceph-mon[36043]: pgmap v295: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 301 op/s 2024-03-12T05:21:39.303 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:21:39.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:21:39.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:21:39.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:38 smithi100 ceph-mon[32210]: pgmap v295: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 301 op/s 2024-03-12T05:21:39.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 -- 172.21.15.100:0/963587451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4107380 msgr2=0x7f8db4107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 --2- 172.21.15.100:0/963587451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4107380 0x7f8db4107760 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009900 tx=0x7f8da802bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 -- 172.21.15.100:0/963587451 shutdown_connections 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 --2- 172.21.15.100:0/963587451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db4100400 0x7f8db410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 --2- 172.21.15.100:0/963587451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db40ffa40 0x7f8db40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 --2- 172.21.15.100:0/963587451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4107380 0x7f8db4107760 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 -- 172.21.15.100:0/963587451 >> 172.21.15.100:0/963587451 conn(0x7f8db40753c0 msgr2=0x7f8db40757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 -- 172.21.15.100:0/963587451 shutdown_connections 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 -- 172.21.15.100:0/963587451 wait complete. 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.420+0000 7f8dbab25640 1 Processor -- start 2024-03-12T05:21:39.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 -- start start 2024-03-12T05:21:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 0x7f8db419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 0x7f8db419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db4107380 0x7f8db41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4197e40 con 0x7f8db4100400 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4197fb0 con 0x7f8db4107380 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8dbab25640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4198170 con 0x7f8db40ffa40 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 0x7f8db419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db88c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 0x7f8db419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 0x7f8db419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59640/0 (socket says 172.21.15.100:59640) 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 -- 172.21.15.100:0/3810377615 learned_addr learned my addr 172.21.15.100:0/3810377615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db88c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 0x7f8db419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43290/0 (socket says 172.21.15.100:43290) 2024-03-12T05:21:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db90c2640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db4107380 0x7f8db41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 -- 172.21.15.100:0/3810377615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 msgr2=0x7f8db419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 0x7f8db419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 -- 172.21.15.100:0/3810377615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db4107380 msgr2=0x7f8db41976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db4107380 0x7f8db41976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 -- 172.21.15.100:0/3810377615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da8009560 con 0x7f8db4100400 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 0x7f8db419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.421+0000 7f8db3fff640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 0x7f8db419dae0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8da000e9f0 tx=0x7f8da000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.422+0000 7f8db1ffb640 1 -- 172.21.15.100:0/3810377615 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0009700 con 0x7f8db4100400 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.422+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db4198450 con 0x7f8db4100400 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.422+0000 7f8db1ffb640 1 -- 172.21.15.100:0/3810377615 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8da000cd80 con 0x7f8db4100400 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.422+0000 7f8db1ffb640 1 -- 172.21.15.100:0/3810377615 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0018920 con 0x7f8db4100400 2024-03-12T05:21:39.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.422+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db41a4520 con 0x7f8db4100400 2024-03-12T05:21:39.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.424+0000 7f8db1ffb640 1 -- 172.21.15.100:0/3810377615 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8da0018a80 con 0x7f8db4100400 2024-03-12T05:21:39.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.424+0000 7f8db1ffb640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8da40a1ba0 0x7f8da40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.424+0000 7f8dbd7d3740 1 Processor -- start 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.424+0000 7f8dbd7d3740 1 -- start start 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8da40a1ba0 0x7f8da40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db1ffb640 1 -- 172.21.15.100:0/3810377615 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8da0014070 con 0x7f8db4100400 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c96e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c977740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c978170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c8dcf90 con 0x559e2c96ec10 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c8da4c0 con 0x559e2c96e260 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c8d5ca0 con 0x559e2c977d10 2024-03-12T05:21:39.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c977740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c977740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59650/0 (socket says 172.21.15.100:59650) 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 -- 172.21.15.100:0/2832762995 learned_addr learned my addr 172.21.15.100:0/2832762995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db88c1640 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c96e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db90c2640 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c978170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8da40a1ba0 0x7f8da40a4060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8da8006010 tx=0x7f8da8004bd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 -- 172.21.15.100:0/2832762995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 msgr2=0x559e2c978170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c978170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 -- 172.21.15.100:0/2832762995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 msgr2=0x559e2c96e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c96e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 -- 172.21.15.100:0/2832762995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2c8980b0 con 0x559e2c96ec10 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8db3fff640 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c977740 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8da0097210 tx=0x7f8da009a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175002b32e82fbf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8d9effd640 1 -- 172.21.15.100:0/2832762995 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00a2fe0 con 0x559e2c96ec10 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8d9effd640 1 -- 172.21.15.100:0/2832762995 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8da0010b80 con 0x559e2c96ec10 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.425+0000 7f8d9effd640 1 -- 172.21.15.100:0/2832762995 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0099ef0 con 0x559e2c96ec10 2024-03-12T05:21:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/2832762995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 msgr2=0x559e2c977740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c977740 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8da0097210 tx=0x7f8da009a260 comp rx=0 tx=0).stop 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/2832762995 shutdown_connections 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c978170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c977740 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/2832762995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c96e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/2832762995 >> 172.21.15.100:0/2832762995 conn(0x559e2c968840 msgr2=0x559e2c968c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/2832762995 shutdown_connections 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/2832762995 wait complete. 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 -- start start 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.426+0000 7f8dbd7d3740 1 Processor -- start 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96e260 0x559e2c97b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c97bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c97c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c87af40 con 0x559e2c96ec10 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c5a94b0 con 0x559e2c977d10 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c85b670 con 0x559e2c96e260 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8dbd7d3740 10 client.0 _ll_register_callbacks cb 0x559e2c8c8f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c97bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db88c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96e260 0x559e2c97b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c97bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59666/0 (socket says 172.21.15.100:59666) 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 -- 172.21.15.100:0/51621334 learned_addr learned my addr 172.21.15.100:0/51621334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db90c2640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c97c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 -- 172.21.15.100:0/51621334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96e260 msgr2=0x559e2c97b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96e260 0x559e2c97b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 -- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 msgr2=0x559e2c97c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c97c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 -- 172.21.15.100:0/51621334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db41a4520 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db88c1640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96e260 0x559e2c97b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8db3fff640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c97bce0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8da00cb960 tx=0x7f8da0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2224c92eb53317 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.427+0000 7f8d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8d93fff640 1 -- 172.21.15.100:0/51621334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0097090 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8d93fff640 1 -- 172.21.15.100:0/51621334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8da00a2fe0 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e2c8aff40 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8d93fff640 1 -- 172.21.15.100:0/51621334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8da0096eb0 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 10 client.17541 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e2c8d4320 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8d93fff640 1 -- 172.21.15.100:0/51621334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e2c8aff40 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8d93fff640 1 -- 172.21.15.100:0/51621334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8da0099e50 con 0x559e2c96ec10 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 10 client.17541 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 20 client.17541 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c98be40 0x559e2c98e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 4 client.17541 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e2c8cc310 con 0x559e2c98be40 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.428+0000 7f8db88c1640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c98be40 0x559e2c98e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.429+0000 7f8d93fff640 10 client.17541 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.429+0000 7f8db88c1640 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c98be40 0x559e2c98e260 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10401813fed9841a server_cookie=f50bb08650b9a7df in_seq=0 out_seq=0 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8d93fff640 1 -- 172.21.15.100:0/51621334 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e2c8cc310 con 0x559e2c98be40 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8d93fff640 10 client.17541 handle_command_reply: tid=0 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 client.17541 shutdown 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 msgr2=0x559e2c97bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c97bce0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8da00cb960 tx=0x7f8da0010f50 comp rx=0 tx=0).stop 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 shutdown_connections 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c98be40 0x559e2c98e260 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c97c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c97bce0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8d93fff640 0 client.17541 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/51621334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96e260 0x559e2c97b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 >> 172.21.15.100:0/51621334 conn(0x559e2c968840 msgr2=0x559e2c96a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 shutdown_connections 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/51621334 wait complete. 2024-03-12T05:21:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.430+0000 7f8dbd7d3740 20 client.17541 trim_cache size 0 max 16384 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 Processor -- start 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 -- start start 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 0x559e2c97ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96ec10 0x559e2c97c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c979100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c964180 con 0x559e2c96e260 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c97e880 con 0x559e2c977d10 2024-03-12T05:21:39.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c988ca0 con 0x559e2c96ec10 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8db88c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 0x559e2c97ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8db88c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 0x559e2c97ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59680/0 (socket says 172.21.15.100:59680) 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8db88c1640 1 -- 172.21.15.100:0/425194851 learned_addr learned my addr 172.21.15.100:0/425194851 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.445+0000 7f8db90c2640 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c979100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db3fff640 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96ec10 0x559e2c97c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db88c1640 1 -- 172.21.15.100:0/425194851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96ec10 msgr2=0x559e2c97c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db88c1640 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96ec10 0x559e2c97c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db88c1640 1 -- 172.21.15.100:0/425194851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 msgr2=0x559e2c979100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db88c1640 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c979100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db88c1640 1 -- 172.21.15.100:0/425194851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2c870c00 con 0x559e2c96e260 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8db88c1640 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 0x559e2c97ded0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009230 tx=0x7f8da8043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6779f93553c9dab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8d9e7fc640 1 -- 172.21.15.100:0/425194851 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8052ea0 con 0x559e2c96e260 2024-03-12T05:21:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8d9e7fc640 1 -- 172.21.15.100:0/425194851 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8da8052610 con 0x559e2c96e260 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.446+0000 7f8d9e7fc640 1 -- 172.21.15.100:0/425194851 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da80053a0 con 0x559e2c96e260 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/425194851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 msgr2=0x559e2c97ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 0x559e2c97ded0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009230 tx=0x7f8da8043040 comp rx=0 tx=0).stop 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/425194851 shutdown_connections 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c977d10 0x559e2c979100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c96ec10 0x559e2c97c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/425194851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96e260 0x559e2c97ded0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/425194851 >> 172.21.15.100:0/425194851 conn(0x559e2c968840 msgr2=0x559e2c97a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/425194851 shutdown_connections 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/425194851 wait complete. 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- start start 2024-03-12T05:21:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 Processor -- start 2024-03-12T05:21:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c982290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c9827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c9db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c9db8f0 con 0x559e2c96ec10 2024-03-12T05:21:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c9dba60 con 0x559e2c96e260 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2c9dbc20 con 0x559e2c977d10 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.447+0000 7f8dbd7d3740 10 client.0 _ll_register_callbacks cb 0x7f8da00963c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c9827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c9827d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59688/0 (socket says 172.21.15.100:59688) 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 -- 172.21.15.100:0/3976908251 learned_addr learned my addr 172.21.15.100:0/3976908251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c982290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db90c2640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c9db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 -- 172.21.15.100:0/3976908251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 msgr2=0x559e2c9db3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c9db3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 -- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 msgr2=0x559e2c982290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c982290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 -- 172.21.15.100:0/3976908251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db410e8f0 con 0x559e2c96ec10 2024-03-12T05:21:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c982290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8db3fff640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c9827d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8da0098af0 tx=0x7f8da009bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417ef7a6676ed1c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8d93fff640 1 -- 172.21.15.100:0/3976908251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00b3320 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e2c8fb900 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8d93fff640 1 -- 172.21.15.100:0/3976908251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8da00a2fe0 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8d93fff640 1 -- 172.21.15.100:0/3976908251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8da00851d0 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8dbd7d3740 10 client.17547 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e2c896910 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.448+0000 7f8d93fff640 1 -- 172.21.15.100:0/3976908251 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e2c8fb900 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8d93fff640 1 -- 172.21.15.100:0/3976908251 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8da0099700 con 0x559e2c96ec10 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8dbd7d3740 10 client.17547 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8dbd7d3740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8dbd7d3740 20 client.17547 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c9e4360 0x559e2c9e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8dbd7d3740 4 client.17547 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559e2c5d5860 con 0x559e2c9e4360 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c9e4360 0x559e2c9e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8d93fff640 10 client.17547 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.449+0000 7f8db88c1640 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c9e4360 0x559e2c9e6750 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5792f6ff850cdb6 server_cookie=3ec31cb27c1666c6 in_seq=0 out_seq=0 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.450+0000 7f8d93fff640 1 -- 172.21.15.100:0/3976908251 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e2c5d5860 con 0x559e2c9e4360 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.450+0000 7f8d93fff640 10 client.17547 handle_command_reply: tid=0 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.450+0000 7f8dbd7d3740 1 client.17547 shutdown 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.450+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 msgr2=0x559e2c9827d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.450+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c9827d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8da0098af0 tx=0x7f8da009bf90 comp rx=0 tx=0).stop 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 shutdown_connections 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559e2c9e4360 0x559e2c9e6750 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e2c977d10 0x559e2c9db3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e2c96ec10 0x559e2c9827d0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 --2- 172.21.15.100:0/3976908251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e2c96e260 0x559e2c982290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 >> 172.21.15.100:0/3976908251 conn(0x559e2c968840 msgr2=0x559e2c969db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8d93fff640 0 client.17547 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 shutdown_connections 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 1 -- 172.21.15.100:0/3976908251 wait complete. 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbd7d3740 20 client.17547 trim_cache size 0 max 16384 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "978d80e3-04cd-41a9-8ec9-aef676336f22", 2024-03-12T05:21:39.453 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:21:39.454 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:21:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8da40a1ba0 msgr2=0x7f8da40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8da40a1ba0 0x7f8da40a4060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8da8006010 tx=0x7f8da8004bd0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 msgr2=0x7f8db419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 0x7f8db419dae0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8da000e9f0 tx=0x7f8da000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 shutdown_connections 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8da40a1ba0 0x7f8da40a4060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db4107380 0x7f8db41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8db4100400 0x7f8db419dae0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 --2- 172.21.15.100:0/3810377615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8db40ffa40 0x7f8db419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 >> 172.21.15.100:0/3810377615 conn(0x7f8db40753c0 msgr2=0x7f8db40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 shutdown_connections 2024-03-12T05:21:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.451+0000 7f8dbab25640 1 -- 172.21.15.100:0/3810377615 wait complete. 2024-03-12T05:21:39.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:21:39.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:21:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 -- 172.21.15.100:0/1441005714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 msgr2=0x7fb5580ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1441005714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb5580ffc20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb548009930 tx=0x7fb54802bc30 comp rx=0 tx=0).stop 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 -- 172.21.15.100:0/1441005714 shutdown_connections 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1441005714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb558100160 0x7fb55810c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1441005714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb5580ffc20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1441005714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5581057b0 0x7fb558105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 -- 172.21.15.100:0/1441005714 >> 172.21.15.100:0/1441005714 conn(0x7fb558076590 msgr2=0x7fb5580fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 -- 172.21.15.100:0/1441005714 shutdown_connections 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.611+0000 7fb5606ae640 1 -- 172.21.15.100:0/1441005714 wait complete. 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 Processor -- start 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 -- start start 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb55819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558100160 0x7fb55819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5581057b0 0x7fb558196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558196be0 con 0x7fb5580ff7a0 2024-03-12T05:21:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558196d50 con 0x7fb5581057b0 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb5606ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558196f10 con 0x7fb558100160 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb55819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb55819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59718/0 (socket says 172.21.15.100:59718) 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 -- 172.21.15.100:0/1599077979 learned_addr learned my addr 172.21.15.100:0/1599077979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5581057b0 0x7fb558196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558100160 0x7fb55819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 -- 172.21.15.100:0/1599077979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558100160 msgr2=0x7fb55819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558100160 0x7fb55819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 -- 172.21.15.100:0/1599077979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5581057b0 msgr2=0x7fb558196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5581057b0 0x7fb558196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 -- 172.21.15.100:0/1599077979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548009560 con 0x7fb5580ff7a0 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.612+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb55819c3e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb55400e9f0 tx=0x7fb55400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.613+0000 7fb54f7fe640 1 -- 172.21.15.100:0/1599077979 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554009720 con 0x7fb5580ff7a0 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.613+0000 7fb54f7fe640 1 -- 172.21.15.100:0/1599077979 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb55400cd80 con 0x7fb5580ff7a0 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.613+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5581971f0 con 0x7fb5580ff7a0 2024-03-12T05:21:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.613+0000 7fb54f7fe640 1 -- 172.21.15.100:0/1599077979 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554010690 con 0x7fb5580ff7a0 2024-03-12T05:21:39.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.613+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5581a3480 con 0x7fb5580ff7a0 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.615+0000 7fb54f7fe640 1 -- 172.21.15.100:0/1599077979 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb5540046c0 con 0x7fb5580ff7a0 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.615+0000 7fb563283740 1 Processor -- start 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.615+0000 7fb563283740 1 -- start start 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb563283740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x5615987428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb563283740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x56159874b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb563283740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56159874b9c0 0x56159874ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb563283740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615986af280 con 0x561598742de0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb563283740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615986ac7b0 con 0x56159874b9c0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb563283740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615986a7f90 con 0x5615987424c0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x56159874b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb54f7fe640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb5440a1c20 0x7fb5440a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55ec4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x5615987428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56159874b9c0 0x56159874ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55ec4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x5615987428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43342/0 (socket says 172.21.15.100:43342) 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x56159874b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59722/0 (socket says 172.21.15.100:59722) 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55ec4b640 1 -- 172.21.15.100:0/2296740266 learned_addr learned my addr 172.21.15.100:0/2296740266 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb5440a1c20 0x7fb5440a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb54f7fe640 1 -- 172.21.15.100:0/1599077979 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb554014070 con 0x7fb5580ff7a0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 -- 172.21.15.100:0/2296740266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 msgr2=0x5615987428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x5615987428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 -- 172.21.15.100:0/2296740266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 msgr2=0x56159874b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x56159874b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 -- 172.21.15.100:0/2296740266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561598666570 con 0x56159874b9c0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x5615987428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.616+0000 7fb55dc49640 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56159874b9c0 0x56159874ddb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb548009230 tx=0x7fb54802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5fd7b35516884d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb5440a1c20 0x7fb5440a40e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb550009fd0 tx=0x7fb550004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb54cff9640 1 -- 172.21.15.100:0/2296740266 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548004800 con 0x56159874b9c0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb54cff9640 1 -- 172.21.15.100:0/2296740266 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb548004960 con 0x56159874b9c0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb54cff9640 1 -- 172.21.15.100:0/2296740266 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548005030 con 0x56159874b9c0 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 -- 172.21.15.100:0/2296740266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56159874b9c0 msgr2=0x56159874ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56159874b9c0 0x56159874ddb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb548009230 tx=0x7fb54802c5d0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 -- 172.21.15.100:0/2296740266 shutdown_connections 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56159874b9c0 0x56159874ddb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x56159874b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 --2- 172.21.15.100:0/2296740266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x5615987428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 -- 172.21.15.100:0/2296740266 >> 172.21.15.100:0/2296740266 conn(0x56159873cba0 msgr2=0x56159873cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 -- 172.21.15.100:0/2296740266 shutdown_connections 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.617+0000 7fb563283740 1 -- 172.21.15.100:0/2296740266 wait complete. 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 -- start start 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 Processor -- start 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615987424c0 0x56159874b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x5615987487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x561598748ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159837b430 con 0x561598742de0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561598337e00 con 0x5615987424c0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159862cfd0 con 0x56159874b9c0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb563283740 10 client.0 _ll_register_callbacks cb 0x561598682fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x5615987487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x5615987487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59738/0 (socket says 172.21.15.100:59738) 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 -- 172.21.15.100:0/1524007041 learned_addr learned my addr 172.21.15.100:0/1524007041 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615987424c0 0x56159874b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x561598748ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 -- 172.21.15.100:0/1524007041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 msgr2=0x561598748ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x561598748ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 -- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615987424c0 msgr2=0x56159874b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615987424c0 0x56159874b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55dc49640 1 -- 172.21.15.100:0/1524007041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55810cf70 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.618+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x561598748ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x5615987487f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb54802c170 tx=0x7fb54802c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f953154795eb8f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb541ffb640 1 -- 172.21.15.100:0/1524007041 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548004ad0 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615983a77e0 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb541ffb640 1 -- 172.21.15.100:0/1524007041 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb548005000 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb541ffb640 1 -- 172.21.15.100:0/1524007041 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5480051c0 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 10 client.17559 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5615986a6610 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb541ffb640 1 -- 172.21.15.100:0/1524007041 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5615983a77e0 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb541ffb640 1 -- 172.21.15.100:0/1524007041 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb54803c050 con 0x561598742de0 2024-03-12T05:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 10 client.17559 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 20 client.17559 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56159875b8b0 0x56159875dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 4 client.17559 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615983e4720 con 0x56159875b8b0 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.619+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56159875b8b0 0x56159875dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb541ffb640 10 client.17559 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56159875b8b0 0x56159875dcd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bbc342053299b8 server_cookie=e68d3614e784f1e in_seq=0 out_seq=0 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb541ffb640 1 -- 172.21.15.100:0/1524007041 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615983e4720 con 0x56159875b8b0 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb541ffb640 10 client.17559 handle_command_reply: tid=0 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb563283740 1 client.17559 shutdown 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 msgr2=0x5615987487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.620+0000 7fb563283740 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x5615987487f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb54802c170 tx=0x7fb54802c740 comp rx=0 tx=0).stop 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 shutdown_connections 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56159875b8b0 0x56159875dcd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x561598748ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561598742de0 0x5615987487f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 --2- 172.21.15.100:0/1524007041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615987424c0 0x56159874b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 >> 172.21.15.100:0/1524007041 conn(0x56159873cba0 msgr2=0x56159873e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb541ffb640 0 client.17559 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 shutdown_connections 2024-03-12T05:21:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 1 -- 172.21.15.100:0/1524007041 wait complete. 2024-03-12T05:21:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.621+0000 7fb563283740 20 client.17559 trim_cache size 0 max 16384 2024-03-12T05:21:39.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.636+0000 7fb563283740 1 Processor -- start 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.636+0000 7fb563283740 1 -- start start 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb563283740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x561598752800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb563283740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x561598752d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb563283740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 0x56159874af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb563283740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561598741f50 con 0x56159874b9c0 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb563283740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159873dc20 con 0x561598742de0 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb563283740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159873d9d0 con 0x5615987424c0 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 0x56159874af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x561598752800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55dc49640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x561598752d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 0x56159874af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59744/0 (socket says 172.21.15.100:59744) 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x561598752800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43350/0 (socket says 172.21.15.100:43350) 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 -- 172.21.15.100:0/2512179849 learned_addr learned my addr 172.21.15.100:0/2512179849 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 -- 172.21.15.100:0/2512179849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 msgr2=0x561598752800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x561598752800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 -- 172.21.15.100:0/2512179849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 msgr2=0x561598752d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x561598752d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 -- 172.21.15.100:0/2512179849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561598642ac0 con 0x56159874b9c0 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55dc49640 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x561598752d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 0x56159874af20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb550013900 tx=0x7fb550009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce1631c6ee00414 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb543fff640 1 -- 172.21.15.100:0/2512179849 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550013e50 con 0x56159874b9c0 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb543fff640 1 -- 172.21.15.100:0/2512179849 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb550010e90 con 0x56159874b9c0 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.637+0000 7fb543fff640 1 -- 172.21.15.100:0/2512179849 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55002bd60 con 0x56159874b9c0 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 -- 172.21.15.100:0/2512179849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 msgr2=0x56159874af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 0x56159874af20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb550013900 tx=0x7fb550009be0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 -- 172.21.15.100:0/2512179849 shutdown_connections 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159874b9c0 0x56159874af20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x561598752d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 --2- 172.21.15.100:0/2512179849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615987424c0 0x561598752800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 -- 172.21.15.100:0/2512179849 >> 172.21.15.100:0/2512179849 conn(0x56159873cba0 msgr2=0x561598743b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 -- 172.21.15.100:0/2512179849 shutdown_connections 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 -- 172.21.15.100:0/2512179849 wait complete. 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 -- start start 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.638+0000 7fb563283740 1 Processor -- start 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 0x561598752930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x56159874ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x56159874f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159874f860 con 0x5615987424c0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159874f9d0 con 0x561598742de0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159874fb90 con 0x56159874b9c0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 10 client.0 _ll_register_callbacks cb 0x7fb554005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 0x561598752930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 0x561598752930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59752/0 (socket says 172.21.15.100:59752) 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 -- 172.21.15.100:0/1989166142 learned_addr learned my addr 172.21.15.100:0/1989166142 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55ec4b640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x56159874f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x56159874ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 -- 172.21.15.100:0/1989166142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 msgr2=0x56159874f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x56159874f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 -- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 msgr2=0x56159874ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x56159874ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 -- 172.21.15.100:0/1989166142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615986a6610 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb55e44a640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 0x561598752930 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb5540cb7e0 tx=0x7fb5540cfde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37e398fd9f3c6df server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb541ffb640 1 -- 172.21.15.100:0/1989166142 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55409a6c0 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb541ffb640 1 -- 172.21.15.100:0/1989166142 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb554083e40 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb548004960 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb541ffb640 1 -- 172.21.15.100:0/1989166142 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb55409e810 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 10 client.17565 fetch_fsmap learned FSMap version 25 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.639+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb5500135c0 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb541ffb640 1 -- 172.21.15.100:0/1989166142 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb548004960 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb541ffb640 1 -- 172.21.15.100:0/1989166142 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb55401f060 con 0x5615987424c0 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb563283740 10 client.17565 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb563283740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:21:39.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb563283740 20 client.17565 populate_metadata read hostname 'smithi100' 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb563283740 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615987b5710 0x5615987b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb563283740 4 client.17565 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561598433890 con 0x5615987b5710 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615987b5710 0x5615987b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb541ffb640 10 client.17565 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb55dc49640 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615987b5710 0x5615987b7b30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=811c383d42f1c89c server_cookie=8ad251eac0b90739 in_seq=0 out_seq=0 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.640+0000 7fb541ffb640 1 -- 172.21.15.100:0/1989166142 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561598433890 con 0x5615987b5710 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb541ffb640 10 client.17565 handle_command_reply: tid=0 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 client.17565 shutdown 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 msgr2=0x561598752930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 0x561598752930 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb5540cb7e0 tx=0x7fb5540cfde0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 shutdown_connections 2024-03-12T05:21:39.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5615987b5710 0x5615987b7b30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56159874b9c0 0x56159874f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561598742de0 0x56159874ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb541ffb640 0 client.17565 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 --2- 172.21.15.100:0/1989166142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615987424c0 0x561598752930 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 >> 172.21.15.100:0/1989166142 conn(0x56159873cba0 msgr2=0x561598747080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 shutdown_connections 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 1 -- 172.21.15.100:0/1989166142 wait complete. 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb563283740 20 client.17565 trim_cache size 0 max 16384 2024-03-12T05:21:39.649 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: "978d80e3-04cd-41a9-8ec9-aef676336f22": { 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "978d80e3-04cd-41a9-8ec9-aef676336f22", 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:21:39.650 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb5440a1c20 msgr2=0x7fb5440a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb5440a1c20 0x7fb5440a40e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb550009fd0 tx=0x7fb550004ab0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 msgr2=0x7fb55819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb55819c3e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb55400e9f0 tx=0x7fb55400edf0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 shutdown_connections 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb5440a1c20 0x7fb5440a40e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5581057b0 0x7fb558196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558100160 0x7fb55819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 --2- 172.21.15.100:0/1599077979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5580ff7a0 0x7fb55819c3e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:21:39.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 >> 172.21.15.100:0/1599077979 conn(0x7fb558076590 msgr2=0x7fb5580fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:21:39.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 shutdown_connections 2024-03-12T05:21:39.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:21:39.641+0000 7fb5606ae640 1 -- 172.21.15.100:0/1599077979 wait complete. 2024-03-12T05:21:39.653 INFO:tasks.cephfs.filesystem:scrub status for tag:978d80e3-04cd-41a9-8ec9-aef676336f22 - {'path': '/', 'tag': '978d80e3-04cd-41a9-8ec9-aef676336f22', 'options': 'recursive,force'} 2024-03-12T05:21:39.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1150633 11.95 MB/sec execute 383 sec latency 141.907 ms 2024-03-12T05:21:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:39 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:21:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:39 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:39 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:21:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:39 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:21:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:39 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:39 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:39 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:21:40.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:39 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:39 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:21:40.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:39 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:21:40.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:39 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:39 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:39 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:21:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:39 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:39 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:21:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:39 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:21:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:39 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:39 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:21:40.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1152341 11.94 MB/sec execute 384 sec latency 25.085 ms 2024-03-12T05:21:41.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:40 smithi156 ceph-mon[36270]: pgmap v296: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-03-12T05:21:41.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:40 smithi161 ceph-mon[36043]: pgmap v296: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-03-12T05:21:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:40 smithi100 ceph-mon[32210]: pgmap v296: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-03-12T05:21:41.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1154784 11.95 MB/sec execute 385 sec latency 27.443 ms 2024-03-12T05:21:42.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1156837 11.93 MB/sec execute 386 sec latency 17.471 ms 2024-03-12T05:21:43.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:42 smithi156 ceph-mon[36270]: pgmap v297: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 307 op/s 2024-03-12T05:21:43.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:42 smithi161 ceph-mon[36043]: pgmap v297: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 307 op/s 2024-03-12T05:21:43.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:42 smithi100 ceph-mon[32210]: pgmap v297: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 307 op/s 2024-03-12T05:21:43.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1158841 11.94 MB/sec execute 387 sec latency 13.853 ms 2024-03-12T05:21:44.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1160858 11.94 MB/sec execute 388 sec latency 85.927 ms 2024-03-12T05:21:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:44 smithi156 ceph-mon[36270]: pgmap v298: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-03-12T05:21:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:44 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:21:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:44 smithi161 ceph-mon[36043]: pgmap v298: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-03-12T05:21:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:44 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:21:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:44 smithi100 ceph-mon[32210]: pgmap v298: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-03-12T05:21:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:44 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:21:45.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1162655 11.93 MB/sec execute 389 sec latency 16.444 ms 2024-03-12T05:21:46.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:45 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:21:46.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:45 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:21:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:45 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:21:46.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1165273 11.94 MB/sec execute 390 sec latency 16.178 ms 2024-03-12T05:21:47.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:46 smithi156 ceph-mon[36270]: pgmap v299: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-03-12T05:21:47.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:46 smithi161 ceph-mon[36043]: pgmap v299: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-03-12T05:21:47.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:46 smithi100 ceph-mon[32210]: pgmap v299: 97 pgs: 97 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-03-12T05:21:47.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1167217 11.92 MB/sec execute 391 sec latency 23.518 ms 2024-03-12T05:21:48.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1169348 11.93 MB/sec execute 392 sec latency 23.905 ms 2024-03-12T05:21:49.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:48 smithi156 ceph-mon[36270]: pgmap v300: 97 pgs: 97 active+clean; 604 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 303 op/s 2024-03-12T05:21:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:48 smithi161 ceph-mon[36043]: pgmap v300: 97 pgs: 97 active+clean; 604 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 303 op/s 2024-03-12T05:21:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:48 smithi100 ceph-mon[32210]: pgmap v300: 97 pgs: 97 active+clean; 604 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 303 op/s 2024-03-12T05:21:49.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1171560 11.93 MB/sec execute 393 sec latency 60.774 ms 2024-03-12T05:21:50.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1173544 11.93 MB/sec execute 394 sec latency 15.788 ms 2024-03-12T05:21:51.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:50 smithi156 ceph-mon[36270]: pgmap v301: 97 pgs: 97 active+clean; 604 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2024-03-12T05:21:51.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:50 smithi161 ceph-mon[36043]: pgmap v301: 97 pgs: 97 active+clean; 604 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2024-03-12T05:21:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:50 smithi100 ceph-mon[32210]: pgmap v301: 97 pgs: 97 active+clean; 604 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2024-03-12T05:21:51.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1175982 11.93 MB/sec execute 395 sec latency 17.069 ms 2024-03-12T05:21:52.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1177249 11.92 MB/sec execute 396 sec latency 25.196 ms 2024-03-12T05:21:53.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:52 smithi156 ceph-mon[36270]: pgmap v302: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 304 op/s 2024-03-12T05:21:53.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:52 smithi161 ceph-mon[36043]: pgmap v302: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 304 op/s 2024-03-12T05:21:53.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:52 smithi100 ceph-mon[32210]: pgmap v302: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 304 op/s 2024-03-12T05:21:53.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1179585 11.92 MB/sec execute 397 sec latency 23.406 ms 2024-03-12T05:21:54.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1181342 11.91 MB/sec execute 398 sec latency 79.717 ms 2024-03-12T05:21:55.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:54 smithi156 ceph-mon[36270]: pgmap v303: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2024-03-12T05:21:55.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:54 smithi161 ceph-mon[36043]: pgmap v303: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2024-03-12T05:21:55.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:54 smithi100 ceph-mon[32210]: pgmap v303: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2024-03-12T05:21:55.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1183288 11.91 MB/sec execute 399 sec latency 75.216 ms 2024-03-12T05:21:56.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1185544 11.91 MB/sec execute 400 sec latency 24.449 ms 2024-03-12T05:21:57.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:56 smithi161 ceph-mon[36043]: pgmap v304: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2024-03-12T05:21:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:56 smithi100 ceph-mon[32210]: pgmap v304: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2024-03-12T05:21:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:56 smithi156 ceph-mon[36270]: pgmap v304: 97 pgs: 97 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2024-03-12T05:21:57.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1187446 11.90 MB/sec execute 401 sec latency 18.323 ms 2024-03-12T05:21:58.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1190060 11.92 MB/sec execute 402 sec latency 16.736 ms 2024-03-12T05:21:59.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:21:58 smithi161 ceph-mon[36043]: pgmap v305: 97 pgs: 97 active+clean; 578 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 289 op/s 2024-03-12T05:21:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:21:58 smithi100 ceph-mon[32210]: pgmap v305: 97 pgs: 97 active+clean; 578 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 289 op/s 2024-03-12T05:21:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:21:58 smithi156 ceph-mon[36270]: pgmap v305: 97 pgs: 97 active+clean; 578 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 289 op/s 2024-03-12T05:21:59.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1191398 11.90 MB/sec execute 403 sec latency 75.018 ms 2024-03-12T05:22:00.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1193773 11.91 MB/sec execute 404 sec latency 33.017 ms 2024-03-12T05:22:01.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:00 smithi161 ceph-mon[36043]: pgmap v306: 97 pgs: 97 active+clean; 578 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 194 op/s 2024-03-12T05:22:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:00 smithi100 ceph-mon[32210]: pgmap v306: 97 pgs: 97 active+clean; 578 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 194 op/s 2024-03-12T05:22:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:00 smithi156 ceph-mon[36270]: pgmap v306: 97 pgs: 97 active+clean; 578 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 194 op/s 2024-03-12T05:22:01.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1194975 11.89 MB/sec execute 405 sec latency 23.221 ms 2024-03-12T05:22:02.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1197337 11.89 MB/sec execute 406 sec latency 24.076 ms 2024-03-12T05:22:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:02 smithi161 ceph-mon[36043]: pgmap v307: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 326 op/s 2024-03-12T05:22:03.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:02 smithi100 ceph-mon[32210]: pgmap v307: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 326 op/s 2024-03-12T05:22:03.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:02 smithi156 ceph-mon[36270]: pgmap v307: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 326 op/s 2024-03-12T05:22:03.802 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1199331 11.88 MB/sec execute 407 sec latency 79.911 ms 2024-03-12T05:22:04.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1201151 11.88 MB/sec execute 408 sec latency 90.834 ms 2024-03-12T05:22:05.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:04 smithi161 ceph-mon[36043]: pgmap v308: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:22:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:04 smithi100 ceph-mon[32210]: pgmap v308: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:22:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:04 smithi156 ceph-mon[36270]: pgmap v308: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:22:05.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1203493 11.88 MB/sec execute 409 sec latency 18.611 ms 2024-03-12T05:22:06.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1205282 11.88 MB/sec execute 410 sec latency 33.509 ms 2024-03-12T05:22:07.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:06 smithi161 ceph-mon[36043]: pgmap v309: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:22:07.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:06 smithi100 ceph-mon[32210]: pgmap v309: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:22:07.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:06 smithi156 ceph-mon[36270]: pgmap v309: 97 pgs: 97 active+clean; 588 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-12T05:22:07.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1207786 11.89 MB/sec execute 411 sec latency 30.164 ms 2024-03-12T05:22:08.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1209389 11.87 MB/sec execute 412 sec latency 25.692 ms 2024-03-12T05:22:09.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:08 smithi161 ceph-mon[36043]: pgmap v310: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 322 op/s 2024-03-12T05:22:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:08 smithi100 ceph-mon[32210]: pgmap v310: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 322 op/s 2024-03-12T05:22:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:08 smithi156 ceph-mon[36270]: pgmap v310: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 322 op/s 2024-03-12T05:22:09.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:22:09.654 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:22:09.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 -- 172.21.15.100:0/267957267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8101d50 msgr2=0x7ff1e8105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/267957267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8101d50 0x7ff1e8105d90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009930 tx=0x7ff1dc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 -- 172.21.15.100:0/267957267 shutdown_connections 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/267957267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e8106470 0x7ff1e810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/267957267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8101d50 0x7ff1e8105d90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/267957267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e81013a0 0x7ff1e8101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.768+0000 7ff1efa5b640 1 -- 172.21.15.100:0/267957267 >> 172.21.15.100:0/267957267 conn(0x7ff1e80fcc00 msgr2=0x7ff1e80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 -- 172.21.15.100:0/267957267 shutdown_connections 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 -- 172.21.15.100:0/267957267 wait complete. 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 Processor -- start 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 -- start start 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e81013a0 0x7ff1e819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e8106470 0x7ff1e819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 0x7ff1e8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8198060 con 0x7ff1e8197670 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e81981d0 con 0x7ff1e8106470 2024-03-12T05:22:09.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1efa5b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8198390 con 0x7ff1e81013a0 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1edff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 0x7ff1e8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1edff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 0x7ff1e8197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57928/0 (socket says 172.21.15.100:57928) 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.769+0000 7ff1edff8640 1 -- 172.21.15.100:0/3623882077 learned_addr learned my addr 172.21.15.100:0/3623882077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1ecff6640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e8106470 0x7ff1e819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e81013a0 0x7ff1e819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1edff8640 1 -- 172.21.15.100:0/3623882077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e81013a0 msgr2=0x7ff1e819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1edff8640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e81013a0 0x7ff1e819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1edff8640 1 -- 172.21.15.100:0/3623882077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e8106470 msgr2=0x7ff1e819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1edff8640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e8106470 0x7ff1e819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1edff8640 1 -- 172.21.15.100:0/3623882077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009560 con 0x7ff1e8197670 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e81013a0 0x7ff1e819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1edff8640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 0x7ff1e8197b20 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800eb10 tx=0x7ff1d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1e67fc640 1 -- 172.21.15.100:0/3623882077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8004b20 con 0x7ff1e8197670 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1e67fc640 1 -- 172.21.15.100:0/3623882077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d8004c80 con 0x7ff1e8197670 2024-03-12T05:22:09.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e81a4330 con 0x7ff1e8197670 2024-03-12T05:22:09.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1e67fc640 1 -- 172.21.15.100:0/3623882077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8010650 con 0x7ff1e8197670 2024-03-12T05:22:09.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.770+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e81a4860 con 0x7ff1e8197670 2024-03-12T05:22:09.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.772+0000 7ff1e67fc640 1 -- 172.21.15.100:0/3623882077 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff1d8005020 con 0x7ff1e8197670 2024-03-12T05:22:09.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 Processor -- start 2024-03-12T05:22:09.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 -- start start 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1e67fc640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1d00a1c20 0x7ff1d00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0ebd0 0x55c018e17270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e177b0 0x55c018e19ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018d7d280 con 0x55c018e0ebd0 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018d7a7b0 con 0x55c018e177b0 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1f262a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018d75f90 con 0x55c018e0e2b0 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1e67fc640 1 -- 172.21.15.100:0/3623882077 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1d8014070 con 0x7ff1e8197670 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ecff6640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1d00a1c20 0x7ff1d00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ecff6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0ebd0 0x55c018e17270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e0e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:57428/0 (socket says 172.21.15.100:57428) 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/3559156748 learned_addr learned my addr 172.21.15.100:0/3559156748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1edff8640 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e177b0 0x55c018e19ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/3559156748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e177b0 msgr2=0x55c018e19ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e177b0 0x55c018e19ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/3559156748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0ebd0 msgr2=0x55c018e17270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0ebd0 0x55c018e17270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.773+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/3559156748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c018d34570 con 0x55c018e0e2b0 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1ecff6640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1d00a1c20 0x7ff1d00a40e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc02bc30 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e0e690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009c30 tx=0x7ff1e0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8b03dc8f2bbfc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1cf7fe640 1 -- 172.21.15.100:0/3559156748 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0009e60 con 0x55c018e0e2b0 2024-03-12T05:22:09.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1cf7fe640 1 -- 172.21.15.100:0/3559156748 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e00035f0 con 0x55c018e0e2b0 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1cf7fe640 1 -- 172.21.15.100:0/3559156748 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00126b0 con 0x55c018e0e2b0 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 -- 172.21.15.100:0/3559156748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 msgr2=0x55c018e0e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e0e690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009c30 tx=0x7ff1e0010740 comp rx=0 tx=0).stop 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 -- 172.21.15.100:0/3559156748 shutdown_connections 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e177b0 0x55c018e19ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0ebd0 0x55c018e17270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 --2- 172.21.15.100:0/3559156748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e0e690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.774+0000 7ff1f262a740 1 -- 172.21.15.100:0/3559156748 >> 172.21.15.100:0/3559156748 conn(0x55c018e08990 msgr2=0x55c018e08d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 -- 172.21.15.100:0/3559156748 shutdown_connections 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 -- 172.21.15.100:0/3559156748 wait complete. 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 -- start start 2024-03-12T05:22:09.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 Processor -- start 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0e2b0 0x55c018e17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0ebd0 0x55c018e145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018a49430 con 0x55c018e177b0 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018a05e00 con 0x55c018e0e2b0 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018cfafd0 con 0x55c018e0ebd0 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1f262a740 10 client.0 _ll_register_callbacks cb 0x55c018d50fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1edff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1edff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e14d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57960/0 (socket says 172.21.15.100:57960) 2024-03-12T05:22:09.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1edff8640 1 -- 172.21.15.100:0/2766819070 learned_addr learned my addr 172.21.15.100:0/2766819070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1ecff6640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0ebd0 0x55c018e145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.775+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0e2b0 0x55c018e17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1edff8640 1 -- 172.21.15.100:0/2766819070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0ebd0 msgr2=0x55c018e145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1edff8640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0ebd0 0x55c018e145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1edff8640 1 -- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0e2b0 msgr2=0x55c018e17260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1edff8640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0e2b0 0x55c018e17260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1edff8640 1 -- 172.21.15.100:0/2766819070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e0009890 con 0x55c018e177b0 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1edff8640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e14d30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff1d809a660 tx=0x7ff1d809aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e3e4c05bef0ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1ccff9640 1 -- 172.21.15.100:0/2766819070 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80855a0 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1ccff9640 1 -- 172.21.15.100:0/2766819070 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1d80cb700 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c018a757e0 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1ccff9640 1 -- 172.21.15.100:0/2766819070 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d8098ac0 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 10 client.17577 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c018d74610 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1ccff9640 1 -- 172.21.15.100:0/2766819070 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d8094030 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1ccff9640 1 -- 172.21.15.100:0/2766819070 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1d801f060 con 0x55c018e177b0 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 10 client.17577 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:09.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 20 client.17577 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e27a20 0x55c018e29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 4 client.17577 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.776+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c018ab2720 con 0x55c018e27a20 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.777+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e27a20 0x55c018e29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.777+0000 7ff1ccff9640 10 client.17577 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.777+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e27a20 0x55c018e29e40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a866f1af5b958da server_cookie=355a4c57212b502c in_seq=0 out_seq=0 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1ccff9640 1 -- 172.21.15.100:0/2766819070 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c018ab2720 con 0x55c018e27a20 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1ccff9640 10 client.17577 handle_command_reply: tid=0 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 client.17577 shutdown 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 msgr2=0x55c018e14d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e14d30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff1d809a660 tx=0x7ff1d809aaa0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 shutdown_connections 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e27a20 0x55c018e29e40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e14d30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0ebd0 0x55c018e145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1ccff9640 0 client.17577 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2766819070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0e2b0 0x55c018e17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 >> 172.21.15.100:0/2766819070 conn(0x55c018e08990 msgr2=0x55c018e0a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 shutdown_connections 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 1 -- 172.21.15.100:0/2766819070 wait complete. 2024-03-12T05:22:09.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.778+0000 7ff1f262a740 20 client.17577 trim_cache size 0 max 16384 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.793+0000 7ff1f262a740 1 Processor -- start 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.793+0000 7ff1f262a740 1 -- start start 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 0x55c018e1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e177b0 0x55c018e16d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1f262a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018e19fa0 con 0x55c018e0e2b0 2024-03-12T05:22:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1f262a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018e099e0 con 0x55c018e0ebd0 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1f262a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018e244a0 con 0x55c018e177b0 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 0x55c018e1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 0x55c018e1e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57962/0 (socket says 172.21.15.100:57962) 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/2353906283 learned_addr learned my addr 172.21.15.100:0/2353906283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ecff6640 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1edff8640 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e177b0 0x55c018e16d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/2353906283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e177b0 msgr2=0x55c018e16d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e177b0 0x55c018e16d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/2353906283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 msgr2=0x55c018e1eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 -- 172.21.15.100:0/2353906283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c018d10ac0 con 0x55c018e0e2b0 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1edff8640 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e177b0 0x55c018e16d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 0x55c018e1e660 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0010c40 tx=0x7ff1e0010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633778e210e810f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ceffd640 1 -- 172.21.15.100:0/2353906283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0021040 con 0x55c018e0e2b0 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ceffd640 1 -- 172.21.15.100:0/2353906283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e0022040 con 0x55c018e0e2b0 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.794+0000 7ff1ceffd640 1 -- 172.21.15.100:0/2353906283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0012450 con 0x55c018e0e2b0 2024-03-12T05:22:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 -- 172.21.15.100:0/2353906283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 msgr2=0x55c018e1e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 0x55c018e1e660 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0010c40 tx=0x7ff1e0010f70 comp rx=0 tx=0).stop 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 -- 172.21.15.100:0/2353906283 shutdown_connections 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e177b0 0x55c018e16d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 --2- 172.21.15.100:0/2353906283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e0e2b0 0x55c018e1e660 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 -- 172.21.15.100:0/2353906283 >> 172.21.15.100:0/2353906283 conn(0x55c018e08990 msgr2=0x55c018e0f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 -- 172.21.15.100:0/2353906283 shutdown_connections 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 -- 172.21.15.100:0/2353906283 wait complete. 2024-03-12T05:22:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.795+0000 7ff1f262a740 1 -- start start 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 Processor -- start 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e1eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018e1dd50 con 0x55c018e177b0 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018e7df30 con 0x55c018e0ebd0 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c018e7e070 con 0x55c018e0e2b0 2024-03-12T05:22:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1f262a740 10 client.0 _ll_register_callbacks cb 0x7ff1e00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1ed7f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e1eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1ecff6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e1d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57974/0 (socket says 172.21.15.100:57974) 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1ed7f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e1eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:57462/0 (socket says 172.21.15.100:57462) 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 -- 172.21.15.100:0/1872428413 learned_addr learned my addr 172.21.15.100:0/1872428413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 -- 172.21.15.100:0/1872428413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 msgr2=0x55c018e1eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e1eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 -- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 msgr2=0x55c018e1cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 -- 172.21.15.100:0/1872428413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c018d34570 con 0x55c018e177b0 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1ecff6640 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.796+0000 7ff1edff8640 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e1d650 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800cf20 tx=0x7ff1d80a2fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e0fff8a206dfd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ccff9640 1 -- 172.21.15.100:0/1872428413 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80cb900 con 0x55c018e177b0 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ccff9640 1 -- 172.21.15.100:0/1872428413 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1d809ae20 con 0x55c018e177b0 2024-03-12T05:22:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1e00035f0 con 0x55c018e177b0 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ccff9640 1 -- 172.21.15.100:0/1872428413 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d8098ac0 con 0x55c018e177b0 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 10 client.17583 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c018d366a0 con 0x55c018e177b0 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ccff9640 1 -- 172.21.15.100:0/1872428413 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e00035f0 con 0x55c018e177b0 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ccff9640 1 -- 172.21.15.100:0/1872428413 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1d801f040 con 0x55c018e177b0 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 10 client.17583 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 20 client.17583 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e7f450 0x55c018e81870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 4 client.17583 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c018b01890 con 0x55c018e7f450 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.797+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e7f450 0x55c018e81870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1ccff9640 10 client.17583 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1ed7f7640 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e7f450 0x55c018e81870 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b75777a6c4d59f server_cookie=200e1efe7df19399 in_seq=0 out_seq=0 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1ccff9640 1 -- 172.21.15.100:0/1872428413 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c018b01890 con 0x55c018e7f450 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1ccff9640 10 client.17583 handle_command_reply: tid=0 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 client.17583 shutdown 2024-03-12T05:22:09.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 msgr2=0x55c018e1d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e1d650 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800cf20 tx=0x7ff1d80a2fe0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 shutdown_connections 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c018e7f450 0x55c018e81870 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c018e177b0 0x55c018e1d650 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1ccff9640 0 client.17583 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c018e0ebd0 0x55c018e1cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 --2- 172.21.15.100:0/1872428413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c018e0e2b0 0x55c018e1eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 >> 172.21.15.100:0/1872428413 conn(0x55c018e08990 msgr2=0x55c018e12e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 shutdown_connections 2024-03-12T05:22:09.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 1 -- 172.21.15.100:0/1872428413 wait complete. 2024-03-12T05:22:09.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.798+0000 7ff1f262a740 20 client.17583 trim_cache size 0 max 16384 2024-03-12T05:22:09.805 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:22:09.805 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:22:09.805 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:22:09.805 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1d00a1c20 msgr2=0x7ff1d00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1d00a1c20 0x7ff1d00a40e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 msgr2=0x7ff1e8197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 0x7ff1e8197b20 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800eb10 tx=0x7ff1d800c2d0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 shutdown_connections 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1d00a1c20 0x7ff1d00a40e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e8197670 0x7ff1e8197b20 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e8106470 0x7ff1e819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 --2- 172.21.15.100:0/3623882077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e81013a0 0x7ff1e819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 >> 172.21.15.100:0/3623882077 conn(0x7ff1e80fcc00 msgr2=0x7ff1e80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 shutdown_connections 2024-03-12T05:22:09.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.799+0000 7ff1efa5b640 1 -- 172.21.15.100:0/3623882077 wait complete. 2024-03-12T05:22:09.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1211525 11.88 MB/sec execute 413 sec latency 63.936 ms 2024-03-12T05:22:09.810 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:22:09.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:22:09.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 -- 172.21.15.100:0/2051074914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35201053c0 msgr2=0x7f352010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2051074914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35201053c0 0x7f352010cf90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3518009cd0 tx=0x7f351802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 -- 172.21.15.100:0/2051074914 shutdown_connections 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2051074914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35201053c0 0x7f352010cf90 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2051074914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520100ca0 0x7f3520104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2051074914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35201002f0 0x7f35201006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 -- 172.21.15.100:0/2051074914 >> 172.21.15.100:0/2051074914 conn(0x7f3520077180 msgr2=0x7f3520077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 -- 172.21.15.100:0/2051074914 shutdown_connections 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.923+0000 7f3524bc9640 1 -- 172.21.15.100:0/2051074914 wait complete. 2024-03-12T05:22:09.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 Processor -- start 2024-03-12T05:22:09.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 -- start start 2024-03-12T05:22:09.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35201002f0 0x7f35201986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35201053c0 0x7f3520198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 0x7f35201a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201a3dc0 con 0x7f3520199150 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201a3f30 con 0x7f35201002f0 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3524bc9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201a40f0 con 0x7f35201053c0 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f351ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 0x7f35201a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f351ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 0x7f35201a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57982/0 (socket says 172.21.15.100:57982) 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f351ed9d640 1 -- 172.21.15.100:0/2039857809 learned_addr learned my addr 172.21.15.100:0/2039857809 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f351e59c640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35201002f0 0x7f35201986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.924+0000 7f3515d9b640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35201053c0 0x7f3520198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f351ed9d640 1 -- 172.21.15.100:0/2039857809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35201053c0 msgr2=0x7f3520198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f351ed9d640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35201053c0 0x7f3520198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f351ed9d640 1 -- 172.21.15.100:0/2039857809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35201002f0 msgr2=0x7f35201986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f351ed9d640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35201002f0 0x7f35201986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f351ed9d640 1 -- 172.21.15.100:0/2039857809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009900 con 0x7f3520199150 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f351ed9d640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 0x7f35201a3750 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3518004af0 tx=0x7f3518032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180328b0 con 0x7f3520199150 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35201a4370 con 0x7f3520199150 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3518003e50 con 0x7f3520199150 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35201a4870 con 0x7f3520199150 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.925+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518004740 con 0x7f3520199150 2024-03-12T05:22:09.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.926+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3520067f70 con 0x7f3520199150 2024-03-12T05:22:09.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.927+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f351800a040 con 0x7f3520199150 2024-03-12T05:22:09.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.928+0000 7f3517fff640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f350c0a1bd0 0x7f350c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:09.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.928+0000 7f351e59c640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f350c0a1bd0 0x7f350c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:09.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.928+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35180f06a0 con 0x7f3520199150 2024-03-12T05:22:09.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.928+0000 7f351e59c640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f350c0a1bd0 0x7f350c0a4090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3510009bd0 tx=0x7f3510005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:09.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:09.931+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f35180b9640 con 0x7f3520199150 2024-03-12T05:22:10.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.209+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3520108650 con 0x7f3520199150 2024-03-12T05:22:10.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.210+0000 7f3517fff640 1 -- 172.21.15.100:0/2039857809 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f35180bcd00 con 0x7f3520199150 2024-03-12T05:22:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:22:10.205 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:22:10.206 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f350c0a1bd0 msgr2=0x7f350c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f350c0a1bd0 0x7f350c0a4090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3510009bd0 tx=0x7f3510005f90 comp rx=0 tx=0).stop 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 msgr2=0x7f35201a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 0x7f35201a3750 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3518004af0 tx=0x7f3518032570 comp rx=0 tx=0).stop 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 shutdown_connections 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f350c0a1bd0 0x7f350c0a4090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3520199150 0x7f35201a3750 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35201053c0 0x7f3520198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 --2- 172.21.15.100:0/2039857809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35201002f0 0x7f35201986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.212+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 >> 172.21.15.100:0/2039857809 conn(0x7f3520077180 msgr2=0x7f35200fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.213+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 shutdown_connections 2024-03-12T05:22:10.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.213+0000 7f3524bc9640 1 -- 172.21.15.100:0/2039857809 wait complete. 2024-03-12T05:22:10.219 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:22:10.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:22:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:10 smithi100 ceph-mon[32210]: pgmap v311: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-03-12T05:22:10.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:10 smithi100 ceph-mon[32210]: osdmap e96: 12 total, 12 up, 12 in 2024-03-12T05:22:10.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.330+0000 7fde3b12e640 1 -- 172.21.15.100:0/1625123765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde340685f0 msgr2=0x7fde340689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.330+0000 7fde3b12e640 1 --2- 172.21.15.100:0/1625123765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde340685f0 0x7fde340689d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fde28009930 tx=0x7fde2802bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- 172.21.15.100:0/1625123765 shutdown_connections 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 --2- 172.21.15.100:0/1625123765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde341064c0 0x7fde3410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 --2- 172.21.15.100:0/1625123765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde34068fa0 0x7fde34105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 --2- 172.21.15.100:0/1625123765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde340685f0 0x7fde340689d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- 172.21.15.100:0/1625123765 >> 172.21.15.100:0/1625123765 conn(0x7fde34077180 msgr2=0x7fde34077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- 172.21.15.100:0/1625123765 shutdown_connections 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- 172.21.15.100:0/1625123765 wait complete. 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 Processor -- start 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- start start 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde340685f0 0x7fde3419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 0x7fde3419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde34197610 0x7fde34197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34198000 con 0x7fde341064c0 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34198170 con 0x7fde340685f0 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.331+0000 7fde3b12e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34198330 con 0x7fde34197610 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 0x7fde3419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 0x7fde3419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57988/0 (socket says 172.21.15.100:57988) 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 -- 172.21.15.100:0/3801890475 learned_addr learned my addr 172.21.15.100:0/3801890475 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde38eca640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde340685f0 0x7fde3419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde396cb640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde34197610 0x7fde34197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 -- 172.21.15.100:0/3801890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde34197610 msgr2=0x7fde34197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde34197610 0x7fde34197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 -- 172.21.15.100:0/3801890475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde340685f0 msgr2=0x7fde3419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde340685f0 0x7fde3419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 -- 172.21.15.100:0/3801890475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28009560 con 0x7fde341064c0 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde33fff640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 0x7fde3419d9a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fde2000d950 tx=0x7fde2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde31ffb640 1 -- 172.21.15.100:0/3801890475 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20009c30 con 0x7fde341064c0 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde31ffb640 1 -- 172.21.15.100:0/3801890475 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde2000b600 con 0x7fde341064c0 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde341a4320 con 0x7fde341064c0 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde31ffb640 1 -- 172.21.15.100:0/3801890475 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20004a00 con 0x7fde341064c0 2024-03-12T05:22:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.332+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde341a4830 con 0x7fde341064c0 2024-03-12T05:22:10.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.334+0000 7fde31ffb640 1 -- 172.21.15.100:0/3801890475 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fde20004020 con 0x7fde341064c0 2024-03-12T05:22:10.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde31ffb640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fde240a1ba0 0x7fde240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 Processor -- start 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 -- start start 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde38eca640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fde240a1ba0 0x7fde240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde31ffb640 1 -- 172.21.15.100:0/3801890475 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fde200cd4f0 con 0x7fde341064c0 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7c250 0x56499ae7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 0x56499ae85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae87b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ae37960 con 0x56499ae7cb70 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499adeb310 con 0x56499ae7c250 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ade8840 con 0x56499ae85750 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 0x56499ae85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 0x56499ae85210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58000/0 (socket says 172.21.15.100:58000) 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 -- 172.21.15.100:0/3872600509 learned_addr learned my addr 172.21.15.100:0/3872600509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde38eca640 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7c250 0x56499ae7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde38eca640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fde240a1ba0 0x7fde240a4060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde28005ab0 tx=0x7fde28004720 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde396cb640 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae87b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 -- 172.21.15.100:0/3872600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 msgr2=0x56499ae87b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae87b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 -- 172.21.15.100:0/3872600509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7c250 msgr2=0x56499ae7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7c250 0x56499ae7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.335+0000 7fde33fff640 1 -- 172.21.15.100:0/3872600509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56499ada6060 con 0x56499ae7cb70 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde38eca640 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7c250 0x56499ae7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde33fff640 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 0x56499ae85210 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fde20096510 tx=0x7fde20099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05efbd676570624 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde1effd640 1 -- 172.21.15.100:0/3872600509 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2009e200 con 0x56499ae7cb70 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde1effd640 1 -- 172.21.15.100:0/3872600509 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde200a2850 con 0x56499ae7cb70 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde1effd640 1 -- 172.21.15.100:0/3872600509 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20096720 con 0x56499ae7cb70 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3872600509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 msgr2=0x56499ae85210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.336+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 0x56499ae85210 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fde20096510 tx=0x7fde20099ce0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3872600509 shutdown_connections 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae87b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7cb70 0x56499ae85210 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3872600509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7c250 0x56499ae7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3872600509 >> 172.21.15.100:0/3872600509 conn(0x56499ae76980 msgr2=0x56499ae76d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3872600509 shutdown_connections 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3872600509 wait complete. 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- start start 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 Processor -- start 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae82cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae83230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ade4020 con 0x56499ae7c250 2024-03-12T05:22:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499aab7430 con 0x56499ae85750 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499aa73e00 con 0x56499ae7cb70 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde3ddd7740 10 client.0 _ll_register_callbacks cb 0x56499ad8bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde38eca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde38eca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae827b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58002/0 (socket says 172.21.15.100:58002) 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.337+0000 7fde38eca640 1 -- 172.21.15.100:0/51402458 learned_addr learned my addr 172.21.15.100:0/51402458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde396cb640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae83230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde33fff640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae82cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde38eca640 1 -- 172.21.15.100:0/51402458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 msgr2=0x56499ae82cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde38eca640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae82cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde38eca640 1 -- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 msgr2=0x56499ae83230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde38eca640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae83230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde38eca640 1 -- 172.21.15.100:0/51402458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde341a4830 con 0x56499ae7c250 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde38eca640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae827b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fde28035000 tx=0x7fde2802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f1a5b0e7a436e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde13fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde13fff640 1 -- 172.21.15.100:0/51402458 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28005000 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde13fff640 1 -- 172.21.15.100:0/51402458 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fde28005160 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56499aae37e0 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde13fff640 1 -- 172.21.15.100:0/51402458 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde28005320 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde3ddd7740 10 client.17601 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.338+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56499ade26a0 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde13fff640 1 -- 172.21.15.100:0/51402458 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56499aae37e0 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde13fff640 1 -- 172.21.15.100:0/51402458 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fde28041580 con 0x56499ae7c250 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde3ddd7740 10 client.17601 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde3ddd7740 10 client.17601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde3ddd7740 20 client.17601 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499ae97fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde3ddd7740 4 client.17601 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56499ab20720 con 0x56499ae95b80 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde33fff640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499ae97fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde13fff640 10 client.17601 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.339+0000 7fde33fff640 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499ae97fa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46f379861992021c server_cookie=a55affc55d31f18 in_seq=0 out_seq=0 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde13fff640 1 -- 172.21.15.100:0/51402458 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56499ab20720 con 0x56499ae95b80 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde13fff640 10 client.17601 handle_command_reply: tid=0 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 client.17601 shutdown 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 msgr2=0x56499ae827b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae827b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fde28035000 tx=0x7fde2802bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 shutdown_connections 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499ae97fa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae83230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde13fff640 0 client.17601 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:10.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae82cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/51402458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae827b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 >> 172.21.15.100:0/51402458 conn(0x56499ae76980 msgr2=0x56499ae786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 shutdown_connections 2024-03-12T05:22:10.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 1 -- 172.21.15.100:0/51402458 wait complete. 2024-03-12T05:22:10.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.340+0000 7fde3ddd7740 20 client.17601 trim_cache size 0 max 16384 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 Processor -- start 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 -- start start 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae84ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae83050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ad69130 con 0x56499ae7c250 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ae77980 con 0x56499ae85750 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ae88140 con 0x56499ae7cb70 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae84ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae84ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58018/0 (socket says 172.21.15.100:58018) 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 -- 172.21.15.100:0/2181494379 learned_addr learned my addr 172.21.15.100:0/2181494379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde396cb640 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde33fff640 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae83050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 -- 172.21.15.100:0/2181494379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 msgr2=0x56499ae83050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae83050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 -- 172.21.15.100:0/2181494379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 msgr2=0x56499ae7fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae7fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde38eca640 1 -- 172.21.15.100:0/2181494379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56499ad7ec20 con 0x56499ae7c250 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.352+0000 7fde396cb640 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde38eca640 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae84ee0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fde280418e0 tx=0x7fde28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6825b2c4e5622f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde1e7fc640 1 -- 172.21.15.100:0/2181494379 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28041660 con 0x56499ae7c250 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde1e7fc640 1 -- 172.21.15.100:0/2181494379 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde2802cb80 con 0x56499ae7c250 2024-03-12T05:22:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde1e7fc640 1 -- 172.21.15.100:0/2181494379 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28003850 con 0x56499ae7c250 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 -- 172.21.15.100:0/2181494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 msgr2=0x56499ae84ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae84ee0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fde280418e0 tx=0x7fde28009780 comp rx=0 tx=0).stop 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 -- 172.21.15.100:0/2181494379 shutdown_connections 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae85750 0x56499ae7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae7cb70 0x56499ae83050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/2181494379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae84ee0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 -- 172.21.15.100:0/2181494379 >> 172.21.15.100:0/2181494379 conn(0x56499ae76980 msgr2=0x56499ae7dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.353+0000 7fde3ddd7740 1 -- 172.21.15.100:0/2181494379 shutdown_connections 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 -- 172.21.15.100:0/2181494379 wait complete. 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 -- start start 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 Processor -- start 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae8c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7cb70 0x56499ae8aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ae8b540 con 0x56499ae7c250 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ae8b6b0 con 0x56499ae7cb70 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ae8b870 con 0x56499ae85750 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde3ddd7740 10 client.0 _ll_register_callbacks cb 0x7fde20009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde38eca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae8c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde38eca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae8c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58028/0 (socket says 172.21.15.100:58028) 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde33fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7cb70 0x56499ae8aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde38eca640 1 -- 172.21.15.100:0/3290963762 learned_addr learned my addr 172.21.15.100:0/3290963762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.354+0000 7fde396cb640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde38eca640 1 -- 172.21.15.100:0/3290963762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 msgr2=0x56499ae8b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde38eca640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae8b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde38eca640 1 -- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7cb70 msgr2=0x56499ae8aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde38eca640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7cb70 0x56499ae8aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde38eca640 1 -- 172.21.15.100:0/3290963762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56499ade26a0 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde33fff640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7cb70 0x56499ae8aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde38eca640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae8c720 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fde2802c170 tx=0x7fde28005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112c61beacd90fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde13fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde13fff640 1 -- 172.21.15.100:0/3290963762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28005000 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde13fff640 1 -- 172.21.15.100:0/3290963762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fde28005160 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56499ab6f890 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde13fff640 1 -- 172.21.15.100:0/3290963762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde28005320 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde3ddd7740 10 client.17607 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56499ada48c0 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde13fff640 1 -- 172.21.15.100:0/3290963762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56499ab6f890 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.355+0000 7fde13fff640 1 -- 172.21.15.100:0/3290963762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fde28034440 con 0x56499ae7c250 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 10 client.17607 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 10 client.17607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 20 client.17607 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:10.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499aeef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 4 client.17607 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56499aa8f700 con 0x56499ae95b80 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde33fff640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499aeef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde13fff640 10 client.17607 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde33fff640 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499aeef120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15328aaefd943cb server_cookie=1a2e6b7b526ba0b5 in_seq=0 out_seq=0 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde13fff640 1 -- 172.21.15.100:0/3290963762 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56499aa8f700 con 0x56499ae95b80 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde13fff640 10 client.17607 handle_command_reply: tid=0 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 1 client.17607 shutdown 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 msgr2=0x56499ae8c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.356+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae8c720 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fde2802c170 tx=0x7fde28005990 comp rx=0 tx=0).stop 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 shutdown_connections 2024-03-12T05:22:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56499ae95b80 0x56499aeef120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56499ae85750 0x56499ae8b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde13fff640 0 client.17607 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56499ae7cb70 0x56499ae8aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 --2- 172.21.15.100:0/3290963762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499ae7c250 0x56499ae8c720 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 >> 172.21.15.100:0/3290963762 conn(0x56499ae76980 msgr2=0x56499ae752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 shutdown_connections 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 1 -- 172.21.15.100:0/3290963762 wait complete. 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3ddd7740 20 client.17607 trim_cache size 0 max 16384 2024-03-12T05:22:10.359 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:22:10.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fde240a1ba0 msgr2=0x7fde240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fde240a1ba0 0x7fde240a4060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde28005ab0 tx=0x7fde28004720 comp rx=0 tx=0).stop 2024-03-12T05:22:10.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 msgr2=0x7fde3419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 0x7fde3419d9a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fde2000d950 tx=0x7fde2000de10 comp rx=0 tx=0).stop 2024-03-12T05:22:10.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 shutdown_connections 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fde240a1ba0 0x7fde240a4060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde34197610 0x7fde34197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde341064c0 0x7fde3419d9a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 --2- 172.21.15.100:0/3801890475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde340685f0 0x7fde3419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 >> 172.21.15.100:0/3801890475 conn(0x7fde34077180 msgr2=0x7fde340fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 shutdown_connections 2024-03-12T05:22:10.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.357+0000 7fde3b12e640 1 -- 172.21.15.100:0/3801890475 wait complete. 2024-03-12T05:22:10.363 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:22:10.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:22:10.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:10 smithi156 ceph-mon[36270]: pgmap v311: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-03-12T05:22:10.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:10 smithi156 ceph-mon[36270]: osdmap e96: 12 total, 12 up, 12 in 2024-03-12T05:22:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:10 smithi161 ceph-mon[36043]: pgmap v311: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-03-12T05:22:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:10 smithi161 ceph-mon[36043]: osdmap e96: 12 total, 12 up, 12 in 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 -- 172.21.15.100:0/3497988922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c100c10 msgr2=0x7f025c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3497988922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c100c10 0x7f025c104c50 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f024c009930 tx=0x7f024c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 -- 172.21.15.100:0/3497988922 shutdown_connections 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3497988922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c105330 0x7f025c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3497988922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c100c10 0x7f025c104c50 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3497988922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f025c100260 0x7f025c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 -- 172.21.15.100:0/3497988922 >> 172.21.15.100:0/3497988922 conn(0x7f025c077180 msgr2=0x7f025c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 -- 172.21.15.100:0/3497988922 shutdown_connections 2024-03-12T05:22:10.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 -- 172.21.15.100:0/3497988922 wait complete. 2024-03-12T05:22:10.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.477+0000 7f0260a2e640 1 Processor -- start 2024-03-12T05:22:10.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 -- start start 2024-03-12T05:22:10.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c100260 0x7f025c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 0x7f025c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f025c197600 0x7f025c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c197ff0 con 0x7f025c105330 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c198160 con 0x7f025c100260 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0260a2e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c198320 con 0x7f025c197600 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 0x7f025c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c100260 0x7f025c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 0x7f025c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56164/0 (socket says 172.21.15.100:56164) 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0259d9b640 1 -- 172.21.15.100:0/3001375503 learned_addr learned my addr 172.21.15.100:0/3001375503 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025ad9d640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f025c197600 0x7f025c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 -- 172.21.15.100:0/3001375503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f025c197600 msgr2=0x7f025c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f025c197600 0x7f025c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 -- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 msgr2=0x7f025c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 0x7f025c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 -- 172.21.15.100:0/3001375503 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254009840 con 0x7f025c100260 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f0259d9b640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 0x7f025c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f025a59c640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c100260 0x7f025c19d450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f025000e9f0 tx=0x7f025000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.478+0000 7f024b7fe640 1 -- 172.21.15.100:0/3001375503 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250009620 con 0x7f025c100260 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.479+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c009560 con 0x7f025c100260 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.479+0000 7f024b7fe640 1 -- 172.21.15.100:0/3001375503 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f025000ccc0 con 0x7f025c100260 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.479+0000 7f024b7fe640 1 -- 172.21.15.100:0/3001375503 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250004a00 con 0x7f025c100260 2024-03-12T05:22:10.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.479+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c1a4610 con 0x7f025c100260 2024-03-12T05:22:10.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f024b7fe640 1 -- 172.21.15.100:0/3001375503 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0250004b60 con 0x7f025c100260 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 Processor -- start 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 -- start start 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 0x55aa99061360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9901c960 con 0x55aa99060f80 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa98fd0310 con 0x55aa990618a0 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.481+0000 7f0263600740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa98fcd840 con 0x55aa9906eb60 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 0x55aa99061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 0x55aa99061360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56176/0 (socket says 172.21.15.100:56176) 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025ad9d640 1 -- 172.21.15.100:0/4240156528 learned_addr learned my addr 172.21.15.100:0/4240156528 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f024b7fe640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f02440a1ba0 0x7f02440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f0259d9b640 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025ad9d640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f02440a1ba0 0x7f02440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f024b7fe640 1 -- 172.21.15.100:0/3001375503 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0250014070 con 0x7f025c100260 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 -- 172.21.15.100:0/4240156528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 msgr2=0x55aa99070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 -- 172.21.15.100:0/4240156528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 msgr2=0x55aa99061360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 0x55aa99061360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 -- 172.21.15.100:0/4240156528 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa98f8b060 con 0x55aa990618a0 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025ad9d640 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 0x55aa99061360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f0259d9b640 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025a59c640 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906e620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02500caf30 tx=0x7f02500053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb3d5a1404d819a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f025ad9d640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f02440a1ba0 0x7f02440a4060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02540049d0 tx=0x7f0254004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f0248ff9640 1 -- 172.21.15.100:0/4240156528 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250085070 con 0x55aa990618a0 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f0248ff9640 1 -- 172.21.15.100:0/4240156528 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0250099f50 con 0x55aa990618a0 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.482+0000 7f0248ff9640 1 -- 172.21.15.100:0/4240156528 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500a2bb0 con 0x55aa990618a0 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 -- 172.21.15.100:0/4240156528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 msgr2=0x55aa9906e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906e620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02500caf30 tx=0x7f02500053b0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 -- 172.21.15.100:0/4240156528 shutdown_connections 2024-03-12T05:22:10.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906e620 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 --2- 172.21.15.100:0/4240156528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa99060f80 0x55aa99061360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 -- 172.21.15.100:0/4240156528 >> 172.21.15.100:0/4240156528 conn(0x55aa9905b980 msgr2=0x55aa9905bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 -- 172.21.15.100:0/4240156528 shutdown_connections 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 -- 172.21.15.100:0/4240156528 wait complete. 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.483+0000 7f0263600740 1 -- start start 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 Processor -- start 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa9906bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 0x55aa9906c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa98fc9020 con 0x55aa9906eb60 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa98c9c430 con 0x55aa990618a0 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa98c58e00 con 0x55aa99060f80 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0263600740 10 client.0 _ll_register_callbacks cb 0x55aa98f70cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 0x55aa9906c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 0x55aa9906c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56188/0 (socket says 172.21.15.100:56188) 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 -- 172.21.15.100:0/2545100742 learned_addr learned my addr 172.21.15.100:0/2545100742 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025a59c640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa9906bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0259d9b640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 -- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 msgr2=0x55aa9906bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa9906bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 -- 172.21.15.100:0/2545100742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 msgr2=0x55aa9906c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 -- 172.21.15.100:0/2545100742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c1a4610 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025a59c640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa9906bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f025ad9d640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 0x55aa9906c610 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0254009ae0 tx=0x7f0254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e974c0fa3aa48c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.484+0000 7f0241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0241ffb640 1 -- 172.21.15.100:0/2545100742 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254022650 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0241ffb640 1 -- 172.21.15.100:0/2545100742 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f02540227b0 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa98cc87e0 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0241ffb640 1 -- 172.21.15.100:0/2545100742 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0254022970 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 10 client.17619 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aa98fc76a0 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0241ffb640 1 -- 172.21.15.100:0/2545100742 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aa98cc87e0 con 0x55aa9906eb60 2024-03-12T05:22:10.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0241ffb640 1 -- 172.21.15.100:0/2545100742 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0254028020 con 0x55aa9906eb60 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 10 client.17619 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 10 client.17619 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 20 client.17619 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa9907ef80 0x55aa99081370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 4 client.17619 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa98d05720 con 0x55aa9907ef80 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.485+0000 7f025a59c640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa9907ef80 0x55aa99081370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f025a59c640 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa9907ef80 0x55aa99081370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c36892b0ff4e5318 server_cookie=b63e36e0041344e0 in_seq=0 out_seq=0 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0241ffb640 10 client.17619 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0241ffb640 1 -- 172.21.15.100:0/2545100742 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa98d05720 con 0x55aa9907ef80 2024-03-12T05:22:10.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0241ffb640 10 client.17619 handle_command_reply: tid=0 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 client.17619 shutdown 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 msgr2=0x55aa9906c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 0x55aa9906c610 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0254009ae0 tx=0x7f0254004a40 comp rx=0 tx=0).stop 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 shutdown_connections 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9906eb60 0x55aa9906c610 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa9907ef80 0x55aa99081370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa990618a0 0x55aa9906c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 --2- 172.21.15.100:0/2545100742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa9906bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0241ffb640 0 client.17619 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.486+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 >> 172.21.15.100:0/2545100742 conn(0x55aa9905b980 msgr2=0x55aa9905d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.487+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 shutdown_connections 2024-03-12T05:22:10.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.487+0000 7f0263600740 1 -- 172.21.15.100:0/2545100742 wait complete. 2024-03-12T05:22:10.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.487+0000 7f0263600740 20 client.17619 trim_cache size 0 max 16384 2024-03-12T05:22:10.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 Processor -- start 2024-03-12T05:22:10.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 -- start start 2024-03-12T05:22:10.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa99060f80 0x55aa9905aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa990759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99075f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa98f4e130 con 0x55aa990618a0 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa99061fa0 con 0x55aa99060f80 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0263600740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa990865a0 con 0x55aa9906eb60 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa990759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa990759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56202/0 (socket says 172.21.15.100:56202) 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 -- 172.21.15.100:0/1859870287 learned_addr learned my addr 172.21.15.100:0/1859870287 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f025a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa99060f80 0x55aa9905aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f025ad9d640 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99075f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 -- 172.21.15.100:0/1859870287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 msgr2=0x55aa99075f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99075f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 -- 172.21.15.100:0/1859870287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa99060f80 msgr2=0x55aa9905aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa99060f80 0x55aa9905aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 -- 172.21.15.100:0/1859870287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa98f63c20 con 0x55aa990618a0 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f025ad9d640 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0259d9b640 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa990759c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f024c009230 tx=0x7f024c004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d7d288752274e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.499+0000 7f0243fff640 1 -- 172.21.15.100:0/1859870287 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c02c7e0 con 0x55aa990618a0 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0243fff640 1 -- 172.21.15.100:0/1859870287 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f024c004510 con 0x55aa990618a0 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0243fff640 1 -- 172.21.15.100:0/1859870287 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c005390 con 0x55aa990618a0 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 -- 172.21.15.100:0/1859870287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 msgr2=0x55aa990759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa990759c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f024c009230 tx=0x7f024c004b90 comp rx=0 tx=0).stop 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 -- 172.21.15.100:0/1859870287 shutdown_connections 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9906eb60 0x55aa99075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa990759c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 --2- 172.21.15.100:0/1859870287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa99060f80 0x55aa9905aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 -- 172.21.15.100:0/1859870287 >> 172.21.15.100:0/1859870287 conn(0x55aa9905b980 msgr2=0x55aa9905a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 -- 172.21.15.100:0/1859870287 shutdown_connections 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 -- 172.21.15.100:0/1859870287 wait complete. 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.500+0000 7f0263600740 1 -- start start 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 Processor -- start 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa99074680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa99074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa9906eb60 0x55aa99075100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa990d55f0 con 0x55aa990618a0 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa990d5760 con 0x55aa9906eb60 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa990d5920 con 0x55aa99060f80 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 10 client.0 _ll_register_callbacks cb 0x7f0250096730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa99074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa99074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56204/0 (socket says 172.21.15.100:56204) 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 -- 172.21.15.100:0/2473185186 learned_addr learned my addr 172.21.15.100:0/2473185186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f025ad9d640 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa9906eb60 0x55aa99075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f025a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa99074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 -- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 msgr2=0x55aa99074680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa99074680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 -- 172.21.15.100:0/2473185186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa9906eb60 msgr2=0x55aa99075100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa9906eb60 0x55aa99075100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 -- 172.21.15.100:0/2473185186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c10d710 con 0x55aa990618a0 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0259d9b640 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa99074bc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f024c009850 tx=0x7f024c0050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14010b6251210825 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0241ffb640 1 -- 172.21.15.100:0/2473185186 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c034440 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0241ffb640 1 -- 172.21.15.100:0/2473185186 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f024c0345a0 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.501+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0250099f50 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0241ffb640 1 -- 172.21.15.100:0/2473185186 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f024c034760 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 10 client.17625 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aa98f898c0 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0241ffb640 1 -- 172.21.15.100:0/2473185186 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0250099f50 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0241ffb640 1 -- 172.21.15.100:0/2473185186 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f024c03c040 con 0x55aa990618a0 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 10 client.17625 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 10 client.17625 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 20 client.17625 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa990d6da0 0x55aa990d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 4 client.17625 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa98d54890 con 0x55aa990d6da0 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.502+0000 7f025a59c640 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa990d6da0 0x55aa990d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0241ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f025a59c640 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa990d6da0 0x55aa990d9190 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ffb752eeba790c server_cookie=8e4de3767debc144 in_seq=0 out_seq=0 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0241ffb640 1 -- 172.21.15.100:0/2473185186 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa98d54890 con 0x55aa990d6da0 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0241ffb640 10 client.17625 handle_command_reply: tid=0 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 client.17625 shutdown 2024-03-12T05:22:10.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 msgr2=0x55aa99074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa99074bc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f024c009850 tx=0x7f024c0050f0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 shutdown_connections 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55aa990d6da0 0x55aa990d9190 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa9906eb60 0x55aa99075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0241ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa990618a0 0x55aa99074bc0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 --2- 172.21.15.100:0/2473185186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa99060f80 0x55aa99074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 >> 172.21.15.100:0/2473185186 conn(0x55aa9905b980 msgr2=0x55aa9905ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 shutdown_connections 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 1 -- 172.21.15.100:0/2473185186 wait complete. 2024-03-12T05:22:10.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0263600740 20 client.17625 trim_cache size 0 max 16384 2024-03-12T05:22:10.510 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:22:10.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f02440a1ba0 msgr2=0x7f02440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f02440a1ba0 0x7f02440a4060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02540049d0 tx=0x7f0254004ab0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c100260 msgr2=0x7f025c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c100260 0x7f025c19d450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f025000e9f0 tx=0x7f025000edf0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 shutdown_connections 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f02440a1ba0 0x7f02440a4060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f025c197600 0x7f025c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c105330 0x7f025c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 --2- 172.21.15.100:0/3001375503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c100260 0x7f025c19d450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.503+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 >> 172.21.15.100:0/3001375503 conn(0x7f025c077180 msgr2=0x7f025c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.504+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 shutdown_connections 2024-03-12T05:22:10.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.504+0000 7f0260a2e640 1 -- 172.21.15.100:0/3001375503 wait complete. 2024-03-12T05:22:10.513 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:22:10.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 -- 172.21.15.100:0/2564194915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954105350 msgr2=0x7f295410cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 --2- 172.21.15.100:0/2564194915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954105350 0x7f295410cf20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f294c009cd0 tx=0x7f294c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 -- 172.21.15.100:0/2564194915 shutdown_connections 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 --2- 172.21.15.100:0/2564194915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954105350 0x7f295410cf20 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 --2- 172.21.15.100:0/2564194915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2954100c30 0x7f2954104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 --2- 172.21.15.100:0/2564194915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954100280 0x7f2954100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 -- 172.21.15.100:0/2564194915 >> 172.21.15.100:0/2564194915 conn(0x7f2954077150 msgr2=0x7f2954077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 -- 172.21.15.100:0/2564194915 shutdown_connections 2024-03-12T05:22:10.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.625+0000 7f295a09d640 1 -- 172.21.15.100:0/2564194915 wait complete. 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 Processor -- start 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 -- start start 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2954100280 0x7f2954198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105350 0x7f2954198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 0x7f29541a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29541a3cb0 con 0x7f2954199030 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29541a3e20 con 0x7f2954105350 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f295a09d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29541a3fe0 con 0x7f2954100280 2024-03-12T05:22:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 0x7f29541a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 0x7f29541a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56222/0 (socket says 172.21.15.100:56222) 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 -- 172.21.15.100:0/3119392588 learned_addr learned my addr 172.21.15.100:0/3119392588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f29537fe640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2954100280 0x7f2954198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2952ffd640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105350 0x7f2954198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 -- 172.21.15.100:0/3119392588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2954100280 msgr2=0x7f2954198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2954100280 0x7f2954198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 -- 172.21.15.100:0/3119392588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105350 msgr2=0x7f2954198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105350 0x7f2954198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.626+0000 7f2953fff640 1 -- 172.21.15.100:0/3119392588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c009900 con 0x7f2954199030 2024-03-12T05:22:10.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.627+0000 7f2953fff640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 0x7f29541a35e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f294c004af0 tx=0x7f294c034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.627+0000 7f2950ff9640 1 -- 172.21.15.100:0/3119392588 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0328c0 con 0x7f2954199030 2024-03-12T05:22:10.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.627+0000 7f2950ff9640 1 -- 172.21.15.100:0/3119392588 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c003e50 con 0x7f2954199030 2024-03-12T05:22:10.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.627+0000 7f2950ff9640 1 -- 172.21.15.100:0/3119392588 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c004860 con 0x7f2954199030 2024-03-12T05:22:10.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.627+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29541a4260 con 0x7f2954199030 2024-03-12T05:22:10.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.627+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29541a4710 con 0x7f2954199030 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.629+0000 7f2950ff9640 1 -- 172.21.15.100:0/3119392588 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f294c00a040 con 0x7f2954199030 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.629+0000 7f295cc74740 1 Processor -- start 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.629+0000 7f295cc74740 1 -- start start 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f295cc74740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65ad5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f295cc74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 0x560b65ade210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f295cc74740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f295cc74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65a90960 con 0x560b65ad5b70 2024-03-12T05:22:10.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f295cc74740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65a44310 con 0x560b65ade750 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f295cc74740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65a41840 con 0x560b65ad5250 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2950ff9640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f293c0a1c20 0x7f293c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f29537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 0x560b65ade210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65ad5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f29537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 0x560b65ade210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56238/0 (socket says 172.21.15.100:56238) 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f29537fe640 1 -- 172.21.15.100:0/2634076911 learned_addr learned my addr 172.21.15.100:0/2634076911 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2953fff640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f293c0a1c20 0x7f293c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 -- 172.21.15.100:0/2634076911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 msgr2=0x560b65ae0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2950ff9640 1 -- 172.21.15.100:0/3119392588 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f294c0f03b0 con 0x7f2954199030 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 -- 172.21.15.100:0/2634076911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 msgr2=0x560b65ade210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 0x560b65ade210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 -- 172.21.15.100:0/2634076911 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b659ff060 con 0x560b65ad5250 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f29537fe640 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 0x560b65ade210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65ad5630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f294400b6c0 tx=0x7f294400b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88535f4911de3d7b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.630+0000 7f2953fff640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f293c0a1c20 0x7f293c0a40e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f294c090300 tx=0x7f294c08f280 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f2939ffb640 1 -- 172.21.15.100:0/2634076911 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944012c40 con 0x560b65ad5250 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f2939ffb640 1 -- 172.21.15.100:0/2634076911 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2944012da0 con 0x560b65ad5250 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f2939ffb640 1 -- 172.21.15.100:0/2634076911 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29440049c0 con 0x560b65ad5250 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 -- 172.21.15.100:0/2634076911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 msgr2=0x560b65ad5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65ad5630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f294400b6c0 tx=0x7f294400b650 comp rx=0 tx=0).stop 2024-03-12T05:22:10.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 -- 172.21.15.100:0/2634076911 shutdown_connections 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5b70 0x560b65ade210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 --2- 172.21.15.100:0/2634076911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65ad5630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 -- 172.21.15.100:0/2634076911 >> 172.21.15.100:0/2634076911 conn(0x560b65acf980 msgr2=0x560b65acfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 -- 172.21.15.100:0/2634076911 shutdown_connections 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.631+0000 7f295cc74740 1 -- 172.21.15.100:0/2634076911 wait complete. 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 -- start start 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 Processor -- start 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65adb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ad5b70 0x560b65adbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 0x560b65adc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65a3d020 con 0x560b65ade750 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65710430 con 0x560b65ad5b70 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b656cce00 con 0x560b65ad5250 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f295cc74740 10 client.0 _ll_register_callbacks cb 0x560b659e4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 0x560b65adc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 0x560b65adc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56250/0 (socket says 172.21.15.100:56250) 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f2952ffd640 1 -- 172.21.15.100:0/2147980959 learned_addr learned my addr 172.21.15.100:0/2147980959 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f29537fe640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ad5b70 0x560b65adbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.632+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65adb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2952ffd640 1 -- 172.21.15.100:0/2147980959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 msgr2=0x560b65adb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65adb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2952ffd640 1 -- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ad5b70 msgr2=0x560b65adbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ad5b70 0x560b65adbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2952ffd640 1 -- 172.21.15.100:0/2147980959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294401a040 con 0x560b65ade750 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f29537fe640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ad5b70 0x560b65adbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 0x560b65adc200 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2944005d00 tx=0x7f2944014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f08498fd60badc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2932ffd640 1 -- 172.21.15.100:0/2147980959 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400b4b0 con 0x560b65ade750 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2932ffd640 1 -- 172.21.15.100:0/2147980959 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2944010740 con 0x560b65ade750 2024-03-12T05:22:10.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b6573c7e0 con 0x560b65ade750 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2932ffd640 1 -- 172.21.15.100:0/2147980959 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2944010900 con 0x560b65ade750 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f295cc74740 10 client.17637 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b65a3b6a0 con 0x560b65ade750 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.633+0000 7f2932ffd640 1 -- 172.21.15.100:0/2147980959 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560b6573c7e0 con 0x560b65ade750 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f2932ffd640 1 -- 172.21.15.100:0/2147980959 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f294402d070 con 0x560b65ade750 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f295cc74740 10 client.17637 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f295cc74740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f295cc74740 20 client.17637 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f295cc74740 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65aee9d0 0x560b65af0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f295cc74740 4 client.17637 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:10.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b65779720 con 0x560b65aee9d0 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f29537fe640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65aee9d0 0x560b65af0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f29537fe640 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65aee9d0 0x560b65af0df0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ccc1eeb66eb79e server_cookie=db36fa4b7ffc6dac in_seq=0 out_seq=0 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.634+0000 7f2932ffd640 10 client.17637 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f2932ffd640 1 -- 172.21.15.100:0/2147980959 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b65779720 con 0x560b65aee9d0 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f2932ffd640 10 client.17637 handle_command_reply: tid=0 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 client.17637 shutdown 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 msgr2=0x560b65adc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 0x560b65adc200 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2944005d00 tx=0x7f2944014040 comp rx=0 tx=0).stop 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 shutdown_connections 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65aee9d0 0x560b65af0df0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ade750 0x560b65adc200 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ad5b70 0x560b65adbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f2932ffd640 0 client.17637 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:10.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 --2- 172.21.15.100:0/2147980959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5250 0x560b65adb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 >> 172.21.15.100:0/2147980959 conn(0x560b65acf980 msgr2=0x560b65ad16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 shutdown_connections 2024-03-12T05:22:10.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 1 -- 172.21.15.100:0/2147980959 wait complete. 2024-03-12T05:22:10.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.635+0000 7f295cc74740 20 client.17637 trim_cache size 0 max 16384 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.647+0000 7f295cc74740 1 Processor -- start 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.647+0000 7f295cc74740 1 -- start start 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f295cc74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65addeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f295cc74740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ad89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f295cc74740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ad8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f295cc74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b659c2130 con 0x560b65ad5250 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f295cc74740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65ae1390 con 0x560b65ade750 2024-03-12T05:22:10.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f295cc74740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65aee7d0 con 0x560b65ad5b70 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65addeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65addeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56266/0 (socket says 172.21.15.100:56266) 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 -- 172.21.15.100:0/2753788973 learned_addr learned my addr 172.21.15.100:0/2753788973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f29537fe640 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ad89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ad8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 -- 172.21.15.100:0/2753788973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 msgr2=0x560b65ad89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ad89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 -- 172.21.15.100:0/2753788973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 msgr2=0x560b65ad8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ad8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 -- 172.21.15.100:0/2753788973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b659d7c20 con 0x560b65ad5250 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f2952ffd640 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65addeb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f29440054b0 tx=0x7f2944012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24defdb6444a21ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f29397fa640 1 -- 172.21.15.100:0/2753788973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400be90 con 0x560b65ad5250 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f29397fa640 1 -- 172.21.15.100:0/2753788973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2944004b60 con 0x560b65ad5250 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.648+0000 7f29397fa640 1 -- 172.21.15.100:0/2753788973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944005810 con 0x560b65ad5250 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 -- 172.21.15.100:0/2753788973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 msgr2=0x560b65addeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65addeb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f29440054b0 tx=0x7f2944012d00 comp rx=0 tx=0).stop 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 -- 172.21.15.100:0/2753788973 shutdown_connections 2024-03-12T05:22:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ad8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ad89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 --2- 172.21.15.100:0/2753788973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65addeb0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 -- 172.21.15.100:0/2753788973 >> 172.21.15.100:0/2753788973 conn(0x560b65acf980 msgr2=0x560b65ada050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 -- 172.21.15.100:0/2753788973 shutdown_connections 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 -- 172.21.15.100:0/2753788973 wait complete. 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.649+0000 7f295cc74740 1 -- start start 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 Processor -- start 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65ae41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ae46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65b44f60 con 0x560b65ad5250 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65b450a0 con 0x560b65ade750 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b65b45260 con 0x560b65ad5b70 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f295cc74740 10 client.0 _ll_register_callbacks cb 0x7f294400ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65ae41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f29537fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ae46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65ae41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56270/0 (socket says 172.21.15.100:56270) 2024-03-12T05:22:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 -- 172.21.15.100:0/1336422339 learned_addr learned my addr 172.21.15.100:0/1336422339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 -- 172.21.15.100:0/1336422339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 msgr2=0x560b65ae46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ae46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 -- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 msgr2=0x560b65ae4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 -- 172.21.15.100:0/1336422339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b65a3b6a0 con 0x560b65ad5250 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f29537fe640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ae46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2952ffd640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65ae41a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2944005010 tx=0x7f294400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a0cdebd7de0c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.650+0000 7f2932ffd640 1 -- 172.21.15.100:0/1336422339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400be90 con 0x560b65ad5250 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f2932ffd640 1 -- 172.21.15.100:0/1336422339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f2944024070 con 0x560b65ad5250 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2944012da0 con 0x560b65ad5250 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f2932ffd640 1 -- 172.21.15.100:0/1336422339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2944004680 con 0x560b65ad5250 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 10 client.17643 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b659fd8c0 con 0x560b65ad5250 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f2932ffd640 1 -- 172.21.15.100:0/1336422339 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2944012da0 con 0x560b65ad5250 2024-03-12T05:22:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f2932ffd640 1 -- 172.21.15.100:0/1336422339 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f294402d030 con 0x560b65ad5250 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 10 client.17643 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 20 client.17643 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65b46530 0x560b65b48920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 4 client.17643 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b657c8890 con 0x560b65b46530 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.651+0000 7f29537fe640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65b46530 0x560b65b48920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f2932ffd640 10 client.17643 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f29537fe640 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65b46530 0x560b65b48920 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8313f365198a7ea server_cookie=df0d82a8e3c7b523 in_seq=0 out_seq=0 2024-03-12T05:22:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f2932ffd640 1 -- 172.21.15.100:0/1336422339 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b657c8890 con 0x560b65b46530 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f2932ffd640 10 client.17643 handle_command_reply: tid=0 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 client.17643 shutdown 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 msgr2=0x560b65ae41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65ae41a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2944005010 tx=0x7f294400b860 comp rx=0 tx=0).stop 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 shutdown_connections 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560b65b46530 0x560b65b48920 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b65ade750 0x560b65ae4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560b65ad5b70 0x560b65ae46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f2932ffd640 0 client.17643 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:10.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 --2- 172.21.15.100:0/1336422339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b65ad5250 0x560b65ae41a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 >> 172.21.15.100:0/1336422339 conn(0x560b65acf980 msgr2=0x560b65ada4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 shutdown_connections 2024-03-12T05:22:10.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 1 -- 172.21.15.100:0/1336422339 wait complete. 2024-03-12T05:22:10.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.652+0000 7f295cc74740 20 client.17643 trim_cache size 0 max 16384 2024-03-12T05:22:10.659 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:22:10.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f293c0a1c20 msgr2=0x7f293c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f293c0a1c20 0x7f293c0a40e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f294c090300 tx=0x7f294c08f280 comp rx=0 tx=0).stop 2024-03-12T05:22:10.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 msgr2=0x7f29541a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:10.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 0x7f29541a35e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f294c004af0 tx=0x7f294c034cc0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 shutdown_connections 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f293c0a1c20 0x7f293c0a40e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2954199030 0x7f29541a35e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105350 0x7f2954198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 --2- 172.21.15.100:0/3119392588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2954100280 0x7f2954198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 >> 172.21.15.100:0/3119392588 conn(0x7f2954077150 msgr2=0x7f29540fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 shutdown_connections 2024-03-12T05:22:10.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:10.653+0000 7f295a09d640 1 -- 172.21.15.100:0/3119392588 wait complete. 2024-03-12T05:22:10.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1213498 11.87 MB/sec execute 414 sec latency 21.839 ms 2024-03-12T05:22:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:11 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2039857809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:22:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:11 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2039857809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:22:11.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:11 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2039857809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:22:11.663 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:22:11.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:22:11.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:22:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.782+0000 7f922bcdd640 1 -- 172.21.15.100:0/1771837222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9224105330 msgr2=0x7f922410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- 172.21.15.100:0/1771837222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9224105330 0x7f922410cf00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f921c009cd0 tx=0x7f921c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- 172.21.15.100:0/1771837222 shutdown_connections 2024-03-12T05:22:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- 172.21.15.100:0/1771837222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9224105330 0x7f922410cf00 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- 172.21.15.100:0/1771837222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9224100c10 0x7f9224104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- 172.21.15.100:0/1771837222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9224100260 0x7f9224100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- 172.21.15.100:0/1771837222 >> 172.21.15.100:0/1771837222 conn(0x7f9224077180 msgr2=0x7f9224077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- 172.21.15.100:0/1771837222 shutdown_connections 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- 172.21.15.100:0/1771837222 wait complete. 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 Processor -- start 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- start start 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9224100260 0x7f9224198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9224105330 0x7f9224198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 0x7f92241a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241a3d20 con 0x7f92241990d0 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241a3e90 con 0x7f9224105330 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.783+0000 7f922bcdd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241a4050 con 0x7f9224100260 2024-03-12T05:22:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 0x7f92241a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 0x7f92241a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56280/0 (socket says 172.21.15.100:56280) 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 -- 172.21.15.100:0/3211029163 learned_addr learned my addr 172.21.15.100:0/3211029163 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f9229a79640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9224100260 0x7f9224198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f9229278640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9224105330 0x7f9224198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 -- 172.21.15.100:0/3211029163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9224100260 msgr2=0x7f9224198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9224100260 0x7f9224198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 -- 172.21.15.100:0/3211029163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9224105330 msgr2=0x7f9224198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9224105330 0x7f9224198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 -- 172.21.15.100:0/3211029163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c009900 con 0x7f92241990d0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922a27a640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 0x7f92241a3680 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f921c0348f0 tx=0x7f921c034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f921affd640 1 -- 172.21.15.100:0/3211029163 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0328c0 con 0x7f92241990d0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92241a42d0 con 0x7f92241990d0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f921affd640 1 -- 172.21.15.100:0/3211029163 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f921c003e50 con 0x7f92241990d0 2024-03-12T05:22:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.784+0000 7f921affd640 1 -- 172.21.15.100:0/3211029163 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c040450 con 0x7f92241990d0 2024-03-12T05:22:11.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.785+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92241a4780 con 0x7f92241990d0 2024-03-12T05:22:11.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.786+0000 7f921affd640 1 -- 172.21.15.100:0/3211029163 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f921c00a040 con 0x7f92241990d0 2024-03-12T05:22:11.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.787+0000 7f922e981740 1 Processor -- start 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f921affd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92100a1ba0 0x7f92100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 -- start start 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92100a1ba0 0x7f92100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52380 0x55571dd52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f921affd640 1 -- 172.21.15.100:0/3211029163 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f921c0f05e0 con 0x7f92241990d0 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 0x55571dd5b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dcc0f90 con 0x55571dd52d30 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dcbe4c0 con 0x55571dd5b8b0 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922e981740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dcb9ca0 con 0x55571dd52380 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 0x55571dd5b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 0x55571dd5b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56296/0 (socket says 172.21.15.100:56296) 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 -- 172.21.15.100:0/750444693 learned_addr learned my addr 172.21.15.100:0/750444693 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922a27a640 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229a79640 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52380 0x55571dd52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92100a1ba0 0x7f92100a4060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9214013450 tx=0x7f9214013850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 -- 172.21.15.100:0/750444693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52380 msgr2=0x55571dd52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52380 0x55571dd52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 -- 172.21.15.100:0/750444693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 msgr2=0x55571dd5dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd5dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 -- 172.21.15.100:0/750444693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55571dc7c0b0 con 0x55571dd52d30 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f922a27a640 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd5dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.788+0000 7f9229278640 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 0x55571dd5b2e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9214011720 tx=0x7f921401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39527abbc6a66c71 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f920ffff640 1 -- 172.21.15.100:0/750444693 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921401f7f0 con 0x55571dd52d30 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f920ffff640 1 -- 172.21.15.100:0/750444693 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9214019070 con 0x55571dd52d30 2024-03-12T05:22:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f920ffff640 1 -- 172.21.15.100:0/750444693 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921401ec60 con 0x55571dd52d30 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 -- 172.21.15.100:0/750444693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 msgr2=0x55571dd5b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 0x55571dd5b2e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9214011720 tx=0x7f921401d000 comp rx=0 tx=0).stop 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 -- 172.21.15.100:0/750444693 shutdown_connections 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd5dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52d30 0x55571dd5b2e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 --2- 172.21.15.100:0/750444693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52380 0x55571dd52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 -- 172.21.15.100:0/750444693 >> 172.21.15.100:0/750444693 conn(0x55571dd4c840 msgr2=0x55571dd4cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 -- 172.21.15.100:0/750444693 shutdown_connections 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.789+0000 7f922e981740 1 -- 172.21.15.100:0/750444693 wait complete. 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 -- start start 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 Processor -- start 2024-03-12T05:22:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52380 0x55571dd588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd58df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 0x55571dd59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dc5ef40 con 0x55571dd5b8b0 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571d98d4b0 con 0x55571dd52380 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dc3f670 con 0x55571dd52d30 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922e981740 10 client.0 _ll_register_callbacks cb 0x55571dcacf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922a27a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 0x55571dd59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922a27a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 0x55571dd59330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56308/0 (socket says 172.21.15.100:56308) 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f922a27a640 1 -- 172.21.15.100:0/2770501236 learned_addr learned my addr 172.21.15.100:0/2770501236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.790+0000 7f9229a79640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52380 0x55571dd588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922a27a640 1 -- 172.21.15.100:0/2770501236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 msgr2=0x55571dd58df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922a27a640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd58df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922a27a640 1 -- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52380 msgr2=0x55571dd588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922a27a640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52380 0x55571dd588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922a27a640 1 -- 172.21.15.100:0/2770501236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92140114a0 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f9229a79640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52380 0x55571dd588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922a27a640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 0x55571dd59330 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f921c0ede30 tx=0x7f921c0bceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e26e91800c33c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f920d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f920d7fa640 1 -- 172.21.15.100:0/2770501236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0c1400 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f920d7fa640 1 -- 172.21.15.100:0/2770501236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f921c0c5a50 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55571dc93f40 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f920d7fa640 1 -- 172.21.15.100:0/2770501236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921c0c1130 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 10 client.17655 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55571dcb8320 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f920d7fa640 1 -- 172.21.15.100:0/2770501236 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55571dc93f40 con 0x55571dd5b8b0 2024-03-12T05:22:11.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f920d7fa640 1 -- 172.21.15.100:0/2770501236 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f921c0bd510 con 0x55571dd5b8b0 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 10 client.17655 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 20 client.17655 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571dd6bc20 0x55571dd6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 4 client.17655 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.791+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55571dcb0310 con 0x55571dd6bc20 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.792+0000 7f9229a79640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571dd6bc20 0x55571dd6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.792+0000 7f9229a79640 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571dd6bc20 0x55571dd6e040 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8de79ea48b5081b server_cookie=392ce2057ea2e409 in_seq=0 out_seq=0 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.792+0000 7f920d7fa640 10 client.17655 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f920d7fa640 1 -- 172.21.15.100:0/2770501236 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55571dcb0310 con 0x55571dd6bc20 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f920d7fa640 10 client.17655 handle_command_reply: tid=0 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 client.17655 shutdown 2024-03-12T05:22:11.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 msgr2=0x55571dd59330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 0x55571dd59330 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f921c0ede30 tx=0x7f921c0bceb0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 shutdown_connections 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571dd6bc20 0x55571dd6e040 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd5b8b0 0x55571dd59330 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd58df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 --2- 172.21.15.100:0/2770501236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52380 0x55571dd588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f920d7fa640 0 client.17655 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 >> 172.21.15.100:0/2770501236 conn(0x55571dd4c840 msgr2=0x55571dd4e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 shutdown_connections 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 1 -- 172.21.15.100:0/2770501236 wait complete. 2024-03-12T05:22:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.793+0000 7f922e981740 20 client.17655 trim_cache size 0 max 16384 2024-03-12T05:22:11.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1215471 11.87 MB/sec execute 415 sec latency 41.434 ms 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 Processor -- start 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 -- start start 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd62880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dd541d0 con 0x55571dd52380 2024-03-12T05:22:11.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dcb0340 con 0x55571dd5b8b0 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922e981740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571dd4c370 con 0x55571dd52d30 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f9229a79640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f922a27a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd62880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f9229278640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f9229a79640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd59150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56312/0 (socket says 172.21.15.100:56312) 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.808+0000 7f9229a79640 1 -- 172.21.15.100:0/1579340987 learned_addr learned my addr 172.21.15.100:0/1579340987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f922a27a640 1 -- 172.21.15.100:0/1579340987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 msgr2=0x55571dd4b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f922a27a640 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd4b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f922a27a640 1 -- 172.21.15.100:0/1579340987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 msgr2=0x55571dd59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f922a27a640 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f922a27a640 1 -- 172.21.15.100:0/1579340987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55571dc54c00 con 0x55571dd5b8b0 2024-03-12T05:22:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f922a27a640 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd62880 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f921c0b9710 tx=0x7f921c0bcf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4127330664721b29 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f920f7fe640 1 -- 172.21.15.100:0/1579340987 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0c57e0 con 0x55571dd5b8b0 2024-03-12T05:22:11.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f920f7fe640 1 -- 172.21.15.100:0/1579340987 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f921c0c5a50 con 0x55571dd5b8b0 2024-03-12T05:22:11.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.809+0000 7f920f7fe640 1 -- 172.21.15.100:0/1579340987 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0bd510 con 0x55571dd5b8b0 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 -- 172.21.15.100:0/1579340987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 msgr2=0x55571dd62880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd62880 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f921c0b9710 tx=0x7f921c0bcf20 comp rx=0 tx=0).stop 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 -- 172.21.15.100:0/1579340987 shutdown_connections 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd5b8b0 0x55571dd62880 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd52d30 0x55571dd4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 --2- 172.21.15.100:0/1579340987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 -- 172.21.15.100:0/1579340987 >> 172.21.15.100:0/1579340987 conn(0x55571dd4c840 msgr2=0x55571dd4b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 -- 172.21.15.100:0/1579340987 shutdown_connections 2024-03-12T05:22:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 -- 172.21.15.100:0/1579340987 wait complete. 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 -- start start 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.810+0000 7f922e981740 1 Processor -- start 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd61a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52d30 0x55571dd61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd5b8b0 0x55571ddc2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ddc2670 con 0x55571dd52380 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ddc27b0 con 0x55571dd52d30 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ddc2920 con 0x55571dd5b8b0 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 10 client.0 _ll_register_callbacks cb 0x7f921400b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd61a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd61a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56324/0 (socket says 172.21.15.100:56324) 2024-03-12T05:22:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 -- 172.21.15.100:0/152773751 learned_addr learned my addr 172.21.15.100:0/152773751 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229278640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52d30 0x55571dd61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922a27a640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd5b8b0 0x55571ddc2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 -- 172.21.15.100:0/152773751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd5b8b0 msgr2=0x55571ddc2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd5b8b0 0x55571ddc2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 -- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52d30 msgr2=0x55571dd61f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52d30 0x55571dd61f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 -- 172.21.15.100:0/152773751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55571dcb8320 con 0x55571dd52380 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229278640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52d30 0x55571dd61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f9229a79640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd61a50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x55571dd6e920 tx=0x7f922001e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4809ccfdd6f3f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f920d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f920d7fa640 1 -- 172.21.15.100:0/152773751 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220003c40 con 0x55571dd52380 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f920d7fa640 1 -- 172.21.15.100:0/152773751 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9220003da0 con 0x55571dd52380 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55571dcdf900 con 0x55571dd52380 2024-03-12T05:22:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f920d7fa640 1 -- 172.21.15.100:0/152773751 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f922000bed0 con 0x55571dd52380 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 10 client.17661 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.811+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55571dc7a910 con 0x55571dd52380 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f920d7fa640 1 -- 172.21.15.100:0/152773751 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9220032080 con 0x55571dd52380 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f920d7fa640 1 -- 172.21.15.100:0/152773751 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9220004c60 con 0x55571dd52380 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f922e981740 10 client.17661 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f922e981740 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f922e981740 20 client.17661 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f922e981740 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571ddc4210 0x55571ddc6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f922e981740 4 client.17661 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55571d9b9860 con 0x55571ddc4210 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f9229278640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571ddc4210 0x55571ddc6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f920d7fa640 10 client.17661 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.812+0000 7f9229278640 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571ddc4210 0x55571ddc6630 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee07653516f1269d server_cookie=8cc12a638889872f in_seq=0 out_seq=0 2024-03-12T05:22:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f920d7fa640 1 -- 172.21.15.100:0/152773751 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55571d9b9860 con 0x55571ddc4210 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f920d7fa640 10 client.17661 handle_command_reply: tid=0 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 client.17661 shutdown 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 msgr2=0x55571dd61a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd61a50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x55571dd6e920 tx=0x7f922001e770 comp rx=0 tx=0).stop 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 shutdown_connections 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55571ddc4210 0x55571ddc6630 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55571dd5b8b0 0x55571ddc2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55571dd52d30 0x55571dd61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 --2- 172.21.15.100:0/152773751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55571dd52380 0x55571dd61a50 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 >> 172.21.15.100:0/152773751 conn(0x55571dd4c840 msgr2=0x55571dd56d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f920d7fa640 0 client.17661 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 shutdown_connections 2024-03-12T05:22:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 1 -- 172.21.15.100:0/152773751 wait complete. 2024-03-12T05:22:11.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922e981740 20 client.17661 trim_cache size 0 max 16384 2024-03-12T05:22:11.816 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:22:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:22:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "3af6084b-4121-4197-bf01-338b71131f90", 2024-03-12T05:22:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:22:11.816 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:22:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92100a1ba0 msgr2=0x7f92100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922bcdd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92100a1ba0 0x7f92100a4060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9214013450 tx=0x7f9214013850 comp rx=0 tx=0).stop 2024-03-12T05:22:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 msgr2=0x7f92241a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.814+0000 7f922bcdd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 0x7f92241a3680 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f921c0348f0 tx=0x7f921c034cc0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 shutdown_connections 2024-03-12T05:22:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92100a1ba0 0x7f92100a4060 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92241990d0 0x7f92241a3680 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9224105330 0x7f9224198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 --2- 172.21.15.100:0/3211029163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9224100260 0x7f9224198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 >> 172.21.15.100:0/3211029163 conn(0x7f9224077180 msgr2=0x7f92240fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 shutdown_connections 2024-03-12T05:22:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.815+0000 7f922bcdd640 1 -- 172.21.15.100:0/3211029163 wait complete. 2024-03-12T05:22:11.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:22:11.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 -- 172.21.15.100:0/715211430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141002d0 msgr2=0x7f8914100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 --2- 172.21.15.100:0/715211430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141002d0 0x7f8914100730 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 -- 172.21.15.100:0/715211430 shutdown_connections 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 --2- 172.21.15.100:0/715211430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8914100c70 0x7f891410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 --2- 172.21.15.100:0/715211430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141002d0 0x7f8914100730 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 --2- 172.21.15.100:0/715211430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89141060a0 0x7f8914106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.934+0000 7f89195d0640 1 -- 172.21.15.100:0/715211430 >> 172.21.15.100:0/715211430 conn(0x7f8914076e10 msgr2=0x7f8914077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 -- 172.21.15.100:0/715211430 shutdown_connections 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 -- 172.21.15.100:0/715211430 wait complete. 2024-03-12T05:22:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 Processor -- start 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 -- start start 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 0x7f891419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8914100c70 0x7f891419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141060a0 0x7f8914197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914197c70 con 0x7f89141060a0 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914197de0 con 0x7f89141002d0 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.935+0000 7f89195d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914197fa0 con 0x7f8914100c70 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 0x7f891419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f89137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141060a0 0x7f8914197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 0x7f891419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44888/0 (socket says 172.21.15.100:44888) 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 -- 172.21.15.100:0/3202992940 learned_addr learned my addr 172.21.15.100:0/3202992940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f89127fc640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8914100c70 0x7f891419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 -- 172.21.15.100:0/3202992940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8914100c70 msgr2=0x7f891419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8914100c70 0x7f891419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 -- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141060a0 msgr2=0x7f8914197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141060a0 0x7f8914197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 -- 172.21.15.100:0/3202992940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c009490 con 0x7f89141002d0 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f89137fe640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141060a0 0x7f8914197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f8912ffd640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 0x7f891419d470 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f890400eb90 tx=0x7f890400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f88fbfff640 1 -- 172.21.15.100:0/3202992940 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400cea0 con 0x7f89141002d0 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8908009560 con 0x7f89141002d0 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f88fbfff640 1 -- 172.21.15.100:0/3202992940 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8904005020 con 0x7f89141002d0 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f88fbfff640 1 -- 172.21.15.100:0/3202992940 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904010670 con 0x7f89141002d0 2024-03-12T05:22:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.936+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89141a4330 con 0x7f89141002d0 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f88fbfff640 1 -- 172.21.15.100:0/3202992940 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8904005190 con 0x7f89141002d0 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 Processor -- start 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 -- start start 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f445eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eb640 0x560f445f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 0x560f445fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4455a280 con 0x560f445f8bc0 2024-03-12T05:22:11.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f445577b0 con 0x560f445eac90 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f891c19c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f44552f90 con 0x560f445eb640 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 0x560f445fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89137fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f445eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 0x560f445fafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56352/0 (socket says 172.21.15.100:56352) 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 -- 172.21.15.100:0/447143896 learned_addr learned my addr 172.21.15.100:0/447143896 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f88fbfff640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f88fc0a1ba0 0x7f88fc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f8912ffd640 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eb640 0x560f445f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 -- 172.21.15.100:0/447143896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eb640 msgr2=0x560f445f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eb640 0x560f445f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 -- 172.21.15.100:0/447143896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 msgr2=0x560f445eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f445eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.939+0000 7f89127fc640 1 -- 172.21.15.100:0/447143896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f44511570 con 0x560f445f8bc0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.940+0000 7f89137fe640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f88fc0a1ba0 0x7f88fc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.940+0000 7f88fbfff640 1 -- 172.21.15.100:0/3202992940 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8904014070 con 0x7f89141002d0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.940+0000 7f89127fc640 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 0x560f445fafb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eab376b1ada7919 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.940+0000 7f88f97fa640 1 -- 172.21.15.100:0/447143896 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080047c0 con 0x560f445f8bc0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.940+0000 7f88f97fa640 1 -- 172.21.15.100:0/447143896 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8908004920 con 0x560f445f8bc0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.940+0000 7f88f97fa640 1 -- 172.21.15.100:0/447143896 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908005150 con 0x560f445f8bc0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- 172.21.15.100:0/447143896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 msgr2=0x560f445fafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 0x560f445fafb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802c6a0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f89137fe640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f88fc0a1ba0 0x7f88fc0a4060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f890c009fd0 tx=0x7f890c0046c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- 172.21.15.100:0/447143896 shutdown_connections 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445f8bc0 0x560f445fafb0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eb640 0x560f445f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- 172.21.15.100:0/447143896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f445eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- 172.21.15.100:0/447143896 >> 172.21.15.100:0/447143896 conn(0x560f445e5990 msgr2=0x560f445e5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- 172.21.15.100:0/447143896 shutdown_connections 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- 172.21.15.100:0/447143896 wait complete. 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- start start 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 Processor -- start 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 0x560f445f8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eb640 0x560f445f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f44226430 con 0x560f445eac90 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f441e2e00 con 0x560f445eb640 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f444d7fd0 con 0x560f445f8bc0 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f891c19c740 10 client.0 _ll_register_callbacks cb 0x560f4452dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:11.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f8912ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 0x560f445f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f8912ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 0x560f445f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56358/0 (socket says 172.21.15.100:56358) 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f8912ffd640 1 -- 172.21.15.100:0/152450935 learned_addr learned my addr 172.21.15.100:0/152450935 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.941+0000 7f89127fc640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eb640 0x560f445f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f89137fe640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f8912ffd640 1 -- 172.21.15.100:0/152450935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 msgr2=0x560f445f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f8912ffd640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f8912ffd640 1 -- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eb640 msgr2=0x560f445f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f8912ffd640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eb640 0x560f445f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f8912ffd640 1 -- 172.21.15.100:0/152450935 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891410d740 con 0x560f445eac90 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f8912ffd640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 0x560f445f8550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f89040049d0 tx=0x7f890409b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429da3066bfe7ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f88f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f88f6ff5640 1 -- 172.21.15.100:0/152450935 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89040ae070 con 0x560f445eac90 2024-03-12T05:22:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f88f6ff5640 1 -- 172.21.15.100:0/152450935 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f890409e890 con 0x560f445eac90 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f442527e0 con 0x560f445eac90 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f88f6ff5640 1 -- 172.21.15.100:0/152450935 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89040b32e0 con 0x560f445eac90 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 10 client.17673 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560f44551610 con 0x560f445eac90 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f88f6ff5640 1 -- 172.21.15.100:0/152450935 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89040b7030 con 0x560f445eac90 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f88f6ff5640 1 -- 172.21.15.100:0/152450935 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f890401f060 con 0x560f445eac90 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 10 client.17673 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 20 client.17673 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f44608d60 0x560f4460b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 4 client.17673 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.942+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f4428f720 con 0x560f44608d60 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.943+0000 7f89127fc640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f44608d60 0x560f4460b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.943+0000 7f88f6ff5640 10 client.17673 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.943+0000 7f89127fc640 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f44608d60 0x560f4460b150 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72c55ec62cc4911 server_cookie=6d231380ff205e7b in_seq=0 out_seq=0 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.943+0000 7f88f6ff5640 1 -- 172.21.15.100:0/152450935 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f4428f720 con 0x560f44608d60 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.943+0000 7f88f6ff5640 10 client.17673 handle_command_reply: tid=0 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 client.17673 shutdown 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 msgr2=0x560f445f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 0x560f445f8550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f89040049d0 tx=0x7f890409b210 comp rx=0 tx=0).stop 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 shutdown_connections 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f44608d60 0x560f4460b150 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eb640 0x560f445f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f88f6ff5640 0 client.17673 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 --2- 172.21.15.100:0/152450935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eac90 0x560f445f8550 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 >> 172.21.15.100:0/152450935 conn(0x560f445e5990 msgr2=0x560f445e7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 shutdown_connections 2024-03-12T05:22:11.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 1 -- 172.21.15.100:0/152450935 wait complete. 2024-03-12T05:22:11.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.944+0000 7f891c19c740 20 client.17673 trim_cache size 0 max 16384 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 Processor -- start 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 -- start start 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f446044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445f8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4428f720 con 0x560f445eb640 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f445e10f0 con 0x560f445eac90 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f891c19c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f445ea5f0 con 0x560f445f8bc0 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56364/0 (socket says 172.21.15.100:56364) 2024-03-12T05:22:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 -- 172.21.15.100:0/2847514220 learned_addr learned my addr 172.21.15.100:0/2847514220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f8912ffd640 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f446044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89137fe640 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 -- 172.21.15.100:0/2847514220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 msgr2=0x560f445f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 -- 172.21.15.100:0/2847514220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 msgr2=0x560f446044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f446044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 -- 172.21.15.100:0/2847514220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f444edac0 con 0x560f445eb640 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f8912ffd640 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f446044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.960+0000 7f89127fc640 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445f8010 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x560f445e6e00 tx=0x7f8908004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389904532972b7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f88f8ff9640 1 -- 172.21.15.100:0/2847514220 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908051b00 con 0x560f445eb640 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f88f8ff9640 1 -- 172.21.15.100:0/2847514220 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89080513f0 con 0x560f445eb640 2024-03-12T05:22:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f88f8ff9640 1 -- 172.21.15.100:0/2847514220 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908005280 con 0x560f445eb640 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 -- 172.21.15.100:0/2847514220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 msgr2=0x560f445f8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445f8010 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x560f445e6e00 tx=0x7f8908004df0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 -- 172.21.15.100:0/2847514220 shutdown_connections 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445f8bc0 0x560f445f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445f8010 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 --2- 172.21.15.100:0/2847514220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445eac90 0x560f446044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 -- 172.21.15.100:0/2847514220 >> 172.21.15.100:0/2847514220 conn(0x560f445e5990 msgr2=0x560f4460fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 -- 172.21.15.100:0/2847514220 shutdown_connections 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.961+0000 7f891c19c740 1 -- 172.21.15.100:0/2847514220 wait complete. 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 -- start start 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 Processor -- start 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eac90 0x560f445ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445f8bc0 0x560f445fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f445fcd40 con 0x560f445eb640 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f445fceb0 con 0x560f445f8bc0 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4465f250 con 0x560f445eac90 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f891c19c740 10 client.0 _ll_register_callbacks cb 0x7f89040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445fc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56380/0 (socket says 172.21.15.100:56380) 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 -- 172.21.15.100:0/2939478793 learned_addr learned my addr 172.21.15.100:0/2939478793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89137fe640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445f8bc0 0x560f445fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f8912ffd640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eac90 0x560f445ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 -- 172.21.15.100:0/2939478793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eac90 msgr2=0x560f445ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eac90 0x560f445ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 -- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445f8bc0 msgr2=0x560f445fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445f8bc0 0x560f445fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.962+0000 7f89127fc640 1 -- 172.21.15.100:0/2939478793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f44511570 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f89127fc640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445fc130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f890802c170 tx=0x7f8908004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299ad7dec339efca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f88f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f88f6ff5640 1 -- 172.21.15.100:0/2939478793 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890802c9f0 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f88f6ff5640 1 -- 172.21.15.100:0/2939478793 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f890802cb50 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8908004920 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f88f6ff5640 1 -- 172.21.15.100:0/2939478793 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8908004f00 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 10 client.17679 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560f445136a0 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f88f6ff5640 1 -- 172.21.15.100:0/2939478793 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8908004920 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f88f6ff5640 1 -- 172.21.15.100:0/2939478793 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f890803c050 con 0x560f445eb640 2024-03-12T05:22:11.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 10 client.17679 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 20 client.17679 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f446604d0 0x560f446628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 4 client.17679 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f442de890 con 0x560f446604d0 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.963+0000 7f8912ffd640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f446604d0 0x560f446628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f88f6ff5640 10 client.17679 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f8912ffd640 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f446604d0 0x560f446628f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9d2084230ff440 server_cookie=294f4e9f86290647 in_seq=0 out_seq=0 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f88f6ff5640 1 -- 172.21.15.100:0/2939478793 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560f442de890 con 0x560f446604d0 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f88f6ff5640 10 client.17679 handle_command_reply: tid=0 2024-03-12T05:22:11.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 client.17679 shutdown 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 msgr2=0x560f445fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445fc130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f890802c170 tx=0x7f8908004c10 comp rx=0 tx=0).stop 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 shutdown_connections 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f446604d0 0x560f446628f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f445f8bc0 0x560f445fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f445eb640 0x560f445fc130 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f88f6ff5640 0 client.17679 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 --2- 172.21.15.100:0/2939478793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f445eac90 0x560f445ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 >> 172.21.15.100:0/2939478793 conn(0x560f445e5990 msgr2=0x560f445f4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 shutdown_connections 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 1 -- 172.21.15.100:0/2939478793 wait complete. 2024-03-12T05:22:11.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.964+0000 7f891c19c740 20 client.17679 trim_cache size 0 max 16384 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "3af6084b-4121-4197-bf01-338b71131f90": { 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "3af6084b-4121-4197-bf01-338b71131f90", 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:22:11.968 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:22:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f88fc0a1ba0 msgr2=0x7f88fc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f88fc0a1ba0 0x7f88fc0a4060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f890c009fd0 tx=0x7f890c0046c0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 msgr2=0x7f891419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 0x7f891419d470 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f890400eb90 tx=0x7f890400ef90 comp rx=0 tx=0).stop 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 shutdown_connections 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f88fc0a1ba0 0x7f88fc0a4060 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89141060a0 0x7f8914197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8914100c70 0x7f891419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 --2- 172.21.15.100:0/3202992940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89141002d0 0x7f891419d470 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 >> 172.21.15.100:0/3202992940 conn(0x7f8914076e10 msgr2=0x7f89141044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 shutdown_connections 2024-03-12T05:22:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:11.965+0000 7f89195d0640 1 -- 172.21.15.100:0/3202992940 wait complete. 2024-03-12T05:22:11.972 INFO:tasks.cephfs.filesystem:scrub status for tag:3af6084b-4121-4197-bf01-338b71131f90 - {'path': '/', 'tag': '3af6084b-4121-4197-bf01-338b71131f90', 'options': 'recursive,force'} 2024-03-12T05:22:12.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:12 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:22:12.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:12 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:12.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:12 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:22:12.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:12 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:12.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:12 smithi100 ceph-mon[32210]: pgmap v313: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-03-12T05:22:12.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:12 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:22:12.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:12 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:12.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:12 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:22:12.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:12 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:12.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:12 smithi156 ceph-mon[36270]: pgmap v313: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-03-12T05:22:12.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:12 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:22:12.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:12 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:12.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:12 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:22:12.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:12 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:12.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:12 smithi161 ceph-mon[36043]: pgmap v313: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-03-12T05:22:12.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1217869 11.87 MB/sec execute 416 sec latency 86.185 ms 2024-03-12T05:22:13.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:13 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:22:13.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:13 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:13.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:13 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:22:13.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:13 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:13.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:13 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:22:13.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:13 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:13.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1219645 11.87 MB/sec execute 417 sec latency 44.311 ms 2024-03-12T05:22:14.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:14 smithi100 ceph-mon[32210]: pgmap v314: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-03-12T05:22:14.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:14 smithi156 ceph-mon[36270]: pgmap v314: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-03-12T05:22:14.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:14 smithi161 ceph-mon[36043]: pgmap v314: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-03-12T05:22:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1221993 11.87 MB/sec execute 418 sec latency 76.757 ms 2024-03-12T05:22:15.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1223668 11.86 MB/sec execute 419 sec latency 17.455 ms 2024-03-12T05:22:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1225893 11.86 MB/sec execute 420 sec latency 17.745 ms 2024-03-12T05:22:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:16 smithi156 ceph-mon[36270]: osdmap e97: 12 total, 12 up, 12 in 2024-03-12T05:22:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:16 smithi156 ceph-mon[36270]: pgmap v316: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-03-12T05:22:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:16 smithi161 ceph-mon[36043]: osdmap e97: 12 total, 12 up, 12 in 2024-03-12T05:22:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:16 smithi161 ceph-mon[36043]: pgmap v316: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-03-12T05:22:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:16 smithi100 ceph-mon[32210]: osdmap e97: 12 total, 12 up, 12 in 2024-03-12T05:22:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:16 smithi100 ceph-mon[32210]: pgmap v316: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-03-12T05:22:17.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1228233 11.86 MB/sec execute 421 sec latency 24.216 ms 2024-03-12T05:22:18.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1230228 11.86 MB/sec execute 422 sec latency 25.550 ms 2024-03-12T05:22:19.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:18 smithi156 ceph-mon[36270]: pgmap v317: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.1 MiB/s wr, 307 op/s 2024-03-12T05:22:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:18 smithi161 ceph-mon[36043]: pgmap v317: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.1 MiB/s wr, 307 op/s 2024-03-12T05:22:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:18 smithi100 ceph-mon[32210]: pgmap v317: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.1 MiB/s wr, 307 op/s 2024-03-12T05:22:19.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1232633 11.87 MB/sec execute 423 sec latency 64.598 ms 2024-03-12T05:22:20.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:19 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:22:20.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:19 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:22:20.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:19 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:22:20.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1234168 11.85 MB/sec execute 424 sec latency 110.614 ms 2024-03-12T05:22:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:20 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:22:21.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:20 smithi156 ceph-mon[36270]: pgmap v318: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 248 op/s 2024-03-12T05:22:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:20 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:22:21.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:20 smithi161 ceph-mon[36043]: pgmap v318: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 248 op/s 2024-03-12T05:22:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:20 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:22:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:20 smithi100 ceph-mon[32210]: pgmap v318: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 248 op/s 2024-03-12T05:22:21.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1236527 11.86 MB/sec execute 425 sec latency 15.977 ms 2024-03-12T05:22:22.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:21 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:22:22.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:21 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:22:22.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:21 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:22:22.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1238425 11.85 MB/sec execute 426 sec latency 39.043 ms 2024-03-12T05:22:23.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:22 smithi156 ceph-mon[36270]: pgmap v319: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-03-12T05:22:23.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:22 smithi161 ceph-mon[36043]: pgmap v319: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-03-12T05:22:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:22 smithi100 ceph-mon[32210]: pgmap v319: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-03-12T05:22:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1240241 11.85 MB/sec execute 427 sec latency 14.522 ms 2024-03-12T05:22:24.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:22:24.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: pgmap v320: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-03-12T05:22:24.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:22:24.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:22:24.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: pgmap v320: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-03-12T05:22:24.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.703 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:22:24.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:22:24.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: pgmap v320: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-03-12T05:22:24.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1242677 11.85 MB/sec execute 428 sec latency 21.672 ms 2024-03-12T05:22:25.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:22:25.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:22:25.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:22:25.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:22:25.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:22:25.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:22:25.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:22:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:22:25.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:25.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:22:25.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:22:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1244209 11.84 MB/sec execute 429 sec latency 84.998 ms 2024-03-12T05:22:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:26 smithi161 ceph-mon[36043]: pgmap v321: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 251 op/s 2024-03-12T05:22:26.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:26 smithi100 ceph-mon[32210]: pgmap v321: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 251 op/s 2024-03-12T05:22:26.619 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:26 smithi156 ceph-mon[36270]: pgmap v321: 97 pgs: 97 active+clean; 559 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 251 op/s 2024-03-12T05:22:26.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1246838 11.85 MB/sec execute 430 sec latency 18.601 ms 2024-03-12T05:22:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1248757 11.84 MB/sec execute 431 sec latency 17.542 ms 2024-03-12T05:22:28.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1250783 11.84 MB/sec execute 432 sec latency 71.475 ms 2024-03-12T05:22:29.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:28 smithi156 ceph-mon[36270]: pgmap v322: 97 pgs: 97 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 307 op/s 2024-03-12T05:22:29.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:28 smithi161 ceph-mon[36043]: pgmap v322: 97 pgs: 97 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 307 op/s 2024-03-12T05:22:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:28 smithi100 ceph-mon[32210]: pgmap v322: 97 pgs: 97 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 307 op/s 2024-03-12T05:22:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1252711 11.83 MB/sec execute 433 sec latency 216.494 ms 2024-03-12T05:22:30.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1254567 11.83 MB/sec execute 434 sec latency 73.219 ms 2024-03-12T05:22:31.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:30 smithi156 ceph-mon[36270]: pgmap v323: 97 pgs: 97 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 200 op/s 2024-03-12T05:22:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:30 smithi161 ceph-mon[36043]: pgmap v323: 97 pgs: 97 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 200 op/s 2024-03-12T05:22:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:30 smithi100 ceph-mon[32210]: pgmap v323: 97 pgs: 97 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 200 op/s 2024-03-12T05:22:31.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1256932 11.84 MB/sec execute 435 sec latency 33.830 ms 2024-03-12T05:22:32.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1258736 11.83 MB/sec execute 436 sec latency 24.544 ms 2024-03-12T05:22:33.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:32 smithi156 ceph-mon[36270]: pgmap v324: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 294 op/s 2024-03-12T05:22:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:32 smithi161 ceph-mon[36043]: pgmap v324: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 294 op/s 2024-03-12T05:22:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:32 smithi100 ceph-mon[32210]: pgmap v324: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 294 op/s 2024-03-12T05:22:33.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1261064 11.84 MB/sec execute 437 sec latency 28.262 ms 2024-03-12T05:22:34.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1262955 11.82 MB/sec execute 438 sec latency 15.232 ms 2024-03-12T05:22:35.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:34 smithi156 ceph-mon[36270]: pgmap v325: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-03-12T05:22:35.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:34 smithi161 ceph-mon[36043]: pgmap v325: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-03-12T05:22:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:34 smithi100 ceph-mon[32210]: pgmap v325: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-03-12T05:22:35.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1264892 11.83 MB/sec execute 439 sec latency 55.361 ms 2024-03-12T05:22:36.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1267226 11.83 MB/sec execute 440 sec latency 23.735 ms 2024-03-12T05:22:37.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:36 smithi156 ceph-mon[36270]: pgmap v326: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-03-12T05:22:37.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:36 smithi161 ceph-mon[36043]: pgmap v326: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-03-12T05:22:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:36 smithi100 ceph-mon[32210]: pgmap v326: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-03-12T05:22:37.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1269129 11.82 MB/sec execute 441 sec latency 40.633 ms 2024-03-12T05:22:38.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1271468 11.83 MB/sec execute 442 sec latency 54.089 ms 2024-03-12T05:22:39.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:38 smithi156 ceph-mon[36270]: pgmap v327: 97 pgs: 97 active+clean; 595 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 286 op/s 2024-03-12T05:22:39.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:38 smithi161 ceph-mon[36043]: pgmap v327: 97 pgs: 97 active+clean; 595 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 286 op/s 2024-03-12T05:22:39.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:38 smithi100 ceph-mon[32210]: pgmap v327: 97 pgs: 97 active+clean; 595 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 286 op/s 2024-03-12T05:22:39.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1272813 11.82 MB/sec execute 443 sec latency 129.093 ms 2024-03-12T05:22:40.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1275084 11.82 MB/sec execute 444 sec latency 61.626 ms 2024-03-12T05:22:41.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:40 smithi156 ceph-mon[36270]: pgmap v328: 97 pgs: 97 active+clean; 595 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 189 op/s 2024-03-12T05:22:41.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:40 smithi161 ceph-mon[36043]: pgmap v328: 97 pgs: 97 active+clean; 595 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 189 op/s 2024-03-12T05:22:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:40 smithi100 ceph-mon[32210]: pgmap v328: 97 pgs: 97 active+clean; 595 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 189 op/s 2024-03-12T05:22:41.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1276629 11.81 MB/sec execute 445 sec latency 17.216 ms 2024-03-12T05:22:41.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:22:41.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 -- 172.21.15.100:0/2716572185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 msgr2=0x7ff03410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 --2- 172.21.15.100:0/2716572185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff03410cf40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009cd0 tx=0x7ff02c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 -- 172.21.15.100:0/2716572185 shutdown_connections 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 --2- 172.21.15.100:0/2716572185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff03410cf40 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 --2- 172.21.15.100:0/2716572185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff034100c50 0x7ff034104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 --2- 172.21.15.100:0/2716572185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 0x7ff034100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.091+0000 7ff038e55640 1 -- 172.21.15.100:0/2716572185 >> 172.21.15.100:0/2716572185 conn(0x7ff034077210 msgr2=0x7ff034077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.092+0000 7ff038e55640 1 -- 172.21.15.100:0/2716572185 shutdown_connections 2024-03-12T05:22:42.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.092+0000 7ff038e55640 1 -- 172.21.15.100:0/2716572185 wait complete. 2024-03-12T05:22:42.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.092+0000 7ff038e55640 1 Processor -- start 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.092+0000 7ff038e55640 1 -- start start 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff038e55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 0x7ff034196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff038e55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff034196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff038e55640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff03419af40 0x7ff03419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff038e55640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419b930 con 0x7ff034105370 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff038e55640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419baa0 con 0x7ff03419af40 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff038e55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419bc10 con 0x7ff0341002a0 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff034196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff034196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56684/0 (socket says 172.21.15.100:56684) 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff02bfff640 1 -- 172.21.15.100:0/4092068561 learned_addr learned my addr 172.21.15.100:0/4092068561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff03259c640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 0x7ff034196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff032d9d640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff03419af40 0x7ff03419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff03259c640 1 -- 172.21.15.100:0/4092068561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff03419af40 msgr2=0x7ff03419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff03259c640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff03419af40 0x7ff03419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff03259c640 1 -- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 msgr2=0x7ff034196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff03259c640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff034196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff03259c640 1 -- 172.21.15.100:0/4092068561 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c009900 con 0x7ff0341002a0 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.093+0000 7ff02bfff640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff034196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.094+0000 7ff03259c640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 0x7ff034196680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb00 tx=0x7ff024009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.094+0000 7ff02b7fe640 1 -- 172.21.15.100:0/4092068561 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024013070 con 0x7ff0341002a0 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.094+0000 7ff02b7fe640 1 -- 172.21.15.100:0/4092068561 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff024005500 con 0x7ff0341002a0 2024-03-12T05:22:42.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.094+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034069520 con 0x7ff0341002a0 2024-03-12T05:22:42.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.094+0000 7ff02b7fe640 1 -- 172.21.15.100:0/4092068561 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024004d00 con 0x7ff0341002a0 2024-03-12T05:22:42.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.094+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034069980 con 0x7ff0341002a0 2024-03-12T05:22:42.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.096+0000 7ff02b7fe640 1 -- 172.21.15.100:0/4092068561 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff02400bc00 con 0x7ff0341002a0 2024-03-12T05:22:42.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.096+0000 7ff03ba2f740 1 Processor -- start 2024-03-12T05:22:42.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.096+0000 7ff03ba2f740 1 -- start start 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.096+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14f9a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.096+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.096+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14fa37b0 0x55bf14fa5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14f09280 con 0x55bf14f9abd0 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14f067b0 con 0x55bf14f9a2b0 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14f01f90 con 0x55bf14fa37b0 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff02b7fe640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0200a1ba0 0x7ff0200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff032d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14f9a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff032d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14f9a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60604/0 (socket says 172.21.15.100:60604) 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff032d9d640 1 -- 172.21.15.100:0/3325297757 learned_addr learned my addr 172.21.15.100:0/3325297757 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff02bfff640 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14fa37b0 0x55bf14fa5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff032d9d640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0200a1ba0 0x7ff0200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff02b7fe640 1 -- 172.21.15.100:0/4092068561 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff0240cef80 con 0x7ff0341002a0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 -- 172.21.15.100:0/3325297757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14fa37b0 msgr2=0x55bf14fa5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14fa37b0 0x55bf14fa5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 -- 172.21.15.100:0/3325297757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 msgr2=0x55bf14f9a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14f9a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 -- 172.21.15.100:0/3325297757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf14ec0570 con 0x55bf14f9abd0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff02bfff640 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14fa37b0 0x55bf14fa5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff032d9d640 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14f9a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff03259c640 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa3270 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff0240cc690 tx=0x7ff02406ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1016967ab5888110 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff032d9d640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0200a1ba0 0x7ff0200a4060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009430 tx=0x7ff02c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff028ff9640 1 -- 172.21.15.100:0/3325297757 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02409cea0 con 0x55bf14f9abd0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.097+0000 7ff028ff9640 1 -- 172.21.15.100:0/3325297757 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02409b4d0 con 0x55bf14f9abd0 2024-03-12T05:22:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff028ff9640 1 -- 172.21.15.100:0/3325297757 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a4310 con 0x55bf14f9abd0 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 -- 172.21.15.100:0/3325297757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 msgr2=0x55bf14fa3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa3270 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff0240cc690 tx=0x7ff02406ee20 comp rx=0 tx=0).stop 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 -- 172.21.15.100:0/3325297757 shutdown_connections 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14fa37b0 0x55bf14fa5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa3270 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/3325297757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14f9a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 -- 172.21.15.100:0/3325297757 >> 172.21.15.100:0/3325297757 conn(0x55bf14f94990 msgr2=0x55bf14f94d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 -- 172.21.15.100:0/3325297757 shutdown_connections 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.098+0000 7ff03ba2f740 1 -- 172.21.15.100:0/3325297757 wait complete. 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 -- start start 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 Processor -- start 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14fa0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14bd5430 con 0x55bf14fa37b0 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14b91e00 con 0x55bf14f9a2b0 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14e86fd0 con 0x55bf14f9abd0 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03ba2f740 10 client.0 _ll_register_callbacks cb 0x55bf14edcfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60614/0 (socket says 172.21.15.100:60614) 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 -- 172.21.15.100:0/107342011 learned_addr learned my addr 172.21.15.100:0/107342011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff02bfff640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14fa0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 -- 172.21.15.100:0/107342011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 msgr2=0x55bf14fa0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14fa0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 -- 172.21.15.100:0/107342011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 msgr2=0x55bf14fa0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff03259c640 1 -- 172.21.15.100:0/107342011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034069980 con 0x55bf14f9a2b0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.099+0000 7ff032d9d640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03259c640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa3270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff024097fd0 tx=0x7ff0240ccb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c321b8f2a977917 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff019ffb640 1 -- 172.21.15.100:0/107342011 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02409cea0 con 0x55bf14f9a2b0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf14c017e0 con 0x55bf14f9a2b0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff019ffb640 1 -- 172.21.15.100:0/107342011 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff024005670 con 0x55bf14f9a2b0 2024-03-12T05:22:42.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff019ffb640 1 -- 172.21.15.100:0/107342011 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0240a4310 con 0x55bf14f9a2b0 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 10 client.27071 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bf14f00610 con 0x55bf14f9a2b0 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff019ffb640 1 -- 172.21.15.100:0/107342011 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bf14c017e0 con 0x55bf14f9a2b0 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff019ffb640 1 -- 172.21.15.100:0/107342011 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff02409afc0 con 0x55bf14f9a2b0 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 10 client.27071 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 10 client.27071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 20 client.27071 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf14fb3770 0x55bf14fb5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 4 client.27071 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.100+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf14c3e720 con 0x55bf14fb3770 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.101+0000 7ff02bfff640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf14fb3770 0x55bf14fb5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.101+0000 7ff019ffb640 10 client.27071 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.101+0000 7ff02bfff640 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf14fb3770 0x55bf14fb5b90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e158ffc5b0159d server_cookie=90ffbe2b483f9b5a in_seq=0 out_seq=0 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff019ffb640 1 -- 172.21.15.100:0/107342011 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf14c3e720 con 0x55bf14fb3770 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff019ffb640 10 client.27071 handle_command_reply: tid=0 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 client.27071 shutdown 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 msgr2=0x55bf14fa3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa3270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff024097fd0 tx=0x7ff0240ccb40 comp rx=0 tx=0).stop 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 shutdown_connections 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf14fb3770 0x55bf14fb5b90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14fa0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff019ffb640 0 client.27071 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/107342011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa3270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 >> 172.21.15.100:0/107342011 conn(0x55bf14f94990 msgr2=0x55bf14f96730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 shutdown_connections 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 1 -- 172.21.15.100:0/107342011 wait complete. 2024-03-12T05:22:42.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.102+0000 7ff03ba2f740 20 client.27071 trim_cache size 0 max 16384 2024-03-12T05:22:42.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.118+0000 7ff03ba2f740 1 Processor -- start 2024-03-12T05:22:42.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 -- start start 2024-03-12T05:22:42.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fafaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14f94d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14f99f10 con 0x55bf14fa37b0 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14fbad60 con 0x55bf14f9a2b0 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14f9a9c0 con 0x55bf14f9abd0 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff03259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fafaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56710/0 (socket says 172.21.15.100:56710) 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 -- 172.21.15.100:0/634596800 learned_addr learned my addr 172.21.15.100:0/634596800 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff02bfff640 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14f94d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 -- 172.21.15.100:0/634596800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 msgr2=0x55bf14f94d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14f94d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 -- 172.21.15.100:0/634596800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 msgr2=0x55bf14fafaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fafaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 -- 172.21.15.100:0/634596800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf14e9cac0 con 0x55bf14fa37b0 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff032d9d640 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa2d30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff02c02c700 tx=0x7ff02c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7324c6b99a541c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.119+0000 7ff01bfff640 1 -- 172.21.15.100:0/634596800 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0328b0 con 0x55bf14fa37b0 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff01bfff640 1 -- 172.21.15.100:0/634596800 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02c049050 con 0x55bf14fa37b0 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff01bfff640 1 -- 172.21.15.100:0/634596800 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c004800 con 0x55bf14fa37b0 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 -- 172.21.15.100:0/634596800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 msgr2=0x55bf14fa2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa2d30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff02c02c700 tx=0x7ff02c009360 comp rx=0 tx=0).stop 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 -- 172.21.15.100:0/634596800 shutdown_connections 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14fa37b0 0x55bf14fa2d30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9abd0 0x55bf14f94d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/634596800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14f9a2b0 0x55bf14fafaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 -- 172.21.15.100:0/634596800 >> 172.21.15.100:0/634596800 conn(0x55bf14f94990 msgr2=0x55bf14f9fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 -- 172.21.15.100:0/634596800 shutdown_connections 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.120+0000 7ff03ba2f740 1 -- 172.21.15.100:0/634596800 wait complete. 2024-03-12T05:22:42.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 -- start start 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 Processor -- start 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14fa37b0 0x55bf15009c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf14faa280 con 0x55bf14f9abd0 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1500a180 con 0x55bf14fa37b0 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1500a2c0 con 0x55bf14f9a2b0 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03ba2f740 10 client.0 _ll_register_callbacks cb 0x7ff02400bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56712/0 (socket says 172.21.15.100:56712) 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 -- 172.21.15.100:0/2038955305 learned_addr learned my addr 172.21.15.100:0/2038955305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff03259c640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff032d9d640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14fa37b0 0x55bf15009c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 -- 172.21.15.100:0/2038955305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9a2b0 msgr2=0x55bf14fa9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 -- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14fa37b0 msgr2=0x55bf15009c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14fa37b0 0x55bf15009c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 -- 172.21.15.100:0/2038955305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c030070 con 0x55bf14f9abd0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff02bfff640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa9980 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff0340fd830 tx=0x7ff01c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18487b5a1c4e4d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.121+0000 7ff019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff019ffb640 1 -- 172.21.15.100:0/2038955305 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c016030 con 0x55bf14f9abd0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff019ffb640 1 -- 172.21.15.100:0/2038955305 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff01c01f5d0 con 0x55bf14f9abd0 2024-03-12T05:22:42.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff02409b4d0 con 0x55bf14f9abd0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff019ffb640 1 -- 172.21.15.100:0/2038955305 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff01c01f790 con 0x55bf14f9abd0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 10 client.17697 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bf14ec26a0 con 0x55bf14f9abd0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff019ffb640 1 -- 172.21.15.100:0/2038955305 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02409b4d0 con 0x55bf14f9abd0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff019ffb640 1 -- 172.21.15.100:0/2038955305 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff01c02d070 con 0x55bf14f9abd0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 10 client.17697 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 20 client.17697 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf1500bb80 0x55bf1500dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 4 client.17697 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf14c8d890 con 0x55bf1500bb80 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.122+0000 7ff03259c640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf1500bb80 0x55bf1500dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff019ffb640 10 client.17697 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03259c640 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf1500bb80 0x55bf1500dfa0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acd1816f29d7b43 server_cookie=fdb8a9e2d6541b02 in_seq=0 out_seq=0 2024-03-12T05:22:42.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff019ffb640 1 -- 172.21.15.100:0/2038955305 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bf14c8d890 con 0x55bf1500bb80 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff019ffb640 10 client.17697 handle_command_reply: tid=0 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 client.17697 shutdown 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 msgr2=0x55bf14fa9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa9980 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff0340fd830 tx=0x7ff01c003a80 comp rx=0 tx=0).stop 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 shutdown_connections 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf1500bb80 0x55bf1500dfa0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf14fa37b0 0x55bf15009c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff019ffb640 0 client.17697 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf14f9abd0 0x55bf14fa9980 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 --2- 172.21.15.100:0/2038955305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf14f9a2b0 0x55bf14fa9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 >> 172.21.15.100:0/2038955305 conn(0x55bf14f94990 msgr2=0x55bf14f9fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 shutdown_connections 2024-03-12T05:22:42.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.123+0000 7ff03ba2f740 1 -- 172.21.15.100:0/2038955305 wait complete. 2024-03-12T05:22:42.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff03ba2f740 20 client.17697 trim_cache size 0 max 16384 2024-03-12T05:22:42.126 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:22:42.126 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:22:42.126 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:22:42.126 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0200a1ba0 msgr2=0x7ff0200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0200a1ba0 0x7ff0200a4060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009430 tx=0x7ff02c005c50 comp rx=0 tx=0).stop 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 msgr2=0x7ff034196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 0x7ff034196680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb00 tx=0x7ff024009e40 comp rx=0 tx=0).stop 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 shutdown_connections 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0200a1ba0 0x7ff0200a4060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff03419af40 0x7ff03419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff034105370 0x7ff034196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 --2- 172.21.15.100:0/4092068561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0341002a0 0x7ff034196680 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 >> 172.21.15.100:0/4092068561 conn(0x7ff034077210 msgr2=0x7ff0340fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 shutdown_connections 2024-03-12T05:22:42.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.124+0000 7ff038e55640 1 -- 172.21.15.100:0/4092068561 wait complete. 2024-03-12T05:22:42.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:22:42.130 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:22:42.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 -- 172.21.15.100:0/476814325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98102d10 msgr2=0x7f9c9810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/476814325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98102d10 0x7f9c9810b1f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009900 tx=0x7f9c8c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:42.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 -- 172.21.15.100:0/476814325 shutdown_connections 2024-03-12T05:22:42.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/476814325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9810b7c0 0x7f9c9810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/476814325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98102d10 0x7f9c9810b1f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/476814325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c981023f0 0x7f9c981027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 -- 172.21.15.100:0/476814325 >> 172.21.15.100:0/476814325 conn(0x7f9c98077180 msgr2=0x7f9c98077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 -- 172.21.15.100:0/476814325 shutdown_connections 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 -- 172.21.15.100:0/476814325 wait complete. 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.245+0000 7f9c9d89a640 1 Processor -- start 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 -- start start 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c981023f0 0x7f9c9819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c98102d10 0x7f9c9819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 0x7f9c98197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98197df0 con 0x7f9c9810b7c0 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98197f60 con 0x7f9c981023f0 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c9d89a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98198120 con 0x7f9c98102d10 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 0x7f9c98197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 0x7f9c98197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56724/0 (socket says 172.21.15.100:56724) 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 -- 172.21.15.100:0/3964175581 learned_addr learned my addr 172.21.15.100:0/3964175581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c96ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c981023f0 0x7f9c9819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c967fc640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c98102d10 0x7f9c9819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 -- 172.21.15.100:0/3964175581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c98102d10 msgr2=0x7f9c9819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c98102d10 0x7f9c9819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 -- 172.21.15.100:0/3964175581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c981023f0 msgr2=0x7f9c9819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c981023f0 0x7f9c9819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c977fe640 1 -- 172.21.15.100:0/3964175581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c009560 con 0x7f9c9810b7c0 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.246+0000 7f9c967fc640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c98102d10 0x7f9c9819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.247+0000 7f9c977fe640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 0x7f9c98197660 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000eaa0 tx=0x7f9c9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.247+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000cdb0 con 0x7f9c9810b7c0 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.247+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c90005020 con 0x7f9c9810b7c0 2024-03-12T05:22:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.247+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c98198400 con 0x7f9c9810b7c0 2024-03-12T05:22:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.247+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900116b0 con 0x7f9c9810b7c0 2024-03-12T05:22:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.247+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c981a4520 con 0x7f9c9810b7c0 2024-03-12T05:22:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.248+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c98068b20 con 0x7f9c9810b7c0 2024-03-12T05:22:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.249+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9c90005520 con 0x7f9c9810b7c0 2024-03-12T05:22:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.249+0000 7f9c9c898640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9c800a1ba0 0x7f9c800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.249+0000 7f9c96ffd640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9c800a1ba0 0x7f9c800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.249+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9c900cda30 con 0x7f9c9810b7c0 2024-03-12T05:22:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.250+0000 7f9c96ffd640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9c800a1ba0 0x7f9c800a4060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009bd0 tx=0x7f9c88005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.254+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c900968c0 con 0x7f9c9810b7c0 2024-03-12T05:22:42.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.496+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c98067f70 con 0x7f9c9810b7c0 2024-03-12T05:22:42.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.497+0000 7f9c9c898640 1 -- 172.21.15.100:0/3964175581 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f9c90099f80 con 0x7f9c9810b7c0 2024-03-12T05:22:42.492 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:22:42.492 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:22:42.492 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9c800a1ba0 msgr2=0x7f9c800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9c800a1ba0 0x7f9c800a4060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009bd0 tx=0x7f9c88005f90 comp rx=0 tx=0).stop 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 msgr2=0x7f9c98197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 0x7f9c98197660 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000eaa0 tx=0x7f9c9000ef60 comp rx=0 tx=0).stop 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 shutdown_connections 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9c800a1ba0 0x7f9c800a4060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c9810b7c0 0x7f9c98197660 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c98102d10 0x7f9c9819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 --2- 172.21.15.100:0/3964175581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c981023f0 0x7f9c9819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 >> 172.21.15.100:0/3964175581 conn(0x7f9c98077180 msgr2=0x7f9c980fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 shutdown_connections 2024-03-12T05:22:42.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.499+0000 7f9c9d89a640 1 -- 172.21.15.100:0/3964175581 wait complete. 2024-03-12T05:22:42.506 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:22:42.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.621+0000 7f9a25af1640 1 -- 172.21.15.100:0/2883205655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a201006a0 msgr2=0x7f9a20105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.621+0000 7f9a25af1640 1 --2- 172.21.15.100:0/2883205655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a201006a0 0x7f9a20105dd0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009930 tx=0x7f9a1402bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- 172.21.15.100:0/2883205655 shutdown_connections 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 --2- 172.21.15.100:0/2883205655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201064b0 0x7f9a2010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 --2- 172.21.15.100:0/2883205655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a201006a0 0x7f9a20105dd0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 --2- 172.21.15.100:0/2883205655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a200ffcf0 0x7f9a201000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- 172.21.15.100:0/2883205655 >> 172.21.15.100:0/2883205655 conn(0x7f9a200752e0 msgr2=0x7f9a200756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- 172.21.15.100:0/2883205655 shutdown_connections 2024-03-12T05:22:42.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- 172.21.15.100:0/2883205655 wait complete. 2024-03-12T05:22:42.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 Processor -- start 2024-03-12T05:22:42.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- start start 2024-03-12T05:22:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 0x7f9a2019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a201064b0 0x7f9a2019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a20197710 0x7f9a20197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201981e0 con 0x7f9a200ffcf0 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20198350 con 0x7f9a20197710 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.622+0000 7f9a25af1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20198510 con 0x7f9a201064b0 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 0x7f9a2019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 0x7f9a2019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56752/0 (socket says 172.21.15.100:56752) 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a20197710 0x7f9a20197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1313357110 learned_addr learned my addr 172.21.15.100:0/1313357110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1effd640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a201064b0 0x7f9a2019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1313357110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a201064b0 msgr2=0x7f9a2019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a201064b0 0x7f9a2019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1313357110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a20197710 msgr2=0x7f9a20197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a20197710 0x7f9a20197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1313357110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a14009560 con 0x7f9a200ffcf0 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a20197710 0x7f9a20197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 0x7f9a2019d650 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b4a0 tx=0x7f9a1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1cff9640 1 -- 172.21.15.100:0/1313357110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10010610 con 0x7f9a200ffcf0 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a201a44b0 con 0x7f9a200ffcf0 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1cff9640 1 -- 172.21.15.100:0/1313357110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a10010c50 con 0x7f9a200ffcf0 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a1cff9640 1 -- 172.21.15.100:0/1313357110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10017e20 con 0x7f9a200ffcf0 2024-03-12T05:22:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.623+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a201a4990 con 0x7f9a200ffcf0 2024-03-12T05:22:42.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a1cff9640 1 -- 172.21.15.100:0/1313357110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9a10017480 con 0x7f9a200ffcf0 2024-03-12T05:22:42.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 Processor -- start 2024-03-12T05:22:42.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 -- start start 2024-03-12T05:22:42.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a3f00 0x5628461a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a1cff9640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1ba0 0x7f9a080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284615f960 con 0x5628461b1b60 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846113310 con 0x5628461a3f00 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.626+0000 7f9a2879e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846110840 con 0x5628461a4820 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1ba0 0x7f9a080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1cff9640 1 -- 172.21.15.100:0/1313357110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a100ccde0 con 0x7f9a200ffcf0 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56756/0 (socket says 172.21.15.100:56756) 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 -- 172.21.15.100:0/909548134 learned_addr learned my addr 172.21.15.100:0/909548134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a3f00 0x5628461a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 -- 172.21.15.100:0/909548134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 msgr2=0x5628461b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 -- 172.21.15.100:0/909548134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a3f00 msgr2=0x5628461a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a3f00 0x5628461a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 -- 172.21.15.100:0/909548134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628460ce060 con 0x5628461b1b60 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1ba0 0x7f9a080a4060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009fd0 tx=0x7f9a18011720 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a1effd640 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461b3f50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9a20073870 tx=0x7f9a1402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae8eaabb76a9972 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a05ffb640 1 -- 172.21.15.100:0/909548134 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14004940 con 0x5628461b1b60 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a05ffb640 1 -- 172.21.15.100:0/909548134 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a14004aa0 con 0x5628461b1b60 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.627+0000 7f9a05ffb640 1 -- 172.21.15.100:0/909548134 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14005210 con 0x5628461b1b60 2024-03-12T05:22:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 -- 172.21.15.100:0/909548134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 msgr2=0x5628461b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461b3f50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9a20073870 tx=0x7f9a1402c640 comp rx=0 tx=0).stop 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 -- 172.21.15.100:0/909548134 shutdown_connections 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461b3f50 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 --2- 172.21.15.100:0/909548134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a3f00 0x5628461a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 -- 172.21.15.100:0/909548134 >> 172.21.15.100:0/909548134 conn(0x56284619e980 msgr2=0x56284619ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 -- 172.21.15.100:0/909548134 shutdown_connections 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 -- 172.21.15.100:0/909548134 wait complete. 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.628+0000 7f9a2879e740 1 -- start start 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 Processor -- start 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 0x5628461aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628461af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284610c020 con 0x5628461a3f00 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845ddf430 con 0x5628461b1b60 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845d9be00 con 0x5628461a4820 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a2879e740 10 client.0 _ll_register_callbacks cb 0x5628460b3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 0x5628461aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 0x5628461aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56772/0 (socket says 172.21.15.100:56772) 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/616130316 learned_addr learned my addr 172.21.15.100:0/616130316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628461af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1effd640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/616130316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 msgr2=0x5628461aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 msgr2=0x5628461af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628461af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/616130316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2010e8d0 con 0x5628461a3f00 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 0x5628461aeab0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000be60 tx=0x7f9a10098080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da041ee12107a7c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.629+0000 7f99feffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f99feffd640 1 -- 172.21.15.100:0/616130316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1009a1c0 con 0x5628461a3f00 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f99feffd640 1 -- 172.21.15.100:0/616130316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a10099ac0 con 0x5628461a3f00 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562845e0b7e0 con 0x5628461a3f00 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f99feffd640 1 -- 172.21.15.100:0/616130316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1009dc10 con 0x5628461a3f00 2024-03-12T05:22:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 10 client.17715 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56284610a6a0 con 0x5628461a3f00 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f99feffd640 1 -- 172.21.15.100:0/616130316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a1007c080 con 0x5628461a3f00 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f99feffd640 1 -- 172.21.15.100:0/616130316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a10096130 con 0x5628461a3f00 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 10 client.17715 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 10 client.17715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 20 client.17715 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5628461c1ee0 0x5628461c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 4 client.17715 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562845e48720 con 0x5628461c1ee0 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.630+0000 7f9a1effd640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5628461c1ee0 0x5628461c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.631+0000 7f99feffd640 10 client.17715 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.631+0000 7f9a1effd640 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5628461c1ee0 0x5628461c42f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=688875218b63b3a7 server_cookie=afcce710ab7cc96 in_seq=0 out_seq=0 2024-03-12T05:22:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f99feffd640 1 -- 172.21.15.100:0/616130316 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562845e48720 con 0x5628461c1ee0 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f99feffd640 10 client.17715 handle_command_reply: tid=0 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 client.17715 shutdown 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 msgr2=0x5628461aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 0x5628461aeab0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000be60 tx=0x7f9a10098080 comp rx=0 tx=0).stop 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 shutdown_connections 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5628461c1ee0 0x5628461c42f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628461af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a4820 0x5628461aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f99feffd640 0 client.17715 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 --2- 172.21.15.100:0/616130316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a3f00 0x5628461aeab0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 >> 172.21.15.100:0/616130316 conn(0x56284619e980 msgr2=0x5628461a06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 shutdown_connections 2024-03-12T05:22:42.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 1 -- 172.21.15.100:0/616130316 wait complete. 2024-03-12T05:22:42.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.632+0000 7f9a2879e740 20 client.17715 trim_cache size 0 max 16384 2024-03-12T05:22:42.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 Processor -- start 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 -- start start 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a4820 0x5628461ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846091130 con 0x5628461b1b60 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628461c94e0 con 0x5628461a4820 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.645+0000 7f9a2879e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284619a2b0 con 0x5628461a3f00 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56776/0 (socket says 172.21.15.100:56776) 2024-03-12T05:22:42.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 -- 172.21.15.100:0/1454642411 learned_addr learned my addr 172.21.15.100:0/1454642411 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1effd640 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a4820 0x5628461ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 -- 172.21.15.100:0/1454642411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 msgr2=0x5628461b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 -- 172.21.15.100:0/1454642411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a4820 msgr2=0x5628461ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a4820 0x5628461ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 -- 172.21.15.100:0/1454642411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628460a6c20 con 0x5628461b1b60 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461af350 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005ed0 tx=0x7f9a18021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8443d6d6f56053 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a057fa640 1 -- 172.21.15.100:0/1454642411 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a180109f0 con 0x5628461b1b60 2024-03-12T05:22:42.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a057fa640 1 -- 172.21.15.100:0/1454642411 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a18010b50 con 0x5628461b1b60 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.646+0000 7f9a057fa640 1 -- 172.21.15.100:0/1454642411 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1801e400 con 0x5628461b1b60 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 -- 172.21.15.100:0/1454642411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 msgr2=0x5628461af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461af350 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005ed0 tx=0x7f9a18021000 comp rx=0 tx=0).stop 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 -- 172.21.15.100:0/1454642411 shutdown_connections 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461b1b60 0x5628461af350 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461a4820 0x5628461ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 --2- 172.21.15.100:0/1454642411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 -- 172.21.15.100:0/1454642411 >> 172.21.15.100:0/1454642411 conn(0x56284619e980 msgr2=0x5628461ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 -- 172.21.15.100:0/1454642411 shutdown_connections 2024-03-12T05:22:42.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 -- 172.21.15.100:0/1454642411 wait complete. 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 -- start start 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.647+0000 7f9a2879e740 1 Processor -- start 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 0x5628461b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628462183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846218920 con 0x5628461a4820 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846218a60 con 0x5628461b1b60 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846218ba0 con 0x5628461a3f00 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 10 client.0 _ll_register_callbacks cb 0x7f9a180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 0x5628461b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 0x5628461b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56780/0 (socket says 172.21.15.100:56780) 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 -- 172.21.15.100:0/281329601 learned_addr learned my addr 172.21.15.100:0/281329601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628462183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 -- 172.21.15.100:0/281329601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 msgr2=0x5628461b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 -- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 msgr2=0x5628462183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628462183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 -- 172.21.15.100:0/281329601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a18019040 con 0x5628461a4820 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a1effd640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 0x5628461b8110 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9a20073870 tx=0x7f9a140045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9668cc091af031a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f99feffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f99feffd640 1 -- 172.21.15.100:0/281329601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1404d040 con 0x5628461a4820 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a14004aa0 con 0x5628461a4820 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.648+0000 7f99feffd640 1 -- 172.21.15.100:0/281329601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a1404f2c0 con 0x5628461a4820 2024-03-12T05:22:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f99feffd640 1 -- 172.21.15.100:0/281329601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a14004920 con 0x5628461a4820 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 10 client.17721 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5628460cc8c0 con 0x5628461a4820 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f99feffd640 1 -- 172.21.15.100:0/281329601 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a14004aa0 con 0x5628461a4820 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f99feffd640 1 -- 172.21.15.100:0/281329601 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a1403c040 con 0x5628461a4820 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 10 client.17721 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 10 client.17721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 20 client.17721 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562846219ef0 0x56284621c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 4 client.17721 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562845e97890 con 0x562846219ef0 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.649+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562846219ef0 0x56284621c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562846219ef0 0x56284621c310 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21602c295c666462 server_cookie=166f1f4f59251425 in_seq=0 out_seq=0 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f99feffd640 10 client.17721 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:42.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f99feffd640 1 -- 172.21.15.100:0/281329601 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562845e97890 con 0x562846219ef0 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f99feffd640 10 client.17721 handle_command_reply: tid=0 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 client.17721 shutdown 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 msgr2=0x5628461b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 0x5628461b8110 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9a20073870 tx=0x7f9a140045a0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 shutdown_connections 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562846219ef0 0x56284621c310 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628461b1b60 0x5628462183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628461a4820 0x5628461b8110 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f99feffd640 0 client.17721 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 --2- 172.21.15.100:0/281329601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628461a3f00 0x5628461b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 >> 172.21.15.100:0/281329601 conn(0x56284619e980 msgr2=0x56284619d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 shutdown_connections 2024-03-12T05:22:42.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 1 -- 172.21.15.100:0/281329601 wait complete. 2024-03-12T05:22:42.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.650+0000 7f9a2879e740 20 client.17721 trim_cache size 0 max 16384 2024-03-12T05:22:42.655 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1ba0 msgr2=0x7f9a080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1ba0 0x7f9a080a4060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009fd0 tx=0x7f9a18011720 comp rx=0 tx=0).stop 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 msgr2=0x7f9a2019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 0x7f9a2019d650 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b4a0 tx=0x7f9a1000b960 comp rx=0 tx=0).stop 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 shutdown_connections 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1ba0 0x7f9a080a4060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a20197710 0x7f9a20197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a201064b0 0x7f9a2019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 --2- 172.21.15.100:0/1313357110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a200ffcf0 0x7f9a2019d650 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 >> 172.21.15.100:0/1313357110 conn(0x7f9a200752e0 msgr2=0x7f9a200fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 shutdown_connections 2024-03-12T05:22:42.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.651+0000 7f9a25af1640 1 -- 172.21.15.100:0/1313357110 wait complete. 2024-03-12T05:22:42.658 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:22:42.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:22:42.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 -- 172.21.15.100:0/2421735489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c1006a0 msgr2=0x7f0c9c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 --2- 172.21.15.100:0/2421735489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c1006a0 0x7f0c9c105dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009900 tx=0x7f0c8c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 -- 172.21.15.100:0/2421735489 shutdown_connections 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 --2- 172.21.15.100:0/2421735489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 --2- 172.21.15.100:0/2421735489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c1006a0 0x7f0c9c105dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 --2- 172.21.15.100:0/2421735489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c0ffcf0 0x7f0c9c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 -- 172.21.15.100:0/2421735489 >> 172.21.15.100:0/2421735489 conn(0x7f0c9c0752e0 msgr2=0x7f0c9c0756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 -- 172.21.15.100:0/2421735489 shutdown_connections 2024-03-12T05:22:42.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.773+0000 7f0ca3779640 1 -- 172.21.15.100:0/2421735489 wait complete. 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 Processor -- start 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 -- start start 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c0ffcf0 0x7f0c9c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c197630 0x7f0c9c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c198100 con 0x7f0c9c1064b0 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c198270 con 0x7f0c9c0ffcf0 2024-03-12T05:22:42.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca3779640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c198430 con 0x7f0c9c197630 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56798/0 (socket says 172.21.15.100:56798) 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 -- 172.21.15.100:0/745834380 learned_addr learned my addr 172.21.15.100:0/745834380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca1515640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c0ffcf0 0x7f0c9c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c197630 0x7f0c9c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 -- 172.21.15.100:0/745834380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c197630 msgr2=0x7f0c9c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c197630 0x7f0c9c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 -- 172.21.15.100:0/745834380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c0ffcf0 msgr2=0x7f0c9c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c0ffcf0 0x7f0c9c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca0d14640 1 -- 172.21.15.100:0/745834380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009560 con 0x7f0c9c1064b0 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.774+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c197630 0x7f0c9c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.775+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c19dab0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c036040 tx=0x7f0c8c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.775+0000 7f0c927fc640 1 -- 172.21.15.100:0/745834380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c004350 con 0x7f0c9c1064b0 2024-03-12T05:22:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.775+0000 7f0c927fc640 1 -- 172.21.15.100:0/745834380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c8c0044b0 con 0x7f0c9c1064b0 2024-03-12T05:22:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.775+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c1a4370 con 0x7f0c9c1064b0 2024-03-12T05:22:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.775+0000 7f0c927fc640 1 -- 172.21.15.100:0/745834380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c003da0 con 0x7f0c9c1064b0 2024-03-12T05:22:42.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.775+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c1a4850 con 0x7f0c9c1064b0 2024-03-12T05:22:42.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.777+0000 7f0c927fc640 1 -- 172.21.15.100:0/745834380 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0c8c004df0 con 0x7f0c9c1064b0 2024-03-12T05:22:42.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 Processor -- start 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 -- start start 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9aa9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0c927fc640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c880a1ba0 0x7f0c880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aabb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9a5b960 con 0x5628f9aa0b70 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9a0f310 con 0x5628f9aa9750 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca6420740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9a0c840 con 0x5628f9aa0250 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca0d14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9aa9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca1515640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c880a1ba0 0x7f0c880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca0d14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9aa9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56810/0 (socket says 172.21.15.100:56810) 2024-03-12T05:22:42.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca0d14640 1 -- 172.21.15.100:0/282002272 learned_addr learned my addr 172.21.15.100:0/282002272 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0c927fc640 1 -- 172.21.15.100:0/745834380 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c8c0ef8a0 con 0x7f0c9c1064b0 2024-03-12T05:22:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.778+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aabb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca0d14640 1 -- 172.21.15.100:0/282002272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 msgr2=0x5628f9aa0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca0d14640 1 -- 172.21.15.100:0/282002272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 msgr2=0x5628f9aabb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aabb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca0d14640 1 -- 172.21.15.100:0/282002272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628f99ca060 con 0x5628f9aa0b70 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca1515640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c880a1ba0 0x7f0c880a4060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009bd0 tx=0x7f0c98005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9aa9210 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0b8a00 tx=0x7f0c8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560ce7c37a5567e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0c877fe640 1 -- 172.21.15.100:0/282002272 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0bbf00 con 0x5628f9aa0b70 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0c877fe640 1 -- 172.21.15.100:0/282002272 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c8c0bcca0 con 0x5628f9aa0b70 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.779+0000 7f0c877fe640 1 -- 172.21.15.100:0/282002272 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0ba720 con 0x5628f9aa0b70 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 -- 172.21.15.100:0/282002272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 msgr2=0x5628f9aa9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9aa9210 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0b8a00 tx=0x7f0c8c000bc0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 -- 172.21.15.100:0/282002272 shutdown_connections 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aabb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9aa9210 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 --2- 172.21.15.100:0/282002272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 -- 172.21.15.100:0/282002272 >> 172.21.15.100:0/282002272 conn(0x5628f9a9a980 msgr2=0x5628f9a9ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 -- 172.21.15.100:0/282002272 shutdown_connections 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 -- 172.21.15.100:0/282002272 wait complete. 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 -- start start 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.780+0000 7f0ca6420740 1 Processor -- start 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 0x5628f9aa6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aa6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aa7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9a08020 con 0x5628f9aa0250 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f96db430 con 0x5628f9aa9750 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9697e00 con 0x5628f9aa0b70 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 0x5628f9aa6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 10 client.0 _ll_register_callbacks cb 0x5628f99afcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 0x5628f9aa6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56826/0 (socket says 172.21.15.100:56826) 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 -- 172.21.15.100:0/82115866 learned_addr learned my addr 172.21.15.100:0/82115866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aa6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aa7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 -- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 msgr2=0x5628f9aa6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aa6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 -- 172.21.15.100:0/82115866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 msgr2=0x5628f9aa7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aa7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 -- 172.21.15.100:0/82115866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c1a4850 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca1515640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 0x5628f9aa6780 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ed00 tx=0x7f0c98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a921234edb23d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0c84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0c84ff9640 1 -- 172.21.15.100:0/82115866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9801ccb0 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0c84ff9640 1 -- 172.21.15.100:0/82115866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c98023070 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.781+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628f97077e0 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0c84ff9640 1 -- 172.21.15.100:0/82115866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c9801ce10 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 10 client.17733 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5628f9a066a0 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0c84ff9640 1 -- 172.21.15.100:0/82115866 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c98033080 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0c84ff9640 1 -- 172.21.15.100:0/82115866 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0c9801d550 con 0x5628f9aa0250 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 10 client.17733 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 10 client.17733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 20 client.17733 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9ab9980 0x5628f9abbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 4 client.17733 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628f9744720 con 0x5628f9ab9980 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.782+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9ab9980 0x5628f9abbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9ab9980 0x5628f9abbda0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2ebafa813072767 server_cookie=19f96f7226222ae5 in_seq=0 out_seq=0 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0c84ff9640 10 client.17733 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0c84ff9640 1 -- 172.21.15.100:0/82115866 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628f9744720 con 0x5628f9ab9980 2024-03-12T05:22:42.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0c84ff9640 10 client.17733 handle_command_reply: tid=0 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 client.17733 shutdown 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 msgr2=0x5628f9aa6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 0x5628f9aa6780 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ed00 tx=0x7f0c98004730 comp rx=0 tx=0).stop 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 shutdown_connections 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9ab9980 0x5628f9abbda0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9aa7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aa6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0c84ff9640 0 client.17733 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 --2- 172.21.15.100:0/82115866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0250 0x5628f9aa6780 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 >> 172.21.15.100:0/82115866 conn(0x5628f9a9a980 msgr2=0x5628f9a9c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 shutdown_connections 2024-03-12T05:22:42.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 1 -- 172.21.15.100:0/82115866 wait complete. 2024-03-12T05:22:42.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.783+0000 7f0ca6420740 20 client.17733 trim_cache size 0 max 16384 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 Processor -- start 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 -- start start 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9a99a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9ab03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f998d130 con 0x5628f9aa0b70 2024-03-12T05:22:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9a9fc60 con 0x5628f9aa9750 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca6420740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9ac0fb0 con 0x5628f9aa0250 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca0d14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9a99a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca0d14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9a99a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56834/0 (socket says 172.21.15.100:56834) 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca0d14640 1 -- 172.21.15.100:0/3930326627 learned_addr learned my addr 172.21.15.100:0/3930326627 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.795+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9ab03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca1515640 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca0d14640 1 -- 172.21.15.100:0/3930326627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 msgr2=0x5628f9aa7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca0d14640 1 -- 172.21.15.100:0/3930326627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 msgr2=0x5628f9ab03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9ab03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca0d14640 1 -- 172.21.15.100:0/3930326627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628f99a2c20 con 0x5628f9aa0b70 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9a99a00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0bc530 tx=0x7f0c8c003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9bed65fb914236 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0c86ffd640 1 -- 172.21.15.100:0/3930326627 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0bc620 con 0x5628f9aa0b70 2024-03-12T05:22:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0c86ffd640 1 -- 172.21.15.100:0/3930326627 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c8c0c4d40 con 0x5628f9aa0b70 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.796+0000 7f0c86ffd640 1 -- 172.21.15.100:0/3930326627 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0b8c10 con 0x5628f9aa0b70 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 -- 172.21.15.100:0/3930326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 msgr2=0x5628f9a99a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9a99a00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0bc530 tx=0x7f0c8c003840 comp rx=0 tx=0).stop 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 -- 172.21.15.100:0/3930326627 shutdown_connections 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa9750 0x5628f9ab03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa0b70 0x5628f9a99a00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 --2- 172.21.15.100:0/3930326627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0250 0x5628f9aa7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 -- 172.21.15.100:0/3930326627 >> 172.21.15.100:0/3930326627 conn(0x5628f9a9a980 msgr2=0x5628f9a992f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 -- 172.21.15.100:0/3930326627 shutdown_connections 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 -- 172.21.15.100:0/3930326627 wait complete. 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 -- start start 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.797+0000 7f0ca6420740 1 Processor -- start 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa0250 0x5628f9aaf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aaf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 0x5628f9b142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9aafed0 con 0x5628f9aa9750 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9b147e0 con 0x5628f9aa0250 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f9b14920 con 0x5628f9aa0b70 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 10 client.0 _ll_register_callbacks cb 0x7f0c8c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 0x5628f9b142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1515640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa0250 0x5628f9aaf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 0x5628f9b142a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56848/0 (socket says 172.21.15.100:56848) 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 -- 172.21.15.100:0/30912255 learned_addr learned my addr 172.21.15.100:0/30912255 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aaf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 -- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 msgr2=0x5628f9aaf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aaf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 -- 172.21.15.100:0/30912255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa0250 msgr2=0x5628f9aaf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa0250 0x5628f9aaf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 -- 172.21.15.100:0/30912255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628f99ca060 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca0d14640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aaf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca1d16640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 0x5628f9b142a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0c94015e20 tx=0x7f0c940118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114c1199ecff37e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0c84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0c84ff9640 1 -- 172.21.15.100:0/30912255 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94003970 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0c84ff9640 1 -- 172.21.15.100:0/30912255 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c94003ad0 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.798+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c8c0bcca0 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0c84ff9640 1 -- 172.21.15.100:0/30912255 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c94003c90 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 10 client.17739 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5628f99c88c0 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0c84ff9640 1 -- 172.21.15.100:0/30912255 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9402a020 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0c84ff9640 1 -- 172.21.15.100:0/30912255 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0c94012530 con 0x5628f9aa9750 2024-03-12T05:22:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 10 client.17739 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 10 client.17739 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 20 client.17739 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9b160f0 0x5628f9b18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 4 client.17739 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628f9793890 con 0x5628f9b160f0 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.799+0000 7f0ca1515640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9b160f0 0x5628f9b18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0c84ff9640 10 client.17739 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca1515640 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9b160f0 0x5628f9b18510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7008cecfe461f34a server_cookie=7a2615be4c248a1b in_seq=0 out_seq=0 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0c84ff9640 1 -- 172.21.15.100:0/30912255 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628f9793890 con 0x5628f9b160f0 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0c84ff9640 10 client.17739 handle_command_reply: tid=0 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 client.17739 shutdown 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 msgr2=0x5628f9b142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 0x5628f9b142a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0c94015e20 tx=0x7f0c940118f0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 shutdown_connections 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628f9aa9750 0x5628f9b142a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628f9aa0b70 0x5628f9aaf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5628f9b160f0 0x5628f9b18510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 --2- 172.21.15.100:0/30912255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628f9aa0250 0x5628f9aaf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0c84ff9640 0 client.17739 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 >> 172.21.15.100:0/30912255 conn(0x5628f9a9a980 msgr2=0x5628f9a99b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 shutdown_connections 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 1 -- 172.21.15.100:0/30912255 wait complete. 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.800+0000 7f0ca6420740 20 client.17739 trim_cache size 0 max 16384 2024-03-12T05:22:42.804 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:22:42.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c880a1ba0 msgr2=0x7f0c880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c880a1ba0 0x7f0c880a4060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009bd0 tx=0x7f0c98005f90 comp rx=0 tx=0).stop 2024-03-12T05:22:42.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 msgr2=0x7f0c9c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c19dab0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c036040 tx=0x7f0c8c02c680 comp rx=0 tx=0).stop 2024-03-12T05:22:42.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 shutdown_connections 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c880a1ba0 0x7f0c880a4060 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c9c197630 0x7f0c9c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c9c1064b0 0x7f0c9c19dab0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 --2- 172.21.15.100:0/745834380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c0ffcf0 0x7f0c9c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 >> 172.21.15.100:0/745834380 conn(0x7f0c9c0752e0 msgr2=0x7f0c9c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 shutdown_connections 2024-03-12T05:22:42.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.801+0000 7f0ca3779640 1 -- 172.21.15.100:0/745834380 wait complete. 2024-03-12T05:22:42.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1279031 11.81 MB/sec execute 446 sec latency 16.445 ms 2024-03-12T05:22:42.807 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:22:42.808 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.920+0000 7fdf40eff640 1 -- 172.21.15.100:0/4057319193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c100c10 msgr2=0x7fdf3c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.920+0000 7fdf40eff640 1 --2- 172.21.15.100:0/4057319193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c100c10 0x7fdf3c104c50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009cd0 tx=0x7fdf3002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 -- 172.21.15.100:0/4057319193 shutdown_connections 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 --2- 172.21.15.100:0/4057319193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3c105330 0x7fdf3c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 --2- 172.21.15.100:0/4057319193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c100c10 0x7fdf3c104c50 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 --2- 172.21.15.100:0/4057319193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c100260 0x7fdf3c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 -- 172.21.15.100:0/4057319193 >> 172.21.15.100:0/4057319193 conn(0x7fdf3c077180 msgr2=0x7fdf3c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 -- 172.21.15.100:0/4057319193 shutdown_connections 2024-03-12T05:22:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 -- 172.21.15.100:0/4057319193 wait complete. 2024-03-12T05:22:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 Processor -- start 2024-03-12T05:22:42.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.921+0000 7fdf40eff640 1 -- start start 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3c100260 0x7fdf3c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c105330 0x7fdf3c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 0x7fdf3c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c19a300 con 0x7fdf3c199850 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c19a470 con 0x7fdf3c105330 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c19a5e0 con 0x7fdf3c100260 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 0x7fdf3c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 0x7fdf3c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56862/0 (socket says 172.21.15.100:56862) 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 -- 172.21.15.100:0/334320521 learned_addr learned my addr 172.21.15.100:0/334320521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c105330 0x7fdf3c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3c100260 0x7fdf3c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 -- 172.21.15.100:0/334320521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3c100260 msgr2=0x7fdf3c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3c100260 0x7fdf3c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 -- 172.21.15.100:0/334320521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c105330 msgr2=0x7fdf3c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c105330 0x7fdf3c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 -- 172.21.15.100:0/334320521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30009900 con 0x7fdf3c199850 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf3ad9d640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 0x7fdf3c199d00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00d750 tx=0x7fdf2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf2b7fe640 1 -- 172.21.15.100:0/334320521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c009bd0 con 0x7fdf3c199850 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf2b7fe640 1 -- 172.21.15.100:0/334320521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c00de60 con 0x7fdf3c199850 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c1a6590 con 0x7fdf3c199850 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.922+0000 7fdf2b7fe640 1 -- 172.21.15.100:0/334320521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c004e70 con 0x7fdf3c199850 2024-03-12T05:22:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.923+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c1a6a20 con 0x7fdf3c199850 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf2b7fe640 1 -- 172.21.15.100:0/334320521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fdf2c005020 con 0x7fdf3c199850 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 Processor -- start 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 -- start start 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ea7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eb2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797e62960 con 0x55a797ea7250 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797e16310 con 0x55a797eb0750 2024-03-12T05:22:42.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf43baa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797e13840 con 0x55a797ea7b70 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ea7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ea7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56878/0 (socket says 172.21.15.100:56878) 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.925+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3316281786 learned_addr learned my addr 172.21.15.100:0/3316281786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf2b7fe640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdf240a1ba0 0x7fdf240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eb2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf3ad9d640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdf240a1ba0 0x7fdf240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3316281786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 msgr2=0x55a797eb0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3316281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 msgr2=0x55a797eb2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eb2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3316281786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a797dd1060 con 0x55a797ea7250 2024-03-12T05:22:42.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf2b7fe640 1 -- 172.21.15.100:0/334320521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdf2c0cd0e0 con 0x7fdf3c199850 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ea7630 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ca0 tx=0x7fdf30005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4396c46eb01a9f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf28ff9640 1 -- 172.21.15.100:0/3316281786 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30034640 con 0x55a797ea7250 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf28ff9640 1 -- 172.21.15.100:0/3316281786 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf300347a0 con 0x55a797ea7250 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.926+0000 7fdf28ff9640 1 -- 172.21.15.100:0/3316281786 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003cb0 con 0x55a797ea7250 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 -- 172.21.15.100:0/3316281786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 msgr2=0x55a797ea7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf3ad9d640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdf240a1ba0 0x7fdf240a4060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0961e0 tx=0x7fdf2c069160 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ea7630 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ca0 tx=0x7fdf30005f50 comp rx=0 tx=0).stop 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 -- 172.21.15.100:0/3316281786 shutdown_connections 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eb2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3316281786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ea7630 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 -- 172.21.15.100:0/3316281786 >> 172.21.15.100:0/3316281786 conn(0x55a797ea1980 msgr2=0x55a797ea1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 -- 172.21.15.100:0/3316281786 shutdown_connections 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 -- 172.21.15.100:0/3316281786 wait complete. 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 -- start start 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.927+0000 7fdf43baa740 1 Processor -- start 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ead7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eadce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797e0f020 con 0x55a797ea7250 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797ae2430 con 0x55a797eb0750 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797a9ee00 con 0x55a797ea7b70 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf43baa740 10 client.0 _ll_register_callbacks cb 0x55a797db6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ead7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ead7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56888/0 (socket says 172.21.15.100:56888) 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 -- 172.21.15.100:0/189819707 learned_addr learned my addr 172.21.15.100:0/189819707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eadce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 -- 172.21.15.100:0/189819707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 msgr2=0x55a797eadce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eadce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 -- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 msgr2=0x55a797eae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 -- 172.21.15.100:0/189819707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c10d710 con 0x55a797ea7250 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ead7a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009df0 tx=0x7fdf30003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e1478fa6d8b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.928+0000 7fdf21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf21ffb640 1 -- 172.21.15.100:0/189819707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30042650 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf21ffb640 1 -- 172.21.15.100:0/189819707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdf300427b0 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a797b4b720 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf21ffb640 1 -- 172.21.15.100:0/189819707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf30042970 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 10 client.17751 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a797e0d6a0 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf21ffb640 1 -- 172.21.15.100:0/189819707 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a797b4b720 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf21ffb640 1 -- 172.21.15.100:0/189819707 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf30061070 con 0x55a797ea7250 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 10 client.17751 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 20 client.17751 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797ec0820 0x55a797ec2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 4 client.17751 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a797b9a890 con 0x55a797ec0820 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.929+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797ec0820 0x55a797ec2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797ec0820 0x55a797ec2c40 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bd14753823817b server_cookie=889a5020f39e6434 in_seq=0 out_seq=0 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf21ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf21ffb640 1 -- 172.21.15.100:0/189819707 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a797b9a890 con 0x55a797ec0820 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf21ffb640 10 client.17751 handle_command_reply: tid=0 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf43baa740 1 client.17751 shutdown 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 msgr2=0x55a797ead7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.930+0000 7fdf43baa740 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ead7a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009df0 tx=0x7fdf30003970 comp rx=0 tx=0).stop 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 shutdown_connections 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797ec0820 0x55a797ec2c40 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eadce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf21ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 --2- 172.21.15.100:0/189819707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7250 0x55a797ead7a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 >> 172.21.15.100:0/189819707 conn(0x55a797ea1980 msgr2=0x55a797ea36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 shutdown_connections 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 1 -- 172.21.15.100:0/189819707 wait complete. 2024-03-12T05:22:42.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.931+0000 7fdf43baa740 20 client.17751 trim_cache size 0 max 16384 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 Processor -- start 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 -- start start 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7250 0x55a797eafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 0x55a797eaaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eaaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797d94130 con 0x55a797ea7b70 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797ebd650 con 0x55a797eb0750 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf43baa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797ec0620 con 0x55a797ea7250 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 0x55a797eaaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 0x55a797eaaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56898/0 (socket says 172.21.15.100:56898) 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 -- 172.21.15.100:0/190428530 learned_addr learned my addr 172.21.15.100:0/190428530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eaaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7250 0x55a797eafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 -- 172.21.15.100:0/190428530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7250 msgr2=0x55a797eafed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7250 0x55a797eafed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 -- 172.21.15.100:0/190428530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 msgr2=0x55a797eaaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eaaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.946+0000 7fdf3a59c640 1 -- 172.21.15.100:0/190428530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a797da9c20 con 0x55a797ea7b70 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.947+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 0x55a797eaaa00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x55a797ec3510 tx=0x7fdf34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72784d9d267f840b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.947+0000 7fdf23fff640 1 -- 172.21.15.100:0/190428530 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34017070 con 0x55a797ea7b70 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.947+0000 7fdf23fff640 1 -- 172.21.15.100:0/190428530 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf34018040 con 0x55a797ea7b70 2024-03-12T05:22:42.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.947+0000 7fdf23fff640 1 -- 172.21.15.100:0/190428530 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34004c30 con 0x55a797ea7b70 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- 172.21.15.100:0/190428530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 msgr2=0x55a797eaaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 0x55a797eaaa00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x55a797ec3510 tx=0x7fdf34005f90 comp rx=0 tx=0).stop 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- 172.21.15.100:0/190428530 shutdown_connections 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797eb0750 0x55a797eaaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797ea7b70 0x55a797eaaa00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- 172.21.15.100:0/190428530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7250 0x55a797eafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- 172.21.15.100:0/190428530 >> 172.21.15.100:0/190428530 conn(0x55a797ea1980 msgr2=0x55a797ea8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- 172.21.15.100:0/190428530 shutdown_connections 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- 172.21.15.100:0/190428530 wait complete. 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- start start 2024-03-12T05:22:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 Processor -- start 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797ea7250 0x55a797eb6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 0x55a797f16cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797eb70f0 con 0x55a797eb0750 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797f17230 con 0x55a797ea7250 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.948+0000 7fdf43baa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a797f17370 con 0x55a797ea7b70 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf43baa740 10 client.0 _ll_register_callbacks cb 0x7fdf34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 0x55a797f16cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 0x55a797f16cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56900/0 (socket says 172.21.15.100:56900) 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3218993353 learned_addr learned my addr 172.21.15.100:0/3218993353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797ea7250 0x55a797eb6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3218993353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 msgr2=0x55a797eb68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797ea7250 msgr2=0x55a797eb6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797ea7250 0x55a797eb6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 -- 172.21.15.100:0/3218993353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf34009e30 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf39d9b640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 0x55a797f16cf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009dc0 tx=0x7fdf30032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf6c4cc260aee92 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.949+0000 7fdf21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.950+0000 7fdf21ffb640 1 -- 172.21.15.100:0/3218993353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300323d0 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.950+0000 7fdf21ffb640 1 -- 172.21.15.100:0/3218993353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdf30004b40 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.950+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a797b0e7e0 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf21ffb640 1 -- 172.21.15.100:0/3218993353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf30004d00 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 10 client.17757 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a797dcf8c0 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf21ffb640 1 -- 172.21.15.100:0/3218993353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a797b0e7e0 con 0x55a797eb0750 2024-03-12T05:22:42.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf21ffb640 1 -- 172.21.15.100:0/3218993353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf30061040 con 0x55a797eb0750 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 10 client.17757 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 20 client.17757 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797f186d0 0x55a797f1aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 4 client.17757 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a797aba700 con 0x55a797f186d0 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.951+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797f186d0 0x55a797f1aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf21ffb640 10 client.17757 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf3a59c640 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797f186d0 0x55a797f1aac0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce3e9deba907fd0 server_cookie=d9f5b61b3d3a2683 in_seq=0 out_seq=0 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf21ffb640 1 -- 172.21.15.100:0/3218993353 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a797aba700 con 0x55a797f186d0 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf21ffb640 10 client.17757 handle_command_reply: tid=0 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 client.17757 shutdown 2024-03-12T05:22:42.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 msgr2=0x55a797f16cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 0x55a797f16cf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009dc0 tx=0x7fdf30032bb0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 shutdown_connections 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a797f186d0 0x55a797f1aac0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a797eb0750 0x55a797f16cf0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a797ea7b70 0x55a797eb68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 --2- 172.21.15.100:0/3218993353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a797ea7250 0x55a797eb6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 >> 172.21.15.100:0/3218993353 conn(0x55a797ea1980 msgr2=0x55a797ea02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf21ffb640 0 client.17757 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 shutdown_connections 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 1 -- 172.21.15.100:0/3218993353 wait complete. 2024-03-12T05:22:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.952+0000 7fdf43baa740 20 client.17757 trim_cache size 0 max 16384 2024-03-12T05:22:42.953 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:22:42.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdf240a1ba0 msgr2=0x7fdf240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdf240a1ba0 0x7fdf240a4060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0961e0 tx=0x7fdf2c069160 comp rx=0 tx=0).stop 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 msgr2=0x7fdf3c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 0x7fdf3c199d00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00d750 tx=0x7fdf2c00dc10 comp rx=0 tx=0).stop 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 shutdown_connections 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdf240a1ba0 0x7fdf240a4060 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf3c199850 0x7fdf3c199d00 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c105330 0x7fdf3c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 --2- 172.21.15.100:0/334320521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf3c100260 0x7fdf3c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 >> 172.21.15.100:0/334320521 conn(0x7fdf3c077180 msgr2=0x7fdf3c068880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 shutdown_connections 2024-03-12T05:22:42.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:42.953+0000 7fdf40eff640 1 -- 172.21.15.100:0/334320521 wait complete. 2024-03-12T05:22:43.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:42 smithi100 ceph-mon[32210]: pgmap v329: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 285 op/s 2024-03-12T05:22:43.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:42 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3964175581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:22:43.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:42 smithi156 ceph-mon[36270]: pgmap v329: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 285 op/s 2024-03-12T05:22:43.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:42 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3964175581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:22:43.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:42 smithi161 ceph-mon[36043]: pgmap v329: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 285 op/s 2024-03-12T05:22:43.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:42 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3964175581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:22:43.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1281358 11.81 MB/sec execute 447 sec latency 21.251 ms 2024-03-12T05:22:43.958 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:22:43.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:22:43.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 -- 172.21.15.100:0/2691137678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4100c10 msgr2=0x7ff6a4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- 172.21.15.100:0/2691137678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4100c10 0x7ff6a4104c50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff694009930 tx=0x7ff69402bc30 comp rx=0 tx=0).stop 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 -- 172.21.15.100:0/2691137678 shutdown_connections 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- 172.21.15.100:0/2691137678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a4105330 0x7ff6a410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- 172.21.15.100:0/2691137678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4100c10 0x7ff6a4104c50 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- 172.21.15.100:0/2691137678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6a4100260 0x7ff6a4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 -- 172.21.15.100:0/2691137678 >> 172.21.15.100:0/2691137678 conn(0x7ff6a4077180 msgr2=0x7ff6a4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 -- 172.21.15.100:0/2691137678 shutdown_connections 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 -- 172.21.15.100:0/2691137678 wait complete. 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 Processor -- start 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 -- start start 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a4100260 0x7ff6a4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 0x7ff6a4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.076+0000 7ff6ab527640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6a4199100 0x7ff6a41a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6ab527640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a3d50 con 0x7ff6a4105330 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6ab527640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a3ec0 con 0x7ff6a4100260 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6ab527640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a4080 con 0x7ff6a4199100 2024-03-12T05:22:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 0x7ff6a4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 0x7ff6a4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56916/0 (socket says 172.21.15.100:56916) 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 -- 172.21.15.100:0/3747730361 learned_addr learned my addr 172.21.15.100:0/3747730361 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a4100260 0x7ff6a4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6a4199100 0x7ff6a41a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 -- 172.21.15.100:0/3747730361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6a4199100 msgr2=0x7ff6a41a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6a4199100 0x7ff6a41a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 -- 172.21.15.100:0/3747730361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a4100260 msgr2=0x7ff6a4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a4100260 0x7ff6a4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 -- 172.21.15.100:0/3747730361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694009560 con 0x7ff6a4105330 2024-03-12T05:22:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 0x7ff6a4198b70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff694009230 tx=0x7ff69402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff69a7fc640 1 -- 172.21.15.100:0/3747730361 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694004350 con 0x7ff6a4105330 2024-03-12T05:22:44.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff69a7fc640 1 -- 172.21.15.100:0/3747730361 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6940044b0 con 0x7ff6a4105330 2024-03-12T05:22:44.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff69a7fc640 1 -- 172.21.15.100:0/3747730361 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694003da0 con 0x7ff6a4105330 2024-03-12T05:22:44.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a41a4300 con 0x7ff6a4105330 2024-03-12T05:22:44.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.077+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a41a47c0 con 0x7ff6a4105330 2024-03-12T05:22:44.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.079+0000 7ff69a7fc640 1 -- 172.21.15.100:0/3747730361 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff694004df0 con 0x7ff6a4105330 2024-03-12T05:22:44.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 Processor -- start 2024-03-12T05:22:44.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 -- start start 2024-03-12T05:22:44.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff69a7fc640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff6900a1ba0 0x7ff6900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7acff90 con 0x55c8c7b61d30 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7acd4c0 con 0x55c8c7b61380 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7ac8ca0 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6a8ac2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff6900a1ba0 0x7ff6900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6a92c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6a8ac2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b6a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56922/0 (socket says 172.21.15.100:56922) 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.080+0000 7ff6a8ac2640 1 -- 172.21.15.100:0/2568836180 learned_addr learned my addr 172.21.15.100:0/2568836180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b6cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff69a7fc640 1 -- 172.21.15.100:0/3747730361 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff6940ef7e0 con 0x7ff6a4105330 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a92c3640 1 -- 172.21.15.100:0/2568836180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b6a8b0 msgr2=0x55c8c7b6cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b6cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a92c3640 1 -- 172.21.15.100:0/2568836180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 msgr2=0x55c8c7b6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a92c3640 1 -- 172.21.15.100:0/2568836180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c7a8b0b0 con 0x55c8c7b61380 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff6900a1ba0 0x7ff6900a4060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff69c005fd0 tx=0x7ff69c009b00 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b61760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009c00 tx=0x7ff6a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1088d76ecdf3a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff68f7fe640 1 -- 172.21.15.100:0/2568836180 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0004230 con 0x55c8c7b61380 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff68f7fe640 1 -- 172.21.15.100:0/2568836180 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6a0009e70 con 0x55c8c7b61380 2024-03-12T05:22:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.081+0000 7ff68f7fe640 1 -- 172.21.15.100:0/2568836180 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0004c40 con 0x55c8c7b61380 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2568836180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 msgr2=0x55c8c7b61760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b61760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009c00 tx=0x7ff6a0011810 comp rx=0 tx=0).stop 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2568836180 shutdown_connections 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b6cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2568836180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b61760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2568836180 >> 172.21.15.100:0/2568836180 conn(0x55c8c7b5b840 msgr2=0x55c8c7b5bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2568836180 shutdown_connections 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2568836180 wait complete. 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 -- start start 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.082+0000 7ff6ae1d0740 1 Processor -- start 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7b67810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b68290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7a6df40 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c779c4b0 con 0x55c8c7b61d30 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7a4e670 con 0x55c8c7b61380 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 10 client.0 _ll_register_callbacks cb 0x55c8c7abbf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b68290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b68290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56932/0 (socket says 172.21.15.100:56932) 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/3554853152 learned_addr learned my addr 172.21.15.100:0/3554853152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7b67810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/3554853152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 msgr2=0x55c8c7b67810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7b67810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 msgr2=0x55c8c7b67d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b67d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/3554853152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0009830 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b68290 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff69c013430 tx=0x7ff69c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75753ba8716c38ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff68cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff68cff9640 1 -- 172.21.15.100:0/3554853152 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c018070 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff68cff9640 1 -- 172.21.15.100:0/3554853152 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff69c00fc90 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.083+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8c7aa2f40 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff68cff9640 1 -- 172.21.15.100:0/3554853152 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c00fe50 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 10 client.17769 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c8c7ac7320 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff68cff9640 1 -- 172.21.15.100:0/3554853152 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c8c7aa2f40 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff68cff9640 1 -- 172.21.15.100:0/3554853152 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff69c005600 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 10 client.17769 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 10 client.17769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:44.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 20 client.17769 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b7ae30 0x55c8c7b7d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 4 client.17769 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8c7abf310 con 0x55c8c7b7ae30 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b7ae30 0x55c8c7b7d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff68cff9640 10 client.17769 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.084+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b7ae30 0x55c8c7b7d250 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714a57484a2910e9 server_cookie=64f694cf5d71f62 in_seq=0 out_seq=0 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff68cff9640 1 -- 172.21.15.100:0/3554853152 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8c7abf310 con 0x55c8c7b7ae30 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff68cff9640 10 client.17769 handle_command_reply: tid=0 2024-03-12T05:22:44.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 client.17769 shutdown 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 msgr2=0x55c8c7b68290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b68290 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff69c013430 tx=0x7ff69c004ab0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 shutdown_connections 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b7ae30 0x55c8c7b7d250 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b6a8b0 0x55c8c7b68290 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b67d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff68cff9640 0 client.17769 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/3554853152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7b67810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.085+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 >> 172.21.15.100:0/3554853152 conn(0x55c8c7b5b840 msgr2=0x55c8c7b5d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.086+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 shutdown_connections 2024-03-12T05:22:44.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.086+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/3554853152 wait complete. 2024-03-12T05:22:44.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.086+0000 7ff6ae1d0740 20 client.17769 trim_cache size 0 max 16384 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 Processor -- start 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 -- start start 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b5b840 0x55c8c7b69f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b64ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7b65d80 con 0x55c8c7b61d30 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7b633f0 con 0x55c8c7b61380 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7b685f0 con 0x55c8c7b5b840 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b64ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b64ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56948/0 (socket says 172.21.15.100:56948) 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/2637425280 learned_addr learned my addr 172.21.15.100:0/2637425280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b5b840 0x55c8c7b69f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/2637425280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b5b840 msgr2=0x55c8c7b69f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b5b840 0x55c8c7b69f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/2637425280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 msgr2=0x55c8c7b680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a9ac4640 1 -- 172.21.15.100:0/2637425280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c7a63c00 con 0x55c8c7b61d30 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.101+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b5b840 0x55c8c7b69f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.102+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b64ab0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009ad0 tx=0x7ff69c0135d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c202ae6f740e9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.102+0000 7ff68effd640 1 -- 172.21.15.100:0/2637425280 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c018070 con 0x55c8c7b61d30 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.102+0000 7ff68effd640 1 -- 172.21.15.100:0/2637425280 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff69c010580 con 0x55c8c7b61d30 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.102+0000 7ff68effd640 1 -- 172.21.15.100:0/2637425280 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010a90 con 0x55c8c7b61d30 2024-03-12T05:22:44.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2637425280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 msgr2=0x55c8c7b64ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b64ab0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009ad0 tx=0x7ff69c0135d0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2637425280 shutdown_connections 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b61d30 0x55c8c7b64ab0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61380 0x55c8c7b680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/2637425280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b5b840 0x55c8c7b69f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2637425280 >> 172.21.15.100:0/2637425280 conn(0x55c8c7b62480 msgr2=0x55c8c7b66100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2637425280 shutdown_connections 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/2637425280 wait complete. 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- start start 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 Processor -- start 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 0x55c8c7bca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7bcac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7b71240 con 0x55c8c7b5b840 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7b713b0 con 0x55c8c7b61d30 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c7b71570 con 0x55c8c7b61380 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.103+0000 7ff6ae1d0740 10 client.0 _ll_register_callbacks cb 0x7ff6a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 0x55c8c7bca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 0x55c8c7bca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56954/0 (socket says 172.21.15.100:56954) 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 -- 172.21.15.100:0/236252114 learned_addr learned my addr 172.21.15.100:0/236252114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a9ac4640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7bcac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 -- 172.21.15.100:0/236252114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 msgr2=0x55c8c7bcac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7bcac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 -- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 msgr2=0x55c8c7b70b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b70b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 -- 172.21.15.100:0/236252114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a001d040 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7bcac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6a92c3640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 0x55c8c7bca730 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009710 tx=0x7ff6a0005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912df3b7fb01fba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff68cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff68cff9640 1 -- 172.21.15.100:0/236252114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a002b070 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8c7aee900 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff68cff9640 1 -- 172.21.15.100:0/236252114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff6a0004630 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff68cff9640 1 -- 172.21.15.100:0/236252114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6a0019720 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6ae1d0740 10 client.17775 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.104+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c8c7ac7320 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff68cff9640 1 -- 172.21.15.100:0/236252114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c8c7aee900 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff68cff9640 1 -- 172.21.15.100:0/236252114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff6a00259c0 con 0x55c8c7b5b840 2024-03-12T05:22:44.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6ae1d0740 10 client.17775 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6ae1d0740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6ae1d0740 20 client.17775 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b6a8b0 0x55c8c7bd5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6ae1d0740 4 client.17775 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c8c77c8860 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b6a8b0 0x55c8c7bd5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff6a8ac2640 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b6a8b0 0x55c8c7bd5550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf0b722174f18f4 server_cookie=4f3b71b6d260dbc5 in_seq=0 out_seq=0 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.105+0000 7ff68cff9640 10 client.17775 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff68cff9640 1 -- 172.21.15.100:0/236252114 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c8c77c8860 con 0x55c8c7b6a8b0 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff68cff9640 10 client.17775 handle_command_reply: tid=0 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 client.17775 shutdown 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 msgr2=0x55c8c7bca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 0x55c8c7bca730 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009710 tx=0x7ff6a0005d70 comp rx=0 tx=0).stop 2024-03-12T05:22:44.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 shutdown_connections 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c8c7b6a8b0 0x55c8c7bd5550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8c7b61d30 0x55c8c7b70b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c7b61380 0x55c8c7bcac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 --2- 172.21.15.100:0/236252114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c7b5b840 0x55c8c7bca730 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 >> 172.21.15.100:0/236252114 conn(0x55c8c7b62480 msgr2=0x55c8c7b5cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.106+0000 7ff68cff9640 0 client.17775 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 shutdown_connections 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ae1d0740 1 -- 172.21.15.100:0/236252114 wait complete. 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ae1d0740 20 client.17775 trim_cache size 0 max 16384 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:22:44.108 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:22:44.109 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "0c851c49-3b90-4404-9b55-a9befa249f90", 2024-03-12T05:22:44.109 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:22:44.109 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:22:44.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff6900a1ba0 msgr2=0x7ff6900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff6900a1ba0 0x7ff6900a4060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff69c005fd0 tx=0x7ff69c009b00 comp rx=0 tx=0).stop 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 msgr2=0x7ff6a4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 0x7ff6a4198b70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff694009230 tx=0x7ff69402c680 comp rx=0 tx=0).stop 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 shutdown_connections 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff6900a1ba0 0x7ff6900a4060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6a4199100 0x7ff6a41a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a4105330 0x7ff6a4198b70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 --2- 172.21.15.100:0/3747730361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a4100260 0x7ff6a4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 >> 172.21.15.100:0/3747730361 conn(0x7ff6a4077180 msgr2=0x7ff6a40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 shutdown_connections 2024-03-12T05:22:44.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.107+0000 7ff6ab527640 1 -- 172.21.15.100:0/3747730361 wait complete. 2024-03-12T05:22:44.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:22:44.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:22:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 -- 172.21.15.100:0/4191922812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144100270 msgr2=0x7f0144100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 --2- 172.21.15.100:0/4191922812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144100270 0x7f0144100650 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0140009cd0 tx=0x7f014002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 -- 172.21.15.100:0/4191922812 shutdown_connections 2024-03-12T05:22:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 --2- 172.21.15.100:0/4191922812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144105340 0x7f014410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 --2- 172.21.15.100:0/4191922812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0144100c20 0x7f0144104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 --2- 172.21.15.100:0/4191922812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144100270 0x7f0144100650 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 -- 172.21.15.100:0/4191922812 >> 172.21.15.100:0/4191922812 conn(0x7f0144077230 msgr2=0x7f0144077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 -- 172.21.15.100:0/4191922812 shutdown_connections 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 -- 172.21.15.100:0/4191922812 wait complete. 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.226+0000 7f014c40f640 1 Processor -- start 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 -- start start 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144100270 0x7f01441985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 0x7f0144198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01441990b0 0x7f01441a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441a3d00 con 0x7f0144105340 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441a3e70 con 0x7f0144100270 2024-03-12T05:22:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441a4030 con 0x7f01441990b0 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 0x7f0144198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 0x7f0144198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56972/0 (socket says 172.21.15.100:56972) 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 -- 172.21.15.100:0/1430892309 learned_addr learned my addr 172.21.15.100:0/1430892309 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014a1ab640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144100270 0x7f01441985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014a9ac640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01441990b0 0x7f01441a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 -- 172.21.15.100:0/1430892309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01441990b0 msgr2=0x7f01441a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01441990b0 0x7f01441a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 -- 172.21.15.100:0/1430892309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144100270 msgr2=0x7f01441985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144100270 0x7f01441985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 -- 172.21.15.100:0/1430892309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009900 con 0x7f0144105340 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014a1ab640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144100270 0x7f01441985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f01499aa640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 0x7f0144198b20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f013400d740 tx=0x7f013400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f013b7fe640 1 -- 172.21.15.100:0/1430892309 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134009c30 con 0x7f0144105340 2024-03-12T05:22:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.227+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01441a4310 con 0x7f0144105340 2024-03-12T05:22:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.228+0000 7f013b7fe640 1 -- 172.21.15.100:0/1430892309 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f013400de50 con 0x7f0144105340 2024-03-12T05:22:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.228+0000 7f013b7fe640 1 -- 172.21.15.100:0/1430892309 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134010460 con 0x7f0144105340 2024-03-12T05:22:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.228+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01441a4820 con 0x7f0144105340 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.230+0000 7f013b7fe640 1 -- 172.21.15.100:0/1430892309 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0134004f10 con 0x7f0144105340 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.230+0000 7f014efd9740 1 Processor -- start 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.230+0000 7f014efd9740 1 -- start start 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.230+0000 7f013b7fe640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01300a1ba0 0x7f01300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014efd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168fe7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014efd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe7bd0 0x55e168ff0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014efd9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168ff2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f013b7fe640 1 -- 172.21.15.100:0/1430892309 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f01340cd240 con 0x7f0144105340 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a9ac640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01300a1ba0 0x7f01300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014efd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168f56280 con 0x55e168fe72b0 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014efd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168f537b0 con 0x55e168fe7bd0 2024-03-12T05:22:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014efd9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168f4ef90 con 0x55e168ff07b0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168fe7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168fe7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56974/0 (socket says 172.21.15.100:56974) 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 -- 172.21.15.100:0/3743587628 learned_addr learned my addr 172.21.15.100:0/3743587628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f01499aa640 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe7bd0 0x55e168ff0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 -- 172.21.15.100:0/3743587628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 msgr2=0x55e168ff2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168ff2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 -- 172.21.15.100:0/3743587628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe7bd0 msgr2=0x55e168ff0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe7bd0 0x55e168ff0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 -- 172.21.15.100:0/3743587628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e168f0d570 con 0x55e168fe72b0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a9ac640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01300a1ba0 0x7f01300a4060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f013c009bf0 tx=0x7f013c009730 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f014a1ab640 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168fe7690 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0140009ca0 tx=0x7f0140005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fa0dadd7490f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.231+0000 7f0138bfe640 1 -- 172.21.15.100:0/3743587628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140032e70 con 0x55e168fe72b0 2024-03-12T05:22:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f0138bfe640 1 -- 172.21.15.100:0/3743587628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01400326b0 con 0x55e168fe72b0 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f0138bfe640 1 -- 172.21.15.100:0/3743587628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140004040 con 0x55e168fe72b0 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 -- 172.21.15.100:0/3743587628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 msgr2=0x55e168fe7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168fe7690 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0140009ca0 tx=0x7f0140005ef0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 -- 172.21.15.100:0/3743587628 shutdown_connections 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168ff2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe7bd0 0x55e168ff0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 --2- 172.21.15.100:0/3743587628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168fe7690 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 -- 172.21.15.100:0/3743587628 >> 172.21.15.100:0/3743587628 conn(0x55e168fe1990 msgr2=0x55e168fe1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 -- 172.21.15.100:0/3743587628 shutdown_connections 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.232+0000 7f014efd9740 1 -- 172.21.15.100:0/3743587628 wait complete. 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 -- start start 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 Processor -- start 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe72b0 0x55e168ff01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168fed540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168fedb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168c22430 con 0x55e168fe7bd0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168bdee00 con 0x55e168fe72b0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168ed3fd0 con 0x55e168ff07b0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014efd9740 10 client.0 _ll_register_callbacks cb 0x55e168f29fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f01499aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168fed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f01499aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168fed540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56980/0 (socket says 172.21.15.100:56980) 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f01499aa640 1 -- 172.21.15.100:0/2714906755 learned_addr learned my addr 172.21.15.100:0/2714906755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014a1ab640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe72b0 0x55e168ff01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.233+0000 7f014a9ac640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168fedb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f01499aa640 1 -- 172.21.15.100:0/2714906755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 msgr2=0x55e168fedb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f01499aa640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168fedb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f01499aa640 1 -- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe72b0 msgr2=0x55e168ff01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f01499aa640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe72b0 0x55e168ff01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f01499aa640 1 -- 172.21.15.100:0/2714906755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014410d720 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f01499aa640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168fed540 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0134099ae0 tx=0x7f013409df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f79177530192d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f012dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f012dffb640 1 -- 172.21.15.100:0/2714906755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134009e50 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f012dffb640 1 -- 172.21.15.100:0/2714906755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f01340964a0 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e168c4e7e0 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f012dffb640 1 -- 172.21.15.100:0/2714906755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0134096660 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 10 client.17787 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e168f4d610 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f012dffb640 1 -- 172.21.15.100:0/2714906755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f013407c020 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f012dffb640 1 -- 172.21.15.100:0/2714906755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f01340b8070 con 0x55e168fe7bd0 2024-03-12T05:22:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 10 client.17787 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 20 client.17787 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169000940 0x55e169002d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 4 client.17787 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.234+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e168c8b720 con 0x55e169000940 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.235+0000 7f014a1ab640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169000940 0x55e169002d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.235+0000 7f014a1ab640 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169000940 0x55e169002d30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ca9ff57c6d3578 server_cookie=34dd32ed7900faad in_seq=0 out_seq=0 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.235+0000 7f012dffb640 10 client.17787 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f012dffb640 1 -- 172.21.15.100:0/2714906755 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e168c8b720 con 0x55e169000940 2024-03-12T05:22:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f012dffb640 10 client.17787 handle_command_reply: tid=0 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 client.17787 shutdown 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 msgr2=0x55e168fed540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168fed540 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0134099ae0 tx=0x7f013409df80 comp rx=0 tx=0).stop 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 shutdown_connections 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169000940 0x55e169002d30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168ff07b0 0x55e168fedb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168fed540 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f012dffb640 0 client.17787 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 --2- 172.21.15.100:0/2714906755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168fe72b0 0x55e168ff01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 >> 172.21.15.100:0/2714906755 conn(0x55e168fe1990 msgr2=0x55e168fe3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 shutdown_connections 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 1 -- 172.21.15.100:0/2714906755 wait complete. 2024-03-12T05:22:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.236+0000 7f014efd9740 20 client.17787 trim_cache size 0 max 16384 2024-03-12T05:22:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.251+0000 7f014efd9740 1 Processor -- start 2024-03-12T05:22:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 -- start start 2024-03-12T05:22:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe72b0 0x55e168fefc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168ff01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168febe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168fe9140 con 0x55e168fe7bd0 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168feba90 con 0x55e168ff07b0 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014efd9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168fe79c0 con 0x55e168fe72b0 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168ff01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168ff01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56984/0 (socket says 172.21.15.100:56984) 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 -- 172.21.15.100:0/1686191104 learned_addr learned my addr 172.21.15.100:0/1686191104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014a1ab640 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe72b0 0x55e168fefc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014a9ac640 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168febe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 -- 172.21.15.100:0/1686191104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe72b0 msgr2=0x55e168fefc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe72b0 0x55e168fefc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 -- 172.21.15.100:0/1686191104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 msgr2=0x55e168febe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168febe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 -- 172.21.15.100:0/1686191104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e168ee9ac0 con 0x55e168fe7bd0 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f014a9ac640 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168febe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f01499aa640 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168ff01b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0134099dc0 tx=0x7f0134098340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb790f7803eca45c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f012ffff640 1 -- 172.21.15.100:0/1686191104 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134009e50 con 0x55e168fe7bd0 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f012ffff640 1 -- 172.21.15.100:0/1686191104 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f013407c070 con 0x55e168fe7bd0 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.252+0000 7f012ffff640 1 -- 172.21.15.100:0/1686191104 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340cad70 con 0x55e168fe7bd0 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 -- 172.21.15.100:0/1686191104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 msgr2=0x55e168ff01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168ff01b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0134099dc0 tx=0x7f0134098340 comp rx=0 tx=0).stop 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 -- 172.21.15.100:0/1686191104 shutdown_connections 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168febe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe7bd0 0x55e168ff01b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 --2- 172.21.15.100:0/1686191104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe72b0 0x55e168fefc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 -- 172.21.15.100:0/1686191104 >> 172.21.15.100:0/1686191104 conn(0x55e168fe1990 msgr2=0x55e1690076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 -- 172.21.15.100:0/1686191104 shutdown_connections 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 -- 172.21.15.100:0/1686191104 wait complete. 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.253+0000 7f014efd9740 1 -- start start 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 Processor -- start 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168ff3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe7bd0 0x55e168ff4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168ff49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e168ff7eb0 con 0x55e168fe72b0 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e169056e50 con 0x55e168ff07b0 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e169056f90 con 0x55e168fe7bd0 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 10 client.0 _ll_register_callbacks cb 0x7f013c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168ff3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168ff3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56996/0 (socket says 172.21.15.100:56996) 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 -- 172.21.15.100:0/1807846818 learned_addr learned my addr 172.21.15.100:0/1807846818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a9ac640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168ff49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f01499aa640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe7bd0 0x55e168ff4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 -- 172.21.15.100:0/1807846818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe7bd0 msgr2=0x55e168ff4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe7bd0 0x55e168ff4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 -- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 msgr2=0x55e168ff49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168ff49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 -- 172.21.15.100:0/1807846818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e168f4d610 con 0x55e168fe72b0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014a1ab640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168ff3f40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0140004af0 tx=0x7f0140003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b591917dfa3dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f012dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f012dffb640 1 -- 172.21.15.100:0/1807846818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140032440 con 0x55e168fe72b0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.254+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01400326b0 con 0x55e168fe72b0 2024-03-12T05:22:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f012dffb640 1 -- 172.21.15.100:0/1807846818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f014004f040 con 0x55e168fe72b0 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f012dffb640 1 -- 172.21.15.100:0/1807846818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0140032820 con 0x55e168fe72b0 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 10 client.17793 fetch_fsmap learned FSMap version 25 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e168f0f6a0 con 0x55e168fe72b0 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f012dffb640 1 -- 172.21.15.100:0/1807846818 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01400326b0 con 0x55e168fe72b0 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f012dffb640 1 -- 172.21.15.100:0/1807846818 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f014004a450 con 0x55e168fe72b0 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 10 client.17793 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 20 client.17793 populate_metadata read hostname 'smithi100' 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169058450 0x55e16905a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 4 client.17793 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e168cda890 con 0x55e169058450 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.255+0000 7f01499aa640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169058450 0x55e16905a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:22:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f01499aa640 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169058450 0x55e16905a870 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d845108fdb73e7f4 server_cookie=ae27abfa247b6418 in_seq=0 out_seq=0 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f012dffb640 10 client.17793 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f012dffb640 1 -- 172.21.15.100:0/1807846818 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e168cda890 con 0x55e169058450 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f012dffb640 10 client.17793 handle_command_reply: tid=0 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 client.17793 shutdown 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 msgr2=0x55e168ff3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168ff3f40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0140004af0 tx=0x7f0140003b20 comp rx=0 tx=0).stop 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 shutdown_connections 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e169058450 0x55e16905a870 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e168ff07b0 0x55e168ff49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e168fe7bd0 0x55e168ff4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f012dffb640 0 client.17793 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 --2- 172.21.15.100:0/1807846818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e168fe72b0 0x55e168ff3f40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 >> 172.21.15.100:0/1807846818 conn(0x55e168fe1990 msgr2=0x55e168febdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 shutdown_connections 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 1 -- 172.21.15.100:0/1807846818 wait complete. 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.256+0000 7f014efd9740 20 client.17793 trim_cache size 0 max 16384 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stdout: "0c851c49-3b90-4404-9b55-a9befa249f90": { 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:22:44.259 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "0c851c49-3b90-4404-9b55-a9befa249f90", 2024-03-12T05:22:44.260 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:22:44.260 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:22:44.260 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:22:44.260 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01300a1ba0 msgr2=0x7f01300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01300a1ba0 0x7f01300a4060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f013c009bf0 tx=0x7f013c009730 comp rx=0 tx=0).stop 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 msgr2=0x7f0144198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 0x7f0144198b20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f013400d740 tx=0x7f013400dc00 comp rx=0 tx=0).stop 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 shutdown_connections 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01300a1ba0 0x7f01300a4060 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01441990b0 0x7f01441a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0144105340 0x7f0144198b20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 --2- 172.21.15.100:0/1430892309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144100270 0x7f01441985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 >> 172.21.15.100:0/1430892309 conn(0x7f0144077230 msgr2=0x7f01440fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 shutdown_connections 2024-03-12T05:22:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:22:44.257+0000 7f014c40f640 1 -- 172.21.15.100:0/1430892309 wait complete. 2024-03-12T05:22:44.262 INFO:tasks.cephfs.filesystem:scrub status for tag:0c851c49-3b90-4404-9b55-a9befa249f90 - {'path': '/', 'tag': '0c851c49-3b90-4404-9b55-a9befa249f90', 'options': 'recursive,force'} 2024-03-12T05:22:44.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1283198 11.81 MB/sec execute 448 sec latency 20.934 ms 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: pgmap v330: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:44 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: pgmap v330: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-03-12T05:22:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:22:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:22:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:22:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:44 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: pgmap v330: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-03-12T05:22:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:22:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:22:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:22:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:44 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:22:45.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1285644 11.82 MB/sec execute 449 sec latency 42.555 ms 2024-03-12T05:22:46.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1287230 11.80 MB/sec execute 450 sec latency 26.895 ms 2024-03-12T05:22:47.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:46 smithi156 ceph-mon[36270]: pgmap v331: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-03-12T05:22:47.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:46 smithi161 ceph-mon[36043]: pgmap v331: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-03-12T05:22:47.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:46 smithi100 ceph-mon[32210]: pgmap v331: 97 pgs: 97 active+clean; 607 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-03-12T05:22:47.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1289656 11.81 MB/sec execute 451 sec latency 17.766 ms 2024-03-12T05:22:48.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1291998 11.81 MB/sec execute 452 sec latency 25.288 ms 2024-03-12T05:22:49.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:48 smithi156 ceph-mon[36270]: pgmap v332: 97 pgs: 97 active+clean; 559 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 290 op/s 2024-03-12T05:22:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:48 smithi161 ceph-mon[36043]: pgmap v332: 97 pgs: 97 active+clean; 559 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 290 op/s 2024-03-12T05:22:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:48 smithi100 ceph-mon[32210]: pgmap v332: 97 pgs: 97 active+clean; 559 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 290 op/s 2024-03-12T05:22:49.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1293901 11.80 MB/sec execute 453 sec latency 16.544 ms 2024-03-12T05:22:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:49 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:22:50.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:49 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:22:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:49 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:22:50.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1296364 11.81 MB/sec execute 454 sec latency 57.314 ms 2024-03-12T05:22:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:50 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:22:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:50 smithi161 ceph-mon[36043]: pgmap v333: 97 pgs: 97 active+clean; 559 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 194 op/s 2024-03-12T05:22:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:50 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:22:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:50 smithi100 ceph-mon[32210]: pgmap v333: 97 pgs: 97 active+clean; 559 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 194 op/s 2024-03-12T05:22:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:50 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:22:51.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:50 smithi156 ceph-mon[36270]: pgmap v333: 97 pgs: 97 active+clean; 559 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 194 op/s 2024-03-12T05:22:51.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1297946 11.80 MB/sec execute 455 sec latency 21.148 ms 2024-03-12T05:22:52.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1300258 11.81 MB/sec execute 456 sec latency 22.809 ms 2024-03-12T05:22:53.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:52 smithi161 ceph-mon[36043]: pgmap v334: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 293 op/s 2024-03-12T05:22:53.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:52 smithi100 ceph-mon[32210]: pgmap v334: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 293 op/s 2024-03-12T05:22:53.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:52 smithi156 ceph-mon[36270]: pgmap v334: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 293 op/s 2024-03-12T05:22:53.804 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1302258 11.80 MB/sec execute 457 sec latency 29.028 ms 2024-03-12T05:22:54.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1304345 11.80 MB/sec execute 458 sec latency 17.532 ms 2024-03-12T05:22:55.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:54 smithi161 ceph-mon[36043]: pgmap v335: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-03-12T05:22:55.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:54 smithi156 ceph-mon[36270]: pgmap v335: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-03-12T05:22:55.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:54 smithi100 ceph-mon[32210]: pgmap v335: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-03-12T05:22:55.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1306713 11.80 MB/sec execute 459 sec latency 69.711 ms 2024-03-12T05:22:56.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1308437 11.80 MB/sec execute 460 sec latency 19.613 ms 2024-03-12T05:22:57.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:56 smithi161 ceph-mon[36043]: pgmap v336: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-03-12T05:22:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:56 smithi100 ceph-mon[32210]: pgmap v336: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-03-12T05:22:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:56 smithi156 ceph-mon[36270]: pgmap v336: 97 pgs: 97 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-03-12T05:22:57.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1310886 11.80 MB/sec execute 461 sec latency 16.237 ms 2024-03-12T05:22:58.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1312845 11.79 MB/sec execute 462 sec latency 103.540 ms 2024-03-12T05:22:59.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:22:58 smithi161 ceph-mon[36043]: pgmap v337: 97 pgs: 97 active+clean; 584 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 296 op/s 2024-03-12T05:22:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:22:58 smithi100 ceph-mon[32210]: pgmap v337: 97 pgs: 97 active+clean; 584 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 296 op/s 2024-03-12T05:22:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:22:58 smithi156 ceph-mon[36270]: pgmap v337: 97 pgs: 97 active+clean; 584 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 296 op/s 2024-03-12T05:22:59.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1314787 11.79 MB/sec execute 463 sec latency 15.793 ms 2024-03-12T05:23:00.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1316288 11.78 MB/sec execute 464 sec latency 58.767 ms 2024-03-12T05:23:01.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:00 smithi161 ceph-mon[36043]: pgmap v338: 97 pgs: 97 active+clean; 584 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 196 op/s 2024-03-12T05:23:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:00 smithi100 ceph-mon[32210]: pgmap v338: 97 pgs: 97 active+clean; 584 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 196 op/s 2024-03-12T05:23:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:00 smithi156 ceph-mon[36270]: pgmap v338: 97 pgs: 97 active+clean; 584 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 196 op/s 2024-03-12T05:23:01.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1318129 11.78 MB/sec execute 465 sec latency 23.126 ms 2024-03-12T05:23:02.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1320562 11.78 MB/sec execute 466 sec latency 22.427 ms 2024-03-12T05:23:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:02 smithi161 ceph-mon[36043]: pgmap v339: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 325 op/s 2024-03-12T05:23:03.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:02 smithi100 ceph-mon[32210]: pgmap v339: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 325 op/s 2024-03-12T05:23:03.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:02 smithi156 ceph-mon[36270]: pgmap v339: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 325 op/s 2024-03-12T05:23:03.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1322594 11.78 MB/sec execute 467 sec latency 14.499 ms 2024-03-12T05:23:04.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1325012 11.79 MB/sec execute 468 sec latency 17.092 ms 2024-03-12T05:23:05.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:04 smithi161 ceph-mon[36043]: pgmap v340: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-03-12T05:23:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:04 smithi100 ceph-mon[32210]: pgmap v340: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-03-12T05:23:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:04 smithi156 ceph-mon[36270]: pgmap v340: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-03-12T05:23:05.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1326935 11.78 MB/sec execute 469 sec latency 25.162 ms 2024-03-12T05:23:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:05 smithi100 ceph-mon[32210]: osdmap e98: 12 total, 12 up, 12 in 2024-03-12T05:23:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:05 smithi100 ceph-mon[32210]: pgmap v342: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-03-12T05:23:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:05 smithi156 ceph-mon[36270]: osdmap e98: 12 total, 12 up, 12 in 2024-03-12T05:23:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:05 smithi156 ceph-mon[36270]: pgmap v342: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-03-12T05:23:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:05 smithi161 ceph-mon[36043]: osdmap e98: 12 total, 12 up, 12 in 2024-03-12T05:23:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:05 smithi161 ceph-mon[36043]: pgmap v342: 97 pgs: 97 active+clean; 595 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-03-12T05:23:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1328740 11.78 MB/sec execute 470 sec latency 69.541 ms 2024-03-12T05:23:07.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1330943 11.78 MB/sec execute 471 sec latency 17.943 ms 2024-03-12T05:23:08.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1332637 11.77 MB/sec execute 472 sec latency 75.619 ms 2024-03-12T05:23:09.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:08 smithi156 ceph-mon[36270]: pgmap v343: 97 pgs: 97 active+clean; 606 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-03-12T05:23:09.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:08 smithi161 ceph-mon[36043]: pgmap v343: 97 pgs: 97 active+clean; 606 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-03-12T05:23:09.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:08 smithi100 ceph-mon[32210]: pgmap v343: 97 pgs: 97 active+clean; 606 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-03-12T05:23:09.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1335237 11.78 MB/sec execute 473 sec latency 40.959 ms 2024-03-12T05:23:10.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1336957 11.77 MB/sec execute 474 sec latency 20.695 ms 2024-03-12T05:23:11.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:10 smithi156 ceph-mon[36270]: pgmap v344: 97 pgs: 97 active+clean; 606 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-03-12T05:23:11.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:10 smithi156 ceph-mon[36270]: osdmap e99: 12 total, 12 up, 12 in 2024-03-12T05:23:11.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:10 smithi161 ceph-mon[36043]: pgmap v344: 97 pgs: 97 active+clean; 606 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-03-12T05:23:11.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:10 smithi161 ceph-mon[36043]: osdmap e99: 12 total, 12 up, 12 in 2024-03-12T05:23:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:10 smithi100 ceph-mon[32210]: pgmap v344: 97 pgs: 97 active+clean; 606 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-03-12T05:23:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:10 smithi100 ceph-mon[32210]: osdmap e99: 12 total, 12 up, 12 in 2024-03-12T05:23:11.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1339094 11.78 MB/sec execute 475 sec latency 119.527 ms 2024-03-12T05:23:12.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1340800 11.76 MB/sec execute 476 sec latency 26.290 ms 2024-03-12T05:23:13.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:12 smithi156 ceph-mon[36270]: pgmap v346: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 299 op/s 2024-03-12T05:23:13.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:12 smithi161 ceph-mon[36043]: pgmap v346: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 299 op/s 2024-03-12T05:23:13.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:12 smithi100 ceph-mon[32210]: pgmap v346: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 299 op/s 2024-03-12T05:23:13.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1342935 11.77 MB/sec execute 477 sec latency 45.793 ms 2024-03-12T05:23:14.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:23:14.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:23:14.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 -- 172.21.15.100:0/3079237490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 msgr2=0x7fc190100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 --2- 172.21.15.100:0/3079237490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190100650 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc180009930 tx=0x7fc18002bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 -- 172.21.15.100:0/3079237490 shutdown_connections 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 --2- 172.21.15.100:0/3079237490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc190105340 0x7fc19010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 --2- 172.21.15.100:0/3079237490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc190100c20 0x7fc190104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 --2- 172.21.15.100:0/3079237490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190100650 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 -- 172.21.15.100:0/3079237490 >> 172.21.15.100:0/3079237490 conn(0x7fc190077230 msgr2=0x7fc190077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 -- 172.21.15.100:0/3079237490 shutdown_connections 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 -- 172.21.15.100:0/3079237490 wait complete. 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 Processor -- start 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.378+0000 7fc195a15640 1 -- start start 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc190105340 0x7fc190102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc190102780 0x7fc1901a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1901a5f40 con 0x7fc190100270 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1901a60b0 con 0x7fc190102780 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1901a6270 con 0x7fc190105340 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49446/0 (socket says 172.21.15.100:49446) 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 -- 172.21.15.100:0/3199912125 learned_addr learned my addr 172.21.15.100:0/3199912125 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc190105340 0x7fc190102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18f7fe640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc190102780 0x7fc1901a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 -- 172.21.15.100:0/3199912125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc190105340 msgr2=0x7fc190102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc190105340 0x7fc190102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 -- 172.21.15.100:0/3199912125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc190102780 msgr2=0x7fc1901a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc190102780 0x7fc1901a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 -- 172.21.15.100:0/3199912125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180009560 con 0x7fc190100270 2024-03-12T05:23:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc18effd640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190101d00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc180009230 tx=0x7fc18002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc194a13640 1 -- 172.21.15.100:0/3199912125 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180004350 con 0x7fc190100270 2024-03-12T05:23:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc194a13640 1 -- 172.21.15.100:0/3199912125 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1800044b0 con 0x7fc190100270 2024-03-12T05:23:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1901a64f0 con 0x7fc190100270 2024-03-12T05:23:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc194a13640 1 -- 172.21.15.100:0/3199912125 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180003d00 con 0x7fc190100270 2024-03-12T05:23:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.379+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1901a6a50 con 0x7fc190100270 2024-03-12T05:23:14.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc194a13640 1 -- 172.21.15.100:0/3199912125 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc180004df0 con 0x7fc190100270 2024-03-12T05:23:14.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 Processor -- start 2024-03-12T05:23:14.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc194a13640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1780a1c50 0x7fc1780a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 -- start start 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc18f7fe640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1780a1c50 0x7fc1780a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442529690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc194a13640 1 -- 172.21.15.100:0/3199912125 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1800ef6a0 con 0x7fc190100270 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564442529bd0 0x564442532270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425327b0 0x564442534ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442498280 con 0x5644425327b0 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644424957b0 con 0x5644425292b0 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc1985ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442490f90 con 0x564442529bd0 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc18effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442529690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc18f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425327b0 0x564442534ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc18effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442529690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45414/0 (socket says 172.21.15.100:45414) 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc18effd640 1 -- 172.21.15.100:0/4134764977 learned_addr learned my addr 172.21.15.100:0/4134764977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.382+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564442529bd0 0x564442532270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18effd640 1 -- 172.21.15.100:0/4134764977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564442529bd0 msgr2=0x564442532270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18effd640 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564442529bd0 0x564442532270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18effd640 1 -- 172.21.15.100:0/4134764977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425327b0 msgr2=0x564442534ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18effd640 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425327b0 0x564442534ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18effd640 1 -- 172.21.15.100:0/4134764977 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56444244f570 con 0x5644425292b0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18f7fe640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1780a1c50 0x7fc1780a4110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc188005fd0 tx=0x7fc188013ae0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc18effd640 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442529690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1800c4610 tx=0x7fc18002c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc68585efb658a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc175ffb640 1 -- 172.21.15.100:0/4134764977 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800bcaa0 con 0x5644425292b0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc175ffb640 1 -- 172.21.15.100:0/4134764977 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1800bc3a0 con 0x5644425292b0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.383+0000 7fc175ffb640 1 -- 172.21.15.100:0/4134764977 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800b8a10 con 0x5644425292b0 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 -- 172.21.15.100:0/4134764977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 msgr2=0x564442529690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442529690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1800c4610 tx=0x7fc18002c740 comp rx=0 tx=0).stop 2024-03-12T05:23:14.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 -- 172.21.15.100:0/4134764977 shutdown_connections 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425327b0 0x564442534ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564442529bd0 0x564442532270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 --2- 172.21.15.100:0/4134764977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442529690 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 -- 172.21.15.100:0/4134764977 >> 172.21.15.100:0/4134764977 conn(0x564442523990 msgr2=0x564442523d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 -- 172.21.15.100:0/4134764977 shutdown_connections 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 -- 172.21.15.100:0/4134764977 wait complete. 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 -- start start 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.384+0000 7fc1985ec740 1 Processor -- start 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442532160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x56444252f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442164430 con 0x564442529bd0 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442120e00 con 0x5644425292b0 2024-03-12T05:23:14.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442415fd0 con 0x5644425327b0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 10 client.0 _ll_register_callbacks cb 0x56444246bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x56444252f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x56444252f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49466/0 (socket says 172.21.15.100:49466) 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 -- 172.21.15.100:0/1244596248 learned_addr learned my addr 172.21.15.100:0/1244596248 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18effd640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442532160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18f7fe640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 -- 172.21.15.100:0/1244596248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 msgr2=0x56444252fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 -- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 msgr2=0x564442532160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442532160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 -- 172.21.15.100:0/1244596248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1901a6a50 con 0x564442529bd0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x56444252f4f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc1840126e0 tx=0x7fc184012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ab63b1af91fcf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc16effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:14.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc16effd640 1 -- 172.21.15.100:0/1244596248 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184017070 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc16effd640 1 -- 172.21.15.100:0/1244596248 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc184012190 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644421907e0 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.385+0000 7fc16effd640 1 -- 172.21.15.100:0/1244596248 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18400f920 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56444248f610 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc16effd640 1 -- 172.21.15.100:0/1244596248 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644421907e0 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc16effd640 1 -- 172.21.15.100:0/1244596248 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc184010b60 con 0x564442529bd0 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 20 client.17805 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442542820 0x564442544c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 4 client.17805 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644421cd720 con 0x564442542820 2024-03-12T05:23:14.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc18effd640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442542820 0x564442544c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc16effd640 10 client.17805 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.386+0000 7fc18effd640 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442542820 0x564442544c40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be3fbb9f9b7b27f server_cookie=fd591b2a54ac6fcb in_seq=0 out_seq=0 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc16effd640 1 -- 172.21.15.100:0/1244596248 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644421cd720 con 0x564442542820 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc16effd640 10 client.17805 handle_command_reply: tid=0 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 client.17805 shutdown 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 msgr2=0x56444252f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x56444252f4f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc1840126e0 tx=0x7fc184012ba0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 shutdown_connections 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442542820 0x564442544c40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x56444252f4f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 --2- 172.21.15.100:0/1244596248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442532160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 >> 172.21.15.100:0/1244596248 conn(0x564442523990 msgr2=0x564442525730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc16effd640 0 client.17805 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:14.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 shutdown_connections 2024-03-12T05:23:14.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 1 -- 172.21.15.100:0/1244596248 wait complete. 2024-03-12T05:23:14.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.387+0000 7fc1985ec740 20 client.17805 trim_cache size 0 max 16384 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.402+0000 7fc1985ec740 1 Processor -- start 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.402+0000 7fc1985ec740 1 -- start start 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442531c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x564442532160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1985ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56444251f0d0 con 0x564442529bd0 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1985ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442524790 con 0x5644425292b0 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1985ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56444253e330 con 0x5644425327b0 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x564442532160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x564442532160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49474/0 (socket says 172.21.15.100:49474) 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 -- 172.21.15.100:0/2994225645 learned_addr learned my addr 172.21.15.100:0/2994225645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18effd640 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442531c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18f7fe640 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 -- 172.21.15.100:0/2994225645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 msgr2=0x56444252a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 -- 172.21.15.100:0/2994225645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 msgr2=0x564442531c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442531c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 -- 172.21.15.100:0/2994225645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56444242bac0 con 0x564442529bd0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x564442532160 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc184001eb0 tx=0x7fc18400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777e2c701b37eb30 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1757fa640 1 -- 172.21.15.100:0/2994225645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184017040 con 0x564442529bd0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1757fa640 1 -- 172.21.15.100:0/2994225645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1840055b0 con 0x564442529bd0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.403+0000 7fc1757fa640 1 -- 172.21.15.100:0/2994225645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400fe20 con 0x564442529bd0 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 -- 172.21.15.100:0/2994225645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 msgr2=0x564442532160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x564442532160 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc184001eb0 tx=0x7fc18400f810 comp rx=0 tx=0).stop 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 -- 172.21.15.100:0/2994225645 shutdown_connections 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x56444252a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564442529bd0 0x564442532160 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2994225645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644425292b0 0x564442531c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 -- 172.21.15.100:0/2994225645 >> 172.21.15.100:0/2994225645 conn(0x564442523990 msgr2=0x5644425495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 -- 172.21.15.100:0/2994225645 shutdown_connections 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 -- 172.21.15.100:0/2994225645 wait complete. 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 -- start start 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.404+0000 7fc1985ec740 1 Processor -- start 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 0x5644425356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564442529bd0 0x564442535c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x564442591d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644425922b0 con 0x5644425292b0 2024-03-12T05:23:14.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564442592420 con 0x564442529bd0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644425925e0 con 0x5644425327b0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 10 client.0 _ll_register_callbacks cb 0x7fc1800b8640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 0x5644425356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 0x5644425356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49488/0 (socket says 172.21.15.100:49488) 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 -- 172.21.15.100:0/2168499024 learned_addr learned my addr 172.21.15.100:0/2168499024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564442529bd0 0x564442535c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18f7fe640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x564442591d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 -- 172.21.15.100:0/2168499024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 msgr2=0x564442591d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x564442591d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 -- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564442529bd0 msgr2=0x564442535c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564442529bd0 0x564442535c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 -- 172.21.15.100:0/2168499024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56444248f610 con 0x5644425292b0 2024-03-12T05:23:14.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564442529bd0 0x564442535c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc18effd640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 0x5644425356e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc1800bc0a0 tx=0x7fc1800dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75122cfca447f6d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc16effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc16effd640 1 -- 172.21.15.100:0/2168499024 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800bcaa0 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc16effd640 1 -- 172.21.15.100:0/2168499024 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1800c04f0 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.405+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1800bc3a0 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc16effd640 1 -- 172.21.15.100:0/2168499024 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1800055d0 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 10 client.17811 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5644424516a0 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc16effd640 1 -- 172.21.15.100:0/2168499024 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1800bc3a0 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc16effd640 1 -- 172.21.15.100:0/2168499024 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc18003c070 con 0x5644425292b0 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 10 client.17811 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 20 client.17811 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:14.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442599a50 0x56444259be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 4 client.17811 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56444221c890 con 0x564442599a50 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442599a50 0x56444259be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc16effd640 10 client.17811 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.406+0000 7fc18e7fc640 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442599a50 0x56444259be70 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894b3703ace709e2 server_cookie=d8aab96dfe42e31 in_seq=0 out_seq=0 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc16effd640 1 -- 172.21.15.100:0/2168499024 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56444221c890 con 0x564442599a50 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc16effd640 10 client.17811 handle_command_reply: tid=0 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 client.17811 shutdown 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 msgr2=0x5644425356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 0x5644425356e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc1800bc0a0 tx=0x7fc1800dc040 comp rx=0 tx=0).stop 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 shutdown_connections 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x564442599a50 0x56444259be70 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644425327b0 0x564442591d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564442529bd0 0x564442535c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc16effd640 0 client.17811 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 --2- 172.21.15.100:0/2168499024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644425292b0 0x5644425356e0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 >> 172.21.15.100:0/2168499024 conn(0x564442523990 msgr2=0x56444252a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 shutdown_connections 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 1 -- 172.21.15.100:0/2168499024 wait complete. 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc1985ec740 20 client.17811 trim_cache size 0 max 16384 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:23:14.409 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:23:14.410 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:23:14.410 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1780a1c50 msgr2=0x7fc1780a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1780a1c50 0x7fc1780a4110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc188005fd0 tx=0x7fc188013ae0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 msgr2=0x7fc190101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190101d00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc180009230 tx=0x7fc18002c650 comp rx=0 tx=0).stop 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 shutdown_connections 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1780a1c50 0x7fc1780a4110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc190102780 0x7fc1901a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc190105340 0x7fc190102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 --2- 172.21.15.100:0/3199912125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc190100270 0x7fc190101d00 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.407+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 >> 172.21.15.100:0/3199912125 conn(0x7fc190077230 msgr2=0x7fc190103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.408+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 shutdown_connections 2024-03-12T05:23:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.408+0000 7fc195a15640 1 -- 172.21.15.100:0/3199912125 wait complete. 2024-03-12T05:23:14.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:23:14.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:23:14.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 -- 172.21.15.100:0/4049532394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 msgr2=0x7fee20100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 --2- 172.21.15.100:0/4049532394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee20100690 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fee18009cd0 tx=0x7fee1802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 -- 172.21.15.100:0/4049532394 shutdown_connections 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 --2- 172.21.15.100:0/4049532394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20105380 0x7fee2010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 --2- 172.21.15.100:0/4049532394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee20100c60 0x7fee20104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 --2- 172.21.15.100:0/4049532394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee20100690 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 -- 172.21.15.100:0/4049532394 >> 172.21.15.100:0/4049532394 conn(0x7fee20077120 msgr2=0x7fee20077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 -- 172.21.15.100:0/4049532394 shutdown_connections 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.523+0000 7fee27c2e640 1 -- 172.21.15.100:0/4049532394 wait complete. 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 Processor -- start 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 -- start start 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee2019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20105380 0x7fee2019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee201998a0 0x7fee20199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2019a290 con 0x7fee201002b0 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2019a400 con 0x7fee20105380 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee27c2e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2019a5c0 con 0x7fee201998a0 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee259ca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee2019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee259ca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee2019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49504/0 (socket says 172.21.15.100:49504) 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee259ca640 1 -- 172.21.15.100:0/2711738547 learned_addr learned my addr 172.21.15.100:0/2711738547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee251c9640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20105380 0x7fee2019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.524+0000 7fee261cb640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee201998a0 0x7fee20199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee259ca640 1 -- 172.21.15.100:0/2711738547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee201998a0 msgr2=0x7fee20199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee259ca640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee201998a0 0x7fee20199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee259ca640 1 -- 172.21.15.100:0/2711738547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20105380 msgr2=0x7fee2019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee259ca640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20105380 0x7fee2019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee259ca640 1 -- 172.21.15.100:0/2711738547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009900 con 0x7fee201002b0 2024-03-12T05:23:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee259ca640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee2019f6f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fee18004af0 tx=0x7fee18034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee180328c0 con 0x7fee201002b0 2024-03-12T05:23:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee18003e50 con 0x7fee201002b0 2024-03-12T05:23:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee201a65b0 con 0x7fee201002b0 2024-03-12T05:23:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18040450 con 0x7fee201002b0 2024-03-12T05:23:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.525+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee201a6a60 con 0x7fee201002b0 2024-03-12T05:23:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.526+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee200683c0 con 0x7fee201002b0 2024-03-12T05:23:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.527+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fee1800a040 con 0x7fee201002b0 2024-03-12T05:23:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.527+0000 7fee16ffd640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fee0c0a1c20 0x7fee0c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.528+0000 7fee251c9640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fee0c0a1c20 0x7fee0c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.528+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fee180f0760 con 0x7fee201002b0 2024-03-12T05:23:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.528+0000 7fee251c9640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fee0c0a1c20 0x7fee0c0a40e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fee2010d310 tx=0x7fee1000b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.531+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee180b9780 con 0x7fee201002b0 2024-03-12T05:23:14.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.760+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee20100c60 con 0x7fee201002b0 2024-03-12T05:23:14.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.761+0000 7fee16ffd640 1 -- 172.21.15.100:0/2711738547 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fee18044090 con 0x7fee201002b0 2024-03-12T05:23:14.755 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:23:14.756 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:23:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fee0c0a1c20 msgr2=0x7fee0c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fee0c0a1c20 0x7fee0c0a40e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fee2010d310 tx=0x7fee1000b410 comp rx=0 tx=0).stop 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 msgr2=0x7fee2019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee2019f6f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fee18004af0 tx=0x7fee18034cc0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 shutdown_connections 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fee0c0a1c20 0x7fee0c0a40e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee201998a0 0x7fee20199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20105380 0x7fee2019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 --2- 172.21.15.100:0/2711738547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee201002b0 0x7fee2019f6f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 >> 172.21.15.100:0/2711738547 conn(0x7fee20077120 msgr2=0x7fee20106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 shutdown_connections 2024-03-12T05:23:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.763+0000 7fee27c2e640 1 -- 172.21.15.100:0/2711738547 wait complete. 2024-03-12T05:23:14.768 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:23:14.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:23:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1345338 11.77 MB/sec execute 478 sec latency 26.726 ms 2024-03-12T05:23:14.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- 172.21.15.100:0/93816933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 msgr2=0x7fad5410d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/93816933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5410d5e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fad50009900 tx=0x7fad5002bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- 172.21.15.100:0/93816933 shutdown_connections 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/93816933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad5410db20 0x7fad5410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/93816933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5410d5e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/93816933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad54068360 0x7fad54068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- 172.21.15.100:0/93816933 >> 172.21.15.100:0/93816933 conn(0x7fad54077180 msgr2=0x7fad54077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- 172.21.15.100:0/93816933 shutdown_connections 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- 172.21.15.100:0/93816933 wait complete. 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 Processor -- start 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- start start 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 0x7fad5419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad5410db20 0x7fad541997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54199e80 con 0x7fad54068c80 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54199ff0 con 0x7fad5410db20 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.884+0000 7fad5bfb6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5419a1b0 con 0x7fad54068360 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59d52640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 0x7fad5419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49514/0 (socket says 172.21.15.100:49514) 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 -- 172.21.15.100:0/2905938187 learned_addr learned my addr 172.21.15.100:0/2905938187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59d52640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 0x7fad5419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50646/0 (socket says 172.21.15.100:50646) 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad5a553640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad5410db20 0x7fad541997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 -- 172.21.15.100:0/2905938187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 msgr2=0x7fad5419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 0x7fad5419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 -- 172.21.15.100:0/2905938187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad5410db20 msgr2=0x7fad541997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad5410db20 0x7fad541997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 -- 172.21.15.100:0/2905938187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad50009560 con 0x7fad54068c80 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59d52640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 0x7fad5419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad59551640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5419fbb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fad50005940 tx=0x7fad5002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad4affd640 1 -- 172.21.15.100:0/2905938187 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50004350 con 0x7fad54068c80 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5419a430 con 0x7fad54068c80 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad4affd640 1 -- 172.21.15.100:0/2905938187 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad500044b0 con 0x7fad54068c80 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad4affd640 1 -- 172.21.15.100:0/2905938187 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50003d30 con 0x7fad54068c80 2024-03-12T05:23:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.885+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad541a66a0 con 0x7fad54068c80 2024-03-12T05:23:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.887+0000 7fad4affd640 1 -- 172.21.15.100:0/2905938187 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fad50004df0 con 0x7fad54068c80 2024-03-12T05:23:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 Processor -- start 2024-03-12T05:23:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 -- start start 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 0x557467fb5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fb7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467f67960 con 0x557467facb70 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467f1b310 con 0x557467fb5750 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5eb87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467f18840 con 0x557467fac250 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad4affd640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad400a1ba0 0x7fad400a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad59d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 0x557467fb5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad59d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 0x557467fb5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49528/0 (socket says 172.21.15.100:49528) 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad59d52640 1 -- 172.21.15.100:0/218046866 learned_addr learned my addr 172.21.15.100:0/218046866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad59551640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fb7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.888+0000 7fad5a553640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad400a1ba0 0x7fad400a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad5a553640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59d52640 1 -- 172.21.15.100:0/218046866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 msgr2=0x557467fac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad4affd640 1 -- 172.21.15.100:0/2905938187 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad500ef7e0 con 0x7fad54068c80 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59d52640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59d52640 1 -- 172.21.15.100:0/218046866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 msgr2=0x557467fb7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59d52640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fb7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59d52640 1 -- 172.21.15.100:0/218046866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557467ed6060 con 0x557467facb70 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad5a553640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59551640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fb7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad59d52640 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 0x557467fb5210 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fad440115f0 tx=0x7fad44011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce46472596c51f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad5a553640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad400a1ba0 0x7fad400a4060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad5406a6b0 tx=0x7fad4c009aa0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad3ffff640 1 -- 172.21.15.100:0/218046866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400f640 con 0x557467facb70 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad3ffff640 1 -- 172.21.15.100:0/218046866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad4400fc80 con 0x557467facb70 2024-03-12T05:23:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.889+0000 7fad3ffff640 1 -- 172.21.15.100:0/218046866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44019d00 con 0x557467facb70 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 -- 172.21.15.100:0/218046866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 msgr2=0x557467fb5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 0x557467fb5210 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fad440115f0 tx=0x7fad44011ab0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 -- 172.21.15.100:0/218046866 shutdown_connections 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fb7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467facb70 0x557467fb5210 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 --2- 172.21.15.100:0/218046866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 -- 172.21.15.100:0/218046866 >> 172.21.15.100:0/218046866 conn(0x557467fa6980 msgr2=0x557467fa6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 -- 172.21.15.100:0/218046866 shutdown_connections 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.890+0000 7fad5eb87740 1 -- 172.21.15.100:0/218046866 wait complete. 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 -- start start 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 Processor -- start 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fac250 0x557467fb27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fb3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467f14020 con 0x557467fb5750 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467be7430 con 0x557467fac250 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467ba3e00 con 0x557467facb70 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5eb87740 10 client.0 _ll_register_callbacks cb 0x557467ebbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5a553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fb3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5a553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fb3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49538/0 (socket says 172.21.15.100:49538) 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad59551640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad5a553640 1 -- 172.21.15.100:0/1989292697 learned_addr learned my addr 172.21.15.100:0/1989292697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.891+0000 7fad59d52640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fac250 0x557467fb27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5a553640 1 -- 172.21.15.100:0/1989292697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 msgr2=0x557467fb2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5a553640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5a553640 1 -- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fac250 msgr2=0x557467fb27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5a553640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fac250 0x557467fb27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5a553640 1 -- 172.21.15.100:0/1989292697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad44009d40 con 0x557467fb5750 2024-03-12T05:23:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad59d52640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fac250 0x557467fb27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad59551640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5a553640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fb3220 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fad4c003f10 tx=0x7fad4c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1552d7cd0e609fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad3d7fa640 1 -- 172.21.15.100:0/1989292697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00f8e0 con 0x557467fb5750 2024-03-12T05:23:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad3d7fa640 1 -- 172.21.15.100:0/1989292697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad4c00fa40 con 0x557467fb5750 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557467c137e0 con 0x557467fb5750 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad3d7fa640 1 -- 172.21.15.100:0/1989292697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad4c00fc00 con 0x557467fb5750 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5eb87740 10 client.17829 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557467f126a0 con 0x557467fb5750 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.892+0000 7fad3d7fa640 1 -- 172.21.15.100:0/1989292697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557467c137e0 con 0x557467fb5750 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad3d7fa640 1 -- 172.21.15.100:0/1989292697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fad4c010430 con 0x557467fb5750 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad5eb87740 10 client.17829 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad5eb87740 10 client.17829 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad5eb87740 20 client.17829 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x557467fc5be0 0x557467fc8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad5eb87740 4 client.17829 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557467c50720 con 0x557467fc5be0 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad59d52640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x557467fc5be0 0x557467fc8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad3d7fa640 10 client.17829 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.893+0000 7fad59d52640 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x557467fc5be0 0x557467fc8000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75a3c28bcde5a162 server_cookie=de13d5ef5838da04 in_seq=0 out_seq=0 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad3d7fa640 1 -- 172.21.15.100:0/1989292697 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557467c50720 con 0x557467fc5be0 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad3d7fa640 10 client.17829 handle_command_reply: tid=0 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 client.17829 shutdown 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 msgr2=0x557467fb3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fb3220 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fad4c003f10 tx=0x7fad4c009500 comp rx=0 tx=0).stop 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 shutdown_connections 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x557467fc5be0 0x557467fc8000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fb3220 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad3d7fa640 0 client.17829 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:14.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1989292697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fac250 0x557467fb27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 >> 172.21.15.100:0/1989292697 conn(0x557467fa6980 msgr2=0x557467fa86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 shutdown_connections 2024-03-12T05:23:14.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 1 -- 172.21.15.100:0/1989292697 wait complete. 2024-03-12T05:23:14.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.894+0000 7fad5eb87740 20 client.17829 trim_cache size 0 max 16384 2024-03-12T05:23:14.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.906+0000 7fad5eb87740 1 Processor -- start 2024-03-12T05:23:14.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.906+0000 7fad5eb87740 1 -- start start 2024-03-12T05:23:14.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.906+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 0x557467fb4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fafa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad5eb87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467e99130 con 0x557467fac250 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad5eb87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467fab540 con 0x557467fb5750 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad5eb87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467fcd2b0 con 0x557467facb70 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 0x557467fb4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 0x557467fb4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49544/0 (socket says 172.21.15.100:49544) 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 -- 172.21.15.100:0/1981766157 learned_addr learned my addr 172.21.15.100:0/1981766157 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad5a553640 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fafa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59551640 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 -- 172.21.15.100:0/1981766157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 msgr2=0x557467fb3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 -- 172.21.15.100:0/1981766157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 msgr2=0x557467fafa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fafa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 -- 172.21.15.100:0/1981766157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557467eaec20 con 0x557467fac250 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad59d52640 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 0x557467fb4ed0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fad44011510 tx=0x7fad440199a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a712c322e5ec9f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad3f7fe640 1 -- 172.21.15.100:0/1981766157 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44019410 con 0x557467fac250 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad3f7fe640 1 -- 172.21.15.100:0/1981766157 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad44019570 con 0x557467fac250 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.907+0000 7fad3f7fe640 1 -- 172.21.15.100:0/1981766157 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44005040 con 0x557467fac250 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 -- 172.21.15.100:0/1981766157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 msgr2=0x557467fb4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 0x557467fb4ed0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fad44011510 tx=0x7fad440199a0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 -- 172.21.15.100:0/1981766157 shutdown_connections 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467fb5750 0x557467fafa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467facb70 0x557467fb3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 --2- 172.21.15.100:0/1981766157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fac250 0x557467fb4ed0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 -- 172.21.15.100:0/1981766157 >> 172.21.15.100:0/1981766157 conn(0x557467fa6980 msgr2=0x557467fada90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 -- 172.21.15.100:0/1981766157 shutdown_connections 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 -- 172.21.15.100:0/1981766157 wait complete. 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.908+0000 7fad5eb87740 1 -- start start 2024-03-12T05:23:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 Processor -- start 2024-03-12T05:23:14.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fbaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467facb70 0x557467fbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fbb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557467fbcf00 con 0x557467fb5750 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746801c110 con 0x557467facb70 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746801c2d0 con 0x557467fac250 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5eb87740 10 client.0 _ll_register_callbacks cb 0x7fad44009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fbb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fbb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49554/0 (socket says 172.21.15.100:49554) 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 -- 172.21.15.100:0/717579186 learned_addr learned my addr 172.21.15.100:0/717579186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad59551640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467facb70 0x557467fbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad59d52640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fbaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 -- 172.21.15.100:0/717579186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 msgr2=0x557467fbaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fbaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 -- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467facb70 msgr2=0x557467fbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467facb70 0x557467fbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.909+0000 7fad5a553640 1 -- 172.21.15.100:0/717579186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557467ed6060 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5a553640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fbb9e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fad4c014010 tx=0x7fad4c003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35d695ce5ee752e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad3d7fa640 1 -- 172.21.15.100:0/717579186 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c010a00 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad3d7fa640 1 -- 172.21.15.100:0/717579186 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad4c010b60 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad4400fc80 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad3d7fa640 1 -- 172.21.15.100:0/717579186 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad4c00f8e0 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 10 client.17835 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557467ed48c0 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad3d7fa640 1 -- 172.21.15.100:0/717579186 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad4400fc80 con 0x557467fb5750 2024-03-12T05:23:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad3d7fa640 1 -- 172.21.15.100:0/717579186 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fad4c004c50 con 0x557467fb5750 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 10 client.17835 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 10 client.17835 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 20 client.17835 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55746801da60 0x55746801fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 4 client.17835 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557467c9f890 con 0x55746801da60 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.910+0000 7fad59d52640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55746801da60 0x55746801fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad59d52640 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55746801da60 0x55746801fe80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c957a4dc0e58b server_cookie=5c25aef56b6d9895 in_seq=0 out_seq=0 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad3d7fa640 10 client.17835 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad3d7fa640 1 -- 172.21.15.100:0/717579186 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557467c9f890 con 0x55746801da60 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad3d7fa640 10 client.17835 handle_command_reply: tid=0 2024-03-12T05:23:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad5eb87740 1 client.17835 shutdown 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 msgr2=0x557467fbb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.911+0000 7fad5eb87740 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fbb9e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fad4c014010 tx=0x7fad4c003e40 comp rx=0 tx=0).stop 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 shutdown_connections 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55746801da60 0x55746801fe80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557467fb5750 0x557467fbb9e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad3d7fa640 0 client.17835 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557467facb70 0x557467fbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 --2- 172.21.15.100:0/717579186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557467fac250 0x557467fbaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 >> 172.21.15.100:0/717579186 conn(0x557467fa6980 msgr2=0x557467fa52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 shutdown_connections 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 1 -- 172.21.15.100:0/717579186 wait complete. 2024-03-12T05:23:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5eb87740 20 client.17835 trim_cache size 0 max 16384 2024-03-12T05:23:14.914 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:23:14.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad400a1ba0 msgr2=0x7fad400a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad400a1ba0 0x7fad400a4060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad5406a6b0 tx=0x7fad4c009aa0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 msgr2=0x7fad5419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5419fbb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fad50005940 tx=0x7fad5002c670 comp rx=0 tx=0).stop 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 shutdown_connections 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad400a1ba0 0x7fad400a4060 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad5410db20 0x7fad541997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54068c80 0x7fad5419fbb0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 --2- 172.21.15.100:0/2905938187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad54068360 0x7fad5419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 >> 172.21.15.100:0/2905938187 conn(0x7fad54077180 msgr2=0x7fad5410f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 shutdown_connections 2024-03-12T05:23:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:14.912+0000 7fad5bfb6640 1 -- 172.21.15.100:0/2905938187 wait complete. 2024-03-12T05:23:14.918 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:23:14.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:23:15.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:14 smithi100 ceph-mon[32210]: pgmap v347: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 268 op/s 2024-03-12T05:23:15.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:14 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2711738547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:23:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/2401485401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0068f80 msgr2=0x7f0bd0105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/2401485401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0068f80 0x7f0bd0105d50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009930 tx=0x7f0bc002bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/2401485401 shutdown_connections 2024-03-12T05:23:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/2401485401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd0106430 0x7f0bd010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/2401485401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0068f80 0x7f0bd0105d50 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/2401485401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 0x7f0bd00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/2401485401 >> 172.21.15.100:0/2401485401 conn(0x7f0bd0077180 msgr2=0x7f0bd0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/2401485401 shutdown_connections 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.071+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/2401485401 wait complete. 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 Processor -- start 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 -- start start 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 0x7f0bd019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 0x7f0bd019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd01975f0 0x7f0bd0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0197fe0 con 0x7f0bd0106430 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0198150 con 0x7f0bd00685d0 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd7a9a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0198310 con 0x7f0bd01975f0 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 0x7f0bd019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 0x7f0bd019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49570/0 (socket says 172.21.15.100:49570) 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5836640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 0x7f0bd019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 -- 172.21.15.100:0/1967064522 learned_addr learned my addr 172.21.15.100:0/1967064522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd6037640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd01975f0 0x7f0bd0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 -- 172.21.15.100:0/1967064522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd01975f0 msgr2=0x7f0bd0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd01975f0 0x7f0bd0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 -- 172.21.15.100:0/1967064522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 msgr2=0x7f0bd019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 0x7f0bd019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5035640 1 -- 172.21.15.100:0/1967064522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0009560 con 0x7f0bd0106430 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.072+0000 7f0bd5836640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 0x7f0bd019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.073+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 0x7f0bd019d980 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009230 tx=0x7f0bc002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.073+0000 7f0bc6ffd640 1 -- 172.21.15.100:0/1967064522 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0004350 con 0x7f0bd0106430 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.073+0000 7f0bc6ffd640 1 -- 172.21.15.100:0/1967064522 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc00044b0 con 0x7f0bd0106430 2024-03-12T05:23:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.073+0000 7f0bc6ffd640 1 -- 172.21.15.100:0/1967064522 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0003da0 con 0x7f0bd0106430 2024-03-12T05:23:15.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.073+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd01a4300 con 0x7f0bd0106430 2024-03-12T05:23:15.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.073+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd01a47b0 con 0x7f0bd0106430 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.075+0000 7f0bc6ffd640 1 -- 172.21.15.100:0/1967064522 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0bc0004df0 con 0x7f0bd0106430 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.075+0000 7f0bda66b740 1 Processor -- start 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 -- start start 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6250 0x5636a52d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52df750 0x5636a52e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a5291960 con 0x5636a52df750 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a5245310 con 0x5636a52d6250 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bda66b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a5242840 con 0x5636a52d6b70 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bc6ffd640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0bbc0a1ba0 0x7f0bbc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5035640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52df750 0x5636a52e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd6037640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6250 0x5636a52d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd6037640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6250 0x5636a52d6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45540/0 (socket says 172.21.15.100:45540) 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd6037640 1 -- 172.21.15.100:0/3613142587 learned_addr learned my addr 172.21.15.100:0/3613142587 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5836640 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd6037640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0bbc0a1ba0 0x7f0bbc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bc6ffd640 1 -- 172.21.15.100:0/1967064522 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0bc00ef790 con 0x7f0bd0106430 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5836640 1 -- 172.21.15.100:0/3613142587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6250 msgr2=0x5636a52d6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5836640 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6250 0x5636a52d6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5836640 1 -- 172.21.15.100:0/3613142587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52df750 msgr2=0x5636a52e1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5836640 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52df750 0x5636a52e1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.076+0000 7f0bd5836640 1 -- 172.21.15.100:0/3613142587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636a5200060 con 0x5636a52d6b70 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.077+0000 7f0bd6037640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0bbc0a1ba0 0x7f0bbc0a4060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80099e0 tx=0x7f0bc8009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.077+0000 7f0bd5836640 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52df210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009fd0 tx=0x7f0bcc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd15b80cc5af8ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.077+0000 7f0bbbfff640 1 -- 172.21.15.100:0/3613142587 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004230 con 0x5636a52d6b70 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.077+0000 7f0bbbfff640 1 -- 172.21.15.100:0/3613142587 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc004390 con 0x5636a52d6b70 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.077+0000 7f0bbbfff640 1 -- 172.21.15.100:0/3613142587 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004df0 con 0x5636a52d6b70 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 -- 172.21.15.100:0/3613142587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 msgr2=0x5636a52df210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52df210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009fd0 tx=0x7f0bcc011810 comp rx=0 tx=0).stop 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 -- 172.21.15.100:0/3613142587 shutdown_connections 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52df750 0x5636a52e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52df210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- 172.21.15.100:0/3613142587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6250 0x5636a52d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 -- 172.21.15.100:0/3613142587 >> 172.21.15.100:0/3613142587 conn(0x5636a52d0980 msgr2=0x5636a52d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 -- 172.21.15.100:0/3613142587 shutdown_connections 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 -- 172.21.15.100:0/3613142587 wait complete. 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 -- start start 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 Processor -- start 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.078+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a523e020 con 0x5636a52d6250 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a4f11430 con 0x5636a52df750 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a4ecde00 con 0x5636a52d6b70 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 10 client.0 _ll_register_callbacks cb 0x5636a51e5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52dc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49592/0 (socket says 172.21.15.100:49592) 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 -- 172.21.15.100:0/1067397565 learned_addr learned my addr 172.21.15.100:0/1067397565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd6037640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 -- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 msgr2=0x5636a52dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 -- 172.21.15.100:0/1067397565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 msgr2=0x5636a52dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 -- 172.21.15.100:0/1067397565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc009c00 con 0x5636a52d6250 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bd5836640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52dc670 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0049d0 tx=0x7f0bcc00fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446099693270f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bb97fa640 1 -- 172.21.15.100:0/1067397565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc019630 con 0x5636a52d6250 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bb97fa640 1 -- 172.21.15.100:0/1067397565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0bcc019790 con 0x5636a52d6250 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636a4f3d7e0 con 0x5636a52d6250 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bb97fa640 1 -- 172.21.15.100:0/1067397565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bcc00fec0 con 0x5636a52d6250 2024-03-12T05:23:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 10 client.17847 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.079+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5636a523c6a0 con 0x5636a52d6250 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bb97fa640 1 -- 172.21.15.100:0/1067397565 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5636a4f3d7e0 con 0x5636a52d6250 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bb97fa640 1 -- 172.21.15.100:0/1067397565 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bcc024b60 con 0x5636a52d6250 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bda66b740 10 client.17847 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bda66b740 10 client.17847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bda66b740 20 client.17847 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a52efa30 0x5636a52f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bda66b740 4 client.17847 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636a4f7a720 con 0x5636a52efa30 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a52efa30 0x5636a52f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bb97fa640 10 client.17847 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.080+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a52efa30 0x5636a52f1e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ca6e44a0f123735 server_cookie=ea28f448a4db78c8 in_seq=0 out_seq=0 2024-03-12T05:23:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bb97fa640 1 -- 172.21.15.100:0/1067397565 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636a4f7a720 con 0x5636a52efa30 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bb97fa640 10 client.17847 handle_command_reply: tid=0 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 client.17847 shutdown 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 msgr2=0x5636a52dc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52dc670 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0049d0 tx=0x7f0bcc00fcd0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 shutdown_connections 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a52efa30 0x5636a52f1e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6b70 0x5636a52dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bb97fa640 0 client.17847 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1067397565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52dc670 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 >> 172.21.15.100:0/1067397565 conn(0x5636a52d0980 msgr2=0x5636a52d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 shutdown_connections 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 1 -- 172.21.15.100:0/1067397565 wait complete. 2024-03-12T05:23:15.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.081+0000 7f0bda66b740 20 client.17847 trim_cache size 0 max 16384 2024-03-12T05:23:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 Processor -- start 2024-03-12T05:23:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 -- start start 2024-03-12T05:23:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6250 0x5636a52deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 0x5636a52db650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a51c3130 con 0x5636a52d6b70 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a52d1980 con 0x5636a52df750 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bda66b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a52d07b0 con 0x5636a52d6250 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 0x5636a52db650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 0x5636a52db650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49600/0 (socket says 172.21.15.100:49600) 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 -- 172.21.15.100:0/1002413165 learned_addr learned my addr 172.21.15.100:0/1002413165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd6037640 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5836640 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6250 0x5636a52deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 -- 172.21.15.100:0/1002413165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6250 msgr2=0x5636a52deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6250 0x5636a52deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 -- 172.21.15.100:0/1002413165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 msgr2=0x5636a52dcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.094+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.095+0000 7f0bd5035640 1 -- 172.21.15.100:0/1002413165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636a51d8c20 con 0x5636a52d6b70 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.095+0000 7f0bd5836640 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6250 0x5636a52deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.095+0000 7f0bd5035640 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 0x5636a52db650 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00ed010 tx=0x7f0bc008f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c8d586db57481 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.095+0000 7f0bbb7fe640 1 -- 172.21.15.100:0/1002413165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00bbdf0 con 0x5636a52d6b70 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.095+0000 7f0bbb7fe640 1 -- 172.21.15.100:0/1002413165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc00dc0b0 con 0x5636a52d6b70 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.095+0000 7f0bbb7fe640 1 -- 172.21.15.100:0/1002413165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00bc9c0 con 0x5636a52d6b70 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- 172.21.15.100:0/1002413165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 msgr2=0x5636a52db650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 0x5636a52db650 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00ed010 tx=0x7f0bc008f740 comp rx=0 tx=0).stop 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- 172.21.15.100:0/1002413165 shutdown_connections 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52df750 0x5636a52dcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6b70 0x5636a52db650 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- 172.21.15.100:0/1002413165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52d6250 0x5636a52deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- 172.21.15.100:0/1002413165 >> 172.21.15.100:0/1002413165 conn(0x5636a52d0980 msgr2=0x5636a52daf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- 172.21.15.100:0/1002413165 shutdown_connections 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- 172.21.15.100:0/1002413165 wait complete. 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- start start 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 Processor -- start 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6b70 0x5636a52e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52df750 0x5636a52e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a5345f40 con 0x5636a52d6250 2024-03-12T05:23:15.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a5346080 con 0x5636a52d6b70 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a53461c0 con 0x5636a52df750 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.096+0000 7f0bda66b740 10 client.0 _ll_register_callbacks cb 0x7f0bcc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52e5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49614/0 (socket says 172.21.15.100:49614) 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 -- 172.21.15.100:0/527318995 learned_addr learned my addr 172.21.15.100:0/527318995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5035640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6b70 0x5636a52e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd6037640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52df750 0x5636a52e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 -- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52df750 msgr2=0x5636a52e5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52df750 0x5636a52e5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 -- 172.21.15.100:0/527318995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6b70 msgr2=0x5636a52e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6b70 0x5636a52e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 -- 172.21.15.100:0/527318995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd01a47b0 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5035640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6b70 0x5636a52e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bd5836640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52e5000 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0058c0 tx=0x7f0bcc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95ba09ff7b2202a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bb97fa640 1 -- 172.21.15.100:0/527318995 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc018710 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bb97fa640 1 -- 172.21.15.100:0/527318995 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0bcc00f850 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bcc004390 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bb97fa640 1 -- 172.21.15.100:0/527318995 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bcc018870 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bda66b740 10 client.17853 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.097+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5636a51fe8c0 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bb97fa640 1 -- 172.21.15.100:0/527318995 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc004390 con 0x5636a52d6250 2024-03-12T05:23:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bb97fa640 1 -- 172.21.15.100:0/527318995 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bcc024ad0 con 0x5636a52d6250 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bda66b740 10 client.17853 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bda66b740 10 client.17853 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bda66b740 20 client.17853 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bda66b740 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a53475f0 0x5636a5349a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bda66b740 4 client.17853 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636a4fc9890 con 0x5636a53475f0 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bd5035640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a53475f0 0x5636a5349a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bb97fa640 10 client.17853 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.098+0000 7f0bd5035640 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a53475f0 0x5636a5349a10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=826653828e523c86 server_cookie=d58c38147b3e1ef6 in_seq=0 out_seq=0 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bb97fa640 1 -- 172.21.15.100:0/527318995 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636a4fc9890 con 0x5636a53475f0 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bb97fa640 10 client.17853 handle_command_reply: tid=0 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 client.17853 shutdown 2024-03-12T05:23:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 msgr2=0x5636a52e5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52e5000 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0058c0 tx=0x7f0bcc005d30 comp rx=0 tx=0).stop 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 shutdown_connections 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636a52df750 0x5636a52e5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5636a53475f0 0x5636a5349a10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636a52d6b70 0x5636a52e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 --2- 172.21.15.100:0/527318995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636a52d6250 0x5636a52e5000 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bb97fa640 0 client.17853 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 >> 172.21.15.100:0/527318995 conn(0x5636a52d0980 msgr2=0x5636a52cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 shutdown_connections 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 1 -- 172.21.15.100:0/527318995 wait complete. 2024-03-12T05:23:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bda66b740 20 client.17853 trim_cache size 0 max 16384 2024-03-12T05:23:15.102 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0bbc0a1ba0 msgr2=0x7f0bbc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0bbc0a1ba0 0x7f0bbc0a4060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80099e0 tx=0x7f0bc8009500 comp rx=0 tx=0).stop 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 msgr2=0x7f0bd019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 0x7f0bd019d980 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009230 tx=0x7f0bc002c680 comp rx=0 tx=0).stop 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 shutdown_connections 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0bbc0a1ba0 0x7f0bbc0a4060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd01975f0 0x7f0bd0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd0106430 0x7f0bd019d980 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 --2- 172.21.15.100:0/1967064522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd00685d0 0x7f0bd019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.099+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 >> 172.21.15.100:0/1967064522 conn(0x7f0bd0077180 msgr2=0x7f0bd00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.100+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 shutdown_connections 2024-03-12T05:23:15.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.100+0000 7f0bd7a9a640 1 -- 172.21.15.100:0/1967064522 wait complete. 2024-03-12T05:23:15.105 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:23:15.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:23:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:14 smithi156 ceph-mon[36270]: pgmap v347: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 268 op/s 2024-03-12T05:23:15.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:14 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2711738547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.218+0000 7fd269d84640 1 -- 172.21.15.100:0/46155986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264100c10 msgr2=0x7fd264104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.218+0000 7fd269d84640 1 --2- 172.21.15.100:0/46155986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264100c10 0x7fd264104c50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd258009930 tx=0x7fd25802bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 -- 172.21.15.100:0/46155986 shutdown_connections 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 --2- 172.21.15.100:0/46155986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd264105330 0x7fd26410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 --2- 172.21.15.100:0/46155986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264100c10 0x7fd264104c50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 --2- 172.21.15.100:0/46155986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd264100260 0x7fd264100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 -- 172.21.15.100:0/46155986 >> 172.21.15.100:0/46155986 conn(0x7fd264077180 msgr2=0x7fd264077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 -- 172.21.15.100:0/46155986 shutdown_connections 2024-03-12T05:23:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 -- 172.21.15.100:0/46155986 wait complete. 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 Processor -- start 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 -- start start 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.219+0000 7fd269d84640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd264100260 0x7fd264101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 0x7fd264102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd264102770 0x7fd2641a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641a5fc0 con 0x7fd264105330 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641a6130 con 0x7fd264100260 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641a62f0 con 0x7fd264102770 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 0x7fd264102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 0x7fd264102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49636/0 (socket says 172.21.15.100:49636) 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 -- 172.21.15.100:0/765254987 learned_addr learned my addr 172.21.15.100:0/765254987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd263fff640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd264102770 0x7fd2641a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd2637fe640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd264100260 0x7fd264101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 -- 172.21.15.100:0/765254987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd264102770 msgr2=0x7fd2641a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd264102770 0x7fd2641a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 -- 172.21.15.100:0/765254987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd264100260 msgr2=0x7fd264101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd264100260 0x7fd264101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 -- 172.21.15.100:0/765254987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258009560 con 0x7fd264105330 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd262ffd640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 0x7fd264102230 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd258009230 tx=0x7fd25802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd260ff9640 1 -- 172.21.15.100:0/765254987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258004350 con 0x7fd264105330 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd260ff9640 1 -- 172.21.15.100:0/765254987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2580044b0 con 0x7fd264105330 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd260ff9640 1 -- 172.21.15.100:0/765254987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258003da0 con 0x7fd264105330 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2641a6570 con 0x7fd264105330 2024-03-12T05:23:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.220+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2641a6a80 con 0x7fd264105330 2024-03-12T05:23:15.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:14 smithi161 ceph-mon[36043]: pgmap v347: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 268 op/s 2024-03-12T05:23:15.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:14 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2711738547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.223+0000 7fd26ca2a740 1 Processor -- start 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.223+0000 7fd26ca2a740 1 -- start start 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.223+0000 7fd260ff9640 1 -- 172.21.15.100:0/765254987 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd258004df0 con 0x7fd264105330 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518db6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518d66960 con 0x558518db4750 2024-03-12T05:23:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518d1a310 con 0x558518dabb70 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518d17840 con 0x558518dab250 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd262ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518db6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45606/0 (socket says 172.21.15.100:45606) 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd262ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518db6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49642/0 (socket says 172.21.15.100:49642) 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 -- 172.21.15.100:0/3353387827 learned_addr learned my addr 172.21.15.100:0/3353387827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd263fff640 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 -- 172.21.15.100:0/3353387827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 msgr2=0x558518dab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 -- 172.21.15.100:0/3353387827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 msgr2=0x558518db6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518db6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 -- 172.21.15.100:0/3353387827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558518cd5060 con 0x558518dabb70 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd260ff9640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd24c0a1bd0 0x7fd24c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd263fff640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd24c0a1bd0 0x7fd24c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd2637fe640 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db4210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd254009cf0 tx=0x7fd254011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6cd3ab1bc769f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd260ff9640 1 -- 172.21.15.100:0/765254987 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd2580ef7d0 con 0x7fd264105330 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd249ffb640 1 -- 172.21.15.100:0/3353387827 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254009ea0 con 0x558518dabb70 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd249ffb640 1 -- 172.21.15.100:0/3353387827 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd254004230 con 0x558518dabb70 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.224+0000 7fd249ffb640 1 -- 172.21.15.100:0/3353387827 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254019880 con 0x558518dabb70 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3353387827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 msgr2=0x558518db4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd263fff640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd24c0a1bd0 0x7fd24c0a4090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005fd0 tx=0x7fd25c009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db4210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd254009cf0 tx=0x7fd254011810 comp rx=0 tx=0).stop 2024-03-12T05:23:15.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3353387827 shutdown_connections 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518db6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db4210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3353387827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3353387827 >> 172.21.15.100:0/3353387827 conn(0x558518da5980 msgr2=0x558518da5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3353387827 shutdown_connections 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3353387827 wait complete. 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.225+0000 7fd26ca2a740 1 -- start start 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 Processor -- start 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 0x558518db1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518db4750 0x558518db21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518d13020 con 0x558518dab250 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585189e6430 con 0x558518dabb70 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585189a2e00 con 0x558518db4750 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd26ca2a740 10 client.0 _ll_register_callbacks cb 0x558518cbacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 0x558518db1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 0x558518db1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49652/0 (socket says 172.21.15.100:49652) 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 -- 172.21.15.100:0/3073732641 learned_addr learned my addr 172.21.15.100:0/3073732641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd263fff640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518db4750 0x558518db21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd262ffd640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 -- 172.21.15.100:0/3073732641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518db4750 msgr2=0x558518db21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518db4750 0x558518db21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 -- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 msgr2=0x558518db1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.226+0000 7fd2637fe640 1 -- 172.21.15.100:0/3073732641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd254009950 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd2637fe640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 0x558518db1740 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd254005df0 tx=0x7fd254009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1195552effc20046 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd242ffd640 1 -- 172.21.15.100:0/3073732641 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254005040 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd242ffd640 1 -- 172.21.15.100:0/3073732641 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd2540051a0 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558518a127e0 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd242ffd640 1 -- 172.21.15.100:0/3073732641 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd254005360 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 10 client.17865 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558518d116a0 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd242ffd640 1 -- 172.21.15.100:0/3073732641 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558518a127e0 con 0x558518dab250 2024-03-12T05:23:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd242ffd640 1 -- 172.21.15.100:0/3073732641 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd254021040 con 0x558518dab250 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 10 client.17865 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 20 client.17865 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518dc4af0 0x558518dc6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 4 client.17865 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.227+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558518a4f720 con 0x558518dc4af0 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.228+0000 7fd262ffd640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518dc4af0 0x558518dc6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.228+0000 7fd242ffd640 10 client.17865 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.228+0000 7fd262ffd640 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518dc4af0 0x558518dc6ee0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffd3539f16ec54a server_cookie=695d2ca4a2c8c95f in_seq=0 out_seq=0 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd242ffd640 1 -- 172.21.15.100:0/3073732641 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558518a4f720 con 0x558518dc4af0 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd242ffd640 10 client.17865 handle_command_reply: tid=0 2024-03-12T05:23:15.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 client.17865 shutdown 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 msgr2=0x558518db1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 0x558518db1740 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd254005df0 tx=0x7fd254009640 comp rx=0 tx=0).stop 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 shutdown_connections 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518dc4af0 0x558518dc6ee0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518db4750 0x558518db21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518db1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd242ffd640 0 client.17865 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/3073732641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dab250 0x558518db1740 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 >> 172.21.15.100:0/3073732641 conn(0x558518da5980 msgr2=0x558518da76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 shutdown_connections 2024-03-12T05:23:15.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 1 -- 172.21.15.100:0/3073732641 wait complete. 2024-03-12T05:23:15.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.229+0000 7fd26ca2a740 20 client.17865 trim_cache size 0 max 16384 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.244+0000 7fd26ca2a740 1 Processor -- start 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.244+0000 7fd26ca2a740 1 -- start start 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518db1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dabb70 0x558518da4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518db4750 0x558518dbb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518c98130 con 0x558518dabb70 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518da6980 con 0x558518db4750 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518dcc130 con 0x558518dab250 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518db1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518db1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50810/0 (socket says 172.21.15.100:50810) 2024-03-12T05:23:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 -- 172.21.15.100:0/1057802097 learned_addr learned my addr 172.21.15.100:0/1057802097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd262ffd640 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dabb70 0x558518da4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 -- 172.21.15.100:0/1057802097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518db4750 msgr2=0x558518dbb600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518db4750 0x558518dbb600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 -- 172.21.15.100:0/1057802097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dabb70 msgr2=0x558518da4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dabb70 0x558518da4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.245+0000 7fd2637fe640 1 -- 172.21.15.100:0/1057802097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558518cadc20 con 0x558518dab250 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.246+0000 7fd2637fe640 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518db1fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd254009730 tx=0x7fd254005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c2b9bc7755489e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.246+0000 7fd2497fa640 1 -- 172.21.15.100:0/1057802097 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254003990 con 0x558518dab250 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.246+0000 7fd2497fa640 1 -- 172.21.15.100:0/1057802097 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd254019be0 con 0x558518dab250 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.246+0000 7fd2497fa640 1 -- 172.21.15.100:0/1057802097 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254019490 con 0x558518dab250 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 -- 172.21.15.100:0/1057802097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 msgr2=0x558518db1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518db1fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd254009730 tx=0x7fd254005b90 comp rx=0 tx=0).stop 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 -- 172.21.15.100:0/1057802097 shutdown_connections 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518db4750 0x558518dbb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518dabb70 0x558518da4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/1057802097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518db1fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 -- 172.21.15.100:0/1057802097 >> 172.21.15.100:0/1057802097 conn(0x558518da5980 msgr2=0x558518da42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 -- 172.21.15.100:0/1057802097 shutdown_connections 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.247+0000 7fd26ca2a740 1 -- 172.21.15.100:0/1057802097 wait complete. 2024-03-12T05:23:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.248+0000 7fd26ca2a740 1 -- start start 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.248+0000 7fd26ca2a740 1 Processor -- start 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.249+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.249+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518dba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd26ca2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518dbaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518e1b010 con 0x558518db4750 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd26ca2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518e1b150 con 0x558518dabb70 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd262ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518dba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd263fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518dbaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd262ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518dba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45618/0 (socket says 172.21.15.100:45618) 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd262ffd640 1 -- 172.21.15.100:0/4269598987 learned_addr learned my addr 172.21.15.100:0/4269598987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558518e1b290 con 0x558518dab250 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd26ca2a740 10 client.0 _ll_register_callbacks cb 0x7fd2540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd2637fe640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd262ffd640 1 -- 172.21.15.100:0/4269598987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 msgr2=0x558518dba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.250+0000 7fd262ffd640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd262ffd640 1 -- 172.21.15.100:0/4269598987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 msgr2=0x558518dbaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd262ffd640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518dbaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd262ffd640 1 -- 172.21.15.100:0/4269598987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558518d116a0 con 0x558518dabb70 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd263fff640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518dbaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd262ffd640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518dba570 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd258005ec0 tx=0x7fd2580b8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a50f9d98b56a10 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:15.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:23:15.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd242ffd640 1 -- 172.21.15.100:0/4269598987 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580bcbd0 con 0x558518dabb70 2024-03-12T05:23:15.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd242ffd640 1 -- 172.21.15.100:0/4269598987 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd2580bc4d0 con 0x558518dabb70 2024-03-12T05:23:15.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.251+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd254004230 con 0x558518dabb70 2024-03-12T05:23:15.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.252+0000 7fd242ffd640 1 -- 172.21.15.100:0/4269598987 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2580c0620 con 0x558518dabb70 2024-03-12T05:23:15.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.252+0000 7fd26ca2a740 10 client.27248 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:15.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.252+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558518cd38c0 con 0x558518dabb70 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd242ffd640 1 -- 172.21.15.100:0/4269598987 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2580c4c70 con 0x558518dabb70 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd242ffd640 1 -- 172.21.15.100:0/4269598987 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2580f7040 con 0x558518dabb70 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd26ca2a740 10 client.27248 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd26ca2a740 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd26ca2a740 20 client.27248 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518e1c820 0x558518e1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd26ca2a740 4 client.27248 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd254004230 con 0x558518e1c820 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.253+0000 7fd2637fe640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518e1c820 0x558518e1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd242ffd640 10 client.27248 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd2637fe640 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518e1c820 0x558518e1ec40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b07ba50c49d28d server_cookie=84f920866baa4cac in_seq=0 out_seq=0 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd242ffd640 1 -- 172.21.15.100:0/4269598987 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd254004230 con 0x558518e1c820 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd242ffd640 10 client.27248 handle_command_reply: tid=0 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 client.27248 shutdown 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 msgr2=0x558518dba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518dba570 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd258005ec0 tx=0x7fd2580b8040 comp rx=0 tx=0).stop 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 shutdown_connections 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558518e1c820 0x558518e1ec40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558518db4750 0x558518dbaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558518dabb70 0x558518dba570 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 --2- 172.21.15.100:0/4269598987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558518dab250 0x558518dba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd242ffd640 0 client.27248 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 >> 172.21.15.100:0/4269598987 conn(0x558518da5980 msgr2=0x558518da4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 shutdown_connections 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 1 -- 172.21.15.100:0/4269598987 wait complete. 2024-03-12T05:23:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.254+0000 7fd26ca2a740 20 client.27248 trim_cache size 0 max 16384 2024-03-12T05:23:15.254 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd24c0a1bd0 msgr2=0x7fd24c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd24c0a1bd0 0x7fd24c0a4090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005fd0 tx=0x7fd25c009500 comp rx=0 tx=0).stop 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 msgr2=0x7fd264102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 0x7fd264102230 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd258009230 tx=0x7fd25802c680 comp rx=0 tx=0).stop 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 shutdown_connections 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd24c0a1bd0 0x7fd24c0a4090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd264102770 0x7fd2641a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd264105330 0x7fd264102230 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 --2- 172.21.15.100:0/765254987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd264100260 0x7fd264101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 >> 172.21.15.100:0/765254987 conn(0x7fd264077180 msgr2=0x7fd2640686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 shutdown_connections 2024-03-12T05:23:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:15.255+0000 7fd269d84640 1 -- 172.21.15.100:0/765254987 wait complete. 2024-03-12T05:23:15.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1347299 11.77 MB/sec execute 479 sec latency 26.616 ms 2024-03-12T05:23:16.261 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:23:16.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:23:16.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 -- 172.21.15.100:0/3826188537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44106450 msgr2=0x7f2f4410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 --2- 172.21.15.100:0/3826188537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44106450 0x7f2f4410e020 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009cd0 tx=0x7f2f3c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 -- 172.21.15.100:0/3826188537 shutdown_connections 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 --2- 172.21.15.100:0/3826188537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44106450 0x7f2f4410e020 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 --2- 172.21.15.100:0/3826188537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44100240 0x7f2f44105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 --2- 172.21.15.100:0/3826188537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f440ff890 0x7f2f440ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.396+0000 7f2f48c64640 1 -- 172.21.15.100:0/3826188537 >> 172.21.15.100:0/3826188537 conn(0x7f2f440765a0 msgr2=0x7f2f440fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.397+0000 7f2f48c64640 1 -- 172.21.15.100:0/3826188537 shutdown_connections 2024-03-12T05:23:16.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.397+0000 7f2f48c64640 1 -- 172.21.15.100:0/3826188537 wait complete. 2024-03-12T05:23:16.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.397+0000 7f2f48c64640 1 Processor -- start 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 -- start start 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f440ff890 0x7f2f44100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 0x7f2f44100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44101280 0x7f2f441a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441a5f60 con 0x7f2f44101280 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441a60d0 con 0x7f2f440ff890 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f48c64640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441a6290 con 0x7f2f44106450 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 0x7f2f44100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f42d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44101280 0x7f2f441a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 0x7f2f44100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50830/0 (socket says 172.21.15.100:50830) 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f39d9b640 1 -- 172.21.15.100:0/4176314499 learned_addr learned my addr 172.21.15.100:0/4176314499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f4259c640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f440ff890 0x7f2f44100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f42d9d640 1 -- 172.21.15.100:0/4176314499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 msgr2=0x7f2f44100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f42d9d640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 0x7f2f44100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f42d9d640 1 -- 172.21.15.100:0/4176314499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f440ff890 msgr2=0x7f2f44100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f42d9d640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f440ff890 0x7f2f44100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.398+0000 7f2f42d9d640 1 -- 172.21.15.100:0/4176314499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c009900 con 0x7f2f44101280 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 0x7f2f44100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f42d9d640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44101280 0x7f2f441a5990 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c02c630 tx=0x7f2f3c034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f3bfff640 1 -- 172.21.15.100:0/4176314499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c032ac0 con 0x7f2f44101280 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f3bfff640 1 -- 172.21.15.100:0/4176314499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f3c032eb0 con 0x7f2f44101280 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f3bfff640 1 -- 172.21.15.100:0/4176314499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c040450 con 0x7f2f44101280 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f441a6510 con 0x7f2f44101280 2024-03-12T05:23:16.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.399+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f441a69e0 con 0x7f2f44101280 2024-03-12T05:23:16.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.402+0000 7f2f3bfff640 1 -- 172.21.15.100:0/4176314499 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2f3c00a040 con 0x7f2f44101280 2024-03-12T05:23:16.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.403+0000 7f2f4b834740 1 Processor -- start 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 -- start start 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 0x5596b3f24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3e87f90 con 0x5596b3f228b0 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3e854c0 con 0x5596b3f19380 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f4b834740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3e80ca0 con 0x5596b3f19d30 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.404+0000 7f2f3bfff640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f300a1c50 0x7f2f300a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f42d9d640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f300a1c50 0x7f2f300a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f3bfff640 1 -- 172.21.15.100:0/4176314499 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2f3c0f0440 con 0x7f2f44101280 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 0x5596b3f24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 0x5596b3f24d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49686/0 (socket says 172.21.15.100:49686) 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 -- 172.21.15.100:0/3263464239 learned_addr learned my addr 172.21.15.100:0/3263464239 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f4259c640 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 -- 172.21.15.100:0/3263464239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 msgr2=0x5596b3f222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 -- 172.21.15.100:0/3263464239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 msgr2=0x5596b3f19760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f19760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.405+0000 7f2f39d9b640 1 -- 172.21.15.100:0/3263464239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596b3e430b0 con 0x5596b3f228b0 2024-03-12T05:23:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.406+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 0x5596b3f24d00 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005e00 tx=0x7f2f2c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e7450ea131b20b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.406+0000 7f2f38d99640 1 -- 172.21.15.100:0/3263464239 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010c00 con 0x5596b3f228b0 2024-03-12T05:23:16.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.406+0000 7f2f38d99640 1 -- 172.21.15.100:0/3263464239 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f2c004510 con 0x5596b3f228b0 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 -- 172.21.15.100:0/3263464239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 msgr2=0x5596b3f24d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 0x5596b3f24d00 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005e00 tx=0x7f2f2c005d90 comp rx=0 tx=0).stop 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f42d9d640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f300a1c50 0x7f2f300a4110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0edd10 tx=0x7f2f3c0b8dc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 -- 172.21.15.100:0/3263464239 shutdown_connections 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f228b0 0x5596b3f24d00 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 --2- 172.21.15.100:0/3263464239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.407+0000 7f2f4b834740 1 -- 172.21.15.100:0/3263464239 >> 172.21.15.100:0/3263464239 conn(0x5596b3f13840 msgr2=0x5596b3f13c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.408+0000 7f2f4b834740 1 -- 172.21.15.100:0/3263464239 shutdown_connections 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.408+0000 7f2f4b834740 1 -- 172.21.15.100:0/3263464239 wait complete. 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.408+0000 7f2f4b834740 1 -- start start 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 Processor -- start 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f1fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f228b0 0x5596b3f201f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3e25f40 con 0x5596b3f19d30 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3b544b0 con 0x5596b3f228b0 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3e06670 con 0x5596b3f19380 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4b834740 10 client.0 _ll_register_callbacks cb 0x5596b3e73f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:16.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50858/0 (socket says 172.21.15.100:50858) 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 -- 172.21.15.100:0/1582097039 learned_addr learned my addr 172.21.15.100:0/1582097039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f228b0 0x5596b3f201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4259c640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f1fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 -- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f228b0 msgr2=0x5596b3f201f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f228b0 0x5596b3f201f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 -- 172.21.15.100:0/1582097039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 msgr2=0x5596b3f1fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f1fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f39d9b640 1 -- 172.21.15.100:0/1582097039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c012a20 con 0x5596b3f19380 2024-03-12T05:23:16.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.409+0000 7f2f4259c640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f1fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1f770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005fd0 tx=0x7f2f2c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d107ad055d8d4a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f29ffb640 1 -- 172.21.15.100:0/1582097039 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010bd0 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f29ffb640 1 -- 172.21.15.100:0/1582097039 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2f2c010d30 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596b3e5af40 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f29ffb640 1 -- 172.21.15.100:0/1582097039 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2c004050 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f4b834740 10 client.27220 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5596b3e7f320 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.410+0000 7f2f29ffb640 1 -- 172.21.15.100:0/1582097039 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5596b3e5af40 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f29ffb640 1 -- 172.21.15.100:0/1582097039 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f2c0035f0 con 0x5596b3f19380 2024-03-12T05:23:16.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4b834740 10 client.27220 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4b834740 10 client.27220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4b834740 20 client.27220 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4b834740 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f32b00 0x5596b3f34f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4b834740 4 client.27220 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596b3e77310 con 0x5596b3f32b00 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4259c640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f32b00 0x5596b3f34f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f29ffb640 10 client.27220 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.411+0000 7f2f4259c640 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f32b00 0x5596b3f34f20 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b327137ec8628cd server_cookie=a3ee4f9955a10291 in_seq=0 out_seq=0 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f29ffb640 1 -- 172.21.15.100:0/1582097039 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596b3e77310 con 0x5596b3f32b00 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f29ffb640 10 client.27220 handle_command_reply: tid=0 2024-03-12T05:23:16.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 client.27220 shutdown 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 msgr2=0x5596b3f1f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1f770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005fd0 tx=0x7f2f2c014040 comp rx=0 tx=0).stop 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 shutdown_connections 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f32b00 0x5596b3f34f20 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f228b0 0x5596b3f201f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f1fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 --2- 172.21.15.100:0/1582097039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1f770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f29ffb640 0 client.27220 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 >> 172.21.15.100:0/1582097039 conn(0x5596b3f13840 msgr2=0x5596b3f157a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 shutdown_connections 2024-03-12T05:23:16.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.412+0000 7f2f4b834740 1 -- 172.21.15.100:0/1582097039 wait complete. 2024-03-12T05:23:16.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.413+0000 7f2f4b834740 20 client.27220 trim_cache size 0 max 16384 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 Processor -- start 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 -- start start 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f13840 0x5596b3f21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3f25500 con 0x5596b3f19d30 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3f250c0 con 0x5596b3f13840 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f4b834740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3f19a90 con 0x5596b3f19380 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.428+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f20010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49696/0 (socket says 172.21.15.100:49696) 2024-03-12T05:23:16.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 -- 172.21.15.100:0/403085340 learned_addr learned my addr 172.21.15.100:0/403085340 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f4259c640 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f13840 0x5596b3f21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 -- 172.21.15.100:0/403085340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 msgr2=0x5596b3f1e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 -- 172.21.15.100:0/403085340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f13840 msgr2=0x5596b3f21ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f13840 0x5596b3f21ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 -- 172.21.15.100:0/403085340 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596b3e1bc00 con 0x5596b3f19d30 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f4259c640 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f20010 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00b4e0 tx=0x7f2f2c012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fa79874e9af829 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f2bfff640 1 -- 172.21.15.100:0/403085340 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00bb90 con 0x5596b3f19d30 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f2bfff640 1 -- 172.21.15.100:0/403085340 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f2c003ca0 con 0x5596b3f19d30 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.429+0000 7f2f2bfff640 1 -- 172.21.15.100:0/403085340 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010bd0 con 0x5596b3f19d30 2024-03-12T05:23:16.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 -- 172.21.15.100:0/403085340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 msgr2=0x5596b3f20010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f20010 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00b4e0 tx=0x7f2f2c012f40 comp rx=0 tx=0).stop 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 -- 172.21.15.100:0/403085340 shutdown_connections 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f19d30 0x5596b3f20010 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19380 0x5596b3f1e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 --2- 172.21.15.100:0/403085340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f13840 0x5596b3f21ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 -- 172.21.15.100:0/403085340 >> 172.21.15.100:0/403085340 conn(0x5596b3f1b140 msgr2=0x5596b3f1e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 -- 172.21.15.100:0/403085340 shutdown_connections 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 -- 172.21.15.100:0/403085340 wait complete. 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 -- start start 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.430+0000 7f2f4b834740 1 Processor -- start 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 0x5596b3f298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3f286f0 con 0x5596b3f13840 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3f28860 con 0x5596b3f19380 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b3f28a20 con 0x5596b3f19d30 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 10 client.0 _ll_register_callbacks cb 0x7f2f2c00bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 0x5596b3f298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 0x5596b3f298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49712/0 (socket says 172.21.15.100:49712) 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 -- 172.21.15.100:0/864032789 learned_addr learned my addr 172.21.15.100:0/864032789 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4259c640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 -- 172.21.15.100:0/864032789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 msgr2=0x5596b3f281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f281b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 -- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 msgr2=0x5596b3f27c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f27c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 -- 172.21.15.100:0/864032789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3400d960 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f39d9b640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 0x5596b3f298d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00b6b0 tx=0x7f2f2c003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b6ecfe9a2a1442 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f29ffb640 1 -- 172.21.15.100:0/864032789 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c012680 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f29ffb640 1 -- 172.21.15.100:0/864032789 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2f2c000e40 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596b3ea6900 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f29ffb640 1 -- 172.21.15.100:0/864032789 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2c010bd0 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 10 client.17889 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.431+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5596b3e7f320 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f29ffb640 1 -- 172.21.15.100:0/864032789 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5596b3ea6900 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f29ffb640 1 -- 172.21.15.100:0/864032789 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f2c003f30 con 0x5596b3f13840 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4b834740 10 client.17889 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4b834740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4b834740 20 client.17889 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4b834740 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f228b0 0x5596b3f8cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4b834740 4 client.17889 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:23:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5596b3b80860 con 0x5596b3f228b0 2024-03-12T05:23:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4259c640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f228b0 0x5596b3f8cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f4259c640 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f228b0 0x5596b3f8cdd0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3634d321b35028bb server_cookie=1797efdf11aacd50 in_seq=0 out_seq=0 2024-03-12T05:23:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.432+0000 7f2f29ffb640 10 client.17889 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.433+0000 7f2f29ffb640 1 -- 172.21.15.100:0/864032789 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5596b3b80860 con 0x5596b3f228b0 2024-03-12T05:23:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.433+0000 7f2f29ffb640 10 client.17889 handle_command_reply: tid=0 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.433+0000 7f2f4b834740 1 client.17889 shutdown 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 msgr2=0x5596b3f298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 0x5596b3f298d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00b6b0 tx=0x7f2f2c003690 comp rx=0 tx=0).stop 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 shutdown_connections 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5596b3f228b0 0x5596b3f8cdd0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596b3f19d30 0x5596b3f281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f29ffb640 0 client.17889 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596b3f19380 0x5596b3f27c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 --2- 172.21.15.100:0/864032789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596b3f13840 0x5596b3f298d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 >> 172.21.15.100:0/864032789 conn(0x5596b3f1b140 msgr2=0x5596b3f12050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 shutdown_connections 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 1 -- 172.21.15.100:0/864032789 wait complete. 2024-03-12T05:23:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f4b834740 20 client.17889 trim_cache size 0 max 16384 2024-03-12T05:23:16.436 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:23:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:23:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c9f96c4c-0407-4eea-abac-2fed25aa0e55", 2024-03-12T05:23:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:23:16.436 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f300a1c50 msgr2=0x7f2f300a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f48c64640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f300a1c50 0x7f2f300a4110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0edd10 tx=0x7f2f3c0b8dc0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44101280 msgr2=0x7f2f441a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.434+0000 7f2f48c64640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44101280 0x7f2f441a5990 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c02c630 tx=0x7f2f3c034d40 comp rx=0 tx=0).stop 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 shutdown_connections 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2f300a1c50 0x7f2f300a4110 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f44101280 0x7f2f441a5990 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f44106450 0x7f2f44100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 --2- 172.21.15.100:0/4176314499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f440ff890 0x7f2f44100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 >> 172.21.15.100:0/4176314499 conn(0x7f2f440765a0 msgr2=0x7f2f441046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 shutdown_connections 2024-03-12T05:23:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.435+0000 7f2f48c64640 1 -- 172.21.15.100:0/4176314499 wait complete. 2024-03-12T05:23:16.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:23:16.440 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:23:16.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 -- 172.21.15.100:0/1599319176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f01023a0 msgr2=0x7f83f0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1599319176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f01023a0 0x7f83f0102780 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009ca0 tx=0x7f83e002c180 comp rx=0 tx=0).stop 2024-03-12T05:23:16.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 -- 172.21.15.100:0/1599319176 shutdown_connections 2024-03-12T05:23:16.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1599319176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83f010b6e0 0x7f83f010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1599319176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0102cc0 0x7f83f010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1599319176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f01023a0 0x7f83f0102780 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 -- 172.21.15.100:0/1599319176 >> 172.21.15.100:0/1599319176 conn(0x7f83f0077230 msgr2=0x7f83f0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 -- 172.21.15.100:0/1599319176 shutdown_connections 2024-03-12T05:23:16.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 -- 172.21.15.100:0/1599319176 wait complete. 2024-03-12T05:23:16.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 Processor -- start 2024-03-12T05:23:16.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.558+0000 7f83f54f2640 1 -- start start 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f01023a0 0x7f83f0197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83f0102cc0 0x7f83f0197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 0x7f83f019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f019c5e0 con 0x7f83f010b6e0 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f019c750 con 0x7f83f0102cc0 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f019c910 con 0x7f83f01023a0 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 0x7f83f019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 0x7f83f019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49734/0 (socket says 172.21.15.100:49734) 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f01023a0 0x7f83f0197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 -- 172.21.15.100:0/1494005543 learned_addr learned my addr 172.21.15.100:0/1494005543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83f0102cc0 0x7f83f0197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 -- 172.21.15.100:0/1494005543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f01023a0 msgr2=0x7f83f0197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f01023a0 0x7f83f0197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 -- 172.21.15.100:0/1494005543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83f0102cc0 msgr2=0x7f83f0197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83f0102cc0 0x7f83f0197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 -- 172.21.15.100:0/1494005543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009900 con 0x7f83f010b6e0 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 0x7f83f019bfe0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e970 tx=0x7f83e800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83d7fff640 1 -- 172.21.15.100:0/1494005543 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8009720 con 0x7f83f010b6e0 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83d7fff640 1 -- 172.21.15.100:0/1494005543 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83e8005500 con 0x7f83f010b6e0 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f019cbf0 con 0x7f83f010b6e0 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.559+0000 7f83d7fff640 1 -- 172.21.15.100:0/1494005543 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8010430 con 0x7f83f010b6e0 2024-03-12T05:23:16.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.560+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f01a4470 con 0x7f83f010b6e0 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83d7fff640 1 -- 172.21.15.100:0/1494005543 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f83e8004ff0 con 0x7f83f010b6e0 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83f819d740 1 Processor -- start 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83f819d740 1 -- start start 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83d7fff640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83d80a1ba0 0x7f83d80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83f819d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83f819d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c8900 0x561c004d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.562+0000 7f83f819d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004d5bc0 0x561c004d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83f819d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c00437280 con 0x561c004c7fe0 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83f819d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004347b0 con 0x561c004c8900 2024-03-12T05:23:16.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83f819d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c0042ff90 con 0x561c004d5bc0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83d80a1ba0 0x7f83d80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83d7fff640 1 -- 172.21.15.100:0/1494005543 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f83e8014070 con 0x7f83f010b6e0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49744/0 (socket says 172.21.15.100:49744) 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 -- 172.21.15.100:0/3380465548 learned_addr learned my addr 172.21.15.100:0/3380465548 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c8900 0x561c004d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004d5bc0 0x561c004d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 -- 172.21.15.100:0/3380465548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004d5bc0 msgr2=0x561c004d7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004d5bc0 0x561c004d7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 -- 172.21.15.100:0/3380465548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c8900 msgr2=0x561c004d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c8900 0x561c004d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 -- 172.21.15.100:0/3380465548 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c003ee570 con 0x561c004c7fe0 2024-03-12T05:23:16.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c8900 0x561c004d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83d80a1ba0 0x7f83d80a4060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f83e4004650 tx=0x7f83e4004560 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83eeffd640 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004c83c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f83e002c790 tx=0x7f83e0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f16ecca57f60f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83d57fa640 1 -- 172.21.15.100:0/3380465548 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0034640 con 0x561c004c7fe0 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83d57fa640 1 -- 172.21.15.100:0/3380465548 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83e0003e50 con 0x561c004c7fe0 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.563+0000 7f83d57fa640 1 -- 172.21.15.100:0/3380465548 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0032d00 con 0x561c004c7fe0 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 -- 172.21.15.100:0/3380465548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 msgr2=0x561c004c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004c83c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f83e002c790 tx=0x7f83e0005f70 comp rx=0 tx=0).stop 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 -- 172.21.15.100:0/3380465548 shutdown_connections 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004d5bc0 0x561c004d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c8900 0x561c004d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 --2- 172.21.15.100:0/3380465548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004c83c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 -- 172.21.15.100:0/3380465548 >> 172.21.15.100:0/3380465548 conn(0x561c004c2990 msgr2=0x561c004c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 -- 172.21.15.100:0/3380465548 shutdown_connections 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 -- 172.21.15.100:0/3380465548 wait complete. 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.564+0000 7f83f819d740 1 -- start start 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 Processor -- start 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c00103430 con 0x561c004c7fe0 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c000bfe00 con 0x561c004d5bc0 2024-03-12T05:23:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c003b4fd0 con 0x561c004c8900 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 10 client.0 _ll_register_callbacks cb 0x561c0040afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004d55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49750/0 (socket says 172.21.15.100:49750) 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 -- 172.21.15.100:0/2168306380 learned_addr learned my addr 172.21.15.100:0/2168306380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 -- 172.21.15.100:0/2168306380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 msgr2=0x561c004d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 -- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 msgr2=0x561c004d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 -- 172.21.15.100:0/2168306380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f010e800 con 0x561c004c7fe0 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83eeffd640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004d55d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f83e0000f80 tx=0x7f83e0032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8ca9f3b9a16ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2168306380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0034850 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2168306380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f83e0032790 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.565+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c0012f7e0 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2168306380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e0032580 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 10 client.17901 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561c0042e610 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2168306380 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561c0012f7e0 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2168306380 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f83e0052030 con 0x561c004c7fe0 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 10 client.17901 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 20 client.17901 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c004e5a60 0x561c004e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 4 client.17901 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:16.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c0016c720 con 0x561c004e5a60 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.566+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c004e5a60 0x561c004e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c004e5a60 0x561c004e7e80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5031289fd600f71b server_cookie=af8ccd6acbd3e331 in_seq=0 out_seq=0 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83d2ff5640 10 client.17901 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2168306380 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c0016c720 con 0x561c004e5a60 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83d2ff5640 10 client.17901 handle_command_reply: tid=0 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 client.17901 shutdown 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 msgr2=0x561c004d55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004d55d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f83e0000f80 tx=0x7f83e0032d30 comp rx=0 tx=0).stop 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 shutdown_connections 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c004e5a60 0x561c004e7e80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 --2- 172.21.15.100:0/2168306380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004d55d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83d2ff5640 0 client.17901 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.567+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 >> 172.21.15.100:0/2168306380 conn(0x561c004c2990 msgr2=0x561c004c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.568+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 shutdown_connections 2024-03-12T05:23:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.568+0000 7f83f819d740 1 -- 172.21.15.100:0/2168306380 wait complete. 2024-03-12T05:23:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.568+0000 7f83f819d740 20 client.17901 trim_cache size 0 max 16384 2024-03-12T05:23:16.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 Processor -- start 2024-03-12T05:23:16.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 -- start start 2024-03-12T05:23:16.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c7fe0 0x561c004d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 0x561c004d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004c3c10 con 0x561c004d5bc0 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004d8580 con 0x561c004c7fe0 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83f819d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004c3790 con 0x561c004c8900 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 0x561c004d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 0x561c004d55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49758/0 (socket says 172.21.15.100:49758) 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50934/0 (socket says 172.21.15.100:50934) 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 -- 172.21.15.100:0/3458801994 learned_addr learned my addr 172.21.15.100:0/3458801994 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.583+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c7fe0 0x561c004d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 -- 172.21.15.100:0/3458801994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 msgr2=0x561c004d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 -- 172.21.15.100:0/3458801994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c7fe0 msgr2=0x561c004d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c7fe0 0x561c004d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 -- 172.21.15.100:0/3458801994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c003caac0 con 0x561c004d5bc0 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:16.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83eeffd640 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c7fe0 0x561c004d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:16.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 0x561c004d55d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c6a0 tx=0x7f83e809a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd1a5ca7c8499d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83d4ff9640 1 -- 172.21.15.100:0/3458801994 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8005670 con 0x561c004d5bc0 2024-03-12T05:23:16.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83d4ff9640 1 -- 172.21.15.100:0/3458801994 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83e809a700 con 0x561c004d5bc0 2024-03-12T05:23:16.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.584+0000 7f83d4ff9640 1 -- 172.21.15.100:0/3458801994 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8098a90 con 0x561c004d5bc0 2024-03-12T05:23:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 -- 172.21.15.100:0/3458801994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 msgr2=0x561c004d55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 0x561c004d55d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c6a0 tx=0x7f83e809a300 comp rx=0 tx=0).stop 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 -- 172.21.15.100:0/3458801994 shutdown_connections 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004d5bc0 0x561c004d55d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- 172.21.15.100:0/3458801994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004c7fe0 0x561c004d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 -- 172.21.15.100:0/3458801994 >> 172.21.15.100:0/3458801994 conn(0x561c004c2990 msgr2=0x561c004d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 -- 172.21.15.100:0/3458801994 shutdown_connections 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 -- 172.21.15.100:0/3458801994 wait complete. 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 -- start start 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 Processor -- start 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.585+0000 7f83f819d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004da120 con 0x561c004c7fe0 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004da290 con 0x561c004d5bc0 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c004da450 con 0x561c004c8900 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 10 client.0 _ll_register_callbacks cb 0x7f83e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004dc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49760/0 (socket says 172.21.15.100:49760) 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 -- 172.21.15.100:0/2185987006 learned_addr learned my addr 172.21.15.100:0/2185987006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83ef7fe640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 -- 172.21.15.100:0/2185987006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 msgr2=0x561c004dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 -- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 msgr2=0x561c004d9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 -- 172.21.15.100:0/2185987006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c0042e610 con 0x561c004c7fe0 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83eeffd640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004dc660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f83e0037040 tx=0x7f83e0009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db66b7407b7a6baf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2185987006 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0032540 con 0x561c004c7fe0 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2185987006 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f83e0034650 con 0x561c004c7fe0 2024-03-12T05:23:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83e0003e50 con 0x561c004c7fe0 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2185987006 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e0034810 con 0x561c004c7fe0 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 10 client.17907 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.586+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f83e4012750 con 0x561c004c7fe0 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2185987006 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83e0003e50 con 0x561c004c7fe0 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2185987006 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f83e0052070 con 0x561c004c7fe0 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 10 client.17907 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 20 client.17907 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c0053dd50 0x561c00540170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:16.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 4 client.17907 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561c001bb890 con 0x561c0053dd50 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c0053dd50 0x561c00540170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83d2ff5640 10 client.17907 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83ee7fc640 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c0053dd50 0x561c00540170 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3792c2a0aa0561d8 server_cookie=e75f39809bf7201c in_seq=0 out_seq=0 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83d2ff5640 1 -- 172.21.15.100:0/2185987006 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561c001bb890 con 0x561c0053dd50 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83d2ff5640 10 client.17907 handle_command_reply: tid=0 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.587+0000 7f83f819d740 1 client.17907 shutdown 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 msgr2=0x561c004dc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004dc660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f83e0037040 tx=0x7f83e0009630 comp rx=0 tx=0).stop 2024-03-12T05:23:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 shutdown_connections 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561c0053dd50 0x561c00540170 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c004d5bc0 0x561c004d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83d2ff5640 0 client.17907 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c004c8900 0x561c004dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 --2- 172.21.15.100:0/2185987006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c004c7fe0 0x561c004dc660 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 >> 172.21.15.100:0/2185987006 conn(0x561c004c2990 msgr2=0x561c004c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 shutdown_connections 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 1 -- 172.21.15.100:0/2185987006 wait complete. 2024-03-12T05:23:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f819d740 20 client.17907 trim_cache size 0 max 16384 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "c9f96c4c-0407-4eea-abac-2fed25aa0e55": { 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c9f96c4c-0407-4eea-abac-2fed25aa0e55", 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:23:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:23:16.594 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:23:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83d80a1ba0 msgr2=0x7f83d80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83d80a1ba0 0x7f83d80a4060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f83e4004650 tx=0x7f83e4004560 comp rx=0 tx=0).stop 2024-03-12T05:23:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 msgr2=0x7f83f019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 0x7f83f019bfe0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e970 tx=0x7f83e800ed70 comp rx=0 tx=0).stop 2024-03-12T05:23:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 shutdown_connections 2024-03-12T05:23:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83d80a1ba0 0x7f83d80a4060 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83f010b6e0 0x7f83f019bfe0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83f0102cc0 0x7f83f0197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 --2- 172.21.15.100:0/1494005543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f01023a0 0x7f83f0197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.588+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 >> 172.21.15.100:0/1494005543 conn(0x7f83f0077230 msgr2=0x7f83f00fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.589+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 shutdown_connections 2024-03-12T05:23:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:16.589+0000 7f83f54f2640 1 -- 172.21.15.100:0/1494005543 wait complete. 2024-03-12T05:23:16.596 INFO:tasks.cephfs.filesystem:scrub status for tag:c9f96c4c-0407-4eea-abac-2fed25aa0e55 - {'path': '/', 'tag': 'c9f96c4c-0407-4eea-abac-2fed25aa0e55', 'options': 'recursive,force'} 2024-03-12T05:23:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1349461 11.77 MB/sec execute 480 sec latency 52.849 ms 2024-03-12T05:23:17.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: pgmap v348: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-03-12T05:23:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:23:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:23:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:23:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:16 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: pgmap v348: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-03-12T05:23:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:23:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:23:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:23:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:16 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: pgmap v348: 97 pgs: 97 active+clean; 619 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-03-12T05:23:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:23:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:23:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:23:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:16 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:17.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1351041 11.76 MB/sec execute 481 sec latency 25.712 ms 2024-03-12T05:23:18.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1353448 11.76 MB/sec execute 482 sec latency 15.012 ms 2024-03-12T05:23:19.203 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:18 smithi156 ceph-mon[36270]: pgmap v349: 97 pgs: 97 active+clean; 569 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-03-12T05:23:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:18 smithi161 ceph-mon[36043]: pgmap v349: 97 pgs: 97 active+clean; 569 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-03-12T05:23:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:18 smithi100 ceph-mon[32210]: pgmap v349: 97 pgs: 97 active+clean; 569 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-03-12T05:23:19.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1355442 11.75 MB/sec execute 483 sec latency 25.231 ms 2024-03-12T05:23:20.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1357487 11.76 MB/sec execute 484 sec latency 17.160 ms 2024-03-12T05:23:21.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:20 smithi156 ceph-mon[36270]: pgmap v350: 97 pgs: 97 active+clean; 569 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-03-12T05:23:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:20 smithi161 ceph-mon[36043]: pgmap v350: 97 pgs: 97 active+clean; 569 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-03-12T05:23:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:20 smithi100 ceph-mon[32210]: pgmap v350: 97 pgs: 97 active+clean; 569 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-03-12T05:23:21.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1359716 11.76 MB/sec execute 485 sec latency 163.639 ms 2024-03-12T05:23:22.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1361545 11.76 MB/sec execute 486 sec latency 14.101 ms 2024-03-12T05:23:23.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:22 smithi156 ceph-mon[36270]: pgmap v351: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 320 op/s 2024-03-12T05:23:23.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:22 smithi161 ceph-mon[36043]: pgmap v351: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 320 op/s 2024-03-12T05:23:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:22 smithi100 ceph-mon[32210]: pgmap v351: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 320 op/s 2024-03-12T05:23:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1363998 11.76 MB/sec execute 487 sec latency 18.542 ms 2024-03-12T05:23:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1365994 11.75 MB/sec execute 488 sec latency 23.019 ms 2024-03-12T05:23:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:24 smithi161 ceph-mon[36043]: pgmap v352: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:23:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:24 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:23:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:24 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:23:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:24 smithi100 ceph-mon[32210]: pgmap v352: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:23:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:24 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:23:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:24 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:23:25.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:24 smithi156 ceph-mon[36270]: pgmap v352: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:23:25.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:24 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:23:25.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:24 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:23:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1368009 11.75 MB/sec execute 489 sec latency 16.410 ms 2024-03-12T05:23:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:25 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:23:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:23:26.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:23:26.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:25 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:23:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:23:26.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:23:26.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:25 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:23:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:23:26.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:23:26.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:23:26.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1370192 11.75 MB/sec execute 490 sec latency 64.022 ms 2024-03-12T05:23:27.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:26 smithi161 ceph-mon[36043]: pgmap v353: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:23:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:26 smithi100 ceph-mon[32210]: pgmap v353: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:23:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:26 smithi156 ceph-mon[36270]: pgmap v353: 97 pgs: 97 active+clean; 592 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:23:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1371909 11.75 MB/sec execute 491 sec latency 26.672 ms 2024-03-12T05:23:28.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1374502 11.76 MB/sec execute 492 sec latency 17.595 ms 2024-03-12T05:23:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:28 smithi100 ceph-mon[32210]: pgmap v354: 97 pgs: 97 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 291 op/s 2024-03-12T05:23:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:28 smithi156 ceph-mon[36270]: pgmap v354: 97 pgs: 97 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 291 op/s 2024-03-12T05:23:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:28 smithi161 ceph-mon[36043]: pgmap v354: 97 pgs: 97 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 291 op/s 2024-03-12T05:23:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1375501 11.77 MB/sec execute 493 sec latency 36.048 ms 2024-03-12T05:23:30.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1377640 11.77 MB/sec execute 494 sec latency 24.990 ms 2024-03-12T05:23:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:30 smithi100 ceph-mon[32210]: pgmap v355: 97 pgs: 97 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 186 op/s 2024-03-12T05:23:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:30 smithi156 ceph-mon[36270]: pgmap v355: 97 pgs: 97 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 186 op/s 2024-03-12T05:23:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:30 smithi161 ceph-mon[36043]: pgmap v355: 97 pgs: 97 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 186 op/s 2024-03-12T05:23:31.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1379273 11.77 MB/sec execute 495 sec latency 98.266 ms 2024-03-12T05:23:32.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1381791 11.77 MB/sec execute 496 sec latency 42.267 ms 2024-03-12T05:23:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:32 smithi100 ceph-mon[32210]: pgmap v356: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 287 op/s 2024-03-12T05:23:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:32 smithi156 ceph-mon[36270]: pgmap v356: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 287 op/s 2024-03-12T05:23:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:32 smithi161 ceph-mon[36043]: pgmap v356: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 287 op/s 2024-03-12T05:23:33.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1383743 11.77 MB/sec execute 497 sec latency 16.072 ms 2024-03-12T05:23:34.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1386096 11.77 MB/sec execute 498 sec latency 21.722 ms 2024-03-12T05:23:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:34 smithi100 ceph-mon[32210]: pgmap v357: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-12T05:23:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:34 smithi156 ceph-mon[36270]: pgmap v357: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-12T05:23:35.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:34 smithi161 ceph-mon[36043]: pgmap v357: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-12T05:23:35.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1388122 11.76 MB/sec execute 499 sec latency 18.078 ms 2024-03-12T05:23:36.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1390101 11.77 MB/sec execute 500 sec latency 119.936 ms 2024-03-12T05:23:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:36 smithi100 ceph-mon[32210]: pgmap v358: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-12T05:23:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:36 smithi156 ceph-mon[36270]: pgmap v358: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-12T05:23:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:36 smithi161 ceph-mon[36043]: pgmap v358: 97 pgs: 97 active+clean; 619 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-12T05:23:37.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1392714 11.78 MB/sec execute 501 sec latency 32.945 ms 2024-03-12T05:23:38.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:37 smithi100 ceph-mon[32210]: pgmap v359: 97 pgs: 97 active+clean; 633 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 299 op/s 2024-03-12T05:23:38.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:37 smithi156 ceph-mon[36270]: pgmap v359: 97 pgs: 97 active+clean; 633 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 299 op/s 2024-03-12T05:23:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:37 smithi161 ceph-mon[36043]: pgmap v359: 97 pgs: 97 active+clean; 633 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 299 op/s 2024-03-12T05:23:38.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1394435 11.77 MB/sec execute 502 sec latency 18.342 ms 2024-03-12T05:23:39.806 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1396815 11.77 MB/sec execute 503 sec latency 19.875 ms 2024-03-12T05:23:40.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1399333 11.77 MB/sec execute 504 sec latency 16.228 ms 2024-03-12T05:23:41.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:40 smithi156 ceph-mon[36270]: pgmap v360: 97 pgs: 97 active+clean; 633 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 199 op/s 2024-03-12T05:23:41.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:40 smithi161 ceph-mon[36043]: pgmap v360: 97 pgs: 97 active+clean; 633 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 199 op/s 2024-03-12T05:23:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:40 smithi100 ceph-mon[32210]: pgmap v360: 97 pgs: 97 active+clean; 633 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 199 op/s 2024-03-12T05:23:41.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1401042 11.77 MB/sec execute 505 sec latency 78.557 ms 2024-03-12T05:23:42.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1403454 11.77 MB/sec execute 506 sec latency 29.715 ms 2024-03-12T05:23:43.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:42 smithi156 ceph-mon[36270]: pgmap v361: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 298 op/s 2024-03-12T05:23:43.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:42 smithi161 ceph-mon[36043]: pgmap v361: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 298 op/s 2024-03-12T05:23:43.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:42 smithi100 ceph-mon[32210]: pgmap v361: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 298 op/s 2024-03-12T05:23:43.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1405153 11.76 MB/sec execute 507 sec latency 17.460 ms 2024-03-12T05:23:44.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1407590 11.77 MB/sec execute 508 sec latency 15.476 ms 2024-03-12T05:23:45.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:44 smithi156 ceph-mon[36270]: pgmap v362: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-03-12T05:23:45.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:44 smithi161 ceph-mon[36043]: pgmap v362: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-03-12T05:23:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:44 smithi100 ceph-mon[32210]: pgmap v362: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-03-12T05:23:45.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1409967 11.77 MB/sec execute 509 sec latency 15.592 ms 2024-03-12T05:23:46.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:23:46.598 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 -- 172.21.15.100:0/4244999984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844100270 msgr2=0x7f5844100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 --2- 172.21.15.100:0/4244999984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844100270 0x7f5844100650 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5834009930 tx=0x7f583402bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 -- 172.21.15.100:0/4244999984 shutdown_connections 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 --2- 172.21.15.100:0/4244999984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5844105340 0x7f584410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 --2- 172.21.15.100:0/4244999984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5844100c20 0x7f5844104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 --2- 172.21.15.100:0/4244999984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844100270 0x7f5844100650 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 -- 172.21.15.100:0/4244999984 >> 172.21.15.100:0/4244999984 conn(0x7f5844077230 msgr2=0x7f5844077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 -- 172.21.15.100:0/4244999984 shutdown_connections 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.716+0000 7f5849fe2640 1 -- 172.21.15.100:0/4244999984 wait complete. 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 Processor -- start 2024-03-12T05:23:46.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 -- start start 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5844100270 0x7f5844101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5844105340 0x7f5844102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 0x7f58441a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58441a5fc0 con 0x7f5844102780 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58441a6130 con 0x7f5844105340 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5849fe2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58441a62f0 con 0x7f5844100270 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 0x7f58441a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 0x7f58441a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48578/0 (socket says 172.21.15.100:48578) 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 -- 172.21.15.100:0/2845557328 learned_addr learned my addr 172.21.15.100:0/2845557328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5844105340 0x7f5844102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f58437fe640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5844100270 0x7f5844101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 -- 172.21.15.100:0/2845557328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5844100270 msgr2=0x7f5844101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5844100270 0x7f5844101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 -- 172.21.15.100:0/2845557328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5844105340 msgr2=0x7f5844102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.717+0000 7f5843fff640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5844105340 0x7f5844102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5843fff640 1 -- 172.21.15.100:0/2845557328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834009560 con 0x7f5844102780 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5843fff640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 0x7f58441a59f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f583c00eb10 tx=0x7f583c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5840ff9640 1 -- 172.21.15.100:0/2845557328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c004b20 con 0x7f5844102780 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58441a65d0 con 0x7f5844102780 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5840ff9640 1 -- 172.21.15.100:0/2845557328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f583c004c80 con 0x7f5844102780 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5840ff9640 1 -- 172.21.15.100:0/2845557328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c010650 con 0x7f5844102780 2024-03-12T05:23:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.718+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58441a6ae0 con 0x7f5844102780 2024-03-12T05:23:46.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.721+0000 7f584cbb1740 1 Processor -- start 2024-03-12T05:23:46.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.721+0000 7f5840ff9640 1 -- 172.21.15.100:0/2845557328 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f583c0107b0 con 0x7f5844102780 2024-03-12T05:23:46.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.721+0000 7f584cbb1740 1 -- start start 2024-03-12T05:23:46.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d55690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d5e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d60ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f584cbb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487cc4280 con 0x557487d552b0 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f584cbb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487cc17b0 con 0x557487d55bd0 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f584cbb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487cbcf90 con 0x557487d5e7b0 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d55690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d55690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48584/0 (socket says 172.21.15.100:48584) 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5842ffd640 1 -- 172.21.15.100:0/3029775150 learned_addr learned my addr 172.21.15.100:0/3029775150 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f58437fe640 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d5e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5842ffd640 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d60ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5840ff9640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f582c0a1bd0 0x7f582c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f58437fe640 1 -- 172.21.15.100:0/3029775150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 msgr2=0x557487d60ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f58437fe640 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d60ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f58437fe640 1 -- 172.21.15.100:0/3029775150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 msgr2=0x557487d55690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f58437fe640 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d55690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f58437fe640 1 -- 172.21.15.100:0/3029775150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557487c7b570 con 0x557487d55bd0 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5842ffd640 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d55690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5843fff640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f582c0a1bd0 0x7f582c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.722+0000 7f5840ff9640 1 -- 172.21.15.100:0/2845557328 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f583c014070 con 0x7f5844102780 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f58437fe640 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d5e270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5834009230 tx=0x7f583402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f768cda0294049 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f5829ffb640 1 -- 172.21.15.100:0/3029775150 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834004800 con 0x557487d55bd0 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f5829ffb640 1 -- 172.21.15.100:0/3029775150 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5834004960 con 0x557487d55bd0 2024-03-12T05:23:46.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f5829ffb640 1 -- 172.21.15.100:0/3029775150 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834005010 con 0x557487d55bd0 2024-03-12T05:23:46.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f584cbb1740 1 -- 172.21.15.100:0/3029775150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 msgr2=0x557487d5e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f584cbb1740 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d5e270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5834009230 tx=0x7f583402c5d0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.723+0000 7f5843fff640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f582c0a1bd0 0x7f582c0a4090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f583c0cba10 tx=0x7f583c065c30 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- 172.21.15.100:0/3029775150 shutdown_connections 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d60ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d5e270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 --2- 172.21.15.100:0/3029775150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d55690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- 172.21.15.100:0/3029775150 >> 172.21.15.100:0/3029775150 conn(0x557487d4f990 msgr2=0x557487d4fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- 172.21.15.100:0/3029775150 shutdown_connections 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- 172.21.15.100:0/3029775150 wait complete. 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- start start 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 Processor -- start 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d5e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d5b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d5e7b0 0x557487d5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487990430 con 0x557487d552b0 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55748794ce00 con 0x557487d5e7b0 2024-03-12T05:23:46.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487c41fd0 con 0x557487d55bd0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.724+0000 7f584cbb1740 10 client.0 _ll_register_callbacks cb 0x557487c97fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d5b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d5b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:52948/0 (socket says 172.21.15.100:52948) 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d5e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 -- 172.21.15.100:0/2849692039 learned_addr learned my addr 172.21.15.100:0/2849692039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d5e7b0 0x557487d5bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 -- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d5e7b0 msgr2=0x557487d5bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d5e7b0 0x557487d5bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 -- 172.21.15.100:0/2849692039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 msgr2=0x557487d5e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d5e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 -- 172.21.15.100:0/2849692039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584410d720 con 0x557487d55bd0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d5e7b0 0x557487d5bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f58437fe640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d5b590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5834005ab0 tx=0x7f5834004330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86eac259a152504e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.725+0000 7f5822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f5822ffd640 1 -- 172.21.15.100:0/2849692039 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834005200 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f5822ffd640 1 -- 172.21.15.100:0/2849692039 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f58340044b0 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574879bc7e0 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f5822ffd640 1 -- 172.21.15.100:0/2849692039 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5834004ad0 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 10 client.27253 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557487cbb610 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f5822ffd640 1 -- 172.21.15.100:0/2849692039 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5574879bc7e0 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f5822ffd640 1 -- 172.21.15.100:0/2849692039 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5834059070 con 0x557487d55bd0 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 10 client.27253 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 10 client.27253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 20 client.27253 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487d70980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 4 client.27253 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574879f9720 con 0x557487d6e500 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.726+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487d70980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.727+0000 7f5822ffd640 10 client.27253 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.727+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487d70980 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b969ed034cea18e4 server_cookie=a66474c000cab75e in_seq=0 out_seq=0 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f5822ffd640 1 -- 172.21.15.100:0/2849692039 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574879f9720 con 0x557487d6e500 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f5822ffd640 10 client.27253 handle_command_reply: tid=0 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 client.27253 shutdown 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 msgr2=0x557487d5b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d5b590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5834005ab0 tx=0x7f5834004330 comp rx=0 tx=0).stop 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 shutdown_connections 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487d70980 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d5e7b0 0x557487d5bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d5b590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f5822ffd640 0 client.27253 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2849692039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d5e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 >> 172.21.15.100:0/2849692039 conn(0x557487d4f990 msgr2=0x557487d51730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 shutdown_connections 2024-03-12T05:23:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 1 -- 172.21.15.100:0/2849692039 wait complete. 2024-03-12T05:23:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.728+0000 7f584cbb1740 20 client.27253 trim_cache size 0 max 16384 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 Processor -- start 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 -- start start 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d5dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487d503e0 con 0x557487d552b0 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574879f9720 con 0x557487d55bd0 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.743+0000 7f584cbb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574879bc7e0 con 0x557487d5e7b0 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d65600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48610/0 (socket says 172.21.15.100:48610) 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 -- 172.21.15.100:0/2090699132 learned_addr learned my addr 172.21.15.100:0/2090699132 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:46.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f58437fe640 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d5dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 -- 172.21.15.100:0/2090699132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 msgr2=0x557487d5dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d5dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 -- 172.21.15.100:0/2090699132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 msgr2=0x557487d65b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d65b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 -- 172.21.15.100:0/2090699132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557487c57ac0 con 0x557487d552b0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f5842ffd640 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d65600 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5838013280 tx=0x7f5838005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975faa9235f2acaa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f58297fa640 1 -- 172.21.15.100:0/2090699132 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583801f070 con 0x557487d552b0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f58297fa640 1 -- 172.21.15.100:0/2090699132 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5838020040 con 0x557487d552b0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.744+0000 7f58297fa640 1 -- 172.21.15.100:0/2090699132 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838023a90 con 0x557487d552b0 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 -- 172.21.15.100:0/2090699132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 msgr2=0x557487d65600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d65600 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5838013280 tx=0x7f5838005f90 comp rx=0 tx=0).stop 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 -- 172.21.15.100:0/2090699132 shutdown_connections 2024-03-12T05:23:46.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d5e7b0 0x557487d5dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d55bd0 0x557487d65b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 --2- 172.21.15.100:0/2090699132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d552b0 0x557487d65600 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 -- 172.21.15.100:0/2090699132 >> 172.21.15.100:0/2090699132 conn(0x557487d4f990 msgr2=0x557487d75320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 -- 172.21.15.100:0/2090699132 shutdown_connections 2024-03-12T05:23:46.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 -- 172.21.15.100:0/2090699132 wait complete. 2024-03-12T05:23:46.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.745+0000 7f584cbb1740 1 -- start start 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 Processor -- start 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d552b0 0x557487d63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 0x557487d649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487d65ed0 con 0x557487d5e7b0 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487dc4ab0 con 0x557487d552b0 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487dc4c20 con 0x557487d55bd0 2024-03-12T05:23:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 10 client.0 _ll_register_callbacks cb 0x7f583c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 0x557487d649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 0x557487d649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48612/0 (socket says 172.21.15.100:48612) 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 -- 172.21.15.100:0/1581676821 learned_addr learned my addr 172.21.15.100:0/1581676821 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d552b0 0x557487d63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f58437fe640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 -- 172.21.15.100:0/1581676821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 msgr2=0x557487d64470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d64470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 -- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d552b0 msgr2=0x557487d63f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d552b0 0x557487d63f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 -- 172.21.15.100:0/1581676821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5838009d90 con 0x557487d5e7b0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5842ffd640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 0x557487d649b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5838013430 tx=0x7f5838026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707b141f7ef5289 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5822ffd640 1 -- 172.21.15.100:0/1581676821 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583801f070 con 0x557487d5e7b0 2024-03-12T05:23:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5834004960 con 0x557487d5e7b0 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.746+0000 7f5822ffd640 1 -- 172.21.15.100:0/1581676821 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5838005250 con 0x557487d5e7b0 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f5822ffd640 1 -- 172.21.15.100:0/1581676821 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5838005410 con 0x557487d5e7b0 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 10 client.17925 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557487c7d6a0 con 0x557487d5e7b0 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f5822ffd640 1 -- 172.21.15.100:0/1581676821 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5834004960 con 0x557487d5e7b0 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f5822ffd640 1 -- 172.21.15.100:0/1581676821 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f583802d020 con 0x557487d5e7b0 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 10 client.17925 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 20 client.17925 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487dc83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 4 client.17925 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557487a48890 con 0x557487d6e500 2024-03-12T05:23:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.747+0000 7f58437fe640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487dc83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f58437fe640 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487dc83c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410cff69f4761a51 server_cookie=7f70c6040bef55c5 in_seq=0 out_seq=0 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5822ffd640 10 client.17925 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5822ffd640 1 -- 172.21.15.100:0/1581676821 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557487a48890 con 0x557487d6e500 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5822ffd640 10 client.17925 handle_command_reply: tid=0 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 client.17925 shutdown 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 msgr2=0x557487d649b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 0x557487d649b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5838013430 tx=0x7f5838026040 comp rx=0 tx=0).stop 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 shutdown_connections 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557487d6e500 0x557487dc83c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557487d5e7b0 0x557487d649b0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5822ffd640 0 client.17925 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557487d55bd0 0x557487d64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 --2- 172.21.15.100:0/1581676821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557487d552b0 0x557487d63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 >> 172.21.15.100:0/1581676821 conn(0x557487d4f990 msgr2=0x557487d5ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 shutdown_connections 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 1 -- 172.21.15.100:0/1581676821 wait complete. 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f584cbb1740 20 client.17925 trim_cache size 0 max 16384 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:23:46.750 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:23:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f582c0a1bd0 msgr2=0x7f582c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5849fe2640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f582c0a1bd0 0x7f582c0a4090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f583c0cba10 tx=0x7f583c065c30 comp rx=0 tx=0).stop 2024-03-12T05:23:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 msgr2=0x7f58441a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.748+0000 7f5849fe2640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 0x7f58441a59f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f583c00eb10 tx=0x7f583c00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 shutdown_connections 2024-03-12T05:23:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f582c0a1bd0 0x7f582c0a4090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5844102780 0x7f58441a59f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5844105340 0x7f5844102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 --2- 172.21.15.100:0/2845557328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5844100270 0x7f5844101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 >> 172.21.15.100:0/2845557328 conn(0x7f5844077230 msgr2=0x7f5844068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 shutdown_connections 2024-03-12T05:23:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.749+0000 7f5849fe2640 1 -- 172.21.15.100:0/2845557328 wait complete. 2024-03-12T05:23:46.754 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:23:46.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:23:46.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1411728 11.76 MB/sec execute 510 sec latency 19.507 ms 2024-03-12T05:23:46.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 -- 172.21.15.100:0/489244715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1002f0 msgr2=0x7fd95c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 --2- 172.21.15.100:0/489244715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1002f0 0x7fd95c1006d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd954009cd0 tx=0x7fd95402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 -- 172.21.15.100:0/489244715 shutdown_connections 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 --2- 172.21.15.100:0/489244715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1053c0 0x7fd95c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 --2- 172.21.15.100:0/489244715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd95c100ca0 0x7fd95c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 --2- 172.21.15.100:0/489244715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1002f0 0x7fd95c1006d0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 -- 172.21.15.100:0/489244715 >> 172.21.15.100:0/489244715 conn(0x7fd95c077180 msgr2=0x7fd95c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 -- 172.21.15.100:0/489244715 shutdown_connections 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 -- 172.21.15.100:0/489244715 wait complete. 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 Processor -- start 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.885+0000 7fd9638d4640 1 -- start start 2024-03-12T05:23:46.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd95c1002f0 0x7fd95c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 0x7fd95c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1976e0 0x7fd95c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c1981b0 con 0x7fd95c1053c0 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c198320 con 0x7fd95c1976e0 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c1984e0 con 0x7fd95c1002f0 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 0x7fd95c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 0x7fd95c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48632/0 (socket says 172.21.15.100:48632) 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 -- 172.21.15.100:0/1619604002 learned_addr learned my addr 172.21.15.100:0/1619604002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd961e71640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1976e0 0x7fd95c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd961670640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd95c1002f0 0x7fd95c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 -- 172.21.15.100:0/1619604002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd95c1002f0 msgr2=0x7fd95c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd95c1002f0 0x7fd95c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 -- 172.21.15.100:0/1619604002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1976e0 msgr2=0x7fd95c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1976e0 0x7fd95c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 -- 172.21.15.100:0/1619604002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954009900 con 0x7fd95c1053c0 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd961e71640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1976e0 0x7fd95c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd960e6f640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 0x7fd95c19db60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d770 tx=0x7fd95000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950009c30 con 0x7fd95c1053c0 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd95000de80 con 0x7fd95c1053c0 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c1a4480 con 0x7fd95c1053c0 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95000f6b0 con 0x7fd95c1053c0 2024-03-12T05:23:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.886+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c1a4960 con 0x7fd95c1053c0 2024-03-12T05:23:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.888+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c1017c0 con 0x7fd95c1053c0 2024-03-12T05:23:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.889+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd950010460 con 0x7fd95c1053c0 2024-03-12T05:23:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.889+0000 7fd95a7fc640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9440a1ba0 0x7fd9440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.889+0000 7fd961670640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9440a1ba0 0x7fd9440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.889+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd9500cd500 con 0x7fd95c1053c0 2024-03-12T05:23:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.890+0000 7fd961670640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9440a1ba0 0x7fd9440a4060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd954004af0 tx=0x7fd954005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:46.894+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd950096440 con 0x7fd95c1053c0 2024-03-12T05:23:47.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.115+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd95c109600 con 0x7fd95c1053c0 2024-03-12T05:23:47.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.116+0000 7fd95a7fc640 1 -- 172.21.15.100:0/1619604002 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fd950099b00 con 0x7fd95c1053c0 2024-03-12T05:23:47.111 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:23:47.111 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:23:47.111 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:23:47.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9440a1ba0 msgr2=0x7fd9440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9440a1ba0 0x7fd9440a4060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd954004af0 tx=0x7fd954005c50 comp rx=0 tx=0).stop 2024-03-12T05:23:47.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 msgr2=0x7fd95c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 0x7fd95c19db60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d770 tx=0x7fd95000dc30 comp rx=0 tx=0).stop 2024-03-12T05:23:47.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 shutdown_connections 2024-03-12T05:23:47.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9440a1ba0 0x7fd9440a4060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd95c1976e0 0x7fd95c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c1053c0 0x7fd95c19db60 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 --2- 172.21.15.100:0/1619604002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd95c1002f0 0x7fd95c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 >> 172.21.15.100:0/1619604002 conn(0x7fd95c077180 msgr2=0x7fd95c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 shutdown_connections 2024-03-12T05:23:47.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.118+0000 7fd9638d4640 1 -- 172.21.15.100:0/1619604002 wait complete. 2024-03-12T05:23:47.123 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:23:47.124 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:23:47.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:46 smithi156 ceph-mon[36270]: pgmap v363: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-03-12T05:23:47.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:46 smithi161 ceph-mon[36043]: pgmap v363: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-03-12T05:23:47.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 -- 172.21.15.100:0/327757397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 msgr2=0x7ff1e010cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/327757397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e010cdf0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009cd0 tx=0x7ff1d802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 -- 172.21.15.100:0/327757397 shutdown_connections 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/327757397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e010cdf0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/327757397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0100b00 0x7ff1e0104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/327757397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 0x7ff1e0100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 -- 172.21.15.100:0/327757397 >> 172.21.15.100:0/327757397 conn(0x7ff1e00fb9d0 msgr2=0x7ff1e00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 -- 172.21.15.100:0/327757397 shutdown_connections 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.236+0000 7ff1e6a28640 1 -- 172.21.15.100:0/327757397 wait complete. 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 Processor -- start 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 -- start start 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 0x7ff1e0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0198ed0 0x7ff1e01a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a3b40 con 0x7ff1e0105220 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a3cb0 con 0x7ff1e0100150 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e6a28640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a3e70 con 0x7ff1e0198ed0 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e5225640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e5225640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e0198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48652/0 (socket says 172.21.15.100:48652) 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e5225640 1 -- 172.21.15.100:0/3782883761 learned_addr learned my addr 172.21.15.100:0/3782883761 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.237+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 0x7ff1e0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5225640 1 -- 172.21.15.100:0/3782883761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0198ed0 msgr2=0x7ff1e01a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0198ed0 0x7ff1e01a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5225640 1 -- 172.21.15.100:0/3782883761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 msgr2=0x7ff1e0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 0x7ff1e0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5225640 1 -- 172.21.15.100:0/3782883761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8009900 con 0x7ff1e0105220 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 0x7ff1e0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e0198990 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c970 tx=0x7ff1d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1d6ffd640 1 -- 172.21.15.100:0/3782883761 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0004440 con 0x7ff1e0105220 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1d6ffd640 1 -- 172.21.15.100:0/3782883761 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d0007d40 con 0x7ff1e0105220 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e01a4150 con 0x7ff1e0105220 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1d6ffd640 1 -- 172.21.15.100:0/3782883761 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000f450 con 0x7ff1e0105220 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.238+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e01a4660 con 0x7ff1e0105220 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.240+0000 7ff1d6ffd640 1 -- 172.21.15.100:0/3782883761 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff1d0007860 con 0x7ff1e0105220 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.240+0000 7ff1ea857740 1 Processor -- start 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 -- start start 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623f1e960 con 0x561623f63440 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623ed2310 con 0x561623f6c940 2024-03-12T05:23:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1ea857740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623ecf840 con 0x561623f63d60 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1d6ffd640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1cc0a1bd0 0x7ff1cc0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e6227640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e5a26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f6c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e6227640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f63820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48666/0 (socket says 172.21.15.100:48666) 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e5225640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e6227640 1 -- 172.21.15.100:0/1522651585 learned_addr learned my addr 172.21.15.100:0/1522651585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e6227640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1cc0a1bd0 0x7ff1cc0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1d6ffd640 1 -- 172.21.15.100:0/3782883761 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1d00cd650 con 0x7ff1e0105220 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e5a26640 1 -- 172.21.15.100:0/1522651585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 msgr2=0x561623f6ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.241+0000 7ff1e5a26640 1 -- 172.21.15.100:0/1522651585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 msgr2=0x561623f63820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f63820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1e5a26640 1 -- 172.21.15.100:0/1522651585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561623e8d060 con 0x561623f63d60 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1e6227640 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f63820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1e6227640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1cc0a1bd0 0x7ff1cc0a4090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009ca0 tx=0x7ff1d8005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f6c400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009fd0 tx=0x7ff1dc015c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed81ec3e2388821 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1cbfff640 1 -- 172.21.15.100:0/1522651585 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00fc60 con 0x561623f63d60 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1cbfff640 1 -- 172.21.15.100:0/1522651585 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1dc00fdc0 con 0x561623f63d60 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.242+0000 7ff1cbfff640 1 -- 172.21.15.100:0/1522651585 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0049c0 con 0x561623f63d60 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 -- 172.21.15.100:0/1522651585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 msgr2=0x561623f6c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f6c400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009fd0 tx=0x7ff1dc015c20 comp rx=0 tx=0).stop 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 -- 172.21.15.100:0/1522651585 shutdown_connections 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f6c400 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 --2- 172.21.15.100:0/1522651585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f63820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 -- 172.21.15.100:0/1522651585 >> 172.21.15.100:0/1522651585 conn(0x561623f5db70 msgr2=0x561623f5df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 -- 172.21.15.100:0/1522651585 shutdown_connections 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 -- 172.21.15.100:0/1522651585 wait complete. 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 -- start start 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.243+0000 7ff1ea857740 1 Processor -- start 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f69970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 0x561623f69eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623ecb020 con 0x561623f63d60 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623b9e430 con 0x561623f6c940 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623b5ae00 con 0x561623f63440 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 10 client.0 _ll_register_callbacks cb 0x561623e72cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 0x561623f69eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5a26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f69970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5a26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f69970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:53014/0 (socket says 172.21.15.100:53014) 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 0x561623f69eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48682/0 (socket says 172.21.15.100:48682) 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5a26640 1 -- 172.21.15.100:0/350135357 learned_addr learned my addr 172.21.15.100:0/350135357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e6227640 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 -- 172.21.15.100:0/350135357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 msgr2=0x561623f69970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f69970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 -- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 msgr2=0x561623f6a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 -- 172.21.15.100:0/350135357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc009c00 con 0x561623f63d60 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1e5225640 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 0x561623f69eb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00cacd0 tx=0x7ff1d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edea96d3d101169 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1c97fa640 1 -- 172.21.15.100:0/350135357 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d009e160 con 0x561623f63d60 2024-03-12T05:23:47.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1c97fa640 1 -- 172.21.15.100:0/350135357 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1d009a1e0 con 0x561623f63d60 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561623bca7e0 con 0x561623f63d60 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1c97fa640 1 -- 172.21.15.100:0/350135357 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d009e330 con 0x561623f63d60 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 10 client.17943 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.244+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561623ec96a0 con 0x561623f63d60 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1c97fa640 1 -- 172.21.15.100:0/350135357 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561623bca7e0 con 0x561623f63d60 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1c97fa640 1 -- 172.21.15.100:0/350135357 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1d0020020 con 0x561623f63d60 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1ea857740 10 client.17943 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1ea857740 10 client.17943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1ea857740 20 client.17943 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1ea857740 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623f7cd90 0x561623f7f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1ea857740 4 client.17943 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:47.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561623c07720 con 0x561623f7cd90 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623f7cd90 0x561623f7f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1c97fa640 10 client.17943 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.245+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623f7cd90 0x561623f7f180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4f426497e2e18ac server_cookie=560c9751d1141291 in_seq=0 out_seq=0 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.246+0000 7ff1c97fa640 1 -- 172.21.15.100:0/350135357 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561623c07720 con 0x561623f7cd90 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1c97fa640 10 client.17943 handle_command_reply: tid=0 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 client.17943 shutdown 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 msgr2=0x561623f69eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 0x561623f69eb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00cacd0 tx=0x7ff1d0000bc0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 shutdown_connections 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623f7cd90 0x561623f7f180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f6a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63d60 0x561623f69eb0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1c97fa640 0 client.17943 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:47.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 --2- 172.21.15.100:0/350135357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 >> 172.21.15.100:0/350135357 conn(0x561623f5db70 msgr2=0x561623f5f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 shutdown_connections 2024-03-12T05:23:47.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 1 -- 172.21.15.100:0/350135357 wait complete. 2024-03-12T05:23:47.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.247+0000 7ff1ea857740 20 client.17943 trim_cache size 0 max 16384 2024-03-12T05:23:47.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 Processor -- start 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 -- start start 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f63d60 0x561623f6a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 0x561623f66bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623e50130 con 0x561623f6c940 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623f84390 con 0x561623f63d60 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1ea857740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623f6f060 con 0x561623f63440 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1e6227640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 0x561623f66bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1e6227640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 0x561623f66bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48688/0 (socket says 172.21.15.100:48688) 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1e6227640 1 -- 172.21.15.100:0/3659797576 learned_addr learned my addr 172.21.15.100:0/3659797576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.259+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f63d60 0x561623f6a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e6227640 1 -- 172.21.15.100:0/3659797576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 msgr2=0x561623f6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e6227640 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f6c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e6227640 1 -- 172.21.15.100:0/3659797576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f63d60 msgr2=0x561623f6a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e6227640 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f63d60 0x561623f6a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e6227640 1 -- 172.21.15.100:0/3659797576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561623e65c20 con 0x561623f6c940 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1e6227640 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 0x561623f66bd0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009cd0 tx=0x7ff1d8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6c5f33b80ea1fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1cb7fe640 1 -- 172.21.15.100:0/3659797576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8034a90 con 0x561623f6c940 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1cb7fe640 1 -- 172.21.15.100:0/3659797576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d8034bf0 con 0x561623f6c940 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.260+0000 7ff1cb7fe640 1 -- 172.21.15.100:0/3659797576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8004040 con 0x561623f6c940 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- 172.21.15.100:0/3659797576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 msgr2=0x561623f66bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 0x561623f66bd0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009cd0 tx=0x7ff1d8040000 comp rx=0 tx=0).stop 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- 172.21.15.100:0/3659797576 shutdown_connections 2024-03-12T05:23:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f6c940 0x561623f66bd0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f63d60 0x561623f6a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3659797576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63440 0x561623f6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- 172.21.15.100:0/3659797576 >> 172.21.15.100:0/3659797576 conn(0x561623f5db70 msgr2=0x561623f68240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- 172.21.15.100:0/3659797576 shutdown_connections 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- 172.21.15.100:0/3659797576 wait complete. 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- start start 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 Processor -- start 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f72450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f72990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f72ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.261+0000 7ff1ea857740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623fd33b0 con 0x561623f63440 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1ea857740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623fd3520 con 0x561623f6c940 2024-03-12T05:23:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1ea857740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623fd36e0 con 0x561623f63d60 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1ea857740 10 client.0 _ll_register_callbacks cb 0x7ff1dc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f72450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f72450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48702/0 (socket says 172.21.15.100:48702) 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 -- 172.21.15.100:0/3349345038 learned_addr learned my addr 172.21.15.100:0/3349345038 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f72990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e6227640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f72ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 -- 172.21.15.100:0/3349345038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 msgr2=0x561623f72990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f72990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 -- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 msgr2=0x561623f72ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f72ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 -- 172.21.15.100:0/3349345038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561623e8d060 con 0x561623f63440 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1e5a26640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f72450 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc005530 tx=0x7ff1dc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1a8fe86d52f6c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1c97fa640 1 -- 172.21.15.100:0/3349345038 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc02d070 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1c97fa640 1 -- 172.21.15.100:0/3349345038 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1dc005130 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1dc00fdc0 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1c97fa640 1 -- 172.21.15.100:0/3349345038 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1dc0052f0 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1ea857740 10 client.17949 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.262+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7ff1d8047040 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1c97fa640 1 -- 172.21.15.100:0/3349345038 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1dc00fdc0 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1c97fa640 1 -- 172.21.15.100:0/3349345038 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1dc01d5c0 con 0x561623f63440 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1ea857740 10 client.17949 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1ea857740 10 client.17949 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1ea857740 20 client.17949 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623fd4e30 0x561623fd7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1ea857740 4 client.17949 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:23:47.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561623c56890 con 0x561623fd4e30 2024-03-12T05:23:47.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623fd4e30 0x561623fd7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1e5225640 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623fd4e30 0x561623fd7220 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d36a79d899246b6e server_cookie=d469a1dff461d51f in_seq=0 out_seq=0 2024-03-12T05:23:47.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.263+0000 7ff1c97fa640 10 client.17949 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1c97fa640 1 -- 172.21.15.100:0/3349345038 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561623c56890 con 0x561623fd4e30 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1c97fa640 10 client.17949 handle_command_reply: tid=0 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 client.17949 shutdown 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 msgr2=0x561623f72450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f72450 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc005530 tx=0x7ff1dc0096c0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 shutdown_connections 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x561623fd4e30 0x561623fd7220 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561623f6c940 0x561623f72ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561623f63d60 0x561623f72990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1c97fa640 0 client.17949 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 --2- 172.21.15.100:0/3349345038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561623f63440 0x561623f72450 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 >> 172.21.15.100:0/3349345038 conn(0x561623f5db70 msgr2=0x561623f5c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 shutdown_connections 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 1 -- 172.21.15.100:0/3349345038 wait complete. 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1ea857740 20 client.17949 trim_cache size 0 max 16384 2024-03-12T05:23:47.266 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:23:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1cc0a1bd0 msgr2=0x7ff1cc0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1cc0a1bd0 0x7ff1cc0a4090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009ca0 tx=0x7ff1d8005c50 comp rx=0 tx=0).stop 2024-03-12T05:23:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 msgr2=0x7ff1e0198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e0198990 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c970 tx=0x7ff1d000ce30 comp rx=0 tx=0).stop 2024-03-12T05:23:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 shutdown_connections 2024-03-12T05:23:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1cc0a1bd0 0x7ff1cc0a4090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1e0198ed0 0x7ff1e01a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e0105220 0x7ff1e0198990 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.265+0000 7ff1e6a28640 1 --2- 172.21.15.100:0/3782883761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e0100150 0x7ff1e0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.266+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 >> 172.21.15.100:0/3782883761 conn(0x7ff1e00fb9d0 msgr2=0x7ff1e00fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.266+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 shutdown_connections 2024-03-12T05:23:47.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.266+0000 7ff1e6a28640 1 -- 172.21.15.100:0/3782883761 wait complete. 2024-03-12T05:23:47.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:46 smithi100 ceph-mon[32210]: pgmap v363: 97 pgs: 97 active+clean; 578 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-03-12T05:23:47.271 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:23:47.271 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:23:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 -- 172.21.15.100:0/1664233722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102420 msgr2=0x7f6e200ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 --2- 172.21.15.100:0/1664233722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102420 0x7f6e200ffb60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009900 tx=0x7f6e1c02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 -- 172.21.15.100:0/1664233722 shutdown_connections 2024-03-12T05:23:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 --2- 172.21.15.100:0/1664233722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 0x7f6e2010fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 --2- 172.21.15.100:0/1664233722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102420 0x7f6e200ffb60 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 --2- 172.21.15.100:0/1664233722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102040 0x7f6e200ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.404+0000 7f6e27502640 1 -- 172.21.15.100:0/1664233722 >> 172.21.15.100:0/1664233722 conn(0x7f6e200765a0 msgr2=0x7f6e200fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.405+0000 7f6e27502640 1 -- 172.21.15.100:0/1664233722 shutdown_connections 2024-03-12T05:23:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.405+0000 7f6e27502640 1 -- 172.21.15.100:0/1664233722 wait complete. 2024-03-12T05:23:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.405+0000 7f6e27502640 1 Processor -- start 2024-03-12T05:23:47.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.405+0000 7f6e27502640 1 -- start start 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e27502640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 0x7f6e2019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e27502640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102040 0x7f6e2019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e27502640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102420 0x7f6e20199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e27502640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e20199eb0 con 0x7f6e20102040 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e27502640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2019a020 con 0x7f6e20100180 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e27502640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2019a1e0 con 0x7f6e20102420 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 0x7f6e2019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 0x7f6e2019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:52908/0 (socket says 172.21.15.100:52908) 2024-03-12T05:23:47.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e2529e640 1 -- 172.21.15.100:0/399469386 learned_addr learned my addr 172.21.15.100:0/399469386 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e25a9f640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102420 0x7f6e20199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.406+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102040 0x7f6e2019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e25a9f640 1 -- 172.21.15.100:0/399469386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 msgr2=0x7f6e2019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e25a9f640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 0x7f6e2019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e25a9f640 1 -- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102040 msgr2=0x7f6e2019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e25a9f640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102040 0x7f6e2019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e25a9f640 1 -- 172.21.15.100:0/399469386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c009560 con 0x7f6e20102420 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e25a9f640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102420 0x7f6e20199860 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800be20 tx=0x7f6e1800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e167fc640 1 -- 172.21.15.100:0/399469386 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800ce70 con 0x7f6e20102420 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e167fc640 1 -- 172.21.15.100:0/399469386 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e18014920 con 0x7f6e20102420 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2019a4c0 con 0x7f6e20102420 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e167fc640 1 -- 172.21.15.100:0/399469386 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18012a80 con 0x7f6e20102420 2024-03-12T05:23:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.407+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e201a6750 con 0x7f6e20102420 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 Processor -- start 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e167fc640 1 -- 172.21.15.100:0/399469386 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6e18014a90 con 0x7f6e20102420 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 -- start start 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee6735630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee673e750 0x55dee6740b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee66f0960 con 0x55dee6735b70 2024-03-12T05:23:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee66a4310 con 0x55dee673e750 2024-03-12T05:23:47.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.411+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee66a1840 con 0x55dee6735250 2024-03-12T05:23:47.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48738/0 (socket says 172.21.15.100:48738) 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 -- 172.21.15.100:0/1671658206 learned_addr learned my addr 172.21.15.100:0/1671658206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee6735630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee673e750 0x55dee6740b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 -- 172.21.15.100:0/1671658206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 msgr2=0x55dee6735630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee6735630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e167fc640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6e0c0a1ce0 0x7f6e0c0a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 -- 172.21.15.100:0/1671658206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee673e750 msgr2=0x55dee6740b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee673e750 0x55dee6740b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.412+0000 7f6e2529e640 1 -- 172.21.15.100:0/1671658206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dee665f060 con 0x55dee6735b70 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.413+0000 7f6e167fc640 1 -- 172.21.15.100:0/399469386 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e180cd1e0 con 0x7f6e20102420 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.413+0000 7f6e2529e640 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673e210 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000bd60 tx=0x7f6e100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168b836d68955eee server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.413+0000 7f6e25a9f640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6e0c0a1ce0 0x7f6e0c0a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.413+0000 7f6e0b7fe640 1 -- 172.21.15.100:0/1671658206 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10011640 con 0x55dee6735b70 2024-03-12T05:23:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.413+0000 7f6e0b7fe640 1 -- 172.21.15.100:0/1671658206 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e10015070 con 0x55dee6735b70 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.414+0000 7f6e0b7fe640 1 -- 172.21.15.100:0/1671658206 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1001bb40 con 0x55dee6735b70 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.414+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/1671658206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 msgr2=0x55dee673e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.414+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673e210 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000bd60 tx=0x7f6e100107a0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/1671658206 shutdown_connections 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee673e750 0x55dee6740b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673e210 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/1671658206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee6735630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/1671658206 >> 172.21.15.100:0/1671658206 conn(0x55dee672f980 msgr2=0x55dee672fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e25a9f640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6e0c0a1ce0 0x7f6e0c0a41a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009f90 tx=0x7f6e180124e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/1671658206 shutdown_connections 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.415+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/1671658206 wait complete. 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.416+0000 7f6e2a1b4740 1 -- start start 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.416+0000 7f6e2a1b4740 1 Processor -- start 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735250 0x55dee673b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee673e750 0x55dee673c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee669d020 con 0x55dee6735b70 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee6370430 con 0x55dee6735250 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee632ce00 con 0x55dee673e750 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2a1b4740 10 client.0 _ll_register_callbacks cb 0x55dee6644cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48742/0 (socket says 172.21.15.100:48742) 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 -- 172.21.15.100:0/3889026958 learned_addr learned my addr 172.21.15.100:0/3889026958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee673e750 0x55dee673c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 -- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee673e750 msgr2=0x55dee673c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee673e750 0x55dee673c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 -- 172.21.15.100:0/3889026958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735250 msgr2=0x55dee673b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735250 0x55dee673b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 -- 172.21.15.100:0/3889026958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1000b990 con 0x55dee6735b70 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e2529e640 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673bb90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f6e10010ca0 tx=0x7f6e10005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6e13be6bee8c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.417+0000 7f6e08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e08ff9640 1 -- 172.21.15.100:0/3889026958 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1001f070 con 0x55dee6735b70 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e08ff9640 1 -- 172.21.15.100:0/3889026958 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e10017070 con 0x55dee6735b70 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dee639c7e0 con 0x55dee6735b70 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e08ff9640 1 -- 172.21.15.100:0/3889026958 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e10004020 con 0x55dee6735b70 2024-03-12T05:23:47.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e2a1b4740 10 client.17961 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dee669b6a0 con 0x55dee6735b70 2024-03-12T05:23:47.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e08ff9640 1 -- 172.21.15.100:0/3889026958 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e10031080 con 0x55dee6735b70 2024-03-12T05:23:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e08ff9640 1 -- 172.21.15.100:0/3889026958 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e100047e0 con 0x55dee6735b70 2024-03-12T05:23:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e2a1b4740 10 client.17961 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e2a1b4740 10 client.17961 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.418+0000 7f6e2a1b4740 20 client.17961 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.419+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee6757340 0x55dee6759760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.419+0000 7f6e2a1b4740 4 client.17961 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.419+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dee63d9720 con 0x55dee6757340 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.419+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee6757340 0x55dee6759760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.419+0000 7f6e08ff9640 10 client.17961 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.419+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee6757340 0x55dee6759760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9070148d1b22935 server_cookie=c522c88248abd647 in_seq=0 out_seq=0 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.420+0000 7f6e08ff9640 1 -- 172.21.15.100:0/3889026958 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dee63d9720 con 0x55dee6757340 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.420+0000 7f6e08ff9640 10 client.17961 handle_command_reply: tid=0 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.420+0000 7f6e2a1b4740 1 client.17961 shutdown 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.420+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 msgr2=0x55dee673bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.420+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673bb90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f6e10010ca0 tx=0x7f6e10005270 comp rx=0 tx=0).stop 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 shutdown_connections 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee673e750 0x55dee673c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735b70 0x55dee673bb90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee6757340 0x55dee6759760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/3889026958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735250 0x55dee673b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e08ff9640 0 client.17961 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 >> 172.21.15.100:0/3889026958 conn(0x55dee672f980 msgr2=0x55dee67316d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 shutdown_connections 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/3889026958 wait complete. 2024-03-12T05:23:47.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.421+0000 7f6e2a1b4740 20 client.17961 trim_cache size 0 max 16384 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 Processor -- start 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 -- start start 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 0x55dee673a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee673bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee672ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee6622130 con 0x55dee672f980 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee672b5f0 con 0x55dee6735b70 2024-03-12T05:23:47.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.434+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee675ea30 con 0x55dee6735250 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 0x55dee673a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 0x55dee673a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48756/0 (socket says 172.21.15.100:48756) 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee673bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 -- 172.21.15.100:0/2771714461 learned_addr learned my addr 172.21.15.100:0/2771714461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee672ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 -- 172.21.15.100:0/2771714461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 msgr2=0x55dee673bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee673bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 -- 172.21.15.100:0/2771714461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 msgr2=0x55dee672ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee672ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 -- 172.21.15.100:0/2771714461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dee6637c20 con 0x55dee672f980 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 0x55dee673a7f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005ab0 tx=0x7f6e1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861a15ba9cab0c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e0affd640 1 -- 172.21.15.100:0/2771714461 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c004330 con 0x55dee672f980 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e0affd640 1 -- 172.21.15.100:0/2771714461 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e1c004490 con 0x55dee672f980 2024-03-12T05:23:47.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.435+0000 7f6e0affd640 1 -- 172.21.15.100:0/2771714461 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0055e0 con 0x55dee672f980 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2771714461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 msgr2=0x55dee673a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 0x55dee673a7f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005ab0 tx=0x7f6e1c009260 comp rx=0 tx=0).stop 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2771714461 shutdown_connections 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee672ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee6735250 0x55dee673bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2771714461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee672f980 0x55dee673a7f0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2771714461 >> 172.21.15.100:0/2771714461 conn(0x55dee672ae80 msgr2=0x55dee672e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2771714461 shutdown_connections 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2771714461 wait complete. 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.436+0000 7f6e2a1b4740 1 -- start start 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 Processor -- start 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee672f980 0x55dee67a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 0x55dee67a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee674d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee674d870 con 0x55dee6735250 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee674d9e0 con 0x55dee6735b70 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dee674dba0 con 0x55dee672f980 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2a1b4740 10 client.0 _ll_register_callbacks cb 0x7f6e1000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 0x55dee67a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e24a9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee672f980 0x55dee67a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 0x55dee67a7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48772/0 (socket says 172.21.15.100:48772) 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 -- 172.21.15.100:0/2249060932 learned_addr learned my addr 172.21.15.100:0/2249060932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee674d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 -- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee672f980 msgr2=0x55dee67a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee672f980 0x55dee67a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 -- 172.21.15.100:0/2249060932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 msgr2=0x55dee674d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee674d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.437+0000 7f6e2529e640 1 -- 172.21.15.100:0/2249060932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e20110a10 con 0x55dee6735250 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2529e640 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 0x55dee67a7140 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6e10000c00 tx=0x7f6e10003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fa8bda12005381 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e08ff9640 1 -- 172.21.15.100:0/2249060932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1001f040 con 0x55dee6735250 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e08ff9640 1 -- 172.21.15.100:0/2249060932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e10017040 con 0x55dee6735250 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e10015070 con 0x55dee6735250 2024-03-12T05:23:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e08ff9640 1 -- 172.21.15.100:0/2249060932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e10004d30 con 0x55dee6735250 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 10 client.17967 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dee665d8c0 con 0x55dee6735250 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e08ff9640 1 -- 172.21.15.100:0/2249060932 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e10015070 con 0x55dee6735250 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e08ff9640 1 -- 172.21.15.100:0/2249060932 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e1001bb90 con 0x55dee6735250 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 10 client.17967 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 10 client.17967 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 20 client.17967 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee673e750 0x55dee67b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 4 client.17967 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.438+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dee6428890 con 0x55dee673e750 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee673e750 0x55dee67b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e08ff9640 10 client.17967 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e24a9d640 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee673e750 0x55dee67b1890 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b543e05a216faf23 server_cookie=86aca6c0c73f0ad4 in_seq=0 out_seq=0 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e08ff9640 1 -- 172.21.15.100:0/2249060932 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dee6428890 con 0x55dee673e750 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e08ff9640 10 client.17967 handle_command_reply: tid=0 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 client.17967 shutdown 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 msgr2=0x55dee67a7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 0x55dee67a7140 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6e10000c00 tx=0x7f6e10003e80 comp rx=0 tx=0).stop 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 shutdown_connections 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dee673e750 0x55dee67b1890 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e08ff9640 0 client.17967 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dee6735b70 0x55dee674d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dee6735250 0x55dee67a7140 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 --2- 172.21.15.100:0/2249060932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dee672f980 0x55dee67a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 >> 172.21.15.100:0/2249060932 conn(0x55dee672ae80 msgr2=0x55dee672eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 shutdown_connections 2024-03-12T05:23:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 1 -- 172.21.15.100:0/2249060932 wait complete. 2024-03-12T05:23:47.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.439+0000 7f6e2a1b4740 20 client.17967 trim_cache size 0 max 16384 2024-03-12T05:23:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6e0c0a1ce0 msgr2=0x7f6e0c0a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6e0c0a1ce0 0x7f6e0c0a41a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009f90 tx=0x7f6e180124e0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102420 msgr2=0x7f6e20199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102420 0x7f6e20199860 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800be20 tx=0x7f6e1800bf10 comp rx=0 tx=0).stop 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 shutdown_connections 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6e0c0a1ce0 0x7f6e0c0a41a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e20102420 0x7f6e20199860 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e20102040 0x7f6e2019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 --2- 172.21.15.100:0/399469386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e20100180 0x7f6e2019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 >> 172.21.15.100:0/399469386 conn(0x7f6e200765a0 msgr2=0x7f6e200fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 shutdown_connections 2024-03-12T05:23:47.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.440+0000 7f6e27502640 1 -- 172.21.15.100:0/399469386 wait complete. 2024-03-12T05:23:47.446 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:23:47.447 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 -- 172.21.15.100:0/2308746335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974100c10 msgr2=0x7fe974104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 --2- 172.21.15.100:0/2308746335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974100c10 0x7fe974104c50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe964009930 tx=0x7fe96402bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 -- 172.21.15.100:0/2308746335 shutdown_connections 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 --2- 172.21.15.100:0/2308746335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974105330 0x7fe97410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 --2- 172.21.15.100:0/2308746335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974100c10 0x7fe974104c50 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 --2- 172.21.15.100:0/2308746335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974100260 0x7fe974100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.560+0000 7fe97bcae640 1 -- 172.21.15.100:0/2308746335 >> 172.21.15.100:0/2308746335 conn(0x7fe974077180 msgr2=0x7fe974077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 -- 172.21.15.100:0/2308746335 shutdown_connections 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 -- 172.21.15.100:0/2308746335 wait complete. 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 Processor -- start 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 -- start start 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974100260 0x7fe97419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 0x7fe97419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974197600 0x7fe974197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974197ff0 con 0x7fe974105330 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974198160 con 0x7fe974100260 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe97bcae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974198320 con 0x7fe974197600 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe979249640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 0x7fe97419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe979a4a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974100260 0x7fe97419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe979249640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 0x7fe97419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48782/0 (socket says 172.21.15.100:48782) 2024-03-12T05:23:47.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.561+0000 7fe979249640 1 -- 172.21.15.100:0/1700618096 learned_addr learned my addr 172.21.15.100:0/1700618096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe97a24b640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974197600 0x7fe974197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979249640 1 -- 172.21.15.100:0/1700618096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974197600 msgr2=0x7fe974197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979249640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974197600 0x7fe974197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979249640 1 -- 172.21.15.100:0/1700618096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974100260 msgr2=0x7fe97419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979249640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974100260 0x7fe97419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979249640 1 -- 172.21.15.100:0/1700618096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964009560 con 0x7fe974105330 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979a4a640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974100260 0x7fe97419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe979249640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 0x7fe97419d990 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe964009230 tx=0x7fe96402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe96affd640 1 -- 172.21.15.100:0/1700618096 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964004350 con 0x7fe974105330 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe96affd640 1 -- 172.21.15.100:0/1700618096 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9640044b0 con 0x7fe974105330 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9741a4310 con 0x7fe974105330 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe96affd640 1 -- 172.21.15.100:0/1700618096 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964003da0 con 0x7fe974105330 2024-03-12T05:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.562+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741a47c0 con 0x7fe974105330 2024-03-12T05:23:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.564+0000 7fe96affd640 1 -- 172.21.15.100:0/1700618096 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe964004df0 con 0x7fe974105330 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 Processor -- start 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 -- start start 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe96affd640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe9600a1ba0 0x7fe9600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d725f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 0x55a9d726c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d726cb60 0x55a9d726ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d721a960 con 0x55a9d725f8a0 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d71ce310 con 0x55a9d726cb60 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97e522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d71cb840 con 0x55a9d725ef80 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97a24b640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe9600a1ba0 0x7fe9600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe979249640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 0x55a9d726c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe979a4a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d725f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe979a4a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d725f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:53120/0 (socket says 172.21.15.100:53120) 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe979a4a640 1 -- 172.21.15.100:0/3743825717 learned_addr learned my addr 172.21.15.100:0/3743825717 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe979249640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 0x55a9d726c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48786/0 (socket says 172.21.15.100:48786) 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe97a24b640 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d726cb60 0x55a9d726ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.565+0000 7fe96affd640 1 -- 172.21.15.100:0/1700618096 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe9640ef720 con 0x7fe974105330 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979a4a640 1 -- 172.21.15.100:0/3743825717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d726cb60 msgr2=0x55a9d726ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979a4a640 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d726cb60 0x55a9d726ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979a4a640 1 -- 172.21.15.100:0/3743825717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 msgr2=0x55a9d726c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979a4a640 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 0x55a9d726c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979a4a640 1 -- 172.21.15.100:0/3743825717 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9d7189060 con 0x55a9d725ef80 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979249640 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 0x55a9d726c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe97a24b640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe9600a1ba0 0x7fe9600a4060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005fd0 tx=0x7fe96c009e90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe979a4a640 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d725f360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe9700115c0 tx=0x7fe9700119c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe90097f4aa6382 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe95ffff640 1 -- 172.21.15.100:0/3743825717 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97000f540 con 0x55a9d725ef80 2024-03-12T05:23:47.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe95ffff640 1 -- 172.21.15.100:0/3743825717 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe97000fb80 con 0x55a9d725ef80 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.566+0000 7fe95ffff640 1 -- 172.21.15.100:0/3743825717 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970019c50 con 0x55a9d725ef80 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 -- 172.21.15.100:0/3743825717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 msgr2=0x55a9d725f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d725f360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe9700115c0 tx=0x7fe9700119c0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 -- 172.21.15.100:0/3743825717 shutdown_connections 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d726cb60 0x55a9d726ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725f8a0 0x55a9d726c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 --2- 172.21.15.100:0/3743825717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d725f360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 -- 172.21.15.100:0/3743825717 >> 172.21.15.100:0/3743825717 conn(0x55a9d7259980 msgr2=0x55a9d7259d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 -- 172.21.15.100:0/3743825717 shutdown_connections 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 -- 172.21.15.100:0/3743825717 wait complete. 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 -- start start 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.567+0000 7fe97e522740 1 Processor -- start 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 0x55a9d7269b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d726cb60 0x55a9d726a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d71c7020 con 0x55a9d725ef80 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d6e9a430 con 0x55a9d725f8a0 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d6e56e00 con 0x55a9d726cb60 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97e522740 10 client.0 _ll_register_callbacks cb 0x55a9d716ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 0x55a9d7269b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979249640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 0x55a9d7269b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48788/0 (socket says 172.21.15.100:48788) 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979a4a640 1 -- 172.21.15.100:0/243556856 learned_addr learned my addr 172.21.15.100:0/243556856 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97a24b640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d726cb60 0x55a9d726a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979249640 1 -- 172.21.15.100:0/243556856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d726cb60 msgr2=0x55a9d726a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979249640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d726cb60 0x55a9d726a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979249640 1 -- 172.21.15.100:0/243556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 msgr2=0x55a9d7269b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979249640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 0x55a9d7269b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979249640 1 -- 172.21.15.100:0/243556856 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970009d40 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe97a24b640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d726cb60 0x55a9d726a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.568+0000 7fe979a4a640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 0x55a9d7269b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe979249640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9640ecf40 tx=0x7fe96404d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2234fb4f00221c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe95d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe95d7fa640 1 -- 172.21.15.100:0/243556856 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640bbd80 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe95d7fa640 1 -- 172.21.15.100:0/243556856 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe9640bcb20 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9d6ec67e0 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe95d7fa640 1 -- 172.21.15.100:0/243556856 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe964085d20 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 10 client.27350 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a9d71c56a0 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe95d7fa640 1 -- 172.21.15.100:0/243556856 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a9d6ec67e0 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe95d7fa640 1 -- 172.21.15.100:0/243556856 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe96403c070 con 0x55a9d725f8a0 2024-03-12T05:23:47.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 10 client.27350 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 10 client.27350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 20 client.27350 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d727f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 4 client.27350 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9d6f03720 con 0x55a9d727ccc0 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.569+0000 7fe979a4a640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d727f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.570+0000 7fe95d7fa640 10 client.27350 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.570+0000 7fe979a4a640 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d727f0b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1356cdcb0269021 server_cookie=5af103c49e9a49d4 in_seq=0 out_seq=0 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.570+0000 7fe95d7fa640 1 -- 172.21.15.100:0/243556856 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9d6f03720 con 0x55a9d727ccc0 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.570+0000 7fe95d7fa640 10 client.27350 handle_command_reply: tid=0 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.570+0000 7fe97e522740 1 client.27350 shutdown 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 msgr2=0x55a9d726a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9640ecf40 tx=0x7fe96404d000 comp rx=0 tx=0).stop 2024-03-12T05:23:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 shutdown_connections 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d727f0b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d726cb60 0x55a9d726a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 --2- 172.21.15.100:0/243556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d725ef80 0x55a9d7269b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe95d7fa640 0 client.27350 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 >> 172.21.15.100:0/243556856 conn(0x55a9d7259980 msgr2=0x55a9d725b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 shutdown_connections 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 1 -- 172.21.15.100:0/243556856 wait complete. 2024-03-12T05:23:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.571+0000 7fe97e522740 20 client.27350 trim_cache size 0 max 16384 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 Processor -- start 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 -- start start 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d726c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7264ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d714c130 con 0x55a9d726cb60 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d725e390 con 0x55a9d725f8a0 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97e522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d72555f0 con 0x55a9d725ef80 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7264ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7264ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48790/0 (socket says 172.21.15.100:48790) 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 -- 172.21.15.100:0/1874028894 learned_addr learned my addr 172.21.15.100:0/1874028894 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe979a4a640 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d726c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe979249640 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 -- 172.21.15.100:0/1874028894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 msgr2=0x55a9d726c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d726c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 -- 172.21.15.100:0/1874028894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 msgr2=0x55a9d726a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.583+0000 7fe97a24b640 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.584+0000 7fe97a24b640 1 -- 172.21.15.100:0/1874028894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9d7161c20 con 0x55a9d726cb60 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.584+0000 7fe979a4a640 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d726c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.584+0000 7fe97a24b640 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7264ea0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009bb0 tx=0x7fe96c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83d057b7b7598f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.584+0000 7fe95f7fe640 1 -- 172.21.15.100:0/1874028894 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c016050 con 0x55a9d726cb60 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.584+0000 7fe95f7fe640 1 -- 172.21.15.100:0/1874028894 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe96c003e00 con 0x55a9d726cb60 2024-03-12T05:23:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.584+0000 7fe95f7fe640 1 -- 172.21.15.100:0/1874028894 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c01fde0 con 0x55a9d726cb60 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 -- 172.21.15.100:0/1874028894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 msgr2=0x55a9d7264ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7264ea0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009bb0 tx=0x7fe96c021000 comp rx=0 tx=0).stop 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 -- 172.21.15.100:0/1874028894 shutdown_connections 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7264ea0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d726a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 --2- 172.21.15.100:0/1874028894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d726c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 -- 172.21.15.100:0/1874028894 >> 172.21.15.100:0/1874028894 conn(0x55a9d7259980 msgr2=0x55a9d7254a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 -- 172.21.15.100:0/1874028894 shutdown_connections 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 -- 172.21.15.100:0/1874028894 wait complete. 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 -- start start 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.585+0000 7fe97e522740 1 Processor -- start 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d7272320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d7272860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7272da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d72742c0 con 0x55a9d726cb60 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d72d31e0 con 0x55a9d725f8a0 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d72d3320 con 0x55a9d725ef80 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 10 client.0 _ll_register_callbacks cb 0x7fe970009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7272da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe979a4a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d7272320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7272da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48794/0 (socket says 172.21.15.100:48794) 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 -- 172.21.15.100:0/3640960485 learned_addr learned my addr 172.21.15.100:0/3640960485 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe979249640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d7272860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 -- 172.21.15.100:0/3640960485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 msgr2=0x55a9d7272320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d7272320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 -- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 msgr2=0x55a9d7272860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d7272860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 -- 172.21.15.100:0/3640960485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9d7189060 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe979249640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d7272860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97a24b640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7272da0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe96c01fa00 tx=0x7fe96c0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bccb240c868075 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe95d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe95d7fa640 1 -- 172.21.15.100:0/3640960485 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c004850 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe95d7fa640 1 -- 172.21.15.100:0/3640960485 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe96c004b20 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.586+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe97000fb80 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe95d7fa640 1 -- 172.21.15.100:0/3640960485 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c016020 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 10 client.17985 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe96c019040 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe95d7fa640 1 -- 172.21.15.100:0/3640960485 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe97000fb80 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe95d7fa640 1 -- 172.21.15.100:0/3640960485 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe96c005320 con 0x55a9d726cb60 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 10 client.17985 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 20 client.17985 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d72d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 4 client.17985 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9d6f52890 con 0x55a9d727ccc0 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.587+0000 7fe979a4a640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d72d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe95d7fa640 10 client.17985 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe979a4a640 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d72d6b20 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d0857d60993daa server_cookie=9cce29e15308ec12 in_seq=0 out_seq=0 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe95d7fa640 1 -- 172.21.15.100:0/3640960485 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9d6f52890 con 0x55a9d727ccc0 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe95d7fa640 10 client.17985 handle_command_reply: tid=0 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 client.17985 shutdown 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 msgr2=0x55a9d7272da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7272da0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe96c01fa00 tx=0x7fe96c0094e0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 shutdown_connections 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a9d727ccc0 0x55a9d72d6b20 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9d726cb60 0x55a9d7272da0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe95d7fa640 0 client.17985 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9d725f8a0 0x55a9d7272860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 --2- 172.21.15.100:0/3640960485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9d725ef80 0x55a9d7272320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 >> 172.21.15.100:0/3640960485 conn(0x55a9d7259980 msgr2=0x55a9d7264ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 shutdown_connections 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 1 -- 172.21.15.100:0/3640960485 wait complete. 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.588+0000 7fe97e522740 20 client.17985 trim_cache size 0 max 16384 2024-03-12T05:23:47.589 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:23:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe9600a1ba0 msgr2=0x7fe9600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe9600a1ba0 0x7fe9600a4060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005fd0 tx=0x7fe96c009e90 comp rx=0 tx=0).stop 2024-03-12T05:23:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 msgr2=0x7fe97419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 0x7fe97419d990 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe964009230 tx=0x7fe96402c680 comp rx=0 tx=0).stop 2024-03-12T05:23:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 shutdown_connections 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe9600a1ba0 0x7fe9600a4060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974197600 0x7fe974197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe974105330 0x7fe97419d990 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 --2- 172.21.15.100:0/1700618096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974100260 0x7fe97419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 >> 172.21.15.100:0/1700618096 conn(0x7fe974077180 msgr2=0x7fe974068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 shutdown_connections 2024-03-12T05:23:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:47.589+0000 7fe97bcae640 1 -- 172.21.15.100:0/1700618096 wait complete. 2024-03-12T05:23:47.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1414048 11.77 MB/sec execute 511 sec latency 84.325 ms 2024-03-12T05:23:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:47 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1619604002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:23:48.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:47 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1619604002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:23:48.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:47 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1619604002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:23:48.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:23:48.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:23:48.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:23:48.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 -- 172.21.15.100:0/3952539337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228106450 msgr2=0x7f622810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 --2- 172.21.15.100:0/3952539337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228106450 0x7f622810e020 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6218009930 tx=0x7f621802bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:48.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 -- 172.21.15.100:0/3952539337 shutdown_connections 2024-03-12T05:23:48.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 --2- 172.21.15.100:0/3952539337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228106450 0x7f622810e020 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 --2- 172.21.15.100:0/3952539337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228101d30 0x7f6228105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 --2- 172.21.15.100:0/3952539337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6228101380 0x7f6228101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 -- 172.21.15.100:0/3952539337 >> 172.21.15.100:0/3952539337 conn(0x7f6228078580 msgr2=0x7f6228078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 -- 172.21.15.100:0/3952539337 shutdown_connections 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.712+0000 7f622d70c640 1 -- 172.21.15.100:0/3952539337 wait complete. 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 Processor -- start 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 -- start start 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 0x7f622819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228106450 0x7f622819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62281975c0 0x7f6228197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228197fb0 con 0x7f6228101380 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228198120 con 0x7f6228106450 2024-03-12T05:23:48.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f622d70c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62281982e0 con 0x7f62281975c0 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 0x7f622819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 0x7f622819d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48802/0 (socket says 172.21.15.100:48802) 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f6226ffd640 1 -- 172.21.15.100:0/2888168725 learned_addr learned my addr 172.21.15.100:0/2888168725 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f62267fc640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228106450 0x7f622819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f62277fe640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62281975c0 0x7f6228197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f6226ffd640 1 -- 172.21.15.100:0/2888168725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62281975c0 msgr2=0x7f6228197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.713+0000 7f6226ffd640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62281975c0 0x7f6228197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f6226ffd640 1 -- 172.21.15.100:0/2888168725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228106450 msgr2=0x7f622819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f6226ffd640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228106450 0x7f622819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f6226ffd640 1 -- 172.21.15.100:0/2888168725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6218009560 con 0x7f6228101380 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f6226ffd640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 0x7f622819d410 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f622000eb90 tx=0x7f622000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f621ffff640 1 -- 172.21.15.100:0/2888168725 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200099b0 con 0x7f6228101380 2024-03-12T05:23:48.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62281a42d0 con 0x7f6228101380 2024-03-12T05:23:48.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f621ffff640 1 -- 172.21.15.100:0/2888168725 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622000ee90 con 0x7f6228101380 2024-03-12T05:23:48.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f621ffff640 1 -- 172.21.15.100:0/2888168725 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200107a0 con 0x7f6228101380 2024-03-12T05:23:48.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.714+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281a47e0 con 0x7f6228101380 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.716+0000 7f621ffff640 1 -- 172.21.15.100:0/2888168725 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6220005020 con 0x7f6228101380 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 Processor -- start 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 -- start start 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f621ffff640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62140a1ba0 0x7f62140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62277fe640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62140a1ba0 0x7f62140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f621ffff640 1 -- 172.21.15.100:0/2888168725 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6220014070 con 0x7f6228101380 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x558686ff7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7d30 0x5586870002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687002d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686f65f90 con 0x558686ff7380 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686f634c0 con 0x5586870008b0 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f62303bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686f5eca0 con 0x558686ff7d30 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x558686ff7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x558686ff7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48808/0 (socket says 172.21.15.100:48808) 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.717+0000 7f6226ffd640 1 -- 172.21.15.100:0/4056520875 learned_addr learned my addr 172.21.15.100:0/4056520875 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f62277fe640 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687002d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f62267fc640 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7d30 0x5586870002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f6226ffd640 1 -- 172.21.15.100:0/4056520875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7d30 msgr2=0x5586870002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f6226ffd640 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7d30 0x5586870002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f62277fe640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62140a1ba0 0x7f62140a4060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6218005940 tx=0x7f62180046c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f6226ffd640 1 -- 172.21.15.100:0/4056520875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 msgr2=0x558687002d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f6226ffd640 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687002d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f6226ffd640 1 -- 172.21.15.100:0/4056520875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558686f210b0 con 0x558686ff7380 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f62267fc640 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7d30 0x5586870002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f6226ffd640 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x558686ff7760 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f62200a2e70 tx=0x7f622009a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86712b3ac5d8ca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f621d7fa640 1 -- 172.21.15.100:0/4056520875 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200a33a0 con 0x558686ff7380 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f621d7fa640 1 -- 172.21.15.100:0/4056520875 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6220098f00 con 0x558686ff7380 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.718+0000 7f621d7fa640 1 -- 172.21.15.100:0/4056520875 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622009a2b0 con 0x558686ff7380 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- 172.21.15.100:0/4056520875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 msgr2=0x558686ff7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x558686ff7760 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f62200a2e70 tx=0x7f622009a620 comp rx=0 tx=0).stop 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- 172.21.15.100:0/4056520875 shutdown_connections 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687002d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7d30 0x5586870002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- 172.21.15.100:0/4056520875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x558686ff7760 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- 172.21.15.100:0/4056520875 >> 172.21.15.100:0/4056520875 conn(0x558686ff1840 msgr2=0x558686ff1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- 172.21.15.100:0/4056520875 shutdown_connections 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- 172.21.15.100:0/4056520875 wait complete. 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- start start 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 Processor -- start 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558686ffe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686f03f40 con 0x558686ff7d30 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686c324b0 con 0x5586870008b0 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.719+0000 7f62303bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686ee4670 con 0x558686ff7380 2024-03-12T05:23:48.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62303bf740 10 client.0 _ll_register_callbacks cb 0x558686f51f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:48.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48820/0 (socket says 172.21.15.100:48820) 2024-03-12T05:23:48.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 -- 172.21.15.100:0/3078620883 learned_addr learned my addr 172.21.15.100:0/3078620883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f6226ffd640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62277fe640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558686ffe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 -- 172.21.15.100:0/3078620883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 msgr2=0x558686ffd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 -- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 msgr2=0x558686ffe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558686ffe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 -- 172.21.15.100:0/3078620883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281a47e0 con 0x558686ff7d30 2024-03-12T05:23:48.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62267fc640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffdde0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f622806aa20 tx=0x7f6210013b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b3f1ec26d0fe43 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f620e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:48.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f620e7fc640 1 -- 172.21.15.100:0/3078620883 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100119c0 con 0x558686ff7d30 2024-03-12T05:23:48.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f620e7fc640 1 -- 172.21.15.100:0/3078620883 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6210011b20 con 0x558686ff7d30 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558686f38f40 con 0x558686ff7d30 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f620e7fc640 1 -- 172.21.15.100:0/3078620883 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6210011ce0 con 0x558686ff7d30 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62303bf740 10 client.17997 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558686f5d320 con 0x558686ff7d30 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.720+0000 7f620e7fc640 1 -- 172.21.15.100:0/3078620883 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558686f38f40 con 0x558686ff7d30 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f620e7fc640 1 -- 172.21.15.100:0/3078620883 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f621001c6a0 con 0x558686ff7d30 2024-03-12T05:23:48.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f62303bf740 10 client.17997 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:48.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f62303bf740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:48.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f62303bf740 20 client.17997 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:48.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f62303bf740 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687010ff0 0x558687013410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f62303bf740 4 client.17997 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:48.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558686f55310 con 0x558687010ff0 2024-03-12T05:23:48.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f6226ffd640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687010ff0 0x558687013410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f6226ffd640 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687010ff0 0x558687013410 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b882e3e3507614b7 server_cookie=ed03673f40d4da76 in_seq=0 out_seq=0 2024-03-12T05:23:48.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.721+0000 7f620e7fc640 10 client.17997 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f620e7fc640 1 -- 172.21.15.100:0/3078620883 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558686f55310 con 0x558687010ff0 2024-03-12T05:23:48.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f620e7fc640 10 client.17997 handle_command_reply: tid=0 2024-03-12T05:23:48.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 client.17997 shutdown 2024-03-12T05:23:48.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 msgr2=0x558686ffdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffdde0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f622806aa20 tx=0x7f6210013b70 comp rx=0 tx=0).stop 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 shutdown_connections 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687010ff0 0x558687013410 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558686ffe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f620e7fc640 0 client.17997 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffdde0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 --2- 172.21.15.100:0/3078620883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 >> 172.21.15.100:0/3078620883 conn(0x558686ff1840 msgr2=0x558686ff37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 shutdown_connections 2024-03-12T05:23:48.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 1 -- 172.21.15.100:0/3078620883 wait complete. 2024-03-12T05:23:48.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.722+0000 7f62303bf740 20 client.17997 trim_cache size 0 max 16384 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.738+0000 7f62303bf740 1 Processor -- start 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.738+0000 7f62303bf740 1 -- start start 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62303bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62303bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62303bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687017d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62303bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686fed180 con 0x558686ff7d30 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62303bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686ff94b0 con 0x5586870008b0 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62303bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686ffbdf0 con 0x558686ff7380 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687017d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48822/0 (socket says 172.21.15.100:48822) 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 -- 172.21.15.100:0/1512605981 learned_addr learned my addr 172.21.15.100:0/1512605981 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 -- 172.21.15.100:0/1512605981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 msgr2=0x558686ffffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 -- 172.21.15.100:0/1512605981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 msgr2=0x558687017d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687017d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 -- 172.21.15.100:0/1512605981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558686ef9c00 con 0x558686ff7d30 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f6226ffd640 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f62267fc640 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffe140 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6210001eb0 tx=0x7f62100097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e300aafc75b45d79 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f621cff9640 1 -- 172.21.15.100:0/1512605981 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210005680 con 0x558686ff7d30 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f621cff9640 1 -- 172.21.15.100:0/1512605981 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62100119c0 con 0x558686ff7d30 2024-03-12T05:23:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.739+0000 7f621cff9640 1 -- 172.21.15.100:0/1512605981 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210003af0 con 0x558686ff7d30 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 -- 172.21.15.100:0/1512605981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 msgr2=0x558686ffe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffe140 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6210001eb0 tx=0x7f62100097a0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 -- 172.21.15.100:0/1512605981 shutdown_connections 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586870008b0 0x558687017d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7d30 0x558686ffe140 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 --2- 172.21.15.100:0/1512605981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558686ff7380 0x558686ffffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 -- 172.21.15.100:0/1512605981 >> 172.21.15.100:0/1512605981 conn(0x558686ff1840 msgr2=0x558686ff8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 -- 172.21.15.100:0/1512605981 shutdown_connections 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.740+0000 7f62303bf740 1 -- 172.21.15.100:0/1512605981 wait complete. 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 -- start start 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 Processor -- start 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x5586870065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558686ff7d30 0x558687006b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586870008b0 0x5586870614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586870072e0 con 0x558686ff7380 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558687007450 con 0x558686ff7d30 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558687061a40 con 0x5586870008b0 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62303bf740 10 client.0 _ll_register_callbacks cb 0x7f6220004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x5586870065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x5586870065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48834/0 (socket says 172.21.15.100:48834) 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 -- 172.21.15.100:0/2397512301 learned_addr learned my addr 172.21.15.100:0/2397512301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62267fc640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558686ff7d30 0x558687006b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f62277fe640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586870008b0 0x5586870614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 -- 172.21.15.100:0/2397512301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586870008b0 msgr2=0x5586870614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586870008b0 0x5586870614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 -- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558686ff7d30 msgr2=0x558687006b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558686ff7d30 0x558687006b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 -- 172.21.15.100:0/2397512301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558686f210b0 con 0x558686ff7380 2024-03-12T05:23:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f6226ffd640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x5586870065d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f622009ab90 tx=0x7f622009ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292e23d303ac0188 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.741+0000 7f620e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f620e7fc640 1 -- 172.21.15.100:0/2397512301 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200cb9a0 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f620e7fc640 1 -- 172.21.15.100:0/2397512301 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62200cbb00 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558686f84900 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f620e7fc640 1 -- 172.21.15.100:0/2397512301 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f622009a050 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 10 client.18003 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558686f5d320 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f620e7fc640 1 -- 172.21.15.100:0/2397512301 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558686f84900 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f620e7fc640 1 -- 172.21.15.100:0/2397512301 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f622001f060 con 0x558686ff7380 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 10 client.18003 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 20 client.18003 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687062e20 0x55868706f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 4 client.18003 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558686c5e860 con 0x558687062e20 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.742+0000 7f62267fc640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687062e20 0x55868706f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.743+0000 7f62267fc640 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687062e20 0x55868706f910 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9161c945122bd8 server_cookie=a652b76be7c80104 in_seq=0 out_seq=0 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.743+0000 7f620e7fc640 10 client.18003 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f620e7fc640 1 -- 172.21.15.100:0/2397512301 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558686c5e860 con 0x558687062e20 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f620e7fc640 10 client.18003 handle_command_reply: tid=0 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 client.18003 shutdown 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 msgr2=0x5586870065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x5586870065d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f622009ab90 tx=0x7f622009ad40 comp rx=0 tx=0).stop 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 shutdown_connections 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558687062e20 0x55868706f910 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586870008b0 0x5586870614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558686ff7d30 0x558687006b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 --2- 172.21.15.100:0/2397512301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558686ff7380 0x5586870065d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f620e7fc640 0 client.18003 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 >> 172.21.15.100:0/2397512301 conn(0x558686ff1840 msgr2=0x558687017d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 shutdown_connections 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 1 -- 172.21.15.100:0/2397512301 wait complete. 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.744+0000 7f62303bf740 20 client.18003 trim_cache size 0 max 16384 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "532befc0-814e-4f04-bbd2-cfc6aecec0d7", 2024-03-12T05:23:48.748 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:23:48.749 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:23:48.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62140a1ba0 msgr2=0x7f62140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62140a1ba0 0x7f62140a4060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6218005940 tx=0x7f62180046c0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 msgr2=0x7f622819d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 0x7f622819d410 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f622000eb90 tx=0x7f622000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 shutdown_connections 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62140a1ba0 0x7f62140a4060 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62281975c0 0x7f6228197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6228106450 0x7f622819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 --2- 172.21.15.100:0/2888168725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6228101380 0x7f622819d410 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 >> 172.21.15.100:0/2888168725 conn(0x7f6228078580 msgr2=0x7f62280ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 shutdown_connections 2024-03-12T05:23:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.745+0000 7f622d70c640 1 -- 172.21.15.100:0/2888168725 wait complete. 2024-03-12T05:23:48.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:23:48.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:23:48.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1415646 11.76 MB/sec execute 512 sec latency 22.435 ms 2024-03-12T05:23:48.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 -- 172.21.15.100:0/1692760850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb80ff8f0 msgr2=0x7febb80ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 --2- 172.21.15.100:0/1692760850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb80ff8f0 0x7febb80ffcd0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7feba8009900 tx=0x7feba802bc30 comp rx=0 tx=0).stop 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 -- 172.21.15.100:0/1692760850 shutdown_connections 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 --2- 172.21.15.100:0/1692760850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8106500 0x7febb810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 --2- 172.21.15.100:0/1692760850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb81002a0 0x7febb8105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 --2- 172.21.15.100:0/1692760850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb80ff8f0 0x7febb80ffcd0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 -- 172.21.15.100:0/1692760850 >> 172.21.15.100:0/1692760850 conn(0x7febb8076640 msgr2=0x7febb80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 -- 172.21.15.100:0/1692760850 shutdown_connections 2024-03-12T05:23:48.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.865+0000 7febbd1b5640 1 -- 172.21.15.100:0/1692760850 wait complete. 2024-03-12T05:23:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 Processor -- start 2024-03-12T05:23:48.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 -- start start 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb80ff8f0 0x7febb819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8106500 0x7febb819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 0x7febb8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8197fe0 con 0x7febb81975f0 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8198150 con 0x7febb8106500 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febbd1b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8198310 con 0x7febb80ff8f0 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febb759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 0x7febb8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febb659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8106500 0x7febb819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febb6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb80ff8f0 0x7febb819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febb759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 0x7febb8197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48848/0 (socket says 172.21.15.100:48848) 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.866+0000 7febb759e640 1 -- 172.21.15.100:0/2569619557 learned_addr learned my addr 172.21.15.100:0/2569619557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb759e640 1 -- 172.21.15.100:0/2569619557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb80ff8f0 msgr2=0x7febb819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb759e640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb80ff8f0 0x7febb819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb759e640 1 -- 172.21.15.100:0/2569619557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8106500 msgr2=0x7febb819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb759e640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8106500 0x7febb819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb759e640 1 -- 172.21.15.100:0/2569619557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8009560 con 0x7febb81975f0 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb80ff8f0 0x7febb819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febb759e640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 0x7febb8197aa0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7febb0009fd0 tx=0x7febb000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7feb9ffff640 1 -- 172.21.15.100:0/2569619557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000bea0 con 0x7febb81975f0 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb81a4300 con 0x7febb81975f0 2024-03-12T05:23:48.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7feb9ffff640 1 -- 172.21.15.100:0/2569619557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb0009940 con 0x7febb81975f0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7feb9ffff640 1 -- 172.21.15.100:0/2569619557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb00106c0 con 0x7febb81975f0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.867+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb81a4810 con 0x7febb81975f0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.869+0000 7feb9ffff640 1 -- 172.21.15.100:0/2569619557 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7febb0004240 con 0x7febb81975f0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.869+0000 7febbfd89740 1 Processor -- start 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 -- start start 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e152b0 0x556848e15690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e1e7b0 0x556848e20ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848d84280 con 0x556848e1e7b0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848d817b0 con 0x556848e15bd0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febbfd89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848d7cf90 con 0x556848e152b0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7feb9ffff640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feba00a1ba0 0x7feba00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e1e7b0 0x556848e20ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:53096/0 (socket says 172.21.15.100:53096) 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb6d9d640 1 -- 172.21.15.100:0/2319886604 learned_addr learned my addr 172.21.15.100:0/2319886604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e152b0 0x556848e15690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb759e640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feba00a1ba0 0x7feba00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 -- 172.21.15.100:0/2319886604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e152b0 msgr2=0x556848e15690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7feb9ffff640 1 -- 172.21.15.100:0/2569619557 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7febb00cd4e0 con 0x7febb81975f0 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e152b0 0x556848e15690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 -- 172.21.15.100:0/2319886604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 msgr2=0x556848e1e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 -- 172.21.15.100:0/2319886604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556848d3b570 con 0x556848e1e7b0 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.870+0000 7febb659c640 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e1e7b0 0x556848e20ba0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7febac005d00 tx=0x7febac005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3ed3f610a3370a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febb759e640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feba00a1ba0 0x7feba00a4060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7febb00caca0 tx=0x7febb0004640 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7feb9d7fa640 1 -- 172.21.15.100:0/2319886604 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac012ca0 con 0x556848e1e7b0 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7feb9d7fa640 1 -- 172.21.15.100:0/2319886604 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febac012e00 con 0x556848e1e7b0 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7feb9d7fa640 1 -- 172.21.15.100:0/2319886604 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0049c0 con 0x556848e1e7b0 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 -- 172.21.15.100:0/2319886604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e1e7b0 msgr2=0x556848e20ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e1e7b0 0x556848e20ba0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7febac005d00 tx=0x7febac005c90 comp rx=0 tx=0).stop 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 -- 172.21.15.100:0/2319886604 shutdown_connections 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e1e7b0 0x556848e20ba0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 --2- 172.21.15.100:0/2319886604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e152b0 0x556848e15690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 -- 172.21.15.100:0/2319886604 >> 172.21.15.100:0/2319886604 conn(0x556848e0f990 msgr2=0x556848e0fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 -- 172.21.15.100:0/2319886604 shutdown_connections 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.871+0000 7febbfd89740 1 -- 172.21.15.100:0/2319886604 wait complete. 2024-03-12T05:23:48.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 -- start start 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 Processor -- start 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e1bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848a50430 con 0x556848e152b0 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848a0ce00 con 0x556848e15bd0 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848d01fd0 con 0x556848e1e7b0 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febbfd89740 10 client.0 _ll_register_callbacks cb 0x556848d57fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e1e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48862/0 (socket says 172.21.15.100:48862) 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 -- 172.21.15.100:0/2776453593 learned_addr learned my addr 172.21.15.100:0/2776453593 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e1bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 -- 172.21.15.100:0/2776453593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 msgr2=0x556848e1bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e1bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 -- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 msgr2=0x556848e1b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.872+0000 7febb659c640 1 -- 172.21.15.100:0/2776453593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac01a040 con 0x556848e152b0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febb659c640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e1e240 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7febac005880 tx=0x7febac0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b03d34364374b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7feb9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7feb9aff5640 1 -- 172.21.15.100:0/2776453593 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac005400 con 0x556848e152b0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7feb9aff5640 1 -- 172.21.15.100:0/2776453593 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7febac000e40 con 0x556848e152b0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556848a7c7e0 con 0x556848e152b0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7feb9aff5640 1 -- 172.21.15.100:0/2776453593 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7febac004260 con 0x556848e152b0 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 10 client.18015 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:48.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556848d7b610 con 0x556848e152b0 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7feb9aff5640 1 -- 172.21.15.100:0/2776453593 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556848a7c7e0 con 0x556848e152b0 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7feb9aff5640 1 -- 172.21.15.100:0/2776453593 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febac004920 con 0x556848e152b0 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 10 client.18015 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 10 client.18015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 20 client.18015 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e2e900 0x556848e30cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 4 client.18015 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.873+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556848ab9720 con 0x556848e2e900 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.874+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e2e900 0x556848e30cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.874+0000 7feb9aff5640 10 client.18015 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.874+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e2e900 0x556848e30cf0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2349ce8e78094830 server_cookie=c2ec270f30455d0 in_seq=0 out_seq=0 2024-03-12T05:23:48.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.874+0000 7feb9aff5640 1 -- 172.21.15.100:0/2776453593 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556848ab9720 con 0x556848e2e900 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.874+0000 7feb9aff5640 10 client.18015 handle_command_reply: tid=0 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.874+0000 7febbfd89740 1 client.18015 shutdown 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 msgr2=0x556848e1e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e1e240 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7febac005880 tx=0x7febac0056a0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 shutdown_connections 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e2e900 0x556848e30cf0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e1bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e1b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7feb9aff5640 0 client.18015 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 --2- 172.21.15.100:0/2776453593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e1e240 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 >> 172.21.15.100:0/2776453593 conn(0x556848e0f990 msgr2=0x556848e11730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 shutdown_connections 2024-03-12T05:23:48.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 1 -- 172.21.15.100:0/2776453593 wait complete. 2024-03-12T05:23:48.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.875+0000 7febbfd89740 20 client.18015 trim_cache size 0 max 16384 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.890+0000 7febbfd89740 1 Processor -- start 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.890+0000 7febbfd89740 1 -- start start 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febbfd89740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e152b0 0x556848e1dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febbfd89740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 0x556848e1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febbfd89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febbfd89740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848e20fa0 con 0x556848e15bd0 2024-03-12T05:23:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febbfd89740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848e16410 con 0x556848e152b0 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febbfd89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848e213f0 con 0x556848e1e7b0 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 0x556848e1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 0x556848e1e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48878/0 (socket says 172.21.15.100:48878) 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 -- 172.21.15.100:0/2279320763 learned_addr learned my addr 172.21.15.100:0/2279320763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb659c640 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb659c640 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e152b0 0x556848e1dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 -- 172.21.15.100:0/2279320763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 msgr2=0x556848e25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e25660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 -- 172.21.15.100:0/2279320763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e152b0 msgr2=0x556848e1dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e152b0 0x556848e1dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 -- 172.21.15.100:0/2279320763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556848d17ac0 con 0x556848e15bd0 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb659c640 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 0x556848e1e240 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7feba802c170 tx=0x7feba8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca69c0a4a064fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7feb9cff9640 1 -- 172.21.15.100:0/2279320763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8044040 con 0x556848e15bd0 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7feb9cff9640 1 -- 172.21.15.100:0/2279320763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feba8045040 con 0x556848e15bd0 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.891+0000 7feb9cff9640 1 -- 172.21.15.100:0/2279320763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8004df0 con 0x556848e15bd0 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 -- 172.21.15.100:0/2279320763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 msgr2=0x556848e1e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 0x556848e1e240 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7feba802c170 tx=0x7feba8005b50 comp rx=0 tx=0).stop 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 -- 172.21.15.100:0/2279320763 shutdown_connections 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e15bd0 0x556848e1e240 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 --2- 172.21.15.100:0/2279320763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e152b0 0x556848e1dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 -- 172.21.15.100:0/2279320763 >> 172.21.15.100:0/2279320763 conn(0x556848e0f990 msgr2=0x556848e168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 -- 172.21.15.100:0/2279320763 shutdown_connections 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 -- 172.21.15.100:0/2279320763 wait complete. 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 -- start start 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.892+0000 7febbfd89740 1 Processor -- start 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e23c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848e25690 con 0x556848e152b0 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848e25800 con 0x556848e15bd0 2024-03-12T05:23:48.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556848e259c0 con 0x556848e1e7b0 2024-03-12T05:23:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 10 client.0 _ll_register_callbacks cb 0x7febac00ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:23:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e23c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e23c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48880/0 (socket says 172.21.15.100:48880) 2024-03-12T05:23:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 -- 172.21.15.100:0/2578667074 learned_addr learned my addr 172.21.15.100:0/2578667074 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 -- 172.21.15.100:0/2578667074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 msgr2=0x556848e250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 -- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 msgr2=0x556848e241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 -- 172.21.15.100:0/2578667074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556848d7b610 con 0x556848e152b0 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febb659c640 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e23c60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7febac015040 tx=0x7febac0123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824eeca48deae642 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:23:48.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7feb9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7feb9aff5640 1 -- 172.21.15.100:0/2578667074 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00b4f0 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7feb9aff5640 1 -- 172.21.15.100:0/2578667074 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7febac000e40 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.893+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febac012e00 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7feb9aff5640 1 -- 172.21.15.100:0/2578667074 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7febac00bb30 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 10 client.18021 fetch_fsmap learned FSMap version 25 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7febb810e850 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7feb9aff5640 1 -- 172.21.15.100:0/2578667074 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febac012e00 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7feb9aff5640 1 -- 172.21.15.100:0/2578667074 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febac003ac0 con 0x556848e152b0 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 10 client.18021 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 20 client.18021 populate_metadata read hostname 'smithi100' 2024-03-12T05:23:48.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e85e30 0x556848e88250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 4 client.18021 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556848b08890 con 0x556848e85e30 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e85e30 0x556848e88250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7feb9aff5640 10 client.18021 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.894+0000 7febb6d9d640 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e85e30 0x556848e88250 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c1864893a89168 server_cookie=c0f4f2c69f0d174b in_seq=0 out_seq=0 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7feb9aff5640 1 -- 172.21.15.100:0/2578667074 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556848b08890 con 0x556848e85e30 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7feb9aff5640 10 client.18021 handle_command_reply: tid=0 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 client.18021 shutdown 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 msgr2=0x556848e23c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e23c60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7febac015040 tx=0x7febac0123f0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 shutdown_connections 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556848e85e30 0x556848e88250 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7feb9aff5640 0 client.18021 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:23:48.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556848e1e7b0 0x556848e250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556848e15bd0 0x556848e241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 --2- 172.21.15.100:0/2578667074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556848e152b0 0x556848e23c60 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 >> 172.21.15.100:0/2578667074 conn(0x556848e0f990 msgr2=0x556848e19e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 shutdown_connections 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 1 -- 172.21.15.100:0/2578667074 wait complete. 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbfd89740 20 client.18021 trim_cache size 0 max 16384 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:23:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "532befc0-814e-4f04-bbd2-cfc6aecec0d7": { 2024-03-12T05:23:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:23:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "532befc0-814e-4f04-bbd2-cfc6aecec0d7", 2024-03-12T05:23:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:23:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:23:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:23:48.901 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:23:48.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feba00a1ba0 msgr2=0x7feba00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbd1b5640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feba00a1ba0 0x7feba00a4060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7febb00caca0 tx=0x7febb0004640 comp rx=0 tx=0).stop 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 msgr2=0x7febb8197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbd1b5640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 0x7febb8197aa0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7febb0009fd0 tx=0x7febb000b260 comp rx=0 tx=0).stop 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.895+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 shutdown_connections 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feba00a1ba0 0x7feba00a4060 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febb81975f0 0x7febb8197aa0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8106500 0x7febb819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 --2- 172.21.15.100:0/2569619557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb80ff8f0 0x7febb819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:23:48.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 >> 172.21.15.100:0/2569619557 conn(0x7febb8076640 msgr2=0x7febb80fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:23:48.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 shutdown_connections 2024-03-12T05:23:48.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:23:48.896+0000 7febbd1b5640 1 -- 172.21.15.100:0/2569619557 wait complete. 2024-03-12T05:23:48.904 INFO:tasks.cephfs.filesystem:scrub status for tag:532befc0-814e-4f04-bbd2-cfc6aecec0d7 - {'path': '/', 'tag': '532befc0-814e-4f04-bbd2-cfc6aecec0d7', 'options': 'recursive,force'} 2024-03-12T05:23:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:48 smithi161 ceph-mon[36043]: pgmap v364: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 291 op/s 2024-03-12T05:23:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:48 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:23:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:48 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:48 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:23:49.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:48 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:48 smithi100 ceph-mon[32210]: pgmap v364: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 291 op/s 2024-03-12T05:23:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:48 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:23:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:48 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:48 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:23:49.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:48 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:48 smithi156 ceph-mon[36270]: pgmap v364: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 291 op/s 2024-03-12T05:23:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:48 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:23:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:48 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:48 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:23:49.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:48 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:49.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1418072 11.77 MB/sec execute 513 sec latency 13.615 ms 2024-03-12T05:23:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:49 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:23:50.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:49 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:50.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:49 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:23:50.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:49 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:49 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:23:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:49 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:23:50.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1420396 11.76 MB/sec execute 514 sec latency 17.435 ms 2024-03-12T05:23:51.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:50 smithi161 ceph-mon[36043]: pgmap v365: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-03-12T05:23:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:50 smithi100 ceph-mon[32210]: pgmap v365: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-03-12T05:23:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:50 smithi156 ceph-mon[36270]: pgmap v365: 97 pgs: 97 active+clean; 587 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-03-12T05:23:51.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1422136 11.76 MB/sec execute 515 sec latency 18.986 ms 2024-03-12T05:23:52.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1424157 11.76 MB/sec execute 516 sec latency 83.063 ms 2024-03-12T05:23:53.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:52 smithi161 ceph-mon[36043]: pgmap v366: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 280 op/s 2024-03-12T05:23:53.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:52 smithi100 ceph-mon[32210]: pgmap v366: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 280 op/s 2024-03-12T05:23:53.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:52 smithi156 ceph-mon[36270]: pgmap v366: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 280 op/s 2024-03-12T05:23:53.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1426175 11.76 MB/sec execute 517 sec latency 15.865 ms 2024-03-12T05:23:54.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1428709 11.76 MB/sec execute 518 sec latency 14.200 ms 2024-03-12T05:23:55.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:54 smithi161 ceph-mon[36043]: pgmap v367: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2024-03-12T05:23:55.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:54 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:23:55.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:54 smithi156 ceph-mon[36270]: pgmap v367: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2024-03-12T05:23:55.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:54 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:23:55.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:54 smithi100 ceph-mon[32210]: pgmap v367: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2024-03-12T05:23:55.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:54 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:23:55.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1431168 11.76 MB/sec execute 519 sec latency 17.269 ms 2024-03-12T05:23:56.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:55 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:23:56.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:55 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:23:56.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:55 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:23:56.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1433034 11.76 MB/sec execute 520 sec latency 19.187 ms 2024-03-12T05:23:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:56 smithi100 ceph-mon[32210]: pgmap v368: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2024-03-12T05:23:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:56 smithi156 ceph-mon[36270]: pgmap v368: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2024-03-12T05:23:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:56 smithi161 ceph-mon[36043]: pgmap v368: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 181 op/s 2024-03-12T05:23:57.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1435363 11.77 MB/sec execute 521 sec latency 122.799 ms 2024-03-12T05:23:58.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1437137 11.76 MB/sec execute 522 sec latency 14.118 ms 2024-03-12T05:23:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:23:58 smithi100 ceph-mon[32210]: pgmap v369: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 282 op/s 2024-03-12T05:23:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:23:58 smithi156 ceph-mon[36270]: pgmap v369: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 282 op/s 2024-03-12T05:23:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:23:58 smithi161 ceph-mon[36043]: pgmap v369: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 282 op/s 2024-03-12T05:23:59.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1439460 11.76 MB/sec execute 523 sec latency 16.939 ms 2024-03-12T05:24:00.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1441189 11.75 MB/sec execute 524 sec latency 23.469 ms 2024-03-12T05:24:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:00 smithi100 ceph-mon[32210]: pgmap v370: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-03-12T05:24:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:00 smithi156 ceph-mon[36270]: pgmap v370: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-03-12T05:24:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:00 smithi161 ceph-mon[36043]: pgmap v370: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-03-12T05:24:01.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1443048 11.75 MB/sec execute 525 sec latency 22.143 ms 2024-03-12T05:24:02.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1444811 11.74 MB/sec execute 526 sec latency 172.014 ms 2024-03-12T05:24:03.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:02 smithi100 ceph-mon[32210]: pgmap v371: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 312 op/s 2024-03-12T05:24:03.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:02 smithi156 ceph-mon[36270]: pgmap v371: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 312 op/s 2024-03-12T05:24:03.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:02 smithi161 ceph-mon[36043]: pgmap v371: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 312 op/s 2024-03-12T05:24:03.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1446736 11.74 MB/sec execute 527 sec latency 15.016 ms 2024-03-12T05:24:04.807 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1449037 11.74 MB/sec execute 528 sec latency 51.050 ms 2024-03-12T05:24:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:04 smithi100 ceph-mon[32210]: pgmap v372: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-03-12T05:24:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:04 smithi156 ceph-mon[36270]: pgmap v372: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-03-12T05:24:05.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:04 smithi161 ceph-mon[36043]: pgmap v372: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-03-12T05:24:05.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1451028 11.74 MB/sec execute 529 sec latency 16.063 ms 2024-03-12T05:24:06.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1453447 11.75 MB/sec execute 530 sec latency 19.586 ms 2024-03-12T05:24:07.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:06 smithi100 ceph-mon[32210]: pgmap v373: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-03-12T05:24:07.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:06 smithi156 ceph-mon[36270]: pgmap v373: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-03-12T05:24:07.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:06 smithi161 ceph-mon[36043]: pgmap v373: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-03-12T05:24:07.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1454813 11.74 MB/sec execute 531 sec latency 85.967 ms 2024-03-12T05:24:08.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:08 smithi100 ceph-mon[32210]: pgmap v374: 97 pgs: 97 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.6 MiB/s wr, 322 op/s 2024-03-12T05:24:08.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:08 smithi156 ceph-mon[36270]: pgmap v374: 97 pgs: 97 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.6 MiB/s wr, 322 op/s 2024-03-12T05:24:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:08 smithi161 ceph-mon[36043]: pgmap v374: 97 pgs: 97 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.6 MiB/s wr, 322 op/s 2024-03-12T05:24:08.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.74 MB/sec execute 532 sec latency 373.687 ms 2024-03-12T05:24:09.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.72 MB/sec execute 533 sec latency 1373.756 ms 2024-03-12T05:24:10.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.69 MB/sec execute 534 sec latency 2373.866 ms 2024-03-12T05:24:11.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:10 smithi100 ceph-mon[32210]: osdmap e100: 12 total, 12 up, 12 in 2024-03-12T05:24:11.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:10 smithi100 ceph-mon[32210]: pgmap v376: 97 pgs: 97 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-03-12T05:24:11.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:10 smithi161 ceph-mon[36043]: osdmap e100: 12 total, 12 up, 12 in 2024-03-12T05:24:11.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:10 smithi161 ceph-mon[36043]: pgmap v376: 97 pgs: 97 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-03-12T05:24:11.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:10 smithi156 ceph-mon[36270]: osdmap e100: 12 total, 12 up, 12 in 2024-03-12T05:24:11.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:10 smithi156 ceph-mon[36270]: pgmap v376: 97 pgs: 97 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-03-12T05:24:11.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.67 MB/sec execute 535 sec latency 3373.991 ms 2024-03-12T05:24:12.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.65 MB/sec execute 536 sec latency 4374.090 ms 2024-03-12T05:24:13.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:12 smithi161 ceph-mon[36043]: pgmap v377: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 181 op/s 2024-03-12T05:24:13.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:12 smithi100 ceph-mon[32210]: pgmap v377: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 181 op/s 2024-03-12T05:24:13.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:12 smithi156 ceph-mon[36270]: pgmap v377: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 181 op/s 2024-03-12T05:24:13.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.63 MB/sec execute 537 sec latency 5374.181 ms 2024-03-12T05:24:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.61 MB/sec execute 538 sec latency 6374.231 ms 2024-03-12T05:24:15.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:14 smithi161 ceph-mon[36043]: pgmap v378: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 181 op/s 2024-03-12T05:24:15.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:14 smithi100 ceph-mon[32210]: pgmap v378: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 181 op/s 2024-03-12T05:24:15.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:14 smithi156 ceph-mon[36270]: pgmap v378: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 181 op/s 2024-03-12T05:24:15.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.58 MB/sec execute 539 sec latency 7374.265 ms 2024-03-12T05:24:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.56 MB/sec execute 540 sec latency 8374.373 ms 2024-03-12T05:24:17.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:16 smithi156 ceph-mon[36270]: osdmap e101: 12 total, 12 up, 12 in 2024-03-12T05:24:17.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:16 smithi156 ceph-mon[36270]: pgmap v380: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 80 op/s 2024-03-12T05:24:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:16 smithi161 ceph-mon[36043]: osdmap e101: 12 total, 12 up, 12 in 2024-03-12T05:24:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:16 smithi161 ceph-mon[36043]: pgmap v380: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 80 op/s 2024-03-12T05:24:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:16 smithi100 ceph-mon[32210]: osdmap e101: 12 total, 12 up, 12 in 2024-03-12T05:24:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:16 smithi100 ceph-mon[32210]: pgmap v380: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 80 op/s 2024-03-12T05:24:17.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456607 11.54 MB/sec execute 541 sec latency 9374.455 ms 2024-03-12T05:24:18.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1456886 11.52 MB/sec execute 542 sec latency 10139.309 ms 2024-03-12T05:24:18.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:24:18.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 -- 172.21.15.100:0/1874081613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228105350 msgr2=0x7fe22810cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 --2- 172.21.15.100:0/1874081613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228105350 0x7fe22810cf20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fe220009cd0 tx=0x7fe22002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 -- 172.21.15.100:0/1874081613 shutdown_connections 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 --2- 172.21.15.100:0/1874081613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228105350 0x7fe22810cf20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 --2- 172.21.15.100:0/1874081613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe228100c30 0x7fe228104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 --2- 172.21.15.100:0/1874081613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe228100280 0x7fe228100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 -- 172.21.15.100:0/1874081613 >> 172.21.15.100:0/1874081613 conn(0x7fe228077170 msgr2=0x7fe228077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 -- 172.21.15.100:0/1874081613 shutdown_connections 2024-03-12T05:24:19.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 -- 172.21.15.100:0/1874081613 wait complete. 2024-03-12T05:24:19.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.031+0000 7fe22d288640 1 Processor -- start 2024-03-12T05:24:19.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 -- start start 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 0x7fe228196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe228105350 0x7fe228196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe22819af40 0x7fe22819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22819b930 con 0x7fe228100280 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22819baa0 con 0x7fe22819af40 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe22d288640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22819bc10 con 0x7fe228105350 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 0x7fe228196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 0x7fe228196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57792/0 (socket says 172.21.15.100:57792) 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 -- 172.21.15.100:0/3123728327 learned_addr learned my addr 172.21.15.100:0/3123728327 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe2277fe640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe22819af40 0x7fe22819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe2267fc640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe228105350 0x7fe228196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 -- 172.21.15.100:0/3123728327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe228105350 msgr2=0x7fe228196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe228105350 0x7fe228196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 -- 172.21.15.100:0/3123728327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe22819af40 msgr2=0x7fe22819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe22819af40 0x7fe22819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 -- 172.21.15.100:0/3123728327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009900 con 0x7fe228100280 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe226ffd640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 0x7fe228196680 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe21800e9f0 tx=0x7fe21800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.032+0000 7fe20ffff640 1 -- 172.21.15.100:0/3123728327 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218009700 con 0x7fe228100280 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.033+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe228069750 con 0x7fe228100280 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.033+0000 7fe20ffff640 1 -- 172.21.15.100:0/3123728327 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe21800cd80 con 0x7fe228100280 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.033+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe228069c50 con 0x7fe228100280 2024-03-12T05:24:19.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.033+0000 7fe20ffff640 1 -- 172.21.15.100:0/3123728327 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218018920 con 0x7fe228100280 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.035+0000 7fe20ffff640 1 -- 172.21.15.100:0/3123728327 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe218018a80 con 0x7fe228100280 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.035+0000 7fe22fe66740 1 Processor -- start 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.035+0000 7fe22fe66740 1 -- start start 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bc9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe20ffff640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2100a1ba0 0x7fe2100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe22fe66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2b38280 con 0x55fbf2bc92b0 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe22fe66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2b357b0 con 0x55fbf2bd27b0 2024-03-12T05:24:19.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe22fe66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2b30f90 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bc9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe226ffd640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2100a1ba0 0x7fe2100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bc9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57808/0 (socket says 172.21.15.100:57808) 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 -- 172.21.15.100:0/2314567630 learned_addr learned my addr 172.21.15.100:0/2314567630 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2267fc640 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe20ffff640 1 -- 172.21.15.100:0/3123728327 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe218014070 con 0x7fe228100280 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 -- 172.21.15.100:0/2314567630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 msgr2=0x55fbf2bd2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 -- 172.21.15.100:0/2314567630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 msgr2=0x55fbf2bd4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 -- 172.21.15.100:0/2314567630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbf2aef570 con 0x55fbf2bc92b0 2024-03-12T05:24:19.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2267fc640 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe226ffd640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2100a1ba0 0x7fe2100a4060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2180cb3a0 tx=0x7fe21808ff50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.036+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bc9690 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe220009890 tx=0x7fe220009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aa5fc3724424ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe20d7fa640 1 -- 172.21.15.100:0/2314567630 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220034640 con 0x55fbf2bc92b0 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe20d7fa640 1 -- 172.21.15.100:0/2314567630 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2200347a0 con 0x55fbf2bc92b0 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe20d7fa640 1 -- 172.21.15.100:0/2314567630 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220003bd0 con 0x55fbf2bc92b0 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 -- 172.21.15.100:0/2314567630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 msgr2=0x55fbf2bc9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bc9690 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe220009890 tx=0x7fe220009820 comp rx=0 tx=0).stop 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 -- 172.21.15.100:0/2314567630 shutdown_connections 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2314567630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bc9690 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 -- 172.21.15.100:0/2314567630 >> 172.21.15.100:0/2314567630 conn(0x55fbf2bc3990 msgr2=0x55fbf2bc3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 -- 172.21.15.100:0/2314567630 shutdown_connections 2024-03-12T05:24:19.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.037+0000 7fe22fe66740 1 -- 172.21.15.100:0/2314567630 wait complete. 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 -- start start 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 Processor -- start 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bcf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2804430 con 0x55fbf2bd27b0 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf27c0e00 con 0x55fbf2bc92b0 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2ab5fd0 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe22fe66740 10 client.0 _ll_register_callbacks cb 0x55fbf2b0bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57810/0 (socket says 172.21.15.100:57810) 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 -- 172.21.15.100:0/3310407222 learned_addr learned my addr 172.21.15.100:0/3310407222 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bcf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2267fc640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 -- 172.21.15.100:0/3310407222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 msgr2=0x55fbf2bcfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 -- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 msgr2=0x55fbf2bcf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bcf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.038+0000 7fe2277fe640 1 -- 172.21.15.100:0/3310407222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22810d730 con 0x55fbf2bd27b0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe2277fe640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd0090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe2200096c0 tx=0x7fe220032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b3dbb67b6c9e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe20aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe20aff5640 1 -- 172.21.15.100:0/3310407222 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220034c90 con 0x55fbf2bd27b0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe20aff5640 1 -- 172.21.15.100:0/3310407222 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe220034df0 con 0x55fbf2bd27b0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbf286d720 con 0x55fbf2bd27b0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe20aff5640 1 -- 172.21.15.100:0/3310407222 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe220055430 con 0x55fbf2bd27b0 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 10 client.18033 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fbf2b2f610 con 0x55fbf2bd27b0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe20aff5640 1 -- 172.21.15.100:0/3310407222 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe220061020 con 0x55fbf2bd27b0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe20aff5640 1 -- 172.21.15.100:0/3310407222 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe220050040 con 0x55fbf2bd27b0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 10 client.18033 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 20 client.18033 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2be25a0 0x55fbf2be4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 4 client.18033 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbf28bc890 con 0x55fbf2be25a0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.039+0000 7fe2267fc640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2be25a0 0x55fbf2be4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.040+0000 7fe20aff5640 10 client.18033 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.040+0000 7fe2267fc640 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2be25a0 0x55fbf2be4990 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4343a7c31091177 server_cookie=d407a155b1c70ed7 in_seq=0 out_seq=0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe20aff5640 1 -- 172.21.15.100:0/3310407222 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fbf28bc890 con 0x55fbf2be25a0 2024-03-12T05:24:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe20aff5640 10 client.18033 handle_command_reply: tid=0 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 client.18033 shutdown 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 msgr2=0x55fbf2bd0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd0090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe2200096c0 tx=0x7fe220032f90 comp rx=0 tx=0).stop 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 shutdown_connections 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2be25a0 0x55fbf2be4990 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd0090 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe20aff5640 0 client.18033 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 --2- 172.21.15.100:0/3310407222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bcf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 >> 172.21.15.100:0/3310407222 conn(0x55fbf2bc3990 msgr2=0x55fbf2bc5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 shutdown_connections 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 1 -- 172.21.15.100:0/3310407222 wait complete. 2024-03-12T05:24:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.041+0000 7fe22fe66740 20 client.18033 trim_cache size 0 max 16384 2024-03-12T05:24:19.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 Processor -- start 2024-03-12T05:24:19.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 -- start start 2024-03-12T05:24:19.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2bc5d70 con 0x55fbf2bd27b0 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2be9b70 con 0x55fbf2bc92b0 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe22fe66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2bbf2e0 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57814/0 (socket says 172.21.15.100:57814) 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 -- 172.21.15.100:0/2811612134 learned_addr learned my addr 172.21.15.100:0/2811612134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2267fc640 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 -- 172.21.15.100:0/2811612134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 msgr2=0x55fbf2bcc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 -- 172.21.15.100:0/2811612134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 msgr2=0x55fbf2bd1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2277fe640 1 -- 172.21.15.100:0/2811612134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbf2acbac0 con 0x55fbf2bd27b0 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.057+0000 7fe2267fc640 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe2277fe640 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bccdd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe220033000 tx=0x7fe220036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36f19729826ae06 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe20cff9640 1 -- 172.21.15.100:0/2811612134 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220043b40 con 0x55fbf2bd27b0 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe20cff9640 1 -- 172.21.15.100:0/2811612134 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe220061030 con 0x55fbf2bd27b0 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe20cff9640 1 -- 172.21.15.100:0/2811612134 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220055430 con 0x55fbf2bd27b0 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe22fe66740 1 -- 172.21.15.100:0/2811612134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 msgr2=0x55fbf2bccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bccdd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe220033000 tx=0x7fe220036040 comp rx=0 tx=0).stop 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe22fe66740 1 -- 172.21.15.100:0/2811612134 shutdown_connections 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bccdd0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bcc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.058+0000 7fe22fe66740 1 --2- 172.21.15.100:0/2811612134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- 172.21.15.100:0/2811612134 >> 172.21.15.100:0/2811612134 conn(0x55fbf2bc3990 msgr2=0x55fbf2bca910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- 172.21.15.100:0/2811612134 shutdown_connections 2024-03-12T05:24:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- 172.21.15.100:0/2811612134 wait complete. 2024-03-12T05:24:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- start start 2024-03-12T05:24:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 Processor -- start 2024-03-12T05:24:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2bd8970 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2bd8ae0 con 0x55fbf2bd27b0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbf2c38a40 con 0x55fbf2bc92b0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe22fe66740 10 client.0 _ll_register_callbacks cb 0x7fe218005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe2267fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe2267fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57820/0 (socket says 172.21.15.100:57820) 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.059+0000 7fe2267fc640 1 -- 172.21.15.100:0/580962106 learned_addr learned my addr 172.21.15.100:0/580962106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2277fe640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2277fe640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2267fc640 1 -- 172.21.15.100:0/580962106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc92b0 msgr2=0x55fbf2bd9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2267fc640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2267fc640 1 -- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 msgr2=0x55fbf2bd8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2267fc640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2267fc640 1 -- 172.21.15.100:0/580962106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbf2b2f610 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2277fe640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe2267fc640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd7d30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe21c004700 tx=0x7fe21c01e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592ebe84b30306e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe20aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe20aff5640 1 -- 172.21.15.100:0/580962106 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c017070 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe20aff5640 1 -- 172.21.15.100:0/580962106 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe21c010d50 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbf2bcf440 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe20aff5640 1 -- 172.21.15.100:0/580962106 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21c027470 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe22fe66740 10 client.18039 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.060+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fbf2af16a0 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe20aff5640 1 -- 172.21.15.100:0/580962106 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fbf2bcf440 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe20aff5640 1 -- 172.21.15.100:0/580962106 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe21c02d070 con 0x55fbf2bc9bd0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe22fe66740 10 client.18039 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe22fe66740 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe22fe66740 20 client.18039 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe22fe66740 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2c3a2a0 0x55fbf2c3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe22fe66740 4 client.18039 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fbf28307e0 con 0x55fbf2c3a2a0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.061+0000 7fe2277fe640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2c3a2a0 0x55fbf2c3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe20aff5640 10 client.18039 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe2277fe640 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2c3a2a0 0x55fbf2c3c6c0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b518913eeb700029 server_cookie=3323e461a63ea070 in_seq=0 out_seq=0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe20aff5640 1 -- 172.21.15.100:0/580962106 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fbf28307e0 con 0x55fbf2c3a2a0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe20aff5640 10 client.18039 handle_command_reply: tid=0 2024-03-12T05:24:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 client.18039 shutdown 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 msgr2=0x55fbf2bd7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd7d30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe21c004700 tx=0x7fe21c01e820 comp rx=0 tx=0).stop 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 shutdown_connections 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fbf2c3a2a0 0x55fbf2c3c6c0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fbf2bd27b0 0x55fbf2bd8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbf2bc9bd0 0x55fbf2bd7d30 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe20aff5640 0 client.18039 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 --2- 172.21.15.100:0/580962106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fbf2bc92b0 0x55fbf2bd9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 >> 172.21.15.100:0/580962106 conn(0x55fbf2bc3990 msgr2=0x55fbf2bcdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 shutdown_connections 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 1 -- 172.21.15.100:0/580962106 wait complete. 2024-03-12T05:24:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.062+0000 7fe22fe66740 20 client.18039 trim_cache size 0 max 16384 2024-03-12T05:24:19.064 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:24:19.064 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:24:19.064 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:24:19.064 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2100a1ba0 msgr2=0x7fe2100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2100a1ba0 0x7fe2100a4060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2180cb3a0 tx=0x7fe21808ff50 comp rx=0 tx=0).stop 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 msgr2=0x7fe228196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 0x7fe228196680 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe21800e9f0 tx=0x7fe21800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 shutdown_connections 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2100a1ba0 0x7fe2100a4060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe22819af40 0x7fe22819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe228105350 0x7fe228196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 --2- 172.21.15.100:0/3123728327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe228100280 0x7fe228196680 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 >> 172.21.15.100:0/3123728327 conn(0x7fe228077170 msgr2=0x7fe2280fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 shutdown_connections 2024-03-12T05:24:19.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.063+0000 7fe22d288640 1 -- 172.21.15.100:0/3123728327 wait complete. 2024-03-12T05:24:19.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:24:19.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:24:19.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.181+0000 7f0b17464640 1 -- 172.21.15.100:0/3388079141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b101023f0 msgr2=0x7f0b101027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.181+0000 7f0b17464640 1 --2- 172.21.15.100:0/3388079141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b101023f0 0x7f0b101027d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009ca0 tx=0x7f0b0c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- 172.21.15.100:0/3388079141 shutdown_connections 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 --2- 172.21.15.100:0/3388079141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1010b7c0 0x7f0b1010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 --2- 172.21.15.100:0/3388079141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b10102d10 0x7f0b1010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 --2- 172.21.15.100:0/3388079141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b101023f0 0x7f0b101027d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- 172.21.15.100:0/3388079141 >> 172.21.15.100:0/3388079141 conn(0x7f0b10077180 msgr2=0x7f0b10077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- 172.21.15.100:0/3388079141 shutdown_connections 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- 172.21.15.100:0/3388079141 wait complete. 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 Processor -- start 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- start start 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b101023f0 0x7f0b1019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b10102d10 0x7f0b1019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 0x7f0b10197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10197c90 con 0x7f0b1010b7c0 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10197e00 con 0x7f0b10102d10 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.182+0000 7f0b17464640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10197fc0 con 0x7f0b101023f0 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 0x7f0b10197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 0x7f0b10197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57834/0 (socket says 172.21.15.100:57834) 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 -- 172.21.15.100:0/2187922658 learned_addr learned my addr 172.21.15.100:0/2187922658 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b149ff640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b10102d10 0x7f0b1019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15200640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b101023f0 0x7f0b1019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 -- 172.21.15.100:0/2187922658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b101023f0 msgr2=0x7f0b1019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b101023f0 0x7f0b1019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 -- 172.21.15.100:0/2187922658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b10102d10 msgr2=0x7f0b1019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b10102d10 0x7f0b1019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 -- 172.21.15.100:0/2187922658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c009900 con 0x7f0b1010b7c0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b15a01640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 0x7f0b10197640 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800d750 tx=0x7f0b0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08009bd0 con 0x7f0b1010b7c0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b0800de60 con 0x7f0b1010b7c0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b101982a0 con 0x7f0b1010b7c0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004e70 con 0x7f0b1010b7c0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.183+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b101a44e0 con 0x7f0b1010b7c0 2024-03-12T05:24:19.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.184+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b10067f70 con 0x7f0b1010b7c0 2024-03-12T05:24:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.185+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0b08005020 con 0x7f0b1010b7c0 2024-03-12T05:24:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.186+0000 7f0b067fc640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0afc0a1c20 0x7f0afc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.186+0000 7f0b15200640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0afc0a1c20 0x7f0afc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.186+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0b080cd0a0 con 0x7f0b1010b7c0 2024-03-12T05:24:19.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.186+0000 7f0b15200640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0afc0a1c20 0x7f0afc0a40e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c004af0 tx=0x7f0b0c005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.191+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b08099b10 con 0x7f0b1010b7c0 2024-03-12T05:24:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:18 smithi161 ceph-mon[36043]: pgmap v381: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 127 op/s 2024-03-12T05:24:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:18 smithi100 ceph-mon[32210]: pgmap v381: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 127 op/s 2024-03-12T05:24:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.432+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b101068b0 con 0x7f0b1010b7c0 2024-03-12T05:24:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.433+0000 7f0b067fc640 1 -- 172.21.15.100:0/2187922658 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f0b08020090 con 0x7f0b1010b7c0 2024-03-12T05:24:19.428 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:24:19.428 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:24:19.428 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:24:19.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0afc0a1c20 msgr2=0x7f0afc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0afc0a1c20 0x7f0afc0a40e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c004af0 tx=0x7f0b0c005c50 comp rx=0 tx=0).stop 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 msgr2=0x7f0b10197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 0x7f0b10197640 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800d750 tx=0x7f0b0800dc10 comp rx=0 tx=0).stop 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 shutdown_connections 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0afc0a1c20 0x7f0afc0a40e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1010b7c0 0x7f0b10197640 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b10102d10 0x7f0b1019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 --2- 172.21.15.100:0/2187922658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b101023f0 0x7f0b1019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 >> 172.21.15.100:0/2187922658 conn(0x7f0b10077180 msgr2=0x7f0b100fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 shutdown_connections 2024-03-12T05:24:19.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.436+0000 7f0b17464640 1 -- 172.21.15.100:0/2187922658 wait complete. 2024-03-12T05:24:19.445 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:24:19.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:24:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:18 smithi156 ceph-mon[36270]: pgmap v381: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 127 op/s 2024-03-12T05:24:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.564+0000 7f7fedde2640 1 -- 172.21.15.100:0/4174239602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe80ff8f0 msgr2=0x7f7fe80ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.564+0000 7f7fedde2640 1 --2- 172.21.15.100:0/4174239602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe80ffcd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009930 tx=0x7f7fd802bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 -- 172.21.15.100:0/4174239602 shutdown_connections 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 --2- 172.21.15.100:0/4174239602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fe8106500 0x7f7fe810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 --2- 172.21.15.100:0/4174239602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe81002a0 0x7f7fe8105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 --2- 172.21.15.100:0/4174239602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe80ffcd0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 -- 172.21.15.100:0/4174239602 >> 172.21.15.100:0/4174239602 conn(0x7f7fe8076640 msgr2=0x7f7fe80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 -- 172.21.15.100:0/4174239602 shutdown_connections 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 -- 172.21.15.100:0/4174239602 wait complete. 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 Processor -- start 2024-03-12T05:24:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.565+0000 7f7fedde2640 1 -- start start 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 0x7f7fe819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fe81976c0 0x7f7fe8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8198190 con 0x7f7fe8106500 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8198300 con 0x7f7fe80ff8f0 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81984c0 con 0x7f7fe81976c0 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 0x7f7fe819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 0x7f7fe819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57860/0 (socket says 172.21.15.100:57860) 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/1883915723 learned_addr learned my addr 172.21.15.100:0/1883915723 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fe81976c0 0x7f7fe8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/1883915723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fe81976c0 msgr2=0x7f7fe8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fe81976c0 0x7f7fe8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/1883915723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe80ff8f0 msgr2=0x7f7fe819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/1883915723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8009560 con 0x7f7fe8106500 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 0x7f7fe819db40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00e9f0 tx=0x7f7fdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe4ff9640 1 -- 172.21.15.100:0/1883915723 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0096a0 con 0x7f7fe8106500 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe81a4460 con 0x7f7fe8106500 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe4ff9640 1 -- 172.21.15.100:0/1883915723 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fdc00ccc0 con 0x7f7fe8106500 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fe4ff9640 1 -- 172.21.15.100:0/1883915723 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018860 con 0x7f7fe8106500 2024-03-12T05:24:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.566+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe81a4940 con 0x7f7fe8106500 2024-03-12T05:24:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe4ff9640 1 -- 172.21.15.100:0/1883915723 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7fdc010aa0 con 0x7f7fe8106500 2024-03-12T05:24:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 Processor -- start 2024-03-12T05:24:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 -- start start 2024-03-12T05:24:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe4ff9640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7fd00a1ba0 0x7f7fd00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604ccb60 0x564e604cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6047a960 con 0x564e604bef00 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6042e310 con 0x564e604ccb60 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6042b840 con 0x564e604bf820 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7fd00a1ba0 0x7f7fd00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604bf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57876/0 (socket says 172.21.15.100:57876) 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe77fe640 1 -- 172.21.15.100:0/3428278160 learned_addr learned my addr 172.21.15.100:0/3428278160 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.569+0000 7f7fe4ff9640 1 -- 172.21.15.100:0/1883915723 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fdc014070 con 0x7f7fe8106500 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604ccb60 0x564e604cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe77fe640 1 -- 172.21.15.100:0/3428278160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 msgr2=0x564e604cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe77fe640 1 -- 172.21.15.100:0/3428278160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604ccb60 msgr2=0x564e604cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604ccb60 0x564e604cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe77fe640 1 -- 172.21.15.100:0/3428278160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e603e9060 con 0x564e604bef00 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604ccb60 0x564e604cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7fd00a1ba0 0x7f7fd00a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0005fd0 tx=0x7f7fe0012810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604bf2e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7fd802c240 tx=0x7f7fd802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce07c9e78aae9a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fcdffb640 1 -- 172.21.15.100:0/3428278160 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd802c7e0 con 0x564e604bef00 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fcdffb640 1 -- 172.21.15.100:0/3428278160 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fd802c940 con 0x564e604bef00 2024-03-12T05:24:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.570+0000 7f7fcdffb640 1 -- 172.21.15.100:0/3428278160 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8005410 con 0x564e604bef00 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 -- 172.21.15.100:0/3428278160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 msgr2=0x564e604bf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604bf2e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7fd802c240 tx=0x7f7fd802c710 comp rx=0 tx=0).stop 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 -- 172.21.15.100:0/3428278160 shutdown_connections 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604ccb60 0x564e604cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/3428278160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604bf2e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 -- 172.21.15.100:0/3428278160 >> 172.21.15.100:0/3428278160 conn(0x564e604b9980 msgr2=0x564e604b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 -- 172.21.15.100:0/3428278160 shutdown_connections 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 -- 172.21.15.100:0/3428278160 wait complete. 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 -- start start 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.571+0000 7f7ff09ba740 1 Processor -- start 2024-03-12T05:24:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bef00 0x564e604c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e60427020 con 0x564e604ccb60 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e600fa430 con 0x564e604bef00 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e600b6e00 con 0x564e604bf820 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 10 client.0 _ll_register_callbacks cb 0x564e603cecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bef00 0x564e604c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57892/0 (socket says 172.21.15.100:57892) 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe7fff640 1 -- 172.21.15.100:0/2849530071 learned_addr learned my addr 172.21.15.100:0/2849530071 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bef00 msgr2=0x564e604c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bef00 0x564e604c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/2849530071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 msgr2=0x564e604ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 -- 172.21.15.100:0/2849530071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe810e890 con 0x564e604bf820 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bef00 0x564e604c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604ca050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0cb290 tx=0x7f7fdc09a450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8284c205a9e7394 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2849530071 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc09e8c0 con 0x564e604bf820 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e601267e0 con 0x564e604bf820 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.572+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2849530071 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fdc0a2f10 con 0x564e604bf820 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2849530071 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fdc0b9660 con 0x564e604bf820 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 10 client.27379 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e604256a0 con 0x564e604bf820 2024-03-12T05:24:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2849530071 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564e601267e0 con 0x564e604bf820 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2849530071 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7fdc096570 con 0x564e604bf820 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 10 client.27379 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 10 client.27379 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 20 client.27379 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e604dcf00 0x564e604df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 4 client.27379 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e60163720 con 0x564e604dcf00 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.573+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e604dcf00 0x564e604df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7fc6ffd640 10 client.27379 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e604dcf00 0x564e604df320 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1e897d3f2f022f6 server_cookie=c1831e5f0874bb72 in_seq=0 out_seq=0 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2849530071 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e60163720 con 0x564e604dcf00 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7fc6ffd640 10 client.27379 handle_command_reply: tid=0 2024-03-12T05:24:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 client.27379 shutdown 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 msgr2=0x564e604ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604ca050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0cb290 tx=0x7f7fdc09a450 comp rx=0 tx=0).stop 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 shutdown_connections 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e604dcf00 0x564e604df320 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7fc6ffd640 0 client.27379 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bf820 0x564e604ca050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.574+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2849530071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bef00 0x564e604c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.575+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 >> 172.21.15.100:0/2849530071 conn(0x564e604b9980 msgr2=0x564e604bb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.575+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 shutdown_connections 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.575+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2849530071 wait complete. 2024-03-12T05:24:19.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.575+0000 7f7ff09ba740 20 client.27379 trim_cache size 0 max 16384 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 Processor -- start 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 -- start start 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604ccb60 0x564e604c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e603ac130 con 0x564e604bef00 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e604bfc60 con 0x564e604bf820 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e604be340 con 0x564e604ccb60 2024-03-12T05:24:19.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57900/0 (socket says 172.21.15.100:57900) 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 -- 172.21.15.100:0/2687083379 learned_addr learned my addr 172.21.15.100:0/2687083379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604ccb60 0x564e604c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 -- 172.21.15.100:0/2687083379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604ccb60 msgr2=0x564e604c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604ccb60 0x564e604c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 -- 172.21.15.100:0/2687083379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 msgr2=0x564e604ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 -- 172.21.15.100:0/2687083379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e603c1c20 con 0x564e604bef00 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.587+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604cc240 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8000c00 tx=0x7f7fd8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ff4119621cb071 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7fcd7fa640 1 -- 172.21.15.100:0/2687083379 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8058070 con 0x564e604bef00 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7fcd7fa640 1 -- 172.21.15.100:0/2687083379 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fd8052be0 con 0x564e604bef00 2024-03-12T05:24:19.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7fcd7fa640 1 -- 172.21.15.100:0/2687083379 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8052700 con 0x564e604bef00 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2687083379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 msgr2=0x564e604cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604cc240 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8000c00 tx=0x7f7fd8041040 comp rx=0 tx=0).stop 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2687083379 shutdown_connections 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604ccb60 0x564e604c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2687083379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604bef00 0x564e604cc240 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2687083379 >> 172.21.15.100:0/2687083379 conn(0x564e604b9980 msgr2=0x564e604c4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2687083379 shutdown_connections 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.588+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2687083379 wait complete. 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 -- start start 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 Processor -- start 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bef00 0x564e604d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e60533410 con 0x564e604ccb60 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e60533550 con 0x564e604bf820 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e60533710 con 0x564e604bef00 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7ff09ba740 10 client.0 _ll_register_callbacks cb 0x7f7fd8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bef00 0x564e604d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57912/0 (socket says 172.21.15.100:57912) 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7fe7fff640 1 -- 172.21.15.100:0/2188526084 learned_addr learned my addr 172.21.15.100:0/2188526084 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.589+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe7fff640 1 -- 172.21.15.100:0/2188526084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bef00 msgr2=0x564e604d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bef00 0x564e604d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe7fff640 1 -- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 msgr2=0x564e604d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe7fff640 1 -- 172.21.15.100:0/2188526084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e603e9060 con 0x564e604ccb60 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bef00 0x564e604d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe6ffd640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fe7fff640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604d3010 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000edc0 tx=0x7f7fe0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f837d7f653ab6180 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2188526084 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0025070 con 0x564e604ccb60 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2188526084 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fe001e590 con 0x564e604ccb60 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fd802c940 con 0x564e604ccb60 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2188526084 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe0017070 con 0x564e604ccb60 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 10 client.18063 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e603e78c0 con 0x564e604ccb60 2024-03-12T05:24:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2188526084 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fd802c940 con 0x564e604ccb60 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2188526084 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7fe0029980 con 0x564e604ccb60 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 10 client.18063 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 10 client.18063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 20 client.18063 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e60534f70 0x564e60537390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 4 client.18063 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:24:19.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.590+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e601b2890 con 0x564e60534f70 2024-03-12T05:24:19.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.605+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e60534f70 0x564e60537390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.605+0000 7f7fe77fe640 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e60534f70 0x564e60537390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e567fb6e85ded72 server_cookie=ac43f85093b9b6a9 in_seq=0 out_seq=0 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.605+0000 7f7fc6ffd640 10 client.18063 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.605+0000 7f7fc6ffd640 1 -- 172.21.15.100:0/2188526084 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e601b2890 con 0x564e60534f70 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.605+0000 7f7fc6ffd640 10 client.18063 handle_command_reply: tid=0 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 client.18063 shutdown 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 msgr2=0x564e604d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604d3010 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000edc0 tx=0x7f7fe0004ab0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 shutdown_connections 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564e60534f70 0x564e60537390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e604ccb60 0x564e604d3010 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e604bf820 0x564e604d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fc6ffd640 0 client.18063 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 --2- 172.21.15.100:0/2188526084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e604bef00 0x564e604d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 >> 172.21.15.100:0/2188526084 conn(0x564e604b9980 msgr2=0x564e604badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 shutdown_connections 2024-03-12T05:24:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 1 -- 172.21.15.100:0/2188526084 wait complete. 2024-03-12T05:24:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7ff09ba740 20 client.18063 trim_cache size 0 max 16384 2024-03-12T05:24:19.602 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:24:19.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7fd00a1ba0 msgr2=0x7f7fd00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7fd00a1ba0 0x7f7fd00a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0005fd0 tx=0x7f7fe0012810 comp rx=0 tx=0).stop 2024-03-12T05:24:19.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 msgr2=0x7f7fe819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 0x7f7fe819db40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00e9f0 tx=0x7f7fdc00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 shutdown_connections 2024-03-12T05:24:19.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7fd00a1ba0 0x7f7fd00a4060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fe81976c0 0x7f7fe8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fe8106500 0x7f7fe819db40 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 --2- 172.21.15.100:0/1883915723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe80ff8f0 0x7f7fe819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.606+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 >> 172.21.15.100:0/1883915723 conn(0x7f7fe8076640 msgr2=0x7f7fe8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.607+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 shutdown_connections 2024-03-12T05:24:19.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.607+0000 7f7fedde2640 1 -- 172.21.15.100:0/1883915723 wait complete. 2024-03-12T05:24:19.613 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:24:19.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 -- 172.21.15.100:0/3907264344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c101d40 msgr2=0x7fa95c105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 --2- 172.21.15.100:0/3907264344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c101d40 0x7fa95c105d80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa944009930 tx=0x7fa94402bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 -- 172.21.15.100:0/3907264344 shutdown_connections 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 --2- 172.21.15.100:0/3907264344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa95c106460 0x7fa95c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 --2- 172.21.15.100:0/3907264344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c101d40 0x7fa95c105d80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 --2- 172.21.15.100:0/3907264344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa95c101390 0x7fa95c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 -- 172.21.15.100:0/3907264344 >> 172.21.15.100:0/3907264344 conn(0x7fa95c0fcbb0 msgr2=0x7fa95c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 -- 172.21.15.100:0/3907264344 shutdown_connections 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.728+0000 7fa960920640 1 -- 172.21.15.100:0/3907264344 wait complete. 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 Processor -- start 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 -- start start 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa95c101390 0x7fa95c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 0x7fa95c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa95c197680 0x7fa95c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c198150 con 0x7fa95c106460 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1982c0 con 0x7fa95c197680 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa960920640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c198480 con 0x7fa95c101390 2024-03-12T05:24:19.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa953fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 0x7fa95c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa953fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 0x7fa95c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57932/0 (socket says 172.21.15.100:57932) 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa953fff640 1 -- 172.21.15.100:0/2943968268 learned_addr learned my addr 172.21.15.100:0/2943968268 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa95a59c640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa95c101390 0x7fa95c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.729+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa95c197680 0x7fa95c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa953fff640 1 -- 172.21.15.100:0/2943968268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa95c101390 msgr2=0x7fa95c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa953fff640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa95c101390 0x7fa95c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa953fff640 1 -- 172.21.15.100:0/2943968268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa95c197680 msgr2=0x7fa95c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa953fff640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa95c197680 0x7fa95c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa953fff640 1 -- 172.21.15.100:0/2943968268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa944009560 con 0x7fa95c106460 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa953fff640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 0x7fa95c19dab0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa944009230 tx=0x7fa944004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa9537fe640 1 -- 172.21.15.100:0/2943968268 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440043d0 con 0x7fa95c106460 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa9537fe640 1 -- 172.21.15.100:0/2943968268 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9440052d0 con 0x7fa95c106460 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c1a4370 con 0x7fa95c106460 2024-03-12T05:24:19.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa9537fe640 1 -- 172.21.15.100:0/2943968268 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944003d80 con 0x7fa95c106460 2024-03-12T05:24:19.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.730+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1a4850 con 0x7fa95c106460 2024-03-12T05:24:19.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9537fe640 1 -- 172.21.15.100:0/2943968268 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fa944005440 con 0x7fa95c106460 2024-03-12T05:24:19.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 Processor -- start 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 -- start start 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 0x559e06f90630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f99210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f99750 0x559e06f9bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06f4b960 con 0x559e06f90b70 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06eff310 con 0x559e06f90250 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9635cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06efc840 con 0x559e06f99750 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa95a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f99210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa95ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 0x559e06f90630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa95ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 0x559e06f90630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49970/0 (socket says 172.21.15.100:49970) 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa95a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f99210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57936/0 (socket says 172.21.15.100:57936) 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa9537fe640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa9480a1ba0 0x7fa9480a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa95ad9d640 1 -- 172.21.15.100:0/316848194 learned_addr learned my addr 172.21.15.100:0/316848194 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.733+0000 7fa953fff640 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f99750 0x559e06f9bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa9480a1ba0 0x7fa9480a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa9537fe640 1 -- 172.21.15.100:0/2943968268 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa9440ef880 con 0x7fa95c106460 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95a59c640 1 -- 172.21.15.100:0/316848194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f99750 msgr2=0x559e06f9bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95a59c640 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f99750 0x559e06f9bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95a59c640 1 -- 172.21.15.100:0/316848194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 msgr2=0x559e06f90630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95a59c640 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 0x559e06f90630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95a59c640 1 -- 172.21.15.100:0/316848194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e06eba060 con 0x559e06f90b70 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 0x559e06f90630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95a59c640 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f99210 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bbf0 tx=0x7fa94c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42294eaf4335960c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa9480a1ba0 0x7fa9480a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9540049d0 tx=0x7fa954004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa950ff9640 1 -- 172.21.15.100:0/316848194 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004030 con 0x559e06f90b70 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa950ff9640 1 -- 172.21.15.100:0/316848194 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa94c015070 con 0x559e06f90b70 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.734+0000 7fa950ff9640 1 -- 172.21.15.100:0/316848194 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0116a0 con 0x559e06f90b70 2024-03-12T05:24:19.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 -- 172.21.15.100:0/316848194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 msgr2=0x559e06f99210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f99210 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bbf0 tx=0x7fa94c0107a0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 -- 172.21.15.100:0/316848194 shutdown_connections 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f99750 0x559e06f9bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f99210 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 --2- 172.21.15.100:0/316848194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90250 0x559e06f90630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 -- 172.21.15.100:0/316848194 >> 172.21.15.100:0/316848194 conn(0x559e06f8a980 msgr2=0x559e06f8ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 -- 172.21.15.100:0/316848194 shutdown_connections 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 -- 172.21.15.100:0/316848194 wait complete. 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 -- start start 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.735+0000 7fa9635cc740 1 Processor -- start 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 0x559e06f96710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90b70 0x559e06f96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f97190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06ef8020 con 0x559e06f90250 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06bcb430 con 0x559e06f99750 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06b87e00 con 0x559e06f90b70 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 0x559e06f96710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 10 client.0 _ll_register_callbacks cb 0x559e06e9fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 0x559e06f96710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57952/0 (socket says 172.21.15.100:57952) 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 -- 172.21.15.100:0/2535431858 learned_addr learned my addr 172.21.15.100:0/2535431858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f97190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa953fff640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90b70 0x559e06f96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 -- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90b70 msgr2=0x559e06f96c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90b70 0x559e06f96c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 -- 172.21.15.100:0/2535431858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 msgr2=0x559e06f97190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f97190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 -- 172.21.15.100:0/2535431858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c00b820 con 0x559e06f90250 2024-03-12T05:24:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa95a59c640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 0x559e06f96710 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bfd0 tx=0x7fa94c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feac8815e79a2eb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa941ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa941ffb640 1 -- 172.21.15.100:0/2535431858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c021070 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa941ffb640 1 -- 172.21.15.100:0/2535431858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa94c017070 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.736+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e06bf77e0 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa941ffb640 1 -- 172.21.15.100:0/2535431858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa94c003880 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 10 client.18075 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e06ef66a0 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa941ffb640 1 -- 172.21.15.100:0/2535431858 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e06bf77e0 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa941ffb640 1 -- 172.21.15.100:0/2535431858 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa94c0046c0 con 0x559e06f90250 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 10 client.18075 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 10 client.18075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 20 client.18075 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e06fa9990 0x559e06fabd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 4 client.18075 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e06c34720 con 0x559e06fa9990 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa953fff640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e06fa9990 0x559e06fabd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa953fff640 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e06fa9990 0x559e06fabd80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22744aa077ce5a89 server_cookie=8443c9e66beb61b2 in_seq=0 out_seq=0 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.737+0000 7fa941ffb640 10 client.18075 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa941ffb640 1 -- 172.21.15.100:0/2535431858 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e06c34720 con 0x559e06fa9990 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa941ffb640 10 client.18075 handle_command_reply: tid=0 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 client.18075 shutdown 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 msgr2=0x559e06f96710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 0x559e06f96710 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bfd0 tx=0x7fa94c0058e0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 shutdown_connections 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e06fa9990 0x559e06fabd80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f97190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90b70 0x559e06f96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 --2- 172.21.15.100:0/2535431858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90250 0x559e06f96710 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 >> 172.21.15.100:0/2535431858 conn(0x559e06f8a980 msgr2=0x559e06f8c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa941ffb640 0 client.18075 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 shutdown_connections 2024-03-12T05:24:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 1 -- 172.21.15.100:0/2535431858 wait complete. 2024-03-12T05:24:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.738+0000 7fa9635cc740 20 client.18075 trim_cache size 0 max 16384 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 Processor -- start 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 -- start start 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06f98e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90b70 0x559e06f96fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 0x559e06f94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06e7d130 con 0x559e06f99750 2024-03-12T05:24:19.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06fa9740 con 0x559e06f90b70 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa9635cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06f8f5d0 con 0x559e06f90250 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa95ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 0x559e06f94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.751+0000 7fa953fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90b70 0x559e06f96fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 0x559e06f94fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57966/0 (socket says 172.21.15.100:57966) 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 -- 172.21.15.100:0/3079512602 learned_addr learned my addr 172.21.15.100:0/3079512602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95a59c640 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06f98e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 -- 172.21.15.100:0/3079512602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 msgr2=0x559e06f98e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06f98e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 -- 172.21.15.100:0/3079512602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90b70 msgr2=0x559e06f96fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90b70 0x559e06f96fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 -- 172.21.15.100:0/3079512602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e06e92c20 con 0x559e06f99750 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa953fff640 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90b70 0x559e06f96fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 0x559e06f94fe0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa9540132a0 tx=0x7fa954005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3a7d5a010dfbb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa943fff640 1 -- 172.21.15.100:0/3079512602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954023960 con 0x559e06f99750 2024-03-12T05:24:19.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa943fff640 1 -- 172.21.15.100:0/3079512602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa954010430 con 0x559e06f99750 2024-03-12T05:24:19.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.752+0000 7fa943fff640 1 -- 172.21.15.100:0/3079512602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95402b640 con 0x559e06f99750 2024-03-12T05:24:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 -- 172.21.15.100:0/3079512602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 msgr2=0x559e06f94fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 0x559e06f94fe0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa9540132a0 tx=0x7fa954005bd0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 -- 172.21.15.100:0/3079512602 shutdown_connections 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f99750 0x559e06f94fe0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f90b70 0x559e06f96fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 --2- 172.21.15.100:0/3079512602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06f98e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 -- 172.21.15.100:0/3079512602 >> 172.21.15.100:0/3079512602 conn(0x559e06f8a980 msgr2=0x559e06fb0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 -- 172.21.15.100:0/3079512602 shutdown_connections 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 -- 172.21.15.100:0/3079512602 wait complete. 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 -- start start 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.753+0000 7fa9635cc740 1 Processor -- start 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06fa0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f9ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f9f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06f9f980 con 0x559e06f90b70 2024-03-12T05:24:19.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06f9faf0 con 0x559e06f99750 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e06fffeb0 con 0x559e06f90250 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 10 client.0 _ll_register_callbacks cb 0x7fa94c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f9ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f9ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57982/0 (socket says 172.21.15.100:57982) 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 -- 172.21.15.100:0/653357286 learned_addr learned my addr 172.21.15.100:0/653357286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa95a59c640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06fa0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa95ad9d640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f9f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 -- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 msgr2=0x559e06fa0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06fa0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 -- 172.21.15.100:0/653357286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 msgr2=0x559e06f9f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f9f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 -- 172.21.15.100:0/653357286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1a4850 con 0x559e06f90b70 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa953fff640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f9ed20 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa944005ec0 tx=0x7fa94402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd7466329415ea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa941ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa941ffb640 1 -- 172.21.15.100:0/653357286 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440bc580 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa941ffb640 1 -- 172.21.15.100:0/653357286 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa9440c06d0 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa94c015070 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa941ffb640 1 -- 172.21.15.100:0/653357286 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9440c4d20 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 10 client.18081 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.754+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fa954009a50 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa941ffb640 1 -- 172.21.15.100:0/653357286 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa94c015070 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa941ffb640 1 -- 172.21.15.100:0/653357286 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa94403c070 con 0x559e06f90b70 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 10 client.18081 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 10 client.18081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 20 client.18081 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e07001460 0x559e07003850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 4 client.18081 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e06c83890 con 0x559e07001460 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa95a59c640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e07001460 0x559e07003850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa95a59c640 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e07001460 0x559e07003850 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e60af18298c35553 server_cookie=72fa2bdfc0c60cef in_seq=0 out_seq=0 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa941ffb640 10 client.18081 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa941ffb640 1 -- 172.21.15.100:0/653357286 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e06c83890 con 0x559e07001460 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa941ffb640 10 client.18081 handle_command_reply: tid=0 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.755+0000 7fa9635cc740 1 client.18081 shutdown 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 msgr2=0x559e06f9ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f9ed20 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa944005ec0 tx=0x7fa94402bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 shutdown_connections 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x559e07001460 0x559e07003850 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e06f99750 0x559e06f9f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa941ffb640 0 client.18081 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e06f90b70 0x559e06f9ed20 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 --2- 172.21.15.100:0/653357286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06f90250 0x559e06fa0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 >> 172.21.15.100:0/653357286 conn(0x559e06f8a980 msgr2=0x559e06f94f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 shutdown_connections 2024-03-12T05:24:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 1 -- 172.21.15.100:0/653357286 wait complete. 2024-03-12T05:24:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa9635cc740 20 client.18081 trim_cache size 0 max 16384 2024-03-12T05:24:19.758 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:24:19.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa9480a1ba0 msgr2=0x7fa9480a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa9480a1ba0 0x7fa9480a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9540049d0 tx=0x7fa954004ab0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 msgr2=0x7fa95c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 0x7fa95c19dab0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa944009230 tx=0x7fa944004c00 comp rx=0 tx=0).stop 2024-03-12T05:24:19.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 shutdown_connections 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa9480a1ba0 0x7fa9480a4060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa95c197680 0x7fa95c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c106460 0x7fa95c19dab0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 --2- 172.21.15.100:0/2943968268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa95c101390 0x7fa95c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 >> 172.21.15.100:0/2943968268 conn(0x7fa95c0fcbb0 msgr2=0x7fa95c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 shutdown_connections 2024-03-12T05:24:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.756+0000 7fa960920640 1 -- 172.21.15.100:0/2943968268 wait complete. 2024-03-12T05:24:19.762 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:24:19.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:24:19.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1458276 11.51 MB/sec execute 543 sec latency 21.487 ms 2024-03-12T05:24:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.876+0000 7f09852df640 1 -- 172.21.15.100:0/155032998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980100260 msgr2=0x7f0980100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.876+0000 7f09852df640 1 --2- 172.21.15.100:0/155032998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980100260 0x7f0980100640 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0970009930 tx=0x7f097002bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.876+0000 7f09852df640 1 -- 172.21.15.100:0/155032998 shutdown_connections 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 --2- 172.21.15.100:0/155032998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0980105330 0x7f098010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 --2- 172.21.15.100:0/155032998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0980100c10 0x7f0980104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 --2- 172.21.15.100:0/155032998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980100260 0x7f0980100640 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- 172.21.15.100:0/155032998 >> 172.21.15.100:0/155032998 conn(0x7f0980077180 msgr2=0x7f0980077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- 172.21.15.100:0/155032998 shutdown_connections 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- 172.21.15.100:0/155032998 wait complete. 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 Processor -- start 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- start start 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0980100260 0x7f09801966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0980105330 0x7f0980196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 0x7f098019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019b9a0 con 0x7f098019afb0 2024-03-12T05:24:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019bb10 con 0x7f0980100260 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.877+0000 7f09852df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019bc80 con 0x7f0980105330 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 0x7f098019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0980100260 0x7f09801966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 0x7f098019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58006/0 (socket says 172.21.15.100:58006) 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 -- 172.21.15.100:0/1822351876 learned_addr learned my addr 172.21.15.100:0/1822351876 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097e7fc640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0980105330 0x7f0980196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 -- 172.21.15.100:0/1822351876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0980105330 msgr2=0x7f0980196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0980105330 0x7f0980196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 -- 172.21.15.100:0/1822351876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0980100260 msgr2=0x7f09801966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0980100260 0x7f09801966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 -- 172.21.15.100:0/1822351876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970009560 con 0x7f098019afb0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f097f7fe640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 0x7f098019b460 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f097800eb10 tx=0x7f097800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f0967fff640 1 -- 172.21.15.100:0/1822351876 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978004b20 con 0x7f098019afb0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f0967fff640 1 -- 172.21.15.100:0/1822351876 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0978004c80 con 0x7f098019afb0 2024-03-12T05:24:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09801a3260 con 0x7f098019afb0 2024-03-12T05:24:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f0967fff640 1 -- 172.21.15.100:0/1822351876 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978010650 con 0x7f098019afb0 2024-03-12T05:24:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.878+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09801a36c0 con 0x7f098019afb0 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0967fff640 1 -- 172.21.15.100:0/1822351876 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0978005020 con 0x7f098019afb0 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 Processor -- start 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 -- start start 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 0x558e235aa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 0x558e235b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e235b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23565960 con 0x558e235b3750 2024-03-12T05:24:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23519310 con 0x558e235aa250 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0987f93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23516840 con 0x558e235aab70 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0967fff640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f09680a1ba0 0x7f09680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 0x558e235aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 0x558e235b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 0x558e235aa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49992/0 (socket says 172.21.15.100:49992) 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 0x558e235b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:36832/0 (socket says 172.21.15.100:36832) 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097e7fc640 1 -- 172.21.15.100:0/2826201690 learned_addr learned my addr 172.21.15.100:0/2826201690 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097e7fc640 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e235b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f097f7fe640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f09680a1ba0 0x7f09680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.881+0000 7f0967fff640 1 -- 172.21.15.100:0/1822351876 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0978014070 con 0x7f098019afb0 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097effd640 1 -- 172.21.15.100:0/2826201690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 msgr2=0x558e235aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097effd640 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 0x558e235aa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097effd640 1 -- 172.21.15.100:0/2826201690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 msgr2=0x558e235b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097effd640 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e235b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097effd640 1 -- 172.21.15.100:0/2826201690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e234d4060 con 0x558e235aab70 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097e7fc640 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 0x558e235aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097f7fe640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f09680a1ba0 0x7f09680a4060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09780cb530 tx=0x7f097806cb60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f097effd640 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 0x558e235b3210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0970009230 tx=0x7f097002c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a12704758eec7e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f09657fa640 1 -- 172.21.15.100:0/2826201690 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970004800 con 0x558e235aab70 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f09657fa640 1 -- 172.21.15.100:0/2826201690 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0970004960 con 0x558e235aab70 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.882+0000 7f09657fa640 1 -- 172.21.15.100:0/2826201690 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970005050 con 0x558e235aab70 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- 172.21.15.100:0/2826201690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 msgr2=0x558e235b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 0x558e235b3210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0970009230 tx=0x7f097002c5d0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- 172.21.15.100:0/2826201690 shutdown_connections 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e235b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aab70 0x558e235b3210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- 172.21.15.100:0/2826201690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aa250 0x558e235aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- 172.21.15.100:0/2826201690 >> 172.21.15.100:0/2826201690 conn(0x558e235a4980 msgr2=0x558e235a4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- 172.21.15.100:0/2826201690 shutdown_connections 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- 172.21.15.100:0/2826201690 wait complete. 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- start start 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 Processor -- start 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23512020 con 0x558e235aab70 2024-03-12T05:24:19.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e231e5430 con 0x558e235b3750 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e231a1e00 con 0x558e235aa250 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f0987f93740 10 client.0 _ll_register_callbacks cb 0x558e234b9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f097effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f097effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58020/0 (socket says 172.21.15.100:58020) 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.883+0000 7f097effd640 1 -- 172.21.15.100:0/582741522 learned_addr learned my addr 172.21.15.100:0/582741522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097e7fc640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097e7fc640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097effd640 1 -- 172.21.15.100:0/582741522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 msgr2=0x558e235b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097effd640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097effd640 1 -- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 msgr2=0x558e235b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097effd640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097effd640 1 -- 172.21.15.100:0/582741522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098010d710 con 0x558e235aab70 2024-03-12T05:24:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097e7fc640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f097effd640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0c10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0970005ab0 tx=0x7f0970004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12cc31f1ecc06e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0962ff5640 1 -- 172.21.15.100:0/582741522 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970005200 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0962ff5640 1 -- 172.21.15.100:0/582741522 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0970005360 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e232117e0 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0962ff5640 1 -- 172.21.15.100:0/582741522 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0970004ad0 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0987f93740 10 client.18093 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558e235106a0 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0962ff5640 1 -- 172.21.15.100:0/582741522 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558e232117e0 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0962ff5640 1 -- 172.21.15.100:0/582741522 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0970059070 con 0x558e235aab70 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0987f93740 10 client.18093 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.884+0000 7f0987f93740 10 client.18093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f0987f93740 20 client.18093 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f0987f93740 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e235c3690 0x558e235c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f0987f93740 4 client.18093 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e2324e720 con 0x558e235c3690 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f097e7fc640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e235c3690 0x558e235c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f097e7fc640 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e235c3690 0x558e235c5ab0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2678bc52e778dd8 server_cookie=8fec9323a9b011ef in_seq=0 out_seq=0 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.885+0000 7f0962ff5640 10 client.18093 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0962ff5640 1 -- 172.21.15.100:0/582741522 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e2324e720 con 0x558e235c3690 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0962ff5640 10 client.18093 handle_command_reply: tid=0 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 client.18093 shutdown 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 msgr2=0x558e235b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0c10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0970005ab0 tx=0x7f0970004370 comp rx=0 tx=0).stop 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 shutdown_connections 2024-03-12T05:24:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e235c3690 0x558e235c5ab0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0962ff5640 0 client.18093 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0c10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 --2- 172.21.15.100:0/582741522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 >> 172.21.15.100:0/582741522 conn(0x558e235a4980 msgr2=0x558e235a66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 shutdown_connections 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 1 -- 172.21.15.100:0/582741522 wait complete. 2024-03-12T05:24:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.886+0000 7f0987f93740 20 client.18093 trim_cache size 0 max 16384 2024-03-12T05:24:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 Processor -- start 2024-03-12T05:24:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 -- start start 2024-03-12T05:24:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235ba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23497130 con 0x558e235aab70 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e235a5780 con 0x558e235b3750 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0987f93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e235a5980 con 0x558e235aa250 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58032/0 (socket says 172.21.15.100:58032) 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 -- 172.21.15.100:0/1951546250 learned_addr learned my addr 172.21.15.100:0/1951546250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097e7fc640 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 -- 172.21.15.100:0/1951546250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 msgr2=0x558e235b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 -- 172.21.15.100:0/1951546250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 msgr2=0x558e235ba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235ba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 -- 172.21.15.100:0/1951546250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e234acc20 con 0x558e235aab70 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097e7fc640 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f097effd640 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0f70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f097002c170 tx=0x7f097002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a89e23f5aa293ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0964ff9640 1 -- 172.21.15.100:0/1951546250 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970004ad0 con 0x558e235aab70 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0964ff9640 1 -- 172.21.15.100:0/1951546250 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0970053e60 con 0x558e235aab70 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.899+0000 7f0964ff9640 1 -- 172.21.15.100:0/1951546250 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970034440 con 0x558e235aab70 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 -- 172.21.15.100:0/1951546250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 msgr2=0x558e235b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0f70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f097002c170 tx=0x7f097002c5d0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 -- 172.21.15.100:0/1951546250 shutdown_connections 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235b3750 0x558e235ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235aab70 0x558e235b0f70 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 --2- 172.21.15.100:0/1951546250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e235b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 -- 172.21.15.100:0/1951546250 >> 172.21.15.100:0/1951546250 conn(0x558e235a4980 msgr2=0x558e235ab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 -- 172.21.15.100:0/1951546250 shutdown_connections 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 -- 172.21.15.100:0/1951546250 wait complete. 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.900+0000 7f0987f93740 1 -- start start 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 Processor -- start 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e23615720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aab70 0x558e23615c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e2360fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e236100f0 con 0x558e235b3750 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23610260 con 0x558e235aab70 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e23610420 con 0x558e235aa250 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0987f93740 10 client.0 _ll_register_callbacks cb 0x7f09780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e2360fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aab70 0x558e23615c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e2360fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58044/0 (socket says 172.21.15.100:58044) 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aab70 0x558e23615c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:50032/0 (socket says 172.21.15.100:50032) 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 -- 172.21.15.100:0/3942430419 learned_addr learned my addr 172.21.15.100:0/3942430419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e23615720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 -- 172.21.15.100:0/3942430419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 msgr2=0x558e23615720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e23615720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 -- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aab70 msgr2=0x558e23615c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aab70 0x558e23615c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 -- 172.21.15.100:0/3942430419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e235106a0 con 0x558e235b3750 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f097e7fc640 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e2360fb30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0974009fd0 tx=0x7f097002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff6d6124ce82af2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.901+0000 7f0962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0962ff5640 1 -- 172.21.15.100:0/3942430419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974016040 con 0x558e235b3750 2024-03-12T05:24:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e235a5300 con 0x558e235b3750 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0962ff5640 1 -- 172.21.15.100:0/3942430419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09740246d0 con 0x558e235b3750 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0962ff5640 1 -- 172.21.15.100:0/3942430419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0974024890 con 0x558e235b3750 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 10 client.18099 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558e234d28c0 con 0x558e235b3750 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0962ff5640 1 -- 172.21.15.100:0/3942430419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558e235a5300 con 0x558e235b3750 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0962ff5640 1 -- 172.21.15.100:0/3942430419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09740225e0 con 0x558e235b3750 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 10 client.18099 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 20 client.18099 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e2361d6e0 0x558e2361fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 4 client.18099 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0970004960 con 0x558e2361d6e0 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.902+0000 7f097effd640 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e2361d6e0 0x558e2361fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0962ff5640 10 client.18099 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f097effd640 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e2361d6e0 0x558e2361fb00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795602801625fdb4 server_cookie=b007e4fbec77e699 in_seq=0 out_seq=0 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0962ff5640 1 -- 172.21.15.100:0/3942430419 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0970004960 con 0x558e2361d6e0 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0962ff5640 10 client.18099 handle_command_reply: tid=0 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 client.18099 shutdown 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 msgr2=0x558e2360fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e2360fb30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0974009fd0 tx=0x7f097002bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 shutdown_connections 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558e2361d6e0 0x558e2361fb00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e235b3750 0x558e2360fb30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e235aab70 0x558e23615c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 --2- 172.21.15.100:0/3942430419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e235aa250 0x558e23615720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 >> 172.21.15.100:0/3942430419 conn(0x558e235a4980 msgr2=0x558e235ab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0962ff5640 0 client.18099 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 shutdown_connections 2024-03-12T05:24:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 1 -- 172.21.15.100:0/3942430419 wait complete. 2024-03-12T05:24:19.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.903+0000 7f0987f93740 20 client.18099 trim_cache size 0 max 16384 2024-03-12T05:24:19.906 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f09680a1ba0 msgr2=0x7f09680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f09680a1ba0 0x7f09680a4060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09780cb530 tx=0x7f097806cb60 comp rx=0 tx=0).stop 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 msgr2=0x7f098019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 0x7f098019b460 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f097800eb10 tx=0x7f097800c2d0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 shutdown_connections 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f09680a1ba0 0x7f09680a4060 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f098019afb0 0x7f098019b460 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0980105330 0x7f0980196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 --2- 172.21.15.100:0/1822351876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0980100260 0x7f09801966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 >> 172.21.15.100:0/1822351876 conn(0x7f0980077180 msgr2=0x7f09800fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:19.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 shutdown_connections 2024-03-12T05:24:19.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:19.904+0000 7f09852df640 1 -- 172.21.15.100:0/1822351876 wait complete. 2024-03-12T05:24:20.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:19 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2187922658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:24:20.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:19 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2187922658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:24:20.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:19 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2187922658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:24:20.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1460572 11.52 MB/sec execute 544 sec latency 14.661 ms 2024-03-12T05:24:20.910 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:24:20.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:24:20.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:24:21.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 -- 172.21.15.100:0/3142737897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744100260 msgr2=0x7f6744100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 --2- 172.21.15.100:0/3142737897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744100260 0x7f6744100640 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6734009930 tx=0x7f673402bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 -- 172.21.15.100:0/3142737897 shutdown_connections 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 --2- 172.21.15.100:0/3142737897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744105330 0x7f674410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 --2- 172.21.15.100:0/3142737897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6744100c10 0x7f6744104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 --2- 172.21.15.100:0/3142737897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744100260 0x7f6744100640 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 -- 172.21.15.100:0/3142737897 >> 172.21.15.100:0/3142737897 conn(0x7f6744077180 msgr2=0x7f6744077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 -- 172.21.15.100:0/3142737897 shutdown_connections 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 -- 172.21.15.100:0/3142737897 wait complete. 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 Processor -- start 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.027+0000 7f6749fef640 1 -- start start 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744100260 0x7f674419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 0x7f674419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6744199830 0x7f6744199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674419a2e0 con 0x7f6744105330 2024-03-12T05:24:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674419a450 con 0x7f6744199830 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674419a5c0 con 0x7f6744100260 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 0x7f674419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 0x7f674419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47642/0 (socket says 172.21.15.100:47642) 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 -- 172.21.15.100:0/107096264 learned_addr learned my addr 172.21.15.100:0/107096264 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6743fff640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6744199830 0x7f6744199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f67437fe640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744100260 0x7f674419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 -- 172.21.15.100:0/107096264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744100260 msgr2=0x7f674419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744100260 0x7f674419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 -- 172.21.15.100:0/107096264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6744199830 msgr2=0x7f6744199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6744199830 0x7f6744199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 -- 172.21.15.100:0/107096264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6734009560 con 0x7f6744105330 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6742ffd640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 0x7f674419fc00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f673800e9c0 tx=0x7f673800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6740ff9640 1 -- 172.21.15.100:0/107096264 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738009700 con 0x7f6744105330 2024-03-12T05:24:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67441a6570 con 0x7f6744105330 2024-03-12T05:24:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6740ff9640 1 -- 172.21.15.100:0/107096264 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f673800cd80 con 0x7f6744105330 2024-03-12T05:24:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6740ff9640 1 -- 172.21.15.100:0/107096264 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738010430 con 0x7f6744105330 2024-03-12T05:24:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.028+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67441a6a80 con 0x7f6744105330 2024-03-12T05:24:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.030+0000 7f6740ff9640 1 -- 172.21.15.100:0/107096264 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6738005000 con 0x7f6744105330 2024-03-12T05:24:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 Processor -- start 2024-03-12T05:24:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 -- start start 2024-03-12T05:24:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f6740ff9640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f672c0a1ba0 0x7f672c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 0x55d03e6c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e637f90 con 0x55d03e6c9380 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e6354c0 con 0x55d03e6d28b0 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f674cbbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e630ca0 con 0x55d03e6c9d30 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f67437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 0x55d03e6c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f6743fff640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f672c0a1ba0 0x7f672c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f67437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 0x55d03e6c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47650/0 (socket says 172.21.15.100:47650) 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f67437fe640 1 -- 172.21.15.100:0/2041044563 learned_addr learned my addr 172.21.15.100:0/2041044563 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f6743fff640 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f6740ff9640 1 -- 172.21.15.100:0/107096264 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6738014070 con 0x7f6744105330 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.031+0000 7f6742ffd640 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f67437fe640 1 -- 172.21.15.100:0/2041044563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 msgr2=0x55d03e6d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f67437fe640 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f67437fe640 1 -- 172.21.15.100:0/2041044563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 msgr2=0x55d03e6d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f67437fe640 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6d4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f67437fe640 1 -- 172.21.15.100:0/2041044563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03e5f30b0 con 0x55d03e6c9380 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f6743fff640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f672c0a1ba0 0x7f672c0a4060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f673c005fd0 tx=0x7f673c009ad0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f67437fe640 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 0x55d03e6c9760 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6734036040 tx=0x7f673402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8236005546062c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f6729ffb640 1 -- 172.21.15.100:0/2041044563 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734004800 con 0x55d03e6c9380 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f6729ffb640 1 -- 172.21.15.100:0/2041044563 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6734004960 con 0x55d03e6c9380 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.032+0000 7f6729ffb640 1 -- 172.21.15.100:0/2041044563 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734005080 con 0x55d03e6c9380 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- 172.21.15.100:0/2041044563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 msgr2=0x55d03e6c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 0x55d03e6c9760 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6734036040 tx=0x7f673402c680 comp rx=0 tx=0).stop 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- 172.21.15.100:0/2041044563 shutdown_connections 2024-03-12T05:24:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- 172.21.15.100:0/2041044563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9380 0x55d03e6c9760 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- 172.21.15.100:0/2041044563 >> 172.21.15.100:0/2041044563 conn(0x55d03e6c3840 msgr2=0x55d03e6c3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- 172.21.15.100:0/2041044563 shutdown_connections 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- 172.21.15.100:0/2041044563 wait complete. 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- start start 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 Processor -- start 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e6d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e5d5f40 con 0x55d03e6d28b0 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e3044b0 con 0x55d03e6c9380 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.033+0000 7f674cbbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e5b6670 con 0x55d03e6c9d30 2024-03-12T05:24:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f674cbbd740 10 client.0 _ll_register_callbacks cb 0x55d03e623f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e6d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e6d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47662/0 (socket says 172.21.15.100:47662) 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 -- 172.21.15.100:0/3437696112 learned_addr learned my addr 172.21.15.100:0/3437696112 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6742ffd640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f67437fe640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 -- 172.21.15.100:0/3437696112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 msgr2=0x55d03e6cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 -- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 msgr2=0x55d03e6cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 -- 172.21.15.100:0/3437696112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674410d710 con 0x55d03e6d28b0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6743fff640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e6d0290 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f673c003970 tx=0x7f673c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d857feec90e62444 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6722ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6722ffd640 1 -- 172.21.15.100:0/3437696112 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c021420 con 0x55d03e6d28b0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6722ffd640 1 -- 172.21.15.100:0/3437696112 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f673c021580 con 0x55d03e6d28b0 2024-03-12T05:24:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d03e60af40 con 0x55d03e6d28b0 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f6722ffd640 1 -- 172.21.15.100:0/3437696112 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f673c021740 con 0x55d03e6d28b0 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f674cbbd740 10 client.18111 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.034+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d03e62f320 con 0x55d03e6d28b0 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f6722ffd640 1 -- 172.21.15.100:0/3437696112 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d03e60af40 con 0x55d03e6d28b0 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f6722ffd640 1 -- 172.21.15.100:0/3437696112 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f673c0109a0 con 0x55d03e6d28b0 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f674cbbd740 10 client.18111 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f674cbbd740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f674cbbd740 20 client.18111 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e6e2e20 0x55d03e6e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f674cbbd740 4 client.18111 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d03e627310 con 0x55d03e6e2e20 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f67437fe640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e6e2e20 0x55d03e6e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f6722ffd640 10 client.18111 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.035+0000 7f67437fe640 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e6e2e20 0x55d03e6e5240 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a788c65b6b1d8d3 server_cookie=3c8fa1be0149bc57 in_seq=0 out_seq=0 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f6722ffd640 1 -- 172.21.15.100:0/3437696112 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d03e627310 con 0x55d03e6e2e20 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f6722ffd640 10 client.18111 handle_command_reply: tid=0 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 client.18111 shutdown 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 msgr2=0x55d03e6d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e6d0290 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f673c003970 tx=0x7f673c004ab0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 shutdown_connections 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e6e2e20 0x55d03e6e5240 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e6d0290 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f6722ffd640 0 client.18111 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3437696112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 >> 172.21.15.100:0/3437696112 conn(0x55d03e6c3840 msgr2=0x55d03e6c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 shutdown_connections 2024-03-12T05:24:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 1 -- 172.21.15.100:0/3437696112 wait complete. 2024-03-12T05:24:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.036+0000 7f674cbbd740 20 client.18111 trim_cache size 0 max 16384 2024-03-12T05:24:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.051+0000 7f674cbbd740 1 Processor -- start 2024-03-12T05:24:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 -- start start 2024-03-12T05:24:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9380 0x55d03e6d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 0x55d03e6d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e60af40 con 0x55d03e6c9d30 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e6c4640 con 0x55d03e6d28b0 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f674cbbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e6c91f0 con 0x55d03e6c9380 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 0x55d03e6d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 0x55d03e6d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47672/0 (socket says 172.21.15.100:47672) 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 -- 172.21.15.100:0/3567585624 learned_addr learned my addr 172.21.15.100:0/3567585624 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6743fff640 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f67437fe640 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9380 0x55d03e6d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 -- 172.21.15.100:0/3567585624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9380 msgr2=0x55d03e6d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9380 0x55d03e6d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 -- 172.21.15.100:0/3567585624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 msgr2=0x55d03e6cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 -- 172.21.15.100:0/3567585624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03e5cbc00 con 0x55d03e6c9d30 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f6742ffd640 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 0x55d03e6d00b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6738004fd0 tx=0x7f67380b8f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec4c28f94704ed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.052+0000 7f67297fa640 1 -- 172.21.15.100:0/3567585624 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380631a0 con 0x55d03e6c9d30 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f67297fa640 1 -- 172.21.15.100:0/3567585624 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f673809af20 con 0x55d03e6c9d30 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f67297fa640 1 -- 172.21.15.100:0/3567585624 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738096e90 con 0x55d03e6c9d30 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 -- 172.21.15.100:0/3567585624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 msgr2=0x55d03e6d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 0x55d03e6d00b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6738004fd0 tx=0x7f67380b8f90 comp rx=0 tx=0).stop 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 -- 172.21.15.100:0/3567585624 shutdown_connections 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6d28b0 0x55d03e6cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6c9d30 0x55d03e6d00b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 --2- 172.21.15.100:0/3567585624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9380 0x55d03e6d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 -- 172.21.15.100:0/3567585624 >> 172.21.15.100:0/3567585624 conn(0x55d03e6c3840 msgr2=0x55d03e6ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 -- 172.21.15.100:0/3567585624 shutdown_connections 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.053+0000 7f674cbbd740 1 -- 172.21.15.100:0/3567585624 wait complete. 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 -- start start 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 Processor -- start 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e733470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e7339b0 con 0x55d03e6d28b0 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e733af0 con 0x55d03e6c9380 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e733cb0 con 0x55d03e6c9d30 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f674cbbd740 10 client.0 _ll_register_callbacks cb 0x7f6734009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e733470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f67437fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e733470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47678/0 (socket says 172.21.15.100:47678) 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 -- 172.21.15.100:0/4243228461 learned_addr learned my addr 172.21.15.100:0/4243228461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6742ffd640 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 -- 172.21.15.100:0/4243228461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 msgr2=0x55d03e6d9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 -- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 msgr2=0x55d03e6d8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6d8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.054+0000 7f6743fff640 1 -- 172.21.15.100:0/4243228461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67441a6a80 con 0x55d03e6d28b0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6743fff640 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e733470 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f673c013010 tx=0x7f673c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a08b5edbb9e1fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6722ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6722ffd640 1 -- 172.21.15.100:0/4243228461 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0107f0 con 0x55d03e6d28b0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6722ffd640 1 -- 172.21.15.100:0/4243228461 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f673c012e00 con 0x55d03e6d28b0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d03e656900 con 0x55d03e6d28b0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6722ffd640 1 -- 172.21.15.100:0/4243228461 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f673c004400 con 0x55d03e6d28b0 2024-03-12T05:24:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 10 client.18117 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d03e5f1910 con 0x55d03e6d28b0 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6722ffd640 1 -- 172.21.15.100:0/4243228461 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d03e656900 con 0x55d03e6d28b0 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f6722ffd640 1 -- 172.21.15.100:0/4243228461 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f673c021420 con 0x55d03e6d28b0 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 10 client.18117 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 20 client.18117 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e73b510 0x55d03e73d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 4 client.18117 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.055+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d03e330860 con 0x55d03e73b510 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.056+0000 7f67437fe640 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e73b510 0x55d03e73d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.056+0000 7f6722ffd640 10 client.18117 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.056+0000 7f67437fe640 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e73b510 0x55d03e73d930 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd28ba3f4517c833 server_cookie=ff627fb2b9d55ad7 in_seq=0 out_seq=0 2024-03-12T05:24:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f6722ffd640 1 -- 172.21.15.100:0/4243228461 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d03e330860 con 0x55d03e73b510 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f6722ffd640 10 client.18117 handle_command_reply: tid=0 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 client.18117 shutdown 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 msgr2=0x55d03e733470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e733470 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f673c013010 tx=0x7f673c009940 comp rx=0 tx=0).stop 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 shutdown_connections 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d03e73b510 0x55d03e73d930 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d03e6d28b0 0x55d03e733470 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d03e6c9d30 0x55d03e6d9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 --2- 172.21.15.100:0/4243228461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d03e6c9380 0x55d03e6d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 >> 172.21.15.100:0/4243228461 conn(0x55d03e6c3840 msgr2=0x55d03e6c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f6722ffd640 0 client.18117 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 shutdown_connections 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 1 -- 172.21.15.100:0/4243228461 wait complete. 2024-03-12T05:24:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.057+0000 7f674cbbd740 20 client.18117 trim_cache size 0 max 16384 2024-03-12T05:24:21.059 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:24:21.059 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:24:21.059 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "52ca2f79-b561-438a-b4c5-1237e10f70d8", 2024-03-12T05:24:21.059 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:24:21.059 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f672c0a1ba0 msgr2=0x7f672c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f672c0a1ba0 0x7f672c0a4060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f673c005fd0 tx=0x7f673c009ad0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 msgr2=0x7f674419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 0x7f674419fc00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f673800e9c0 tx=0x7f673800ee80 comp rx=0 tx=0).stop 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 shutdown_connections 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f672c0a1ba0 0x7f672c0a4060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6744199830 0x7f6744199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744105330 0x7f674419fc00 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 --2- 172.21.15.100:0/107096264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744100260 0x7f674419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 >> 172.21.15.100:0/107096264 conn(0x7f6744077180 msgr2=0x7f67440ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 shutdown_connections 2024-03-12T05:24:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.058+0000 7f6749fef640 1 -- 172.21.15.100:0/107096264 wait complete. 2024-03-12T05:24:21.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:24:21.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 -- 172.21.15.100:0/2176372919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc100c20 msgr2=0x7f0efc104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2176372919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc100c20 0x7f0efc104c60 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009930 tx=0x7f0ef002bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 -- 172.21.15.100:0/2176372919 shutdown_connections 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2176372919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0efc105340 0x7f0efc10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2176372919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc100c20 0x7f0efc104c60 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2176372919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc100270 0x7f0efc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 -- 172.21.15.100:0/2176372919 >> 172.21.15.100:0/2176372919 conn(0x7f0efc077230 msgr2=0x7f0efc077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 -- 172.21.15.100:0/2176372919 shutdown_connections 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 -- 172.21.15.100:0/2176372919 wait complete. 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.179+0000 7f0f01c15640 1 Processor -- start 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 -- start start 2024-03-12T05:24:21.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0efc100270 0x7f0efc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 0x7f0efc19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc1997b0 0x7f0efc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19a260 con 0x7f0efc105340 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19a3d0 con 0x7f0efc1997b0 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0f01c15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19a540 con 0x7f0efc100270 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 0x7f0efc19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 0x7f0efc19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47696/0 (socket says 172.21.15.100:47696) 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 -- 172.21.15.100:0/2981357348 learned_addr learned my addr 172.21.15.100:0/2981357348 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0efc100270 0x7f0efc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc1997b0 0x7f0efc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 -- 172.21.15.100:0/2981357348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0efc100270 msgr2=0x7f0efc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0efc100270 0x7f0efc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 -- 172.21.15.100:0/2981357348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc1997b0 msgr2=0x7f0efc199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc1997b0 0x7f0efc199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 -- 172.21.15.100:0/2981357348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0009560 con 0x7f0efc105340 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.180+0000 7f0efaffd640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 0x7f0efc19fb80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005ab0 tx=0x7f0ef002c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.181+0000 7f0ef8ff9640 1 -- 172.21.15.100:0/2981357348 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00042d0 con 0x7f0efc105340 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.181+0000 7f0ef8ff9640 1 -- 172.21.15.100:0/2981357348 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ef0004430 con 0x7f0efc105340 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.181+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc1a6490 con 0x7f0efc105340 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.181+0000 7f0ef8ff9640 1 -- 172.21.15.100:0/2981357348 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0003be0 con 0x7f0efc105340 2024-03-12T05:24:21.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.181+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc1a6970 con 0x7f0efc105340 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0ef8ff9640 1 -- 172.21.15.100:0/2981357348 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0ef0003d40 con 0x7f0efc105340 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 Processor -- start 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 -- start start 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 0x560bea4ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4ad7b0 0x560bea4afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea413280 con 0x560bea4a4bd0 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea4107b0 con 0x560bea4ad7b0 2024-03-12T05:24:21.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0f047eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea40bf90 con 0x560bea4a42b0 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 0x560bea4ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 0x560bea4ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47706/0 (socket says 172.21.15.100:47706) 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0efb7fe640 1 -- 172.21.15.100:0/823436718 learned_addr learned my addr 172.21.15.100:0/823436718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0efbfff640 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.183+0000 7f0efaffd640 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4ad7b0 0x560bea4afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0ef8ff9640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ee40a1c20 0x7f0ee40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efb7fe640 1 -- 172.21.15.100:0/823436718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 msgr2=0x560bea4a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efb7fe640 1 -- 172.21.15.100:0/823436718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4ad7b0 msgr2=0x560bea4afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4ad7b0 0x560bea4afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efb7fe640 1 -- 172.21.15.100:0/823436718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bea3ca570 con 0x560bea4a4bd0 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ee40a1c20 0x7f0ee40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0ef8ff9640 1 -- 172.21.15.100:0/2981357348 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ef00efd40 con 0x7f0efc105340 2024-03-12T05:24:21.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 0x560bea4ad270 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009cf0 tx=0x7f0eec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11a6414b2db34df server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0ee1ffb640 1 -- 172.21.15.100:0/823436718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec009ea0 con 0x560bea4a4bd0 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0ee1ffb640 1 -- 172.21.15.100:0/823436718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eec004230 con 0x560bea4a4bd0 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0ee1ffb640 1 -- 172.21.15.100:0/823436718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec019820 con 0x560bea4a4bd0 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.184+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ee40a1c20 0x7f0ee40a40e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005c40 tx=0x7f0ef4004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 -- 172.21.15.100:0/823436718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 msgr2=0x560bea4ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 0x560bea4ad270 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009cf0 tx=0x7f0eec011810 comp rx=0 tx=0).stop 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 -- 172.21.15.100:0/823436718 shutdown_connections 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4ad7b0 0x560bea4afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a4bd0 0x560bea4ad270 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 --2- 172.21.15.100:0/823436718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 -- 172.21.15.100:0/823436718 >> 172.21.15.100:0/823436718 conn(0x560bea49e990 msgr2=0x560bea49ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 -- 172.21.15.100:0/823436718 shutdown_connections 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 -- 172.21.15.100:0/823436718 wait complete. 2024-03-12T05:24:21.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.185+0000 7f0f047eb740 1 -- start start 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 Processor -- start 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 0x560bea4ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a4bd0 0x560bea4aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea0df430 con 0x560bea4ad7b0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea09be00 con 0x560bea4a42b0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea390fd0 con 0x560bea4a4bd0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0f047eb740 10 client.0 _ll_register_callbacks cb 0x560bea3e6fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 0x560bea4ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a4bd0 0x560bea4aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47710/0 (socket says 172.21.15.100:47710) 2024-03-12T05:24:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 -- 172.21.15.100:0/2266792465 learned_addr learned my addr 172.21.15.100:0/2266792465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 0x560bea4ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:43992/0 (socket says 172.21.15.100:43992) 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 -- 172.21.15.100:0/2266792465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a4bd0 msgr2=0x560bea4aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a4bd0 0x560bea4aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 -- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 msgr2=0x560bea4ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 0x560bea4ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 -- 172.21.15.100:0/2266792465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec009950 con 0x560bea4ad7b0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 0x560bea4ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4aacb0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400fac0 tx=0x7f0ef4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfced3d8e872390 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.186+0000 7f0edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0edaffd640 1 -- 172.21.15.100:0/2266792465 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4022070 con 0x560bea4ad7b0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0edaffd640 1 -- 172.21.15.100:0/2266792465 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ef401d5f0 con 0x560bea4ad7b0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bea10b7e0 con 0x560bea4ad7b0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0edaffd640 1 -- 172.21.15.100:0/2266792465 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ef401d7b0 con 0x560bea4ad7b0 2024-03-12T05:24:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 10 client.18129 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bea40a610 con 0x560bea4ad7b0 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0edaffd640 1 -- 172.21.15.100:0/2266792465 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560bea10b7e0 con 0x560bea4ad7b0 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0edaffd640 1 -- 172.21.15.100:0/2266792465 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ef4026a90 con 0x560bea4ad7b0 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 10 client.18129 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 20 client.18129 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea4bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 4 client.18129 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bea148720 con 0x560bea4bd8f0 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.187+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea4bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.188+0000 7f0edaffd640 10 client.18129 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.188+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea4bfd10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb430786ee3d0f4c server_cookie=e62a10d90ecba7f4 in_seq=0 out_seq=0 2024-03-12T05:24:21.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.188+0000 7f0edaffd640 1 -- 172.21.15.100:0/2266792465 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560bea148720 con 0x560bea4bd8f0 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.188+0000 7f0edaffd640 10 client.18129 handle_command_reply: tid=0 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 client.18129 shutdown 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 msgr2=0x560bea4aacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4aacb0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400fac0 tx=0x7f0ef4004ab0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 shutdown_connections 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea4bfd10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0edaffd640 0 client.18129 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4aacb0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a4bd0 0x560bea4aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2266792465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a42b0 0x560bea4ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 >> 172.21.15.100:0/2266792465 conn(0x560bea49e990 msgr2=0x560bea4a0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 shutdown_connections 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 1 -- 172.21.15.100:0/2266792465 wait complete. 2024-03-12T05:24:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.189+0000 7f0f047eb740 20 client.18129 trim_cache size 0 max 16384 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 Processor -- start 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 -- start start 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea4b9dd0 con 0x560bea4ad7b0 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea4ba440 con 0x560bea4a4bd0 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.205+0000 7f0f047eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea4a3f40 con 0x560bea4a42b0 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:48112/0 (socket says 172.21.15.100:48112) 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47712/0 (socket says 172.21.15.100:47712) 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 -- 172.21.15.100:0/3385210181 learned_addr learned my addr 172.21.15.100:0/3385210181 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efaffd640 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 -- 172.21.15.100:0/3385210181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 msgr2=0x560bea4acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 -- 172.21.15.100:0/3385210181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 msgr2=0x560bea4ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 -- 172.21.15.100:0/3385210181 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bea3a6ac0 con 0x560bea4a42b0 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efbfff640 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4c4690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0eec004440 tx=0x7f0eec019420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0cf3749253aca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0ee17fa640 1 -- 172.21.15.100:0/3385210181 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec029070 con 0x560bea4a42b0 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0ee17fa640 1 -- 172.21.15.100:0/3385210181 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eec02e070 con 0x560bea4a42b0 2024-03-12T05:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.206+0000 7f0ee17fa640 1 -- 172.21.15.100:0/3385210181 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0035f0 con 0x560bea4a42b0 2024-03-12T05:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 -- 172.21.15.100:0/3385210181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 msgr2=0x560bea4c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4c4690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0eec004440 tx=0x7f0eec019420 comp rx=0 tx=0).stop 2024-03-12T05:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 -- 172.21.15.100:0/3385210181 shutdown_connections 2024-03-12T05:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4ad7b0 0x560bea4ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 --2- 172.21.15.100:0/3385210181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4a42b0 0x560bea4c4690 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 -- 172.21.15.100:0/3385210181 >> 172.21.15.100:0/3385210181 conn(0x560bea49e990 msgr2=0x560bea49ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 -- 172.21.15.100:0/3385210181 shutdown_connections 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 -- 172.21.15.100:0/3385210181 wait complete. 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.207+0000 7f0f047eb740 1 -- start start 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 Processor -- start 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 0x560bea4b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4ad7b0 0x560bea4b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea513e10 con 0x560bea4a42b0 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea513f50 con 0x560bea4a4bd0 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bea5140e0 con 0x560bea4ad7b0 2024-03-12T05:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0f047eb740 10 client.0 _ll_register_callbacks cb 0x7f0eec0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 0x560bea4b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 0x560bea4b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47724/0 (socket says 172.21.15.100:47724) 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 -- 172.21.15.100:0/2530448521 learned_addr learned my addr 172.21.15.100:0/2530448521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efaffd640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efbfff640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4ad7b0 0x560bea4b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 -- 172.21.15.100:0/2530448521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4ad7b0 msgr2=0x560bea4b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4ad7b0 0x560bea4b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 -- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 msgr2=0x560bea4b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 -- 172.21.15.100:0/2530448521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bea40a610 con 0x560bea4a42b0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0efb7fe640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 0x560bea4b1130 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0eec005f50 tx=0x7f0eec011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e663e061b8b3c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.208+0000 7f0edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0edaffd640 1 -- 172.21.15.100:0/2530448521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec029070 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0edaffd640 1 -- 172.21.15.100:0/2530448521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0eec0054f0 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eec004230 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0edaffd640 1 -- 172.21.15.100:0/2530448521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eec009ea0 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 10 client.18135 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bea3ca570 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0edaffd640 1 -- 172.21.15.100:0/2530448521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eec004230 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0edaffd640 1 -- 172.21.15.100:0/2530448521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0eec021040 con 0x560bea4a42b0 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 10 client.18135 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 20 client.18135 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea5176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 4 client.18135 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560bea197890 con 0x560bea4bd8f0 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.209+0000 7f0efaffd640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea5176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0efaffd640 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea5176a0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9eb6b73960cebf7 server_cookie=49f51e334f502633 in_seq=0 out_seq=0 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0edaffd640 10 client.18135 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0edaffd640 1 -- 172.21.15.100:0/2530448521 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560bea197890 con 0x560bea4bd8f0 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0edaffd640 10 client.18135 handle_command_reply: tid=0 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 client.18135 shutdown 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 msgr2=0x560bea4b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 0x560bea4b1130 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0eec005f50 tx=0x7f0eec011e50 comp rx=0 tx=0).stop 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 shutdown_connections 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560bea4bd8f0 0x560bea5176a0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bea4ad7b0 0x560bea4b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560bea4a4bd0 0x560bea4b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0edaffd640 0 client.18135 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 --2- 172.21.15.100:0/2530448521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bea4a42b0 0x560bea4b1130 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 >> 172.21.15.100:0/2530448521 conn(0x560bea49e990 msgr2=0x560bea4c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 shutdown_connections 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 1 -- 172.21.15.100:0/2530448521 wait complete. 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.210+0000 7f0f047eb740 20 client.18135 trim_cache size 0 max 16384 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "52ca2f79-b561-438a-b4c5-1237e10f70d8": { 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:24:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "52ca2f79-b561-438a-b4c5-1237e10f70d8", 2024-03-12T05:24:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:24:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:24:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:24:21.213 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ee40a1c20 msgr2=0x7f0ee40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ee40a1c20 0x7f0ee40a40e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005c40 tx=0x7f0ef4004a40 comp rx=0 tx=0).stop 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 msgr2=0x7f0efc19fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 0x7f0efc19fb80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005ab0 tx=0x7f0ef002c5f0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 shutdown_connections 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ee40a1c20 0x7f0ee40a40e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc1997b0 0x7f0efc199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc105340 0x7f0efc19fb80 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 --2- 172.21.15.100:0/2981357348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0efc100270 0x7f0efc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 >> 172.21.15.100:0/2981357348 conn(0x7f0efc077230 msgr2=0x7f0efc0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 shutdown_connections 2024-03-12T05:24:21.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:21.211+0000 7f0f01c15640 1 -- 172.21.15.100:0/2981357348 wait complete. 2024-03-12T05:24:21.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:20 smithi100 ceph-mon[32210]: pgmap v382: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 103 op/s 2024-03-12T05:24:21.216 INFO:tasks.cephfs.filesystem:scrub status for tag:52ca2f79-b561-438a-b4c5-1237e10f70d8 - {'path': '/', 'tag': '52ca2f79-b561-438a-b4c5-1237e10f70d8', 'options': 'recursive,force'} 2024-03-12T05:24:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:20 smithi161 ceph-mon[36043]: pgmap v382: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 103 op/s 2024-03-12T05:24:21.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:20 smithi156 ceph-mon[36270]: pgmap v382: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 103 op/s 2024-03-12T05:24:21.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1462409 11.51 MB/sec execute 545 sec latency 18.298 ms 2024-03-12T05:24:22.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:21 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:24:22.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:21 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:21 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:24:22.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:21 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:24:22.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:21 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:21 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:21 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:24:22.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:21 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:21 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:24:22.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:21 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:24:22.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:21 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:21 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:21 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:24:22.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:21 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:21 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:24:22.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:21 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:24:22.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:21 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:21 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:22.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1464291 11.51 MB/sec execute 546 sec latency 60.384 ms 2024-03-12T05:24:23.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:22 smithi161 ceph-mon[36043]: pgmap v383: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:22 smithi100 ceph-mon[32210]: pgmap v383: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:23.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:22 smithi156 ceph-mon[36270]: pgmap v383: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1466188 11.50 MB/sec execute 547 sec latency 39.803 ms 2024-03-12T05:24:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1468093 11.50 MB/sec execute 548 sec latency 14.746 ms 2024-03-12T05:24:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:24 smithi161 ceph-mon[36043]: pgmap v384: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:24 smithi156 ceph-mon[36270]: pgmap v384: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:24 smithi100 ceph-mon[32210]: pgmap v384: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1470671 11.51 MB/sec execute 549 sec latency 17.348 ms 2024-03-12T05:24:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:25 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:24:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:25 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:24:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:25 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:24:26.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1472056 11.50 MB/sec execute 550 sec latency 52.327 ms 2024-03-12T05:24:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:26 smithi100 ceph-mon[32210]: pgmap v385: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:26 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:26 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:26 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:26 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:26 smithi156 ceph-mon[36270]: pgmap v385: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:26 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:26 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:26 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:26 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:26 smithi161 ceph-mon[36043]: pgmap v385: 97 pgs: 97 active+clean; 593 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 131 op/s 2024-03-12T05:24:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:26 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:26 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:26 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:26 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1474502 11.51 MB/sec execute 551 sec latency 73.629 ms 2024-03-12T05:24:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:27 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:24:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:27 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:24:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:27 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:27 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:24:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:27 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:24:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:27 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:27 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:24:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:27 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:24:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:27 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:24:28.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1475881 11.50 MB/sec execute 552 sec latency 114.192 ms 2024-03-12T05:24:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:28 smithi100 ceph-mon[32210]: pgmap v386: 97 pgs: 97 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 203 op/s 2024-03-12T05:24:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:28 smithi156 ceph-mon[36270]: pgmap v386: 97 pgs: 97 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 203 op/s 2024-03-12T05:24:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:28 smithi161 ceph-mon[36043]: pgmap v386: 97 pgs: 97 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 203 op/s 2024-03-12T05:24:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1478224 11.50 MB/sec execute 553 sec latency 17.089 ms 2024-03-12T05:24:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:29 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:24:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:29 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:24:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:29 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:24:30.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1479785 11.49 MB/sec execute 554 sec latency 23.239 ms 2024-03-12T05:24:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:30 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:24:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:30 smithi100 ceph-mon[32210]: pgmap v387: 97 pgs: 97 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 169 op/s 2024-03-12T05:24:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:30 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:24:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:30 smithi156 ceph-mon[36270]: pgmap v387: 97 pgs: 97 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 169 op/s 2024-03-12T05:24:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:30 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:24:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:30 smithi161 ceph-mon[36043]: pgmap v387: 97 pgs: 97 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 169 op/s 2024-03-12T05:24:31.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1482044 11.50 MB/sec execute 555 sec latency 15.593 ms 2024-03-12T05:24:32.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1484241 11.49 MB/sec execute 556 sec latency 26.212 ms 2024-03-12T05:24:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:32 smithi100 ceph-mon[32210]: pgmap v388: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.9 MiB/s wr, 264 op/s 2024-03-12T05:24:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:32 smithi156 ceph-mon[36270]: pgmap v388: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.9 MiB/s wr, 264 op/s 2024-03-12T05:24:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:32 smithi161 ceph-mon[36043]: pgmap v388: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.9 MiB/s wr, 264 op/s 2024-03-12T05:24:33.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1485626 11.49 MB/sec execute 557 sec latency 63.237 ms 2024-03-12T05:24:34.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1488028 11.49 MB/sec execute 558 sec latency 19.913 ms 2024-03-12T05:24:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:34 smithi100 ceph-mon[32210]: pgmap v389: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-03-12T05:24:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:34 smithi156 ceph-mon[36270]: pgmap v389: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-03-12T05:24:35.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:34 smithi161 ceph-mon[36043]: pgmap v389: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-03-12T05:24:35.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1489828 11.49 MB/sec execute 559 sec latency 23.222 ms 2024-03-12T05:24:36.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:36 smithi100 ceph-mon[32210]: pgmap v390: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-03-12T05:24:36.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:36 smithi156 ceph-mon[36270]: pgmap v390: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-03-12T05:24:36.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:36 smithi161 ceph-mon[36043]: pgmap v390: 97 pgs: 97 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-03-12T05:24:36.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1492355 11.49 MB/sec execute 560 sec latency 36.264 ms 2024-03-12T05:24:37.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1493933 11.48 MB/sec execute 561 sec latency 23.993 ms 2024-03-12T05:24:38.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1495993 11.49 MB/sec execute 562 sec latency 61.198 ms 2024-03-12T05:24:39.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:38 smithi161 ceph-mon[36043]: pgmap v391: 97 pgs: 97 active+clean; 634 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 282 op/s 2024-03-12T05:24:39.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:38 smithi100 ceph-mon[32210]: pgmap v391: 97 pgs: 97 active+clean; 634 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 282 op/s 2024-03-12T05:24:39.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:38 smithi156 ceph-mon[36270]: pgmap v391: 97 pgs: 97 active+clean; 634 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 282 op/s 2024-03-12T05:24:39.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1497925 11.48 MB/sec execute 563 sec latency 16.722 ms 2024-03-12T05:24:40.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1499815 11.48 MB/sec execute 564 sec latency 20.368 ms 2024-03-12T05:24:41.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:40 smithi161 ceph-mon[36043]: pgmap v392: 97 pgs: 97 active+clean; 634 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 188 op/s 2024-03-12T05:24:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:40 smithi100 ceph-mon[32210]: pgmap v392: 97 pgs: 97 active+clean; 634 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 188 op/s 2024-03-12T05:24:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:40 smithi156 ceph-mon[36270]: pgmap v392: 97 pgs: 97 active+clean; 634 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 188 op/s 2024-03-12T05:24:41.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1501822 11.48 MB/sec execute 565 sec latency 73.024 ms 2024-03-12T05:24:42.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1503643 11.48 MB/sec execute 566 sec latency 28.851 ms 2024-03-12T05:24:43.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:42 smithi161 ceph-mon[36043]: pgmap v393: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 286 op/s 2024-03-12T05:24:43.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:42 smithi100 ceph-mon[32210]: pgmap v393: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 286 op/s 2024-03-12T05:24:43.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:42 smithi156 ceph-mon[36270]: pgmap v393: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 286 op/s 2024-03-12T05:24:43.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1505758 11.48 MB/sec execute 567 sec latency 24.607 ms 2024-03-12T05:24:44.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1507478 11.47 MB/sec execute 568 sec latency 25.572 ms 2024-03-12T05:24:45.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:44 smithi161 ceph-mon[36043]: pgmap v394: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:24:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:44 smithi100 ceph-mon[32210]: pgmap v394: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:24:45.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:44 smithi156 ceph-mon[36270]: pgmap v394: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:24:45.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1509924 11.48 MB/sec execute 569 sec latency 18.462 ms 2024-03-12T05:24:46.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1511382 11.47 MB/sec execute 570 sec latency 25.685 ms 2024-03-12T05:24:47.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:46 smithi161 ceph-mon[36043]: pgmap v395: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:24:47.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:46 smithi100 ceph-mon[32210]: pgmap v395: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:24:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:46 smithi156 ceph-mon[36270]: pgmap v395: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-03-12T05:24:47.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1513719 11.47 MB/sec execute 571 sec latency 18.397 ms 2024-03-12T05:24:48.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1515695 11.47 MB/sec execute 572 sec latency 18.958 ms 2024-03-12T05:24:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:48 smithi100 ceph-mon[32210]: pgmap v396: 97 pgs: 97 active+clean; 586 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 268 op/s 2024-03-12T05:24:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:48 smithi156 ceph-mon[36270]: pgmap v396: 97 pgs: 97 active+clean; 586 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 268 op/s 2024-03-12T05:24:49.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:48 smithi161 ceph-mon[36043]: pgmap v396: 97 pgs: 97 active+clean; 586 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 268 op/s 2024-03-12T05:24:49.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1517604 11.47 MB/sec execute 573 sec latency 41.860 ms 2024-03-12T05:24:50.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1519870 11.47 MB/sec execute 574 sec latency 27.365 ms 2024-03-12T05:24:51.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:24:51.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:24:51.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:50 smithi100 ceph-mon[32210]: pgmap v397: 97 pgs: 97 active+clean; 586 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 174 op/s 2024-03-12T05:24:51.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 -- 172.21.15.100:0/1914164445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 msgr2=0x7f364810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 --2- 172.21.15.100:0/1914164445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364810cf10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3640009cd0 tx=0x7f364002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 -- 172.21.15.100:0/1914164445 shutdown_connections 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 --2- 172.21.15.100:0/1914164445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364810cf10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 --2- 172.21.15.100:0/1914164445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100c20 0x7f3648104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 --2- 172.21.15.100:0/1914164445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3648100270 0x7f3648100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 -- 172.21.15.100:0/1914164445 >> 172.21.15.100:0/1914164445 conn(0x7f3648077230 msgr2=0x7f3648077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 -- 172.21.15.100:0/1914164445 shutdown_connections 2024-03-12T05:24:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 -- 172.21.15.100:0/1914164445 wait complete. 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.333+0000 7f364eb0d640 1 Processor -- start 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 -- start start 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100270 0x7f364819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36481975a0 0x7f3648197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648197f90 con 0x7f3648105340 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648198100 con 0x7f36481975a0 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364eb0d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36481982c0 con 0x7f3648100270 2024-03-12T05:24:51.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100270 0x7f364819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100270 0x7f364819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41462/0 (socket says 172.21.15.100:41462) 2024-03-12T05:24:51.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364d0aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36481975a0 0x7f3648197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f364c8a9640 1 -- 172.21.15.100:0/2000780447 learned_addr learned my addr 172.21.15.100:0/2000780447 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 -- 172.21.15.100:0/2000780447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100270 msgr2=0x7f364819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100270 0x7f364819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 -- 172.21.15.100:0/2000780447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36481975a0 msgr2=0x7f3648197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36481975a0 0x7f3648197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 -- 172.21.15.100:0/2000780447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009900 con 0x7f3648105340 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.334+0000 7f3647fff640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364819d930 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f364806a8f0 tx=0x7f363400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.335+0000 7f3645ffb640 1 -- 172.21.15.100:0/2000780447 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634004020 con 0x7f3648105340 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.335+0000 7f3645ffb640 1 -- 172.21.15.100:0/2000780447 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3634004180 con 0x7f3648105340 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.335+0000 7f3645ffb640 1 -- 172.21.15.100:0/2000780447 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634004a00 con 0x7f3648105340 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.335+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36481a42b0 con 0x7f3648105340 2024-03-12T05:24:51.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.335+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36481a4770 con 0x7f3648105340 2024-03-12T05:24:51.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.337+0000 7f3645ffb640 1 -- 172.21.15.100:0/2000780447 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f36340042f0 con 0x7f3648105340 2024-03-12T05:24:51.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.337+0000 7f36517b9740 1 Processor -- start 2024-03-12T05:24:51.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.337+0000 7f36517b9740 1 -- start start 2024-03-12T05:24:51.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3645ffb640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36380a1ba0 0x7f36380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f36517b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b22b0 0x558b588b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f36517b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b2bd0 0x558b588bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f36517b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f36517b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b58821280 con 0x558b588b2bd0 2024-03-12T05:24:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f36517b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5881e7b0 con 0x558b588b22b0 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f36517b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b58819f90 con 0x558b588bb7b0 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f364d0aa640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36380a1ba0 0x7f36380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b22b0 0x558b588b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b2bd0 0x558b588bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b22b0 0x558b588b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:33966/0 (socket says 172.21.15.100:33966) 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f364c8a9640 1 -- 172.21.15.100:0/1069348895 learned_addr learned my addr 172.21.15.100:0/1069348895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f364d0aa640 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3645ffb640 1 -- 172.21.15.100:0/2000780447 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f36340cd3b0 con 0x7f3648105340 2024-03-12T05:24:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 -- 172.21.15.100:0/1069348895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 msgr2=0x558b588bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 -- 172.21.15.100:0/1069348895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b22b0 msgr2=0x558b588b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b22b0 0x558b588b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 -- 172.21.15.100:0/1069348895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b587d8570 con 0x558b588b2bd0 2024-03-12T05:24:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f364d0aa640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36380a1ba0 0x7f36380a4060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3640005cc0 tx=0x7f3640005f80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3647fff640 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b2bd0 0x558b588bb270 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f36340caac0 tx=0x7f3634050060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2149e3666ef17a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.338+0000 7f3632ffd640 1 -- 172.21.15.100:0/1069348895 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634099900 con 0x558b588b2bd0 2024-03-12T05:24:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f3632ffd640 1 -- 172.21.15.100:0/1069348895 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36340b2eb0 con 0x558b588b2bd0 2024-03-12T05:24:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f3632ffd640 1 -- 172.21.15.100:0/1069348895 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340982a0 con 0x558b588b2bd0 2024-03-12T05:24:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 -- 172.21.15.100:0/1069348895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b2bd0 msgr2=0x558b588bb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b2bd0 0x558b588bb270 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f36340caac0 tx=0x7f3634050060 comp rx=0 tx=0).stop 2024-03-12T05:24:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 -- 172.21.15.100:0/1069348895 shutdown_connections 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b2bd0 0x558b588bb270 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 --2- 172.21.15.100:0/1069348895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b22b0 0x558b588b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 -- 172.21.15.100:0/1069348895 >> 172.21.15.100:0/1069348895 conn(0x558b588ac990 msgr2=0x558b588acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 -- 172.21.15.100:0/1069348895 shutdown_connections 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.339+0000 7f36517b9740 1 -- 172.21.15.100:0/1069348895 wait complete. 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 -- start start 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 Processor -- start 2024-03-12T05:24:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b2bd0 0x558b588b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588bb7b0 0x558b588b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b584ed430 con 0x558b588b22b0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b584a9e00 con 0x558b588bb7b0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5879efd0 con 0x558b588b2bd0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f36517b9740 10 client.0 _ll_register_callbacks cb 0x558b587f4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588bb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57122/0 (socket says 172.21.15.100:57122) 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 -- 172.21.15.100:0/24262114 learned_addr learned my addr 172.21.15.100:0/24262114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364d0aa640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588bb7b0 0x558b588b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f3647fff640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b2bd0 0x558b588b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 -- 172.21.15.100:0/24262114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b2bd0 msgr2=0x558b588b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b2bd0 0x558b588b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 -- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588bb7b0 msgr2=0x558b588b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588bb7b0 0x558b588b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.340+0000 7f364c8a9640 1 -- 172.21.15.100:0/24262114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36481a4770 con 0x558b588b22b0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f364c8a9640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588bb1d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f363c015be0 tx=0x7f363c00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9596262d169076 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3627fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3627fff640 1 -- 172.21.15.100:0/24262114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c021070 con 0x558b588b22b0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3627fff640 1 -- 172.21.15.100:0/24262114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f363c004020 con 0x558b588b22b0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b585197e0 con 0x558b588b22b0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3627fff640 1 -- 172.21.15.100:0/24262114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363c018050 con 0x558b588b22b0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 10 client.18147 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558b58818610 con 0x558b588b22b0 2024-03-12T05:24:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3627fff640 1 -- 172.21.15.100:0/24262114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b585197e0 con 0x558b588b22b0 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3627fff640 1 -- 172.21.15.100:0/24262114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f363c024020 con 0x558b588b22b0 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 10 client.18147 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 10 client.18147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 20 client.18147 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b588cb990 0x558b588cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 4 client.18147 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b58556720 con 0x558b588cb990 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.341+0000 7f3647fff640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b588cb990 0x558b588cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.342+0000 7f3627fff640 10 client.18147 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.342+0000 7f3647fff640 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b588cb990 0x558b588cddb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bfe77890edf988 server_cookie=10d72de47aaf847a in_seq=0 out_seq=0 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.342+0000 7f3627fff640 1 -- 172.21.15.100:0/24262114 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b58556720 con 0x558b588cb990 2024-03-12T05:24:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.342+0000 7f3627fff640 10 client.18147 handle_command_reply: tid=0 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.342+0000 7f36517b9740 1 client.18147 shutdown 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 msgr2=0x558b588bb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588bb1d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f363c015be0 tx=0x7f363c00e780 comp rx=0 tx=0).stop 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 shutdown_connections 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b588cb990 0x558b588cddb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588bb7b0 0x558b588b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b2bd0 0x558b588b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f3627fff640 0 client.18147 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 --2- 172.21.15.100:0/24262114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588bb1d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 >> 172.21.15.100:0/24262114 conn(0x558b588ac990 msgr2=0x558b588ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 shutdown_connections 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 1 -- 172.21.15.100:0/24262114 wait complete. 2024-03-12T05:24:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.343+0000 7f36517b9740 20 client.18147 trim_cache size 0 max 16384 2024-03-12T05:24:51.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 Processor -- start 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 -- start start 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b22b0 0x558b588bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 0x558b588b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b588ad9e0 con 0x558b588bb7b0 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b588b1cf0 con 0x558b588b2bd0 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f36517b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b588ac7f0 con 0x558b588b22b0 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f364d0aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 0x558b588b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f364d0aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 0x558b588b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57132/0 (socket says 172.21.15.100:57132) 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f364d0aa640 1 -- 172.21.15.100:0/1429482655 learned_addr learned my addr 172.21.15.100:0/1429482655 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f3647fff640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f364c8a9640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b22b0 0x558b588bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f3647fff640 1 -- 172.21.15.100:0/1429482655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b22b0 msgr2=0x558b588bac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.359+0000 7f3647fff640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b22b0 0x558b588bac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f3647fff640 1 -- 172.21.15.100:0/1429482655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 msgr2=0x558b588b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f3647fff640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 0x558b588b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f3647fff640 1 -- 172.21.15.100:0/1429482655 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b587b4ac0 con 0x558b588b2bd0 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f364d0aa640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 0x558b588b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f364c8a9640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b22b0 0x558b588bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f3647fff640 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bb1d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f363409a160 tx=0x7f36340052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572e8bdf85092d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f36327fc640 1 -- 172.21.15.100:0/1429482655 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340b2cd0 con 0x558b588b2bd0 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f36327fc640 1 -- 172.21.15.100:0/1429482655 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f363409a6a0 con 0x558b588b2bd0 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.360+0000 7f36327fc640 1 -- 172.21.15.100:0/1429482655 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634096610 con 0x558b588b2bd0 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 -- 172.21.15.100:0/1429482655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 msgr2=0x558b588bb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bb1d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f363409a160 tx=0x7f36340052d0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 -- 172.21.15.100:0/1429482655 shutdown_connections 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588bb7b0 0x558b588b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bb1d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 --2- 172.21.15.100:0/1429482655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588b22b0 0x558b588bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 -- 172.21.15.100:0/1429482655 >> 172.21.15.100:0/1429482655 conn(0x558b588ac990 msgr2=0x558b588b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 -- 172.21.15.100:0/1429482655 shutdown_connections 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 -- 172.21.15.100:0/1429482655 wait complete. 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 -- start start 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.361+0000 7f36517b9740 1 Processor -- start 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b588bf7e0 con 0x558b588b22b0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b588bf950 con 0x558b588b2bd0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b588bfb10 con 0x558b588bb7b0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f36517b9740 10 client.0 _ll_register_callbacks cb 0x7f363c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364d0aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f3647fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57140/0 (socket says 172.21.15.100:57140) 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 -- 172.21.15.100:0/2236506358 learned_addr learned my addr 172.21.15.100:0/2236506358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364d0aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41500/0 (socket says 172.21.15.100:41500) 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 -- 172.21.15.100:0/2236506358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 msgr2=0x558b588bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 -- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 msgr2=0x558b588bec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 -- 172.21.15.100:0/2236506358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b58818610 con 0x558b588b22b0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364d0aa640 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f364c8a9640 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588c2910 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f363c0139c0 tx=0x7f363c013d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f6b98f5c477e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f3627fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.362+0000 7f3627fff640 1 -- 172.21.15.100:0/2236506358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c021040 con 0x558b588b22b0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f3627fff640 1 -- 172.21.15.100:0/2236506358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f363c005450 con 0x558b588b22b0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36340b2eb0 con 0x558b588b22b0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f3627fff640 1 -- 172.21.15.100:0/2236506358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363c018040 con 0x558b588b22b0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 10 client.18153 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f364810d720 con 0x558b588b22b0 2024-03-12T05:24:51.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f3627fff640 1 -- 172.21.15.100:0/2236506358 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36340b2eb0 con 0x558b588b22b0 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f3627fff640 1 -- 172.21.15.100:0/2236506358 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f363c024040 con 0x558b588b22b0 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 10 client.18153 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 20 client.18153 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b58923570 0x558b58925990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 4 client.18153 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b585a5890 con 0x558b58923570 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.363+0000 7f3647fff640 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b58923570 0x558b58925990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f3627fff640 10 client.18153 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:51.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f3647fff640 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b58923570 0x558b58925990 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83db28f343508209 server_cookie=14a84fe9998becd6 in_seq=0 out_seq=0 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f3627fff640 1 -- 172.21.15.100:0/2236506358 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558b585a5890 con 0x558b58923570 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f3627fff640 10 client.18153 handle_command_reply: tid=0 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 client.18153 shutdown 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 msgr2=0x558b588c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588c2910 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f363c0139c0 tx=0x7f363c013d40 comp rx=0 tx=0).stop 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 shutdown_connections 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558b58923570 0x558b58925990 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558b588bb7b0 0x558b588bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b588b2bd0 0x558b588bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f3627fff640 0 client.18153 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:51.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 --2- 172.21.15.100:0/2236506358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b588b22b0 0x558b588c2910 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 >> 172.21.15.100:0/2236506358 conn(0x558b588ac990 msgr2=0x558b588b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 shutdown_connections 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 1 -- 172.21.15.100:0/2236506358 wait complete. 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.364+0000 7f36517b9740 20 client.18153 trim_cache size 0 max 16384 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:24:51.371 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:24:51.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36380a1ba0 msgr2=0x7f36380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36380a1ba0 0x7f36380a4060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3640005cc0 tx=0x7f3640005f80 comp rx=0 tx=0).stop 2024-03-12T05:24:51.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 msgr2=0x7f364819d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364819d930 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f364806a8f0 tx=0x7f363400dbb0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 shutdown_connections 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36380a1ba0 0x7f36380a4060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36481975a0 0x7f3648197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3648105340 0x7f364819d930 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 --2- 172.21.15.100:0/2000780447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3648100270 0x7f364819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 >> 172.21.15.100:0/2000780447 conn(0x7f3648077230 msgr2=0x7f3648103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 shutdown_connections 2024-03-12T05:24:51.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.365+0000 7f364eb0d640 1 -- 172.21.15.100:0/2000780447 wait complete. 2024-03-12T05:24:51.374 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:24:51.375 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:24:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:50 smithi156 ceph-mon[36270]: pgmap v397: 97 pgs: 97 active+clean; 586 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 174 op/s 2024-03-12T05:24:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:50 smithi161 ceph-mon[36043]: pgmap v397: 97 pgs: 97 active+clean; 586 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 174 op/s 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 -- 172.21.15.100:0/2751041448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea40fff70 msgr2=0x7f0ea4100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 --2- 172.21.15.100:0/2751041448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea40fff70 0x7f0ea4100350 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e9402bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 -- 172.21.15.100:0/2751041448 shutdown_connections 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 --2- 172.21.15.100:0/2751041448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ea410db20 0x7f0ea410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 --2- 172.21.15.100:0/2751041448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100920 0x7f0ea410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 --2- 172.21.15.100:0/2751041448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea40fff70 0x7f0ea4100350 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 -- 172.21.15.100:0/2751041448 >> 172.21.15.100:0/2751041448 conn(0x7f0ea4077180 msgr2=0x7f0ea4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 -- 172.21.15.100:0/2751041448 shutdown_connections 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.527+0000 7f0eaa664640 1 -- 172.21.15.100:0/2751041448 wait complete. 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 Processor -- start 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 -- start start 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ea40fff70 0x7f0ea419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100920 0x7f0ea419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 0x7f0ea41998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4199f00 con 0x7f0ea410db20 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419a070 con 0x7f0ea4100920 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0eaa664640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419a230 con 0x7f0ea40fff70 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0ea8c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 0x7f0ea41998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0ea8c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 0x7f0ea41998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57162/0 (socket says 172.21.15.100:57162) 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0ea8c01640 1 -- 172.21.15.100:0/856187769 learned_addr learned my addr 172.21.15.100:0/856187769 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.528+0000 7f0ea37fe640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100920 0x7f0ea419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea3fff640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ea40fff70 0x7f0ea419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea8c01640 1 -- 172.21.15.100:0/856187769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ea40fff70 msgr2=0x7f0ea419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea8c01640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ea40fff70 0x7f0ea419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea8c01640 1 -- 172.21.15.100:0/856187769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100920 msgr2=0x7f0ea419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea8c01640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100920 0x7f0ea419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea8c01640 1 -- 172.21.15.100:0/856187769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e94009560 con 0x7f0ea410db20 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea8c01640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 0x7f0ea41998b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9d0 tx=0x7f0e9c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00cd80 con 0x7f0ea410db20 2024-03-12T05:24:51.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea419a510 con 0x7f0ea410db20 2024-03-12T05:24:51.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c005020 con 0x7f0ea410db20 2024-03-12T05:24:51.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0116b0 con 0x7f0ea410db20 2024-03-12T05:24:51.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.529+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea41a67a0 con 0x7f0ea410db20 2024-03-12T05:24:51.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.530+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea4067f70 con 0x7f0ea410db20 2024-03-12T05:24:51.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.531+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0e9c005500 con 0x7f0ea410db20 2024-03-12T05:24:51.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.531+0000 7f0ea17fa640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e8c0a1ba0 0x7f0e8c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.532+0000 7f0ea3fff640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e8c0a1ba0 0x7f0e8c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.532+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0e9c0cd920 con 0x7f0ea410db20 2024-03-12T05:24:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.532+0000 7f0ea3fff640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e8c0a1ba0 0x7f0e8c0a4060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009230 tx=0x7f0e94005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.536+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0e9c096860 con 0x7f0ea410db20 2024-03-12T05:24:51.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.779+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ea4108cb0 con 0x7f0ea410db20 2024-03-12T05:24:51.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.780+0000 7f0ea17fa640 1 -- 172.21.15.100:0/856187769 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f0e9c099f20 con 0x7f0ea410db20 2024-03-12T05:24:51.775 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:24:51.775 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:15:59.702888+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:24:51.775 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-03-12T05:24:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e8c0a1ba0 msgr2=0x7f0e8c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e8c0a1ba0 0x7f0e8c0a4060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009230 tx=0x7f0e94005fd0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 msgr2=0x7f0ea41998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 0x7f0ea41998b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9d0 tx=0x7f0e9c00ee90 comp rx=0 tx=0).stop 2024-03-12T05:24:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 shutdown_connections 2024-03-12T05:24:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e8c0a1ba0 0x7f0e8c0a4060 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea410db20 0x7f0ea41998b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100920 0x7f0ea419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 --2- 172.21.15.100:0/856187769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ea40fff70 0x7f0ea419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 >> 172.21.15.100:0/856187769 conn(0x7f0ea4077180 msgr2=0x7f0ea410f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 shutdown_connections 2024-03-12T05:24:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.782+0000 7f0eaa664640 1 -- 172.21.15.100:0/856187769 wait complete. 2024-03-12T05:24:51.789 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:24:51.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:24:51.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1521481 11.46 MB/sec execute 575 sec latency 33.646 ms 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 -- 172.21.15.100:0/3119637680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 msgr2=0x7ff95010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 --2- 172.21.15.100:0/3119637680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95010cfb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff948009cd0 tx=0x7ff94802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 -- 172.21.15.100:0/3119637680 shutdown_connections 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 --2- 172.21.15.100:0/3119637680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95010cfb0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 --2- 172.21.15.100:0/3119637680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff950100cc0 0x7ff950104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 --2- 172.21.15.100:0/3119637680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff950100310 0x7ff9501006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 -- 172.21.15.100:0/3119637680 >> 172.21.15.100:0/3119637680 conn(0x7ff9500771e0 msgr2=0x7ff9500775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 -- 172.21.15.100:0/3119637680 shutdown_connections 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.906+0000 7ff95548f640 1 -- 172.21.15.100:0/3119637680 wait complete. 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 Processor -- start 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 -- start start 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff950100310 0x7ff95019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff950197660 0x7ff950197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950198130 con 0x7ff9501053e0 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9501982a0 con 0x7ff950100310 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff95548f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950198460 con 0x7ff950197660 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff950100310 0x7ff95019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57172/0 (socket says 172.21.15.100:57172) 2024-03-12T05:24:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 -- 172.21.15.100:0/1093069675 learned_addr learned my addr 172.21.15.100:0/1093069675 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff950197660 0x7ff950197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 -- 172.21.15.100:0/1093069675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff950197660 msgr2=0x7ff950197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff950197660 0x7ff950197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 -- 172.21.15.100:0/1093069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff950100310 msgr2=0x7ff95019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff950100310 0x7ff95019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.907+0000 7ff94e7fc640 1 -- 172.21.15.100:0/1093069675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948009900 con 0x7ff9501053e0 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.908+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95019dae0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff94400e9c0 tx=0x7ff94400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.908+0000 7ff937fff640 1 -- 172.21.15.100:0/1093069675 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944009700 con 0x7ff9501053e0 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.908+0000 7ff937fff640 1 -- 172.21.15.100:0/1093069675 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff94400cd80 con 0x7ff9501053e0 2024-03-12T05:24:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.908+0000 7ff937fff640 1 -- 172.21.15.100:0/1093069675 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944010430 con 0x7ff9501053e0 2024-03-12T05:24:51.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.908+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9501a4400 con 0x7ff9501053e0 2024-03-12T05:24:51.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.908+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9501a48e0 con 0x7ff9501053e0 2024-03-12T05:24:51.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.910+0000 7ff937fff640 1 -- 172.21.15.100:0/1093069675 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff944005000 con 0x7ff9501053e0 2024-03-12T05:24:51.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.910+0000 7ff937fff640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff9380a1ba0 0x7ff9380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.910+0000 7ff95805d740 1 Processor -- start 2024-03-12T05:24:51.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.910+0000 7ff95805d740 1 -- start start 2024-03-12T05:24:51.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94effd640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff9380a1ba0 0x7ff9380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff937fff640 1 -- 172.21.15.100:0/1093069675 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff944014070 con 0x7ff9501053e0 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff95805d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3416b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff95805d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 0x562e34174210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff95805d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e34174750 0x562e34176b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff95805d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34126960 con 0x562e3416bb70 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff95805d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340da310 con 0x562e3416b250 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff95805d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340d7840 con 0x562e34174750 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 0x562e34174210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 0x562e34174210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57178/0 (socket says 172.21.15.100:57178) 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3416b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 -- 172.21.15.100:0/2511048478 learned_addr learned my addr 172.21.15.100:0/2511048478 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94effd640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff9380a1ba0 0x7ff9380a4060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff940009c20 tx=0x7ff940005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 -- 172.21.15.100:0/2511048478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e34174750 msgr2=0x562e34176b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e34174750 0x562e34176b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 -- 172.21.15.100:0/2511048478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 msgr2=0x562e3416b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3416b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 -- 172.21.15.100:0/2511048478 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e34095060 con 0x562e3416bb70 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.911+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 0x562e34174210 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff944096cc0 tx=0x7ff944096500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f8d13b78ff3a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.912+0000 7ff9357fa640 1 -- 172.21.15.100:0/2511048478 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94409e9b0 con 0x562e3416bb70 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.912+0000 7ff9357fa640 1 -- 172.21.15.100:0/2511048478 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9440a3000 con 0x562e3416bb70 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.912+0000 7ff9357fa640 1 -- 172.21.15.100:0/2511048478 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944005510 con 0x562e3416bb70 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- 172.21.15.100:0/2511048478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 msgr2=0x562e34174210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 0x562e34174210 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff944096cc0 tx=0x7ff944096500 comp rx=0 tx=0).stop 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- 172.21.15.100:0/2511048478 shutdown_connections 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e34174750 0x562e34176b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416bb70 0x562e34174210 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- 172.21.15.100:0/2511048478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3416b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- 172.21.15.100:0/2511048478 >> 172.21.15.100:0/2511048478 conn(0x562e34165980 msgr2=0x562e34165d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- 172.21.15.100:0/2511048478 shutdown_connections 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- 172.21.15.100:0/2511048478 wait complete. 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- start start 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 Processor -- start 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416b250 0x562e341717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34171d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 0x562e34172270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340d3020 con 0x562e3416b250 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e33da6430 con 0x562e34174750 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e33d62e00 con 0x562e3416bb70 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff95805d740 10 client.0 _ll_register_callbacks cb 0x562e3407acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff94effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416b250 0x562e341717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 0x562e34172270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff94e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34171d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.913+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 0x562e34172270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34048/0 (socket says 172.21.15.100:34048) 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 -- 172.21.15.100:0/476086884 learned_addr learned my addr 172.21.15.100:0/476086884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 -- 172.21.15.100:0/476086884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 msgr2=0x562e34171d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34171d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 -- 172.21.15.100:0/476086884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416b250 msgr2=0x562e341717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416b250 0x562e341717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 -- 172.21.15.100:0/476086884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9501a48e0 con 0x562e34174750 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94effd640 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416b250 0x562e341717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 0x562e34172270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff948004af0 tx=0x7ff948034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3656a13f098e943f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff932ff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff932ff5640 1 -- 172.21.15.100:0/476086884 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480347c0 con 0x562e34174750 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e33dd27e0 con 0x562e34174750 2024-03-12T05:24:51.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.914+0000 7ff932ff5640 1 -- 172.21.15.100:0/476086884 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff948034c90 con 0x562e34174750 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff932ff5640 1 -- 172.21.15.100:0/476086884 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff948034e50 con 0x562e34174750 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 10 client.27539 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562e340d16a0 con 0x562e34174750 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff932ff5640 1 -- 172.21.15.100:0/476086884 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562e33dd27e0 con 0x562e34174750 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff932ff5640 1 -- 172.21.15.100:0/476086884 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff948004270 con 0x562e34174750 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 10 client.27539 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 10 client.27539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 20 client.27539 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e34184b00 0x562e34186ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 4 client.27539 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e33e0f720 con 0x562e34184b00 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.915+0000 7ff94effd640 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e34184b00 0x562e34186ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.916+0000 7ff932ff5640 10 client.27539 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.916+0000 7ff94effd640 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e34184b00 0x562e34186ef0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4050a2f0dab17396 server_cookie=2707f1ac58f78b0d in_seq=0 out_seq=0 2024-03-12T05:24:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff932ff5640 1 -- 172.21.15.100:0/476086884 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e33e0f720 con 0x562e34184b00 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff932ff5640 10 client.27539 handle_command_reply: tid=0 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 client.27539 shutdown 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 msgr2=0x562e34172270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 0x562e34172270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff948004af0 tx=0x7ff948034440 comp rx=0 tx=0).stop 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 shutdown_connections 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e34184b00 0x562e34186ef0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e34174750 0x562e34172270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34171d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff932ff5640 0 client.27539 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 --2- 172.21.15.100:0/476086884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3416b250 0x562e341717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 >> 172.21.15.100:0/476086884 conn(0x562e34165980 msgr2=0x562e341676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 shutdown_connections 2024-03-12T05:24:51.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 1 -- 172.21.15.100:0/476086884 wait complete. 2024-03-12T05:24:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.917+0000 7ff95805d740 20 client.27539 trim_cache size 0 max 16384 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 Processor -- start 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 -- start start 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e34173f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34172090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e3416cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34058130 con 0x562e34174750 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.930+0000 7ff95805d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34184960 con 0x562e3416b250 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff95805d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3418c0e0 con 0x562e3416bb70 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e3416cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e34173f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e3416cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57188/0 (socket says 172.21.15.100:57188) 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e34173f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34060/0 (socket says 172.21.15.100:34060) 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 -- 172.21.15.100:0/1351144090 learned_addr learned my addr 172.21.15.100:0/1351144090 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34172090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 -- 172.21.15.100:0/1351144090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 msgr2=0x562e34172090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34172090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 -- 172.21.15.100:0/1351144090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 msgr2=0x562e34173f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e34173f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 -- 172.21.15.100:0/1351144090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e3406dc20 con 0x562e34174750 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34172090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff94f7fe640 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e3416cad0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff948002230 tx=0x7ff94802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c2ad0ec6d614fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff934ff9640 1 -- 172.21.15.100:0/1351144090 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480347c0 con 0x562e34174750 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff934ff9640 1 -- 172.21.15.100:0/1351144090 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9480326b0 con 0x562e34174750 2024-03-12T05:24:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.931+0000 7ff934ff9640 1 -- 172.21.15.100:0/1351144090 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948049a70 con 0x562e34174750 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 -- 172.21.15.100:0/1351144090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 msgr2=0x562e3416cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e3416cad0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff948002230 tx=0x7ff94802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 -- 172.21.15.100:0/1351144090 shutdown_connections 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e3416cad0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e34172090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 --2- 172.21.15.100:0/1351144090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e34173f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 -- 172.21.15.100:0/1351144090 >> 172.21.15.100:0/1351144090 conn(0x562e34165980 msgr2=0x562e34165d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 -- 172.21.15.100:0/1351144090 shutdown_connections 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.932+0000 7ff95805d740 1 -- 172.21.15.100:0/1351144090 wait complete. 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 -- start start 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 Processor -- start 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3417a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e3417a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e341dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3417b160 con 0x562e34174750 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e341db500 con 0x562e3416b250 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e341db640 con 0x562e3416bb70 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff95805d740 10 client.0 _ll_register_callbacks cb 0x7ff94400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e341dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e341dafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57200/0 (socket says 172.21.15.100:57200) 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3417a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e3417a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94f7fe640 1 -- 172.21.15.100:0/1981554114 learned_addr learned my addr 172.21.15.100:0/1981554114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94effd640 1 -- 172.21.15.100:0/1981554114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 msgr2=0x562e3417a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94effd640 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e3417a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94effd640 1 -- 172.21.15.100:0/1981554114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 msgr2=0x562e341dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94effd640 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e341dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.933+0000 7ff94effd640 1 -- 172.21.15.100:0/1981554114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e34095060 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff94effd640 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3417a350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x562e34172c50 tx=0x7ff940013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68183473f9d8388d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff932ff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff932ff5640 1 -- 172.21.15.100:0/1981554114 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94001cb60 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff932ff5640 1 -- 172.21.15.100:0/1981554114 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff94001ccc0 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9440a3000 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff932ff5640 1 -- 172.21.15.100:0/1981554114 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94001ce80 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 10 client.27545 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562e340d16a0 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff932ff5640 1 -- 172.21.15.100:0/1981554114 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff940040080 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff932ff5640 1 -- 172.21.15.100:0/1981554114 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff9400035f0 con 0x562e3416b250 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 10 client.27545 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 10 client.27545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 20 client.27545 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e341dcdf0 0x562e341df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 4 client.27545 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:24:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.934+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e33e5e890 con 0x562e341dcdf0 2024-03-12T05:24:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.935+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e341dcdf0 0x562e341df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.935+0000 7ff932ff5640 10 client.27545 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.935+0000 7ff94e7fc640 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e341dcdf0 0x562e341df210 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f74dba1e42e96f server_cookie=ec450ef0229ee5dd in_seq=0 out_seq=0 2024-03-12T05:24:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.936+0000 7ff932ff5640 1 -- 172.21.15.100:0/1981554114 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e33e5e890 con 0x562e341dcdf0 2024-03-12T05:24:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.936+0000 7ff932ff5640 10 client.27545 handle_command_reply: tid=0 2024-03-12T05:24:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.936+0000 7ff95805d740 1 client.27545 shutdown 2024-03-12T05:24:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.936+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 msgr2=0x562e3417a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.936+0000 7ff95805d740 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3417a350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x562e34172c50 tx=0x7ff940013040 comp rx=0 tx=0).stop 2024-03-12T05:24:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 shutdown_connections 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x562e341dcdf0 0x562e341df210 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e34174750 0x562e341dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff932ff5640 0 client.27545 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e3416bb70 0x562e3417a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 --2- 172.21.15.100:0/1981554114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e3416b250 0x562e3417a350 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 >> 172.21.15.100:0/1981554114 conn(0x562e34165980 msgr2=0x562e3416fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 shutdown_connections 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 1 -- 172.21.15.100:0/1981554114 wait complete. 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95805d740 20 client.27545 trim_cache size 0 max 16384 2024-03-12T05:24:51.938 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:24:51.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff9380a1ba0 msgr2=0x7ff9380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff9380a1ba0 0x7ff9380a4060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff940009c20 tx=0x7ff940005f90 comp rx=0 tx=0).stop 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 msgr2=0x7ff95019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95019dae0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff94400e9c0 tx=0x7ff94400ee80 comp rx=0 tx=0).stop 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 shutdown_connections 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff9380a1ba0 0x7ff9380a4060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff950197660 0x7ff950197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9501053e0 0x7ff95019dae0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 --2- 172.21.15.100:0/1093069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff950100310 0x7ff95019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 >> 172.21.15.100:0/1093069675 conn(0x7ff9500771e0 msgr2=0x7ff950069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 shutdown_connections 2024-03-12T05:24:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:51.937+0000 7ff95548f640 1 -- 172.21.15.100:0/1093069675 wait complete. 2024-03-12T05:24:51.944 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:24:51.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:24:52.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/856187769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:24:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.101+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3723149763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc8100c10 msgr2=0x7f1fc8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3723149763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc8100c10 0x7f1fc8104c50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009930 tx=0x7f1fbc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3723149763 shutdown_connections 2024-03-12T05:24:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3723149763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8105330 0x7f1fc810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3723149763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc8100c10 0x7f1fc8104c50 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3723149763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 0x7f1fc8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3723149763 >> 172.21.15.100:0/3723149763 conn(0x7f1fc8077180 msgr2=0x7f1fc8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3723149763 shutdown_connections 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3723149763 wait complete. 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 Processor -- start 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.102+0000 7f1fccfbc640 1 -- start start 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 0x7f1fc8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8105330 0x7f1fc8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 0x7f1fc819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc819b9c0 con 0x7f1fc819afd0 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc819bb30 con 0x7f1fc8100260 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc819bca0 con 0x7f1fc8105330 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 0x7f1fc819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 0x7f1fc8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 0x7f1fc8196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34076/0 (socket says 172.21.15.100:34076) 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 0x7f1fc819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57208/0 (socket says 172.21.15.100:57208) 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc659c640 1 -- 172.21.15.100:0/3738266438 learned_addr learned my addr 172.21.15.100:0/3738266438 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8105330 0x7f1fc8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 -- 172.21.15.100:0/3738266438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8105330 msgr2=0x7f1fc8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8105330 0x7f1fc8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 -- 172.21.15.100:0/3738266438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 msgr2=0x7f1fc8196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 0x7f1fc8196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 -- 172.21.15.100:0/3738266438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009560 con 0x7f1fc819afd0 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fc6d9d640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 0x7f1fc819b480 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c710 tx=0x7f1fb800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fb77fe640 1 -- 172.21.15.100:0/3738266438 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8004220 con 0x7f1fc819afd0 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fb77fe640 1 -- 172.21.15.100:0/3738266438 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8007e90 con 0x7f1fc819afd0 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc81a3280 con 0x7f1fc819afd0 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fb77fe640 1 -- 172.21.15.100:0/3738266438 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8003da0 con 0x7f1fc819afd0 2024-03-12T05:24:52.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.103+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc81a3700 con 0x7f1fc819afd0 2024-03-12T05:24:52.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fb77fe640 1 -- 172.21.15.100:0/3738266438 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1fb80079b0 con 0x7f1fc819afd0 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 Processor -- start 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 -- start start 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040250 0x563a32040630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32049210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 0x563a3204bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31ffb960 con 0x563a32049750 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31faf310 con 0x563a32040b70 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31fac840 con 0x563a32040250 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.106+0000 7f1fb77fe640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1fb00a1ba0 0x7f1fb00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 0x563a3204bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 0x563a3204bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57220/0 (socket says 172.21.15.100:57220) 2024-03-12T05:24:52.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32049210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2432336093 learned_addr learned my addr 172.21.15.100:0/2432336093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc6d9d640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1fb00a1ba0 0x7f1fb00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040250 0x563a32040630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fb77fe640 1 -- 172.21.15.100:0/3738266438 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1fb80cd2d0 con 0x7f1fc819afd0 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2432336093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040250 msgr2=0x563a32040630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040250 0x563a32040630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2432336093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 msgr2=0x563a32049210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32049210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2432336093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a31f6a060 con 0x563a32049750 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc659c640 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32049210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:52.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc6d9d640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1fb00a1ba0 0x7f1fb00a4060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8096940 tx=0x7f1fb8003800 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 0x563a3204bb40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009930 tx=0x7f1fbc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd46a4947cc1e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fb4ff9640 1 -- 172.21.15.100:0/2432336093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc02c920 con 0x563a32049750 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fb4ff9640 1 -- 172.21.15.100:0/2432336093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fbc004510 con 0x563a32049750 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.107+0000 7f1fb4ff9640 1 -- 172.21.15.100:0/2432336093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc003c30 con 0x563a32049750 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2432336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 msgr2=0x563a3204bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 0x563a3204bb40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009930 tx=0x7f1fbc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2432336093 shutdown_connections 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32049750 0x563a3204bb40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32049210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2432336093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040250 0x563a32040630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2432336093 >> 172.21.15.100:0/2432336093 conn(0x563a3203a980 msgr2=0x563a3203ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2432336093 shutdown_connections 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2432336093 wait complete. 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 -- start start 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.108+0000 7f1fcfb91740 1 Processor -- start 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32046d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32047250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31fa8020 con 0x563a32040250 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31c7b430 con 0x563a32040b70 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31c37e00 con 0x563a32049750 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 10 client.0 _ll_register_callbacks cb 0x563a31f4fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:52.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57232/0 (socket says 172.21.15.100:57232) 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2610163334 learned_addr learned my addr 172.21.15.100:0/2610163334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc659c640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32046d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32047250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 msgr2=0x563a32047250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32047250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2610163334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 msgr2=0x563a32046d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32046d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/2610163334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc810d710 con 0x563a32040250 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc659c640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32046d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:52.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320467d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc02c420 tx=0x7f1fbc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1194e9f1c8c2c9a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fadffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fadffb640 1 -- 172.21.15.100:0/2610163334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc041760 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fadffb640 1 -- 172.21.15.100:0/2610163334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1fbc000e40 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.109+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a31ca77e0 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fadffb640 1 -- 172.21.15.100:0/2610163334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fbc0418c0 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 10 client.18189 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563a31fa66a0 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fadffb640 1 -- 172.21.15.100:0/2610163334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563a31ca77e0 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fadffb640 1 -- 172.21.15.100:0/2610163334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fbc05f070 con 0x563a32040250 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 10 client.18189 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 10 client.18189 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 20 client.18189 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a32059850 0x563a3205bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 4 client.18189 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.110+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a31ce4720 con 0x563a32059850 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.111+0000 7f1fc659c640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a32059850 0x563a3205bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.111+0000 7f1fadffb640 10 client.18189 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.111+0000 7f1fc659c640 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a32059850 0x563a3205bc40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=213b6ff305785aa1 server_cookie=71557ed3829ea022 in_seq=0 out_seq=0 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fadffb640 1 -- 172.21.15.100:0/2610163334 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a31ce4720 con 0x563a32059850 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fadffb640 10 client.18189 handle_command_reply: tid=0 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 client.18189 shutdown 2024-03-12T05:24:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 msgr2=0x563a320467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320467d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc02c420 tx=0x7f1fbc005fd0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 shutdown_connections 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32047250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a32059850 0x563a3205bc40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040b70 0x563a32046d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fadffb640 0 client.18189 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/2610163334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320467d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 >> 172.21.15.100:0/2610163334 conn(0x563a3203a980 msgr2=0x563a3203c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 shutdown_connections 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 1 -- 172.21.15.100:0/2610163334 wait complete. 2024-03-12T05:24:52.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.112+0000 7f1fcfb91740 20 client.18189 trim_cache size 0 max 16384 2024-03-12T05:24:52.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.128+0000 7f1fcfb91740 1 Processor -- start 2024-03-12T05:24:52.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.128+0000 7f1fcfb91740 1 -- start start 2024-03-12T05:24:52.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040250 0x563a32048f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 0x563a32041a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32041fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31f2d130 con 0x563a32040b70 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3203b980 con 0x563a32040250 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a32060e50 con 0x563a32049750 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 0x563a32041a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 0x563a32041a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57242/0 (socket says 172.21.15.100:57242) 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 -- 172.21.15.100:0/4131521349 learned_addr learned my addr 172.21.15.100:0/4131521349 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040250 0x563a32048f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32041fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 -- 172.21.15.100:0/4131521349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 msgr2=0x563a32041fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32041fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 -- 172.21.15.100:0/4131521349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040250 msgr2=0x563a32048f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040250 0x563a32048f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 -- 172.21.15.100:0/4131521349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a31f42c20 con 0x563a32040b70 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1fc659c640 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 0x563a32041a90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x563a32046030 tx=0x7f1fc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac406acac17f64f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.129+0000 7f1faffff640 1 -- 172.21.15.100:0/4131521349 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0019070 con 0x563a32040b70 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1faffff640 1 -- 172.21.15.100:0/4131521349 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fc001a040 con 0x563a32040b70 2024-03-12T05:24:52.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1faffff640 1 -- 172.21.15.100:0/4131521349 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00208d0 con 0x563a32040b70 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 -- 172.21.15.100:0/4131521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 msgr2=0x563a32041a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 0x563a32041a90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x563a32046030 tx=0x7f1fc0004a40 comp rx=0 tx=0).stop 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 -- 172.21.15.100:0/4131521349 shutdown_connections 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32049750 0x563a32041fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040b70 0x563a32041a90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/4131521349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32040250 0x563a32048f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 -- 172.21.15.100:0/4131521349 >> 172.21.15.100:0/4131521349 conn(0x563a3203a980 msgr2=0x563a32035a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 -- 172.21.15.100:0/4131521349 shutdown_connections 2024-03-12T05:24:52.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.130+0000 7f1fcfb91740 1 -- 172.21.15.100:0/4131521349 wait complete. 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 -- start start 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 Processor -- start 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040b70 0x563a320a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32049750 0x563a3204fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3204ffc0 con 0x563a32040250 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a32050130 con 0x563a32049750 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a320502f0 con 0x563a32040b70 2024-03-12T05:24:52.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fcfb91740 10 client.0 _ll_register_callbacks cb 0x7f1fbc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320a91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57246/0 (socket says 172.21.15.100:57246) 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/95309688 learned_addr learned my addr 172.21.15.100:0/95309688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc659c640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040b70 0x563a320a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32049750 0x563a3204fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040b70 msgr2=0x563a320a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040b70 0x563a320a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/95309688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32049750 msgr2=0x563a3204fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32049750 0x563a3204fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 -- 172.21.15.100:0/95309688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc0013bd0 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fc5d9b640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320a91c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x563a320478e0 tx=0x7f1fbc02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276800179c1cb903 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fadffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.131+0000 7f1fadffb640 1 -- 172.21.15.100:0/95309688 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc004ff0 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fadffb640 1 -- 172.21.15.100:0/95309688 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1fbc000e40 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a32046600 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fadffb640 1 -- 172.21.15.100:0/95309688 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fbc0057c0 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 10 client.18195 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563a31f688c0 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fadffb640 1 -- 172.21.15.100:0/95309688 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563a32046600 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fadffb640 1 -- 172.21.15.100:0/95309688 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fbc05f070 con 0x563a32040250 2024-03-12T05:24:52.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 10 client.18195 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 10 client.18195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 20 client.18195 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a320b1d10 0x563a320b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 4 client.18195 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f1fbc004510 con 0x563a320b1d10 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.132+0000 7f1fc659c640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a320b1d10 0x563a320b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fc659c640 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a320b1d10 0x563a320b4100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa6c42f729927c66 server_cookie=34d0bcd614f954b in_seq=0 out_seq=0 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fadffb640 10 client.18195 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fadffb640 1 -- 172.21.15.100:0/95309688 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f1fbc004510 con 0x563a320b1d10 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fadffb640 10 client.18195 handle_command_reply: tid=0 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 client.18195 shutdown 2024-03-12T05:24:52.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 msgr2=0x563a320a91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320a91c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x563a320478e0 tx=0x7f1fbc02c710 comp rx=0 tx=0).stop 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 shutdown_connections 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563a320b1d10 0x563a320b4100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a32049750 0x563a3204fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a32040b70 0x563a320a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fadffb640 0 client.18195 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 --2- 172.21.15.100:0/95309688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a32040250 0x563a320a91c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 >> 172.21.15.100:0/95309688 conn(0x563a3203a980 msgr2=0x563a32041a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 shutdown_connections 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 1 -- 172.21.15.100:0/95309688 wait complete. 2024-03-12T05:24:52.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fcfb91740 20 client.18195 trim_cache size 0 max 16384 2024-03-12T05:24:52.139 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:24:52.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1fb00a1ba0 msgr2=0x7f1fb00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1fb00a1ba0 0x7f1fb00a4060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8096940 tx=0x7f1fb8003800 comp rx=0 tx=0).stop 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 msgr2=0x7f1fc819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 0x7f1fc819b480 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c710 tx=0x7f1fb800cbd0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 shutdown_connections 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1fb00a1ba0 0x7f1fb00a4060 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc819afd0 0x7f1fc819b480 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8105330 0x7f1fc8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.133+0000 7f1fccfbc640 1 --2- 172.21.15.100:0/3738266438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fc8100260 0x7f1fc8196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.134+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 >> 172.21.15.100:0/3738266438 conn(0x7f1fc8077180 msgr2=0x7f1fc80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.134+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 shutdown_connections 2024-03-12T05:24:52.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.134+0000 7f1fccfbc640 1 -- 172.21.15.100:0/3738266438 wait complete. 2024-03-12T05:24:52.142 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:24:52.142 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:24:52.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 -- 172.21.15.100:0/1208215111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14100570 msgr2=0x7fec14105ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1208215111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14100570 0x7fec14105ca0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fec08009930 tx=0x7fec0802bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 -- 172.21.15.100:0/1208215111 shutdown_connections 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1208215111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 0x7fec1410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1208215111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14100570 0x7fec14105ca0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1208215111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 0x7fec140fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.255+0000 7fec18fa0640 1 -- 172.21.15.100:0/1208215111 >> 172.21.15.100:0/1208215111 conn(0x7fec140752a0 msgr2=0x7fec140756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.256+0000 7fec18fa0640 1 -- 172.21.15.100:0/1208215111 shutdown_connections 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.256+0000 7fec18fa0640 1 -- 172.21.15.100:0/1208215111 wait complete. 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.256+0000 7fec18fa0640 1 Processor -- start 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.256+0000 7fec18fa0640 1 -- start start 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.256+0000 7fec18fa0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 0x7fec141976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 0x7fec14197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 0x7fec1419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c970 con 0x7fec1419bf80 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419cae0 con 0x7fec140ffbc0 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419cc50 con 0x7fec14106380 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 0x7fec1419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 0x7fec1419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57266/0 (socket says 172.21.15.100:57266) 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 -- 172.21.15.100:0/1201377553 learned_addr learned my addr 172.21.15.100:0/1201377553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec137fe640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 0x7fec141976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec12ffd640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 0x7fec14197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 -- 172.21.15.100:0/1201377553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 msgr2=0x7fec14197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 0x7fec14197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 -- 172.21.15.100:0/1201377553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 msgr2=0x7fec141976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 0x7fec141976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 -- 172.21.15.100:0/1201377553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08009560 con 0x7fec1419bf80 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec137fe640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 0x7fec141976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:52.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec12ffd640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 0x7fec14197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:52.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec13fff640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 0x7fec1419c430 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00eb10 tx=0x7fec0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec10ff9640 1 -- 172.21.15.100:0/1201377553 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c004b20 con 0x7fec1419bf80 2024-03-12T05:24:52.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec10ff9640 1 -- 172.21.15.100:0/1201377553 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec0c004c80 con 0x7fec1419bf80 2024-03-12T05:24:52.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec141a4230 con 0x7fec1419bf80 2024-03-12T05:24:52.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec10ff9640 1 -- 172.21.15.100:0/1201377553 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c010650 con 0x7fec1419bf80 2024-03-12T05:24:52.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.257+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141a46b0 con 0x7fec1419bf80 2024-03-12T05:24:52.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.260+0000 7fec10ff9640 1 -- 172.21.15.100:0/1201377553 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fec0c005020 con 0x7fec1419bf80 2024-03-12T05:24:52.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 Processor -- start 2024-03-12T05:24:52.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 -- start start 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec10ff9640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7febfc0a1ba0 0x7febfc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 0x56252ee73820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee7c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee7ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ee2e960 con 0x56252ee73440 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ede2310 con 0x56252ee7c940 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec1cdce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252eddf840 con 0x56252ee73d60 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec12ffd640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7febfc0a1ba0 0x7febfc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 0x56252ee73820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 0x56252ee73820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57280/0 (socket says 172.21.15.100:57280) 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec13fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee7ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec137fe640 1 -- 172.21.15.100:0/483310916 learned_addr learned my addr 172.21.15.100:0/483310916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec12ffd640 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee7c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec10ff9640 1 -- 172.21.15.100:0/1201377553 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec0c014070 con 0x7fec1419bf80 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec13fff640 1 -- 172.21.15.100:0/483310916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 msgr2=0x56252ee7c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec13fff640 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee7c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec13fff640 1 -- 172.21.15.100:0/483310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 msgr2=0x56252ee73820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec13fff640 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 0x56252ee73820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec13fff640 1 -- 172.21.15.100:0/483310916 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56252ed9d060 con 0x56252ee7c940 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.261+0000 7fec137fe640 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 0x56252ee73820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec13fff640 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee7ed30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0cb440 tx=0x7fec0c00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2dd38fcc8390e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec12ffd640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7febfc0a1ba0 0x7febfc0a4060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec08000c00 tx=0x7fec08005d90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7febf9ffb640 1 -- 172.21.15.100:0/483310916 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c078360 con 0x56252ee7c940 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7febf9ffb640 1 -- 172.21.15.100:0/483310916 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec0c078180 con 0x56252ee7c940 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7febf9ffb640 1 -- 172.21.15.100:0/483310916 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c09ea70 con 0x56252ee7c940 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 -- 172.21.15.100:0/483310916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 msgr2=0x56252ee7ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee7ed30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0cb440 tx=0x7fec0c00cf70 comp rx=0 tx=0).stop 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 -- 172.21.15.100:0/483310916 shutdown_connections 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee7ed30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee7c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 --2- 172.21.15.100:0/483310916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73440 0x56252ee73820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.262+0000 7fec1cdce740 1 -- 172.21.15.100:0/483310916 >> 172.21.15.100:0/483310916 conn(0x56252ee6db70 msgr2=0x56252ee6df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 -- 172.21.15.100:0/483310916 shutdown_connections 2024-03-12T05:24:52.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 -- 172.21.15.100:0/483310916 wait complete. 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 -- start start 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 Processor -- start 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73440 0x56252ee799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee79f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee7a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252eddb020 con 0x56252ee7c940 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252eaae430 con 0x56252ee73440 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ea6ae00 con 0x56252ee73d60 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.263+0000 7fec1cdce740 10 client.0 _ll_register_callbacks cb 0x56252ed82cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee7a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec12ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee79f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee7a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57286/0 (socket says 172.21.15.100:57286) 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 -- 172.21.15.100:0/3492946645 learned_addr learned my addr 172.21.15.100:0/3492946645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec137fe640 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73440 0x56252ee799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 -- 172.21.15.100:0/3492946645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 msgr2=0x56252ee79f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee79f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 -- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73440 msgr2=0x56252ee799c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73440 0x56252ee799c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 -- 172.21.15.100:0/3492946645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141a46b0 con 0x56252ee7c940 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec13fff640 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee7a440 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0972f0 tx=0x7fec0c0205e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d86f8313a0689c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7febf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7febf2ffd640 1 -- 172.21.15.100:0/3492946645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c078320 con 0x56252ee7c940 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7febf2ffd640 1 -- 172.21.15.100:0/3492946645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec0c09b020 con 0x56252ee7c940 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.264+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56252eada7e0 con 0x56252ee7c940 2024-03-12T05:24:52.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7febf2ffd640 1 -- 172.21.15.100:0/3492946645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec0c0203a0 con 0x56252ee7c940 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 10 client.18207 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56252edd96a0 con 0x56252ee7c940 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7febf2ffd640 1 -- 172.21.15.100:0/3492946645 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56252eada7e0 con 0x56252ee7c940 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7febf2ffd640 1 -- 172.21.15.100:0/3492946645 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec0c01f060 con 0x56252ee7c940 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 10 client.18207 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 20 client.18207 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252ee8ca50 0x56252ee8ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 4 client.18207 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56252eb17720 con 0x56252ee8ca50 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.265+0000 7fec137fe640 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252ee8ca50 0x56252ee8ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.266+0000 7fec137fe640 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252ee8ca50 0x56252ee8ee40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca26a73a36ce307 server_cookie=83ed5d08993c9e1 in_seq=0 out_seq=0 2024-03-12T05:24:52.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.266+0000 7febf2ffd640 10 client.18207 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.266+0000 7febf2ffd640 1 -- 172.21.15.100:0/3492946645 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56252eb17720 con 0x56252ee8ca50 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.266+0000 7febf2ffd640 10 client.18207 handle_command_reply: tid=0 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.266+0000 7fec1cdce740 1 client.18207 shutdown 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 msgr2=0x56252ee7a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee7a440 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0972f0 tx=0x7fec0c0205e0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 shutdown_connections 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252ee8ca50 0x56252ee8ee40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee7a440 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73d60 0x56252ee79f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7febf2ffd640 0 client.18207 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 --2- 172.21.15.100:0/3492946645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73440 0x56252ee799c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 >> 172.21.15.100:0/3492946645 conn(0x56252ee6db70 msgr2=0x56252ee6f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 shutdown_connections 2024-03-12T05:24:52.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 1 -- 172.21.15.100:0/3492946645 wait complete. 2024-03-12T05:24:52.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.267+0000 7fec1cdce740 20 client.18207 trim_cache size 0 max 16384 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 Processor -- start 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 -- start start 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee7c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73d60 0x56252ee76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee77180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ed60130 con 0x56252ee7c940 2024-03-12T05:24:52.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ee72de0 con 0x56252ee73d60 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec1cdce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ee75390 con 0x56252ee73440 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee77180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee77180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57300/0 (socket says 172.21.15.100:57300) 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 -- 172.21.15.100:0/1752449932 learned_addr learned my addr 172.21.15.100:0/1752449932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec137fe640 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee7c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec12ffd640 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73d60 0x56252ee76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 -- 172.21.15.100:0/1752449932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 msgr2=0x56252ee7c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee7c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 -- 172.21.15.100:0/1752449932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73d60 msgr2=0x56252ee76c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73d60 0x56252ee76c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 -- 172.21.15.100:0/1752449932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56252ed75c20 con 0x56252ee7c940 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.281+0000 7fec13fff640 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee77180 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fec0c096d80 tx=0x7fec0c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe270f857486ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7febf97fa640 1 -- 172.21.15.100:0/1752449932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c09a460 con 0x56252ee7c940 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7febf97fa640 1 -- 172.21.15.100:0/1752449932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec0c0203a0 con 0x56252ee7c940 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7febf97fa640 1 -- 172.21.15.100:0/1752449932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c096f90 con 0x56252ee7c940 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 -- 172.21.15.100:0/1752449932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 msgr2=0x56252ee77180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee77180 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fec0c096d80 tx=0x7fec0c00cf70 comp rx=0 tx=0).stop 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 -- 172.21.15.100:0/1752449932 shutdown_connections 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee7c940 0x56252ee77180 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee73d60 0x56252ee76c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 --2- 172.21.15.100:0/1752449932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee7c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 -- 172.21.15.100:0/1752449932 >> 172.21.15.100:0/1752449932 conn(0x56252ee6db70 msgr2=0x56252ee78290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 -- 172.21.15.100:0/1752449932 shutdown_connections 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.282+0000 7fec1cdce740 1 -- 172.21.15.100:0/1752449932 wait complete. 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 -- start start 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 Processor -- start 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee823b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 0x56252ee828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee82e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252eee2fa0 con 0x56252ee73d60 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252eee3110 con 0x56252ee7c940 2024-03-12T05:24:52.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252eee32d0 con 0x56252ee73440 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec1cdce740 10 client.0 _ll_register_callbacks cb 0x7fec0c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec12ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 0x56252ee828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec12ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 0x56252ee828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57316/0 (socket says 172.21.15.100:57316) 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec13fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee82e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec12ffd640 1 -- 172.21.15.100:0/949223356 learned_addr learned my addr 172.21.15.100:0/949223356 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.283+0000 7fec137fe640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec12ffd640 1 -- 172.21.15.100:0/949223356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 msgr2=0x56252ee823b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec12ffd640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee823b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec12ffd640 1 -- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 msgr2=0x56252ee82e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec12ffd640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee82e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec12ffd640 1 -- 172.21.15.100:0/949223356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56252edd96a0 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec13fff640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee82e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec12ffd640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 0x56252ee828f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fec140737a0 tx=0x7fec08005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0ed441c8e83305 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7febf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7febf2ffd640 1 -- 172.21.15.100:0/949223356 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08005550 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7febf2ffd640 1 -- 172.21.15.100:0/949223356 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec080056b0 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec0c078180 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7febf2ffd640 1 -- 172.21.15.100:0/949223356 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec080035f0 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 10 client.18213 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56252ed9b8c0 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7febf2ffd640 1 -- 172.21.15.100:0/949223356 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec08064020 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7febf2ffd640 1 -- 172.21.15.100:0/949223356 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec08034870 con 0x56252ee73d60 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 10 client.18213 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:52.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 20 client.18213 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252eee4b10 0x56252eee6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 4 client.18213 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.284+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56252eb66890 con 0x56252eee4b10 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec137fe640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252eee4b10 0x56252eee6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec137fe640 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252eee4b10 0x56252eee6f30 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f38e69524881a8 server_cookie=738d9f7a5f8d87af in_seq=0 out_seq=0 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7febf2ffd640 10 client.18213 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7febf2ffd640 1 -- 172.21.15.100:0/949223356 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56252eb66890 con 0x56252eee4b10 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7febf2ffd640 10 client.18213 handle_command_reply: tid=0 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 client.18213 shutdown 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 msgr2=0x56252ee828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 0x56252ee828f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fec140737a0 tx=0x7fec08005fd0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 shutdown_connections 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56252eee4b10 0x56252eee6f30 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56252ee7c940 0x56252ee82e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7febf2ffd640 0 client.18213 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56252ee73d60 0x56252ee828f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 --2- 172.21.15.100:0/949223356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ee73440 0x56252ee823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 >> 172.21.15.100:0/949223356 conn(0x56252ee6db70 msgr2=0x56252ee6c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 shutdown_connections 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 1 -- 172.21.15.100:0/949223356 wait complete. 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.285+0000 7fec1cdce740 20 client.18213 trim_cache size 0 max 16384 2024-03-12T05:24:52.290 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:24:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7febfc0a1ba0 msgr2=0x7febfc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7febfc0a1ba0 0x7febfc0a4060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec08000c00 tx=0x7fec08005d90 comp rx=0 tx=0).stop 2024-03-12T05:24:52.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 msgr2=0x7fec1419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 0x7fec1419c430 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00eb10 tx=0x7fec0c00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 shutdown_connections 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7febfc0a1ba0 0x7febfc0a4060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1419bf80 0x7fec1419c430 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec14106380 0x7fec14197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 --2- 172.21.15.100:0/1201377553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec140ffbc0 0x7fec141976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 >> 172.21.15.100:0/1201377553 conn(0x7fec140752a0 msgr2=0x7fec140fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 shutdown_connections 2024-03-12T05:24:52.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:52.286+0000 7fec18fa0640 1 -- 172.21.15.100:0/1201377553 wait complete. 2024-03-12T05:24:52.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/856187769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:24:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/856187769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:24:52.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1524042 11.47 MB/sec execute 576 sec latency 19.544 ms 2024-03-12T05:24:53.294 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:24:53.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:24:53.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:24:53.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:52 smithi100 ceph-mon[32210]: pgmap v398: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 250 op/s 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.424+0000 7fdda87b5640 1 -- 172.21.15.100:0/22664151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105330 msgr2=0x7fdda010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.424+0000 7fdda87b5640 1 --2- 172.21.15.100:0/22664151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105330 0x7fdda010cf00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009cd0 tx=0x7fdd9802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.424+0000 7fdda87b5640 1 -- 172.21.15.100:0/22664151 shutdown_connections 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 --2- 172.21.15.100:0/22664151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105330 0x7fdda010cf00 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 --2- 172.21.15.100:0/22664151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda0100c10 0x7fdda0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 --2- 172.21.15.100:0/22664151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda0100260 0x7fdda0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- 172.21.15.100:0/22664151 >> 172.21.15.100:0/22664151 conn(0x7fdda0077180 msgr2=0x7fdda0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- 172.21.15.100:0/22664151 shutdown_connections 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- 172.21.15.100:0/22664151 wait complete. 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 Processor -- start 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- start start 2024-03-12T05:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 0x7fdda0196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda0105330 0x7fdda0196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda019af20 0x7fdda019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019b910 con 0x7fdda0100260 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019ba80 con 0x7fdda0105330 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda87b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019bbf0 con 0x7fdda019af20 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda6551640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 0x7fdda0196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda6551640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 0x7fdda0196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57324/0 (socket says 172.21.15.100:57324) 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda6551640 1 -- 172.21.15.100:0/635145558 learned_addr learned my addr 172.21.15.100:0/635145558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.425+0000 7fdda5d50640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda0105330 0x7fdda0196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6d52640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda019af20 0x7fdda019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6551640 1 -- 172.21.15.100:0/635145558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda019af20 msgr2=0x7fdda019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6551640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda019af20 0x7fdda019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6551640 1 -- 172.21.15.100:0/635145558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda0105330 msgr2=0x7fdda0196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6551640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda0105330 0x7fdda0196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6551640 1 -- 172.21.15.100:0/635145558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98009900 con 0x7fdda0100260 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda6551640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 0x7fdda0196660 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000d950 tx=0x7fdd9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdd977fe640 1 -- 172.21.15.100:0/635145558 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90009a30 con 0x7fdda0100260 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda01a31d0 con 0x7fdda0100260 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdd977fe640 1 -- 172.21.15.100:0/635145558 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd90009b90 con 0x7fdda0100260 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdd977fe640 1 -- 172.21.15.100:0/635145558 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90004a00 con 0x7fdda0100260 2024-03-12T05:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.426+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01a3680 con 0x7fdda0100260 2024-03-12T05:24:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.428+0000 7fdd977fe640 1 -- 172.21.15.100:0/635145558 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fdd90009d00 con 0x7fdda0100260 2024-03-12T05:24:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 Processor -- start 2024-03-12T05:24:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 -- start start 2024-03-12T05:24:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdd977fe640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdd8c0a1ba0 0x7fdd8c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9018760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d90212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d9023d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6d52640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdd8c0a1ba0 0x7fdd8c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8f86f90 con 0x5647d9018d30 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8f844c0 con 0x5647d9018380 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fddab381740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8f7fca0 con 0x5647d90218b0 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdd977fe640 1 -- 172.21.15.100:0/635145558 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdd900cd130 con 0x7fdda0100260 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda5d50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d90212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6551640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9018760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda5d50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d90212e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57332/0 (socket says 172.21.15.100:57332) 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda5d50640 1 -- 172.21.15.100:0/424176516 learned_addr learned my addr 172.21.15.100:0/424176516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6d52640 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d9023d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6551640 1 -- 172.21.15.100:0/424176516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 msgr2=0x5647d9023d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6551640 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d9023d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6551640 1 -- 172.21.15.100:0/424176516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 msgr2=0x5647d90212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6551640 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d90212e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda6551640 1 -- 172.21.15.100:0/424176516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647d8f420b0 con 0x5647d9018380 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.429+0000 7fdda5d50640 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d90212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.430+0000 7fdda6d52640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdd8c0a1ba0 0x7fdd8c0a4060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005cc0 tx=0x7fdd98032ef0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.430+0000 7fdda6551640 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9018760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd900ca8e0 tx=0x7fdd90099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6359744d7e97529 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.430+0000 7fdd94ff9640 1 -- 172.21.15.100:0/424176516 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd900a2560 con 0x5647d9018380 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.430+0000 7fdd94ff9640 1 -- 172.21.15.100:0/424176516 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd90005680 con 0x5647d9018380 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.430+0000 7fdd94ff9640 1 -- 172.21.15.100:0/424176516 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90099470 con 0x5647d9018380 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- 172.21.15.100:0/424176516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 msgr2=0x5647d9018760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9018760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd900ca8e0 tx=0x7fdd90099a30 comp rx=0 tx=0).stop 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- 172.21.15.100:0/424176516 shutdown_connections 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d9023d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d90212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- 172.21.15.100:0/424176516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9018760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- 172.21.15.100:0/424176516 >> 172.21.15.100:0/424176516 conn(0x5647d9012840 msgr2=0x5647d9012c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- 172.21.15.100:0/424176516 shutdown_connections 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- 172.21.15.100:0/424176516 wait complete. 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- start start 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 Processor -- start 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9024870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d9024db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d902f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8f24f40 con 0x5647d9018d30 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8c534b0 con 0x5647d9018380 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d8f05670 con 0x5647d90218b0 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fddab381740 10 client.0 _ll_register_callbacks cb 0x5647d8f72f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fdda6551640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9024870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fdda5d50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d9024db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fdda5d50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d9024db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57346/0 (socket says 172.21.15.100:57346) 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fdda6551640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9024870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34196/0 (socket says 172.21.15.100:34196) 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fdda5d50640 1 -- 172.21.15.100:0/2613224497 learned_addr learned my addr 172.21.15.100:0/2613224497 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.431+0000 7fdda6d52640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d902f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda5d50640 1 -- 172.21.15.100:0/2613224497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 msgr2=0x5647d902f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda5d50640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d902f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda5d50640 1 -- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 msgr2=0x5647d9024870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda5d50640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9024870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda5d50640 1 -- 172.21.15.100:0/2613224497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01a3680 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda6551640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9024870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdda5d50640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d9024db0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c012700 tx=0x7fdd9c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace1896762846cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdd89ffb640 1 -- 172.21.15.100:0/2613224497 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c017070 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647d8f59f40 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdd89ffb640 1 -- 172.21.15.100:0/2613224497 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdd9c00f920 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdd89ffb640 1 -- 172.21.15.100:0/2613224497 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd9c00fae0 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 10 client.18225 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5647d8f7e320 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdd89ffb640 1 -- 172.21.15.100:0/2613224497 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5647d8f59f40 con 0x5647d9018d30 2024-03-12T05:24:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fdd89ffb640 1 -- 172.21.15.100:0/2613224497 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdd9c010cb0 con 0x5647d9018d30 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 10 client.18225 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 10 client.18225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 20 client.18225 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.432+0000 7fddab381740 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d9034090 0x5647d90364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.433+0000 7fddab381740 4 client.18225 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.433+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647d8f76310 con 0x5647d9034090 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.433+0000 7fdda6551640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d9034090 0x5647d90364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.433+0000 7fdd89ffb640 10 client.18225 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.433+0000 7fdda6551640 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d9034090 0x5647d90364b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f56678925f7672c server_cookie=53df3514f6b073b3 in_seq=0 out_seq=0 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fdd89ffb640 1 -- 172.21.15.100:0/2613224497 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647d8f76310 con 0x5647d9034090 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fdd89ffb640 10 client.18225 handle_command_reply: tid=0 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 client.18225 shutdown 2024-03-12T05:24:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 msgr2=0x5647d9024db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d9024db0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c012700 tx=0x7fdd9c012bc0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 shutdown_connections 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d9034090 0x5647d90364b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d90218b0 0x5647d902f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d9018d30 0x5647d9024db0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fdd89ffb640 0 client.18225 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 --2- 172.21.15.100:0/2613224497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9024870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 >> 172.21.15.100:0/2613224497 conn(0x5647d9012840 msgr2=0x5647d90147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 shutdown_connections 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 1 -- 172.21.15.100:0/2613224497 wait complete. 2024-03-12T05:24:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.434+0000 7fddab381740 20 client.18225 trim_cache size 0 max 16384 2024-03-12T05:24:53.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 Processor -- start 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 -- start start 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9029770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9029cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d902f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d9012060 con 0x5647d90218b0 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d9030b50 con 0x5647d9018380 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fddab381740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d9012670 con 0x5647d9018d30 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d902f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d902f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57352/0 (socket says 172.21.15.100:57352) 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 -- 172.21.15.100:0/2053799192 learned_addr learned my addr 172.21.15.100:0/2053799192 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6551640 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9029770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda5d50640 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9029cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 -- 172.21.15.100:0/2053799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 msgr2=0x5647d9029cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9029cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 -- 172.21.15.100:0/2053799192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 msgr2=0x5647d9029770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9029770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.450+0000 7fdda6d52640 1 -- 172.21.15.100:0/2053799192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647d8f1ac00 con 0x5647d90218b0 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.451+0000 7fdda6d52640 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d902f230 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdd98032650 tx=0x7fdd98043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f218b9a5a22924ce server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.451+0000 7fdd8bfff640 1 -- 172.21.15.100:0/2053799192 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98034c90 con 0x5647d90218b0 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.451+0000 7fdd8bfff640 1 -- 172.21.15.100:0/2053799192 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd980347a0 con 0x5647d90218b0 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.451+0000 7fdd8bfff640 1 -- 172.21.15.100:0/2053799192 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98004330 con 0x5647d90218b0 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- 172.21.15.100:0/2053799192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 msgr2=0x5647d902f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d902f230 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdd98032650 tx=0x7fdd98043040 comp rx=0 tx=0).stop 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- 172.21.15.100:0/2053799192 shutdown_connections 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d902f230 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9029cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- 172.21.15.100:0/2053799192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9029770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- 172.21.15.100:0/2053799192 >> 172.21.15.100:0/2053799192 conn(0x5647d9012840 msgr2=0x5647d90247f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- 172.21.15.100:0/2053799192 shutdown_connections 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- 172.21.15.100:0/2053799192 wait complete. 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- start start 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 Processor -- start 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9028950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9028e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d908a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d908ab00 con 0x5647d90218b0 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d908ac40 con 0x5647d9018380 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.452+0000 7fddab381740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d908ae00 con 0x5647d9018d30 2024-03-12T05:24:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fddab381740 10 client.0 _ll_register_callbacks cb 0x7fdd90096120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d908a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6551640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9028950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d908a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57360/0 (socket says 172.21.15.100:57360) 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda5d50640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9028e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 -- 172.21.15.100:0/3496377948 learned_addr learned my addr 172.21.15.100:0/3496377948 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 -- 172.21.15.100:0/3496377948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 msgr2=0x5647d9028e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9028e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 -- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 msgr2=0x5647d9028950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9028950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 -- 172.21.15.100:0/3496377948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda010d710 con 0x5647d90218b0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdda6d52640 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d908a5c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fdd9802c700 tx=0x7fdd9802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74710b2781642f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdd89ffb640 1 -- 172.21.15.100:0/3496377948 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd980042b0 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdd89ffb640 1 -- 172.21.15.100:0/3496377948 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdd9805a070 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647d8c7f860 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fdd89ffb640 1 -- 172.21.15.100:0/3496377948 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd980347a0 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fddab381740 10 client.18231 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.453+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5647d8f40910 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fdd89ffb640 1 -- 172.21.15.100:0/3496377948 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5647d8c7f860 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fdd89ffb640 1 -- 172.21.15.100:0/3496377948 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdd98041670 con 0x5647d90218b0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fddab381740 10 client.18231 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fddab381740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fddab381740 20 client.18231 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fddab381740 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d908c3d0 0x5647d908e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fddab381740 4 client.18231 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5647d8cd0410 con 0x5647d908c3d0 2024-03-12T05:24:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fdda6551640 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d908c3d0 0x5647d908e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fdd89ffb640 10 client.18231 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.454+0000 7fdda6551640 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d908c3d0 0x5647d908e7f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d3ff64d8fae24c server_cookie=a2b036abb2ec4424 in_seq=0 out_seq=0 2024-03-12T05:24:53.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:52 smithi156 ceph-mon[36270]: pgmap v398: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 250 op/s 2024-03-12T05:24:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fdd89ffb640 1 -- 172.21.15.100:0/3496377948 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5647d8cd0410 con 0x5647d908c3d0 2024-03-12T05:24:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fdd89ffb640 10 client.18231 handle_command_reply: tid=0 2024-03-12T05:24:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 client.18231 shutdown 2024-03-12T05:24:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 msgr2=0x5647d908a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d908a5c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fdd9802c700 tx=0x7fdd9802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 shutdown_connections 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5647d908c3d0 0x5647d908e7f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647d90218b0 0x5647d908a5c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fdd89ffb640 0 client.18231 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647d9018d30 0x5647d9028e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 --2- 172.21.15.100:0/3496377948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647d9018380 0x5647d9028950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 >> 172.21.15.100:0/3496377948 conn(0x5647d9012840 msgr2=0x5647d901f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 shutdown_connections 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 1 -- 172.21.15.100:0/3496377948 wait complete. 2024-03-12T05:24:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.456+0000 7fddab381740 20 client.18231 trim_cache size 0 max 16384 2024-03-12T05:24:53.459 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:24:53.459 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:24:53.459 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "8c85e42a-ea3e-4dcb-9915-981db42e29e2", 2024-03-12T05:24:53.459 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:24:53.459 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdd8c0a1ba0 msgr2=0x7fdd8c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdd8c0a1ba0 0x7fdd8c0a4060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005cc0 tx=0x7fdd98032ef0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 msgr2=0x7fdda0196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 0x7fdda0196660 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000d950 tx=0x7fdd9000de10 comp rx=0 tx=0).stop 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 shutdown_connections 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fdd8c0a1ba0 0x7fdd8c0a4060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda019af20 0x7fdda019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda0105330 0x7fdda0196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 --2- 172.21.15.100:0/635145558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0100260 0x7fdda0196660 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 >> 172.21.15.100:0/635145558 conn(0x7fdda0077180 msgr2=0x7fdda00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 shutdown_connections 2024-03-12T05:24:53.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.457+0000 7fdda87b5640 1 -- 172.21.15.100:0/635145558 wait complete. 2024-03-12T05:24:53.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:24:53.463 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:24:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:52 smithi161 ceph-mon[36043]: pgmap v398: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 250 op/s 2024-03-12T05:24:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 -- 172.21.15.100:0/2752627479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564100270 msgr2=0x7f4564100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 --2- 172.21.15.100:0/2752627479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564100270 0x7f4564100650 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4558009930 tx=0x7f455802bc30 comp rx=0 tx=0).stop 2024-03-12T05:24:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 -- 172.21.15.100:0/2752627479 shutdown_connections 2024-03-12T05:24:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 --2- 172.21.15.100:0/2752627479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4564105340 0x7f456410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 --2- 172.21.15.100:0/2752627479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4564100c20 0x7f4564104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 --2- 172.21.15.100:0/2752627479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564100270 0x7f4564100650 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 -- 172.21.15.100:0/2752627479 >> 172.21.15.100:0/2752627479 conn(0x7f4564077230 msgr2=0x7f4564077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 -- 172.21.15.100:0/2752627479 shutdown_connections 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.638+0000 7f456b057640 1 -- 172.21.15.100:0/2752627479 wait complete. 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 Processor -- start 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 -- start start 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4564100270 0x7f4564198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 0x7f4564198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4564199110 0x7f45641a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641a3d60 con 0x7f4564105340 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641a3ed0 con 0x7f4564199110 2024-03-12T05:24:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f456b057640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641a4090 con 0x7f4564100270 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 0x7f4564198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 0x7f4564198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57382/0 (socket says 172.21.15.100:57382) 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 -- 172.21.15.100:0/3953600110 learned_addr learned my addr 172.21.15.100:0/3953600110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f45695f4640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4564199110 0x7f45641a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4568df3640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4564100270 0x7f4564198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 -- 172.21.15.100:0/3953600110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4564100270 msgr2=0x7f4564198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4564100270 0x7f4564198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 -- 172.21.15.100:0/3953600110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4564199110 msgr2=0x7f45641a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4564199110 0x7f45641a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.639+0000 7f4563fff640 1 -- 172.21.15.100:0/3953600110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009560 con 0x7f4564105340 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.640+0000 7f4563fff640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 0x7f4564198b80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f455000d740 tx=0x7f455000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.640+0000 7f4561ffb640 1 -- 172.21.15.100:0/3953600110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550009c30 con 0x7f4564105340 2024-03-12T05:24:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.640+0000 7f4561ffb640 1 -- 172.21.15.100:0/3953600110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f455000de50 con 0x7f4564105340 2024-03-12T05:24:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.640+0000 7f4561ffb640 1 -- 172.21.15.100:0/3953600110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550010460 con 0x7f4564105340 2024-03-12T05:24:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.640+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45641a4370 con 0x7f4564105340 2024-03-12T05:24:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.640+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45641a4880 con 0x7f4564105340 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.642+0000 7f4561ffb640 1 -- 172.21.15.100:0/3953600110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4550004f10 con 0x7f4564105340 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.642+0000 7f456dc20740 1 Processor -- start 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.642+0000 7f456dc20740 1 -- start start 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f456dc20740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb17340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f456dc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f456dc20740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f4561ffb640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45540a1ca0 0x7f45540a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f456dc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fda86280 con 0x5652fdb17880 2024-03-12T05:24:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f456dc20740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fda837b0 con 0x5652fdb16f60 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f456dc20740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fda7ef90 con 0x5652fdb24bc0 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb17340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f4568df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb17340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34228/0 (socket says 172.21.15.100:34228) 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 -- 172.21.15.100:0/2307795974 learned_addr learned my addr 172.21.15.100:0/2307795974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45540a1ca0 0x7f45540a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f4563fff640 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f4561ffb640 1 -- 172.21.15.100:0/3953600110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f45500cd1b0 con 0x7f4564105340 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 -- 172.21.15.100:0/2307795974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 msgr2=0x5652fdb26fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb26fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 -- 172.21.15.100:0/2307795974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 msgr2=0x5652fdb24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb24680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 -- 172.21.15.100:0/2307795974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652fda3d570 con 0x5652fdb16f60 2024-03-12T05:24:53.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f4568df3640 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:53.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45540a1ca0 0x7f45540a4160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f455c010c40 tx=0x7f455c017000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.643+0000 7f45695f4640 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb17340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f455c012cf0 tx=0x7f455c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9761bc5345a19176 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f454effd640 1 -- 172.21.15.100:0/2307795974 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c012ea0 con 0x5652fdb16f60 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f454effd640 1 -- 172.21.15.100:0/2307795974 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f455c003e50 con 0x5652fdb16f60 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f454effd640 1 -- 172.21.15.100:0/2307795974 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c00de40 con 0x5652fdb16f60 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 -- 172.21.15.100:0/2307795974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 msgr2=0x5652fdb17340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb17340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f455c012cf0 tx=0x7f455c010740 comp rx=0 tx=0).stop 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 -- 172.21.15.100:0/2307795974 shutdown_connections 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 --2- 172.21.15.100:0/2307795974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb17340 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 -- 172.21.15.100:0/2307795974 >> 172.21.15.100:0/2307795974 conn(0x5652fdb11990 msgr2=0x5652fdb11d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.644+0000 7f456dc20740 1 -- 172.21.15.100:0/2307795974 shutdown_connections 2024-03-12T05:24:53.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 -- 172.21.15.100:0/2307795974 wait complete. 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 -- start start 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 Processor -- start 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb21990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fd752430 con 0x5652fdb17880 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fd70ee00 con 0x5652fdb16f60 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fda03fd0 con 0x5652fdb24bc0 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f456dc20740 10 client.0 _ll_register_callbacks cb 0x5652fda59fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f4563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb21990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f4563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb21990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57396/0 (socket says 172.21.15.100:57396) 2024-03-12T05:24:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.645+0000 7f4563fff640 1 -- 172.21.15.100:0/4114999616 learned_addr learned my addr 172.21.15.100:0/4114999616 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4568df3640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f45695f4640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4563fff640 1 -- 172.21.15.100:0/4114999616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 msgr2=0x5652fdb220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4563fff640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4563fff640 1 -- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 msgr2=0x5652fdb24600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4563fff640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb24600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4563fff640 1 -- 172.21.15.100:0/4114999616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c01a040 con 0x5652fdb17880 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4568df3640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4563fff640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb21990 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4550009610 tx=0x7f455009a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbca9a14356c795e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4543fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4543fff640 1 -- 172.21.15.100:0/4114999616 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500a24e0 con 0x5652fdb17880 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4543fff640 1 -- 172.21.15.100:0/4114999616 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4550098040 con 0x5652fdb17880 2024-03-12T05:24:53.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652fd77e7e0 con 0x5652fdb17880 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f4543fff640 1 -- 172.21.15.100:0/4114999616 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45500b8170 con 0x5652fdb17880 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f456dc20740 10 client.18243 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.646+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652fda7d610 con 0x5652fdb17880 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f4543fff640 1 -- 172.21.15.100:0/4114999616 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f455007c080 con 0x5652fdb17880 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f4543fff640 1 -- 172.21.15.100:0/4114999616 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45500a2050 con 0x5652fdb17880 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f456dc20740 10 client.18243 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f456dc20740 10 client.18243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f456dc20740 20 client.18243 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f456dc20740 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb34d30 0x5652fdb37150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f456dc20740 4 client.18243 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652fd7bb720 con 0x5652fdb34d30 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f4568df3640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb34d30 0x5652fdb37150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f4543fff640 10 client.18243 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.647+0000 7f4568df3640 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb34d30 0x5652fdb37150 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e6f478a5deb68f server_cookie=6e7e61639c5fcad8 in_seq=0 out_seq=0 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f4543fff640 1 -- 172.21.15.100:0/4114999616 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652fd7bb720 con 0x5652fdb34d30 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f4543fff640 10 client.18243 handle_command_reply: tid=0 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 client.18243 shutdown 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 msgr2=0x5652fdb21990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb21990 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4550009610 tx=0x7f455009a440 comp rx=0 tx=0).stop 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 shutdown_connections 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb34d30 0x5652fdb37150 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb21990 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f4543fff640 0 client.18243 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 --2- 172.21.15.100:0/4114999616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.648+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 >> 172.21.15.100:0/4114999616 conn(0x5652fdb11990 msgr2=0x5652fdb13730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.649+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 shutdown_connections 2024-03-12T05:24:53.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.649+0000 7f456dc20740 1 -- 172.21.15.100:0/4114999616 wait complete. 2024-03-12T05:24:53.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.649+0000 7f456dc20740 20 client.18243 trim_cache size 0 max 16384 2024-03-12T05:24:53.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 Processor -- start 2024-03-12T05:24:53.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 -- start start 2024-03-12T05:24:53.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fdb1c850 con 0x5652fdb17880 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fdb16600 con 0x5652fdb16f60 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f456dc20740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fdb3c340 con 0x5652fdb24bc0 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4568df3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb2bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57404/0 (socket says 172.21.15.100:57404) 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f45695f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 -- 172.21.15.100:0/1316094247 learned_addr learned my addr 172.21.15.100:0/1316094247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 -- 172.21.15.100:0/1316094247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 msgr2=0x5652fdb240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 -- 172.21.15.100:0/1316094247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 msgr2=0x5652fdb2baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb2baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.664+0000 7f4563fff640 1 -- 172.21.15.100:0/1316094247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652fda19ac0 con 0x5652fdb17880 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f4568df3640 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb2baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f4563fff640 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb2bfe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f455009a5f0 tx=0x7f45500ca940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b31c72398a2b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f454e7fc640 1 -- 172.21.15.100:0/1316094247 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500046c0 con 0x5652fdb17880 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f454e7fc640 1 -- 172.21.15.100:0/1316094247 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5652fd77e7e0 con 0x5652fdb17880 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f454e7fc640 1 -- 172.21.15.100:0/1316094247 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550096530 con 0x5652fdb17880 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f456dc20740 1 -- 172.21.15.100:0/1316094247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 msgr2=0x5652fdb2bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.665+0000 7f456dc20740 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb2bfe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f455009a5f0 tx=0x7f45500ca940 comp rx=0 tx=0).stop 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- 172.21.15.100:0/1316094247 shutdown_connections 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb17880 0x5652fdb2bfe0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 --2- 172.21.15.100:0/1316094247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb16f60 0x5652fdb2baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- 172.21.15.100:0/1316094247 >> 172.21.15.100:0/1316094247 conn(0x5652fdb11990 msgr2=0x5652fdb1ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- 172.21.15.100:0/1316094247 shutdown_connections 2024-03-12T05:24:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- 172.21.15.100:0/1316094247 wait complete. 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- start start 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 Processor -- start 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 0x5652fdb2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb17880 0x5652fdb2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fdb2ab50 con 0x5652fdb16f60 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fdb2acc0 con 0x5652fdb17880 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652fdb2ae80 con 0x5652fdb24bc0 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f456dc20740 10 client.0 _ll_register_callbacks cb 0x7f455c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f4568df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 0x5652fdb2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f4568df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 0x5652fdb2bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57412/0 (socket says 172.21.15.100:57412) 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f4568df3640 1 -- 172.21.15.100:0/2216748591 learned_addr learned my addr 172.21.15.100:0/2216748591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:24:53.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f4563fff640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb17880 0x5652fdb2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.666+0000 7f45695f4640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4568df3640 1 -- 172.21.15.100:0/2216748591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 msgr2=0x5652fdb2a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4568df3640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb2a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4568df3640 1 -- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb17880 msgr2=0x5652fdb2a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4568df3640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb17880 0x5652fdb2a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4568df3640 1 -- 172.21.15.100:0/2216748591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4558030070 con 0x5652fdb16f60 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4563fff640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb17880 0x5652fdb2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4568df3640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 0x5652fdb2bcb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4558036040 tx=0x7f455802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51492207eb8f484a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4543fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4543fff640 1 -- 172.21.15.100:0/2216748591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455802cbd0 con 0x5652fdb16f60 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4543fff640 1 -- 172.21.15.100:0/2216748591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4558045040 con 0x5652fdb16f60 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f455c003e50 con 0x5652fdb16f60 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4543fff640 1 -- 172.21.15.100:0/2216748591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4558004040 con 0x5652fdb16f60 2024-03-12T05:24:53.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f456dc20740 10 client.18249 fetch_fsmap learned FSMap version 25 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652fda7d610 con 0x5652fdb16f60 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.667+0000 7f4543fff640 1 -- 172.21.15.100:0/2216748591 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f455805e080 con 0x5652fdb16f60 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4543fff640 1 -- 172.21.15.100:0/2216748591 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f455803c040 con 0x5652fdb16f60 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 10 client.18249 fetch_fsmap finished waiting for FSMap version 25 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 20 client.18249 populate_metadata read hostname 'smithi100' 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb8c8a0 0x5652fdb8ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 4 client.18249 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652fd80a890 con 0x5652fdb8c8a0 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4563fff640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb8c8a0 0x5652fdb8ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4543fff640 10 client.18249 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4563fff640 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb8c8a0 0x5652fdb8ecc0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0b1c4fe0b67fe3 server_cookie=5f86e675a4ee68c in_seq=0 out_seq=0 2024-03-12T05:24:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4543fff640 1 -- 172.21.15.100:0/2216748591 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652fd80a890 con 0x5652fdb8c8a0 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4543fff640 10 client.18249 handle_command_reply: tid=0 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 client.18249 shutdown 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 msgr2=0x5652fdb2bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 0x5652fdb2bcb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4558036040 tx=0x7f455802c680 comp rx=0 tx=0).stop 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 shutdown_connections 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5652fdb8c8a0 0x5652fdb8ecc0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652fdb24bc0 0x5652fdb2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f4543fff640 0 client.18249 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652fdb17880 0x5652fdb2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 --2- 172.21.15.100:0/2216748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652fdb16f60 0x5652fdb2bcb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.668+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 >> 172.21.15.100:0/2216748591 conn(0x5652fdb11990 msgr2=0x5652fdb20220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 shutdown_connections 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456dc20740 1 -- 172.21.15.100:0/2216748591 wait complete. 2024-03-12T05:24:53.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456dc20740 20 client.18249 trim_cache size 0 max 16384 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: "8c85e42a-ea3e-4dcb-9915-981db42e29e2": { 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "8c85e42a-ea3e-4dcb-9915-981db42e29e2", 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:24:53.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:24:53.673 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:24:53.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45540a1ca0 msgr2=0x7f45540a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45540a1ca0 0x7f45540a4160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f455c010c40 tx=0x7f455c017000 comp rx=0 tx=0).stop 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 msgr2=0x7f4564198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 0x7f4564198b80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f455000d740 tx=0x7f455000dc00 comp rx=0 tx=0).stop 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 shutdown_connections 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f45540a1ca0 0x7f45540a4160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4564199110 0x7f45641a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4564105340 0x7f4564198b80 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 --2- 172.21.15.100:0/3953600110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4564100270 0x7f4564198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 >> 172.21.15.100:0/3953600110 conn(0x7f4564077230 msgr2=0x7f45640fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 shutdown_connections 2024-03-12T05:24:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:24:53.669+0000 7f456b057640 1 -- 172.21.15.100:0/3953600110 wait complete. 2024-03-12T05:24:53.676 INFO:tasks.cephfs.filesystem:scrub status for tag:8c85e42a-ea3e-4dcb-9915-981db42e29e2 - {'path': '/', 'tag': '8c85e42a-ea3e-4dcb-9915-981db42e29e2', 'options': 'recursive,force'} 2024-03-12T05:24:53.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1525284 11.46 MB/sec execute 577 sec latency 64.554 ms 2024-03-12T05:24:54.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:53 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:24:54.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:53 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:53 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:24:54.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:53 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:24:54.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:53 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:53 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:53 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:24:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:53 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:53 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:24:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:53 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:24:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:53 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:53 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:53 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:24:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:53 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:53 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:24:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:53 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:24:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:53 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:53 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:24:54.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1527697 11.47 MB/sec execute 578 sec latency 17.389 ms 2024-03-12T05:24:55.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:54 smithi100 ceph-mon[32210]: pgmap v399: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 152 op/s 2024-03-12T05:24:55.353 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:54 smithi156 ceph-mon[36270]: pgmap v399: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 152 op/s 2024-03-12T05:24:55.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:54 smithi161 ceph-mon[36043]: pgmap v399: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 152 op/s 2024-03-12T05:24:55.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1529112 11.46 MB/sec execute 579 sec latency 24.437 ms 2024-03-12T05:24:56.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1531450 11.46 MB/sec execute 580 sec latency 19.181 ms 2024-03-12T05:24:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:56 smithi100 ceph-mon[32210]: pgmap v400: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 152 op/s 2024-03-12T05:24:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:56 smithi156 ceph-mon[36270]: pgmap v400: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 152 op/s 2024-03-12T05:24:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:56 smithi161 ceph-mon[36043]: pgmap v400: 97 pgs: 97 active+clean; 601 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 152 op/s 2024-03-12T05:24:57.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1533494 11.45 MB/sec execute 581 sec latency 17.303 ms 2024-03-12T05:24:58.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:58 smithi100 ceph-mon[32210]: pgmap v401: 97 pgs: 97 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 244 op/s 2024-03-12T05:24:58.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:58 smithi156 ceph-mon[36270]: pgmap v401: 97 pgs: 97 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 244 op/s 2024-03-12T05:24:58.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:58 smithi161 ceph-mon[36043]: pgmap v401: 97 pgs: 97 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 244 op/s 2024-03-12T05:24:58.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1535127 11.45 MB/sec execute 582 sec latency 94.601 ms 2024-03-12T05:24:59.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1537107 11.45 MB/sec execute 583 sec latency 39.314 ms 2024-03-12T05:25:00.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:24:59 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:25:00.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:24:59 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:25:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:24:59 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:25:00.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1538311 11.45 MB/sec execute 584 sec latency 574.639 ms 2024-03-12T05:25:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:00 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:25:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:00 smithi100 ceph-mon[32210]: pgmap v402: 97 pgs: 97 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 167 op/s 2024-03-12T05:25:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:00 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:25:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:00 smithi156 ceph-mon[36270]: pgmap v402: 97 pgs: 97 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 167 op/s 2024-03-12T05:25:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:00 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:25:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:00 smithi161 ceph-mon[36043]: pgmap v402: 97 pgs: 97 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 167 op/s 2024-03-12T05:25:01.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1538311 11.43 MB/sec execute 585 sec latency 1574.748 ms 2024-03-12T05:25:02.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1538311 11.41 MB/sec execute 586 sec latency 2574.802 ms 2024-03-12T05:25:03.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:02 smithi100 ceph-mon[32210]: pgmap v403: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 239 op/s 2024-03-12T05:25:03.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:02 smithi156 ceph-mon[36270]: pgmap v403: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 239 op/s 2024-03-12T05:25:03.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:02 smithi161 ceph-mon[36043]: pgmap v403: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 239 op/s 2024-03-12T05:25:03.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1538460 11.39 MB/sec execute 587 sec latency 3287.219 ms 2024-03-12T05:25:04.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1539919 11.38 MB/sec execute 588 sec latency 33.706 ms 2024-03-12T05:25:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:04 smithi100 ceph-mon[32210]: pgmap v404: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-03-12T05:25:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:04 smithi156 ceph-mon[36270]: pgmap v404: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-03-12T05:25:05.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:04 smithi161 ceph-mon[36043]: pgmap v404: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-03-12T05:25:05.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1542205 11.38 MB/sec execute 589 sec latency 16.457 ms 2024-03-12T05:25:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:06 smithi100 ceph-mon[32210]: pgmap v405: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-03-12T05:25:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:06 smithi156 ceph-mon[36270]: pgmap v405: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-03-12T05:25:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:06 smithi161 ceph-mon[36043]: pgmap v405: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-03-12T05:25:06.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1544512 11.38 MB/sec execute 590 sec latency 15.658 ms 2024-03-12T05:25:07.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1546283 11.38 MB/sec execute 591 sec latency 20.010 ms 2024-03-12T05:25:08.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1548816 11.39 MB/sec execute 592 sec latency 115.483 ms 2024-03-12T05:25:09.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:08 smithi161 ceph-mon[36043]: pgmap v406: 97 pgs: 97 active+clean; 633 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 251 op/s 2024-03-12T05:25:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:08 smithi100 ceph-mon[32210]: pgmap v406: 97 pgs: 97 active+clean; 633 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 251 op/s 2024-03-12T05:25:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:08 smithi156 ceph-mon[36270]: pgmap v406: 97 pgs: 97 active+clean; 633 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 251 op/s 2024-03-12T05:25:09.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1550264 11.38 MB/sec execute 593 sec latency 120.492 ms 2024-03-12T05:25:10.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1552485 11.38 MB/sec execute 594 sec latency 19.742 ms 2024-03-12T05:25:11.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:10 smithi161 ceph-mon[36043]: pgmap v407: 97 pgs: 97 active+clean; 633 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 159 op/s 2024-03-12T05:25:11.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:10 smithi161 ceph-mon[36043]: osdmap e102: 12 total, 12 up, 12 in 2024-03-12T05:25:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:10 smithi100 ceph-mon[32210]: pgmap v407: 97 pgs: 97 active+clean; 633 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 159 op/s 2024-03-12T05:25:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:10 smithi100 ceph-mon[32210]: osdmap e102: 12 total, 12 up, 12 in 2024-03-12T05:25:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:10 smithi156 ceph-mon[36270]: pgmap v407: 97 pgs: 97 active+clean; 633 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 159 op/s 2024-03-12T05:25:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:10 smithi156 ceph-mon[36270]: osdmap e102: 12 total, 12 up, 12 in 2024-03-12T05:25:11.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1553786 11.37 MB/sec execute 595 sec latency 174.578 ms 2024-03-12T05:25:12.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1556162 11.38 MB/sec execute 596 sec latency 195.134 ms 2024-03-12T05:25:13.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:12 smithi100 ceph-mon[32210]: pgmap v409: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 216 op/s 2024-03-12T05:25:13.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:12 smithi156 ceph-mon[36270]: pgmap v409: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 216 op/s 2024-03-12T05:25:13.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:12 smithi161 ceph-mon[36043]: pgmap v409: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 216 op/s 2024-03-12T05:25:13.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1557720 11.37 MB/sec execute 597 sec latency 17.588 ms 2024-03-12T05:25:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1559736 11.37 MB/sec execute 598 sec latency 107.229 ms 2024-03-12T05:25:15.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:14 smithi100 ceph-mon[32210]: pgmap v410: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 216 op/s 2024-03-12T05:25:15.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:14 smithi156 ceph-mon[36270]: pgmap v410: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 216 op/s 2024-03-12T05:25:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:14 smithi161 ceph-mon[36043]: pgmap v410: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 216 op/s 2024-03-12T05:25:15.811 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1561314 11.36 MB/sec execute 599 sec latency 17.427 ms 2024-03-12T05:25:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: 1 cleanup 600 sec 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: 0 cleanup 600 sec 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Operation Count AvgLat MaxLat 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: ---------------------------------------- 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: NTCreateX 217010 0.979 216.488 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Close 159386 0.002 0.092 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Rename 9186 4.385 141.896 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Unlink 43830 1.737 3287.206 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Deltree 6 115.115 265.712 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Mkdir 3 0.008 0.014 2024-03-12T05:25:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: Qpathinfo 196646 0.942 10139.293 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: Qfileinfo 34470 0.002 0.032 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: Qfsinfo 36077 0.124 6.628 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: Sfileinfo 17662 0.009 5.817 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: Find 76049 0.321 11707.248 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: WriteX 108268 0.041 58.756 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: ReadX 340054 0.009 11.244 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: LockX 706 0.947 63.292 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: UnlockX 706 0.822 71.727 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: Flush 15202 2.915 86.175 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stdout:Throughput 11.3621 MB/sec 1 clients 1 procs max_latency=11707.258 ms 2024-03-12T05:25:17.129 INFO:tasks.workunit.client.0.smithi100.stderr:+ dbench 10 2024-03-12T05:25:17.130 INFO:tasks.workunit.client.0.smithi100.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-12T05:25:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-03-12T05:25:17.130 INFO:tasks.workunit.client.0.smithi100.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-12T05:25:17.130 INFO:tasks.workunit.client.0.smithi100.stdout:8 of 10 processes prepared for launch 0 sec 2024-03-12T05:25:17.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:16 smithi156 ceph-mon[36270]: osdmap e103: 12 total, 12 up, 12 in 2024-03-12T05:25:17.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:16 smithi156 ceph-mon[36270]: pgmap v412: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 139 op/s 2024-03-12T05:25:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:16 smithi161 ceph-mon[36043]: osdmap e103: 12 total, 12 up, 12 in 2024-03-12T05:25:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:16 smithi161 ceph-mon[36043]: pgmap v412: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 139 op/s 2024-03-12T05:25:17.229 INFO:tasks.workunit.client.0.smithi100.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-12T05:25:17.230 INFO:tasks.workunit.client.0.smithi100.stdout:releasing clients 2024-03-12T05:25:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:16 smithi100 ceph-mon[32210]: osdmap e103: 12 total, 12 up, 12 in 2024-03-12T05:25:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:16 smithi100 ceph-mon[32210]: pgmap v412: 97 pgs: 97 active+clean; 626 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 139 op/s 2024-03-12T05:25:18.229 INFO:tasks.workunit.client.0.smithi100.stdout: 10 395 140.60 MB/sec warmup 1 sec latency 93.986 ms 2024-03-12T05:25:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:18 smithi161 ceph-mon[36043]: pgmap v413: 97 pgs: 97 active+clean; 640 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.2 MiB/s wr, 274 op/s 2024-03-12T05:25:19.229 INFO:tasks.workunit.client.0.smithi100.stdout: 10 726 102.51 MB/sec warmup 2 sec latency 295.606 ms 2024-03-12T05:25:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:18 smithi100 ceph-mon[32210]: pgmap v413: 97 pgs: 97 active+clean; 640 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.2 MiB/s wr, 274 op/s 2024-03-12T05:25:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:18 smithi156 ceph-mon[36270]: pgmap v413: 97 pgs: 97 active+clean; 640 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.2 MiB/s wr, 274 op/s 2024-03-12T05:25:20.229 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1566 84.02 MB/sec warmup 3 sec latency 404.666 ms 2024-03-12T05:25:20.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:19 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:25:20.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:19 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:25:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:19 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:25:21.229 INFO:tasks.workunit.client.0.smithi100.stdout: 10 2637 76.16 MB/sec warmup 4 sec latency 155.156 ms 2024-03-12T05:25:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:20 smithi100 ceph-mon[32210]: pgmap v414: 97 pgs: 97 active+clean; 640 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 220 op/s 2024-03-12T05:25:21.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:20 smithi156 ceph-mon[36270]: pgmap v414: 97 pgs: 97 active+clean; 640 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 220 op/s 2024-03-12T05:25:21.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:20 smithi161 ceph-mon[36043]: pgmap v414: 97 pgs: 97 active+clean; 640 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 220 op/s 2024-03-12T05:25:22.229 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3404 72.79 MB/sec warmup 5 sec latency 115.635 ms 2024-03-12T05:25:23.229 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4166 67.85 MB/sec warmup 6 sec latency 145.314 ms 2024-03-12T05:25:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:22 smithi100 ceph-mon[32210]: pgmap v415: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2024-03-12T05:25:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:22 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:25:23.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:22 smithi156 ceph-mon[36270]: pgmap v415: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2024-03-12T05:25:23.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:22 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:25:23.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:22 smithi161 ceph-mon[36043]: pgmap v415: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2024-03-12T05:25:23.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:22 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:25:23.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:25:23.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:25:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 -- 172.21.15.100:0/2767870638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d8069b10 msgr2=0x7f27d8069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/2767870638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d8069b10 0x7f27d8069f90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009900 tx=0x7f27c802bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 -- 172.21.15.100:0/2767870638 shutdown_connections 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/2767870638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d810da70 0x7f27d810fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/2767870638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d8069b10 0x7f27d8069f90 secure :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009900 tx=0x7f27c802bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/2767870638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d8102100 0x7f27d8069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.801+0000 7f27dfe4e640 1 -- 172.21.15.100:0/2767870638 >> 172.21.15.100:0/2767870638 conn(0x7f27d8076e10 msgr2=0x7f27d8077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.802+0000 7f27dfe4e640 1 -- 172.21.15.100:0/2767870638 shutdown_connections 2024-03-12T05:25:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.802+0000 7f27dfe4e640 1 -- 172.21.15.100:0/2767870638 wait complete. 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 Processor -- start 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 -- start start 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d8102100 0x7f27d819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d810da70 0x7f27d819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 0x7f27d8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819a3c0 con 0x7f27d81999d0 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819a530 con 0x7f27d8102100 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dfe4e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819a6f0 con 0x7f27d810da70 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27de3eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 0x7f27d8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27de3eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 0x7f27d8199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37996/0 (socket says 172.21.15.100:37996) 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27de3eb640 1 -- 172.21.15.100:0/1522461837 learned_addr learned my addr 172.21.15.100:0/1522461837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.803+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d810da70 0x7f27d819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27ddbea640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d8102100 0x7f27d819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27de3eb640 1 -- 172.21.15.100:0/1522461837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d810da70 msgr2=0x7f27d819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27de3eb640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d810da70 0x7f27d819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27de3eb640 1 -- 172.21.15.100:0/1522461837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d8102100 msgr2=0x7f27d819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27de3eb640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d8102100 0x7f27d819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27de3eb640 1 -- 172.21.15.100:0/1522461837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8009560 con 0x7f27d81999d0 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27ddbea640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d8102100 0x7f27d819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27de3eb640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 0x7f27d8199e80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f27d000b9e0 tx=0x7f27d000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27ceffd640 1 -- 172.21.15.100:0/1522461837 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d000cb20 con 0x7f27d81999d0 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d81a66e0 con 0x7f27d81999d0 2024-03-12T05:25:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27ceffd640 1 -- 172.21.15.100:0/1522461837 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27d000cc80 con 0x7f27d81999d0 2024-03-12T05:25:23.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27ceffd640 1 -- 172.21.15.100:0/1522461837 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d00126d0 con 0x7f27d81999d0 2024-03-12T05:25:23.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.804+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d81a6c40 con 0x7f27d81999d0 2024-03-12T05:25:23.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.808+0000 7f27ceffd640 1 -- 172.21.15.100:0/1522461837 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f27d00149b0 con 0x7f27d81999d0 2024-03-12T05:25:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.809+0000 7f27e2a29740 1 Processor -- start 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 -- start start 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27ceffd640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27c40a1ca0 0x7f27c40a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369610ac0 0x563369610ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369619a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369577280 con 0x563369610ac0 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633695747b0 con 0x563369619fc0 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27e2a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336956ff90 con 0x5633696113e0 2024-03-12T05:25:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27de3eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27de3eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37512/0 (socket says 172.21.15.100:37512) 2024-03-12T05:25:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27de3eb640 1 -- 172.21.15.100:0/3302086326 learned_addr learned my addr 172.21.15.100:0/3302086326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27c40a1ca0 0x7f27c40a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.810+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369619a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369610ac0 0x563369610ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 -- 172.21.15.100:0/3302086326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 msgr2=0x563369619a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369619a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 -- 172.21.15.100:0/3302086326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 msgr2=0x56336961c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 -- 172.21.15.100:0/3302086326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56336952e570 con 0x563369610ac0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ceffd640 1 -- 172.21.15.100:0/1522461837 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f27d00ccf70 con 0x7f27d81999d0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.811+0000 7f27ddbea640 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369610ac0 0x563369610ea0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009c30 tx=0x7f27d4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9980a6b63481fd41 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.812+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27c40a1ca0 0x7f27c40a4160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f27c8036040 tx=0x7f27c8005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.812+0000 7f27c3fff640 1 -- 172.21.15.100:0/3302086326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4004030 con 0x563369610ac0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.812+0000 7f27c3fff640 1 -- 172.21.15.100:0/3302086326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27d4003610 con 0x563369610ac0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.812+0000 7f27c3fff640 1 -- 172.21.15.100:0/3302086326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40126b0 con 0x563369610ac0 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 -- 172.21.15.100:0/3302086326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369610ac0 msgr2=0x563369610ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369610ac0 0x563369610ea0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009c30 tx=0x7f27d4010740 comp rx=0 tx=0).stop 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 -- 172.21.15.100:0/3302086326 shutdown_connections 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369619a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 --2- 172.21.15.100:0/3302086326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369610ac0 0x563369610ea0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 -- 172.21.15.100:0/3302086326 >> 172.21.15.100:0/3302086326 conn(0x56336960b1a0 msgr2=0x56336960b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 -- 172.21.15.100:0/3302086326 shutdown_connections 2024-03-12T05:25:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 -- 172.21.15.100:0/3302086326 wait complete. 2024-03-12T05:25:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.813+0000 7f27e2a29740 1 -- start start 2024-03-12T05:25:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 Processor -- start 2024-03-12T05:25:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369616ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369617400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369625f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369243430 con 0x563369617940 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633691ffe00 con 0x563369619fc0 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.814+0000 7f27e2a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633694f4fd0 con 0x5633696113e0 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27e2a29740 10 client.0 _ll_register_callbacks cb 0x56336954afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369617400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27de3eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369625f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369617400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37514/0 (socket says 172.21.15.100:37514) 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 -- 172.21.15.100:0/161584866 learned_addr learned my addr 172.21.15.100:0/161584866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 -- 172.21.15.100:0/161584866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 msgr2=0x563369616ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369616ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 -- 172.21.15.100:0/161584866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 msgr2=0x563369625f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369625f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.815+0000 7f27dd3e9640 1 -- 172.21.15.100:0/161584866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d4009890 con 0x563369619fc0 2024-03-12T05:25:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.816+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369617400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27d80781b0 tx=0x7f27c8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2698845d33d284c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.816+0000 7f27c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.816+0000 7f27c17fa640 1 -- 172.21.15.100:0/161584866 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8004410 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.816+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56336926f7e0 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.816+0000 7f27c17fa640 1 -- 172.21.15.100:0/161584866 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f27c8048050 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27c17fa640 1 -- 172.21.15.100:0/161584866 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c8004df0 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 10 client.27623 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56336956e610 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27c17fa640 1 -- 172.21.15.100:0/161584866 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56336926f7e0 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27c17fa640 1 -- 172.21.15.100:0/161584866 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f27c803c030 con 0x563369619fc0 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 10 client.27623 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 10 client.27623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 20 client.27623 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56336962a790 0x56336962cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 4 client.27623 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:23.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.817+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633692ac720 con 0x56336962a790 2024-03-12T05:25:23.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.818+0000 7f27ddbea640 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56336962a790 0x56336962cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.819+0000 7f27c17fa640 10 client.27623 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:23.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.819+0000 7f27ddbea640 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56336962a790 0x56336962cbd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de598588153a2fd server_cookie=fa4283bf60a8e6b2 in_seq=0 out_seq=0 2024-03-12T05:25:23.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.821+0000 7f27c17fa640 1 -- 172.21.15.100:0/161584866 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633692ac720 con 0x56336962a790 2024-03-12T05:25:23.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.821+0000 7f27c17fa640 10 client.27623 handle_command_reply: tid=0 2024-03-12T05:25:23.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.821+0000 7f27e2a29740 1 client.27623 shutdown 2024-03-12T05:25:23.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 msgr2=0x563369617400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369617400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27d80781b0 tx=0x7f27c8005e80 comp rx=0 tx=0).stop 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 shutdown_connections 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56336962a790 0x56336962cbd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369625f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369617400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 --2- 172.21.15.100:0/161584866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369616ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 >> 172.21.15.100:0/161584866 conn(0x56336960b1a0 msgr2=0x56336960cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27c17fa640 0 client.27623 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 shutdown_connections 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 1 -- 172.21.15.100:0/161584866 wait complete. 2024-03-12T05:25:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.822+0000 7f27e2a29740 20 client.27623 trim_cache size 0 max 16384 2024-03-12T05:25:23.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.846+0000 7f27e2a29740 1 Processor -- start 2024-03-12T05:25:23.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.846+0000 7f27e2a29740 1 -- start start 2024-03-12T05:25:23.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.846+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369614140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369614680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27ddbea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369614140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369614680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27ddbea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369614140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41296/0 (socket says 172.21.15.100:41296) 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369616750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27e2a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633695fe0f0 con 0x563369617940 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27e2a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369619c90 con 0x563369619fc0 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27ddbea640 1 -- 172.21.15.100:0/849242086 learned_addr learned my addr 172.21.15.100:0/849242086 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27e2a29740 1 -- 172.21.15.100:0/849242086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369617e80 con 0x5633696113e0 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369614680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38008/0 (socket says 172.21.15.100:38008) 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27de3eb640 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369616750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 -- 172.21.15.100:0/849242086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 msgr2=0x563369614140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369614140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 -- 172.21.15.100:0/849242086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 msgr2=0x563369616750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369616750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.847+0000 7f27dd3e9640 1 -- 172.21.15.100:0/849242086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56336950aac0 con 0x563369617940 2024-03-12T05:25:23.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.848+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369614680 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f27c8003bb0 tx=0x7f27c8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3286e381a169bf0f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.849+0000 7f27c37fe640 1 -- 172.21.15.100:0/849242086 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c802c7e0 con 0x563369617940 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.849+0000 7f27c37fe640 1 -- 172.21.15.100:0/849242086 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27c8004df0 con 0x563369617940 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.849+0000 7f27c37fe640 1 -- 172.21.15.100:0/849242086 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8041810 con 0x563369617940 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 -- 172.21.15.100:0/849242086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 msgr2=0x563369614680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369614680 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f27c8003bb0 tx=0x7f27c8005850 comp rx=0 tx=0).stop 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 -- 172.21.15.100:0/849242086 shutdown_connections 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x563369616750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x563369614680 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 --2- 172.21.15.100:0/849242086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369614140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 -- 172.21.15.100:0/849242086 >> 172.21.15.100:0/849242086 conn(0x56336960b1a0 msgr2=0x563369615790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 -- 172.21.15.100:0/849242086 shutdown_connections 2024-03-12T05:25:23.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 -- 172.21.15.100:0/849242086 wait complete. 2024-03-12T05:25:23.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 -- start start 2024-03-12T05:25:23.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.850+0000 7f27e2a29740 1 Processor -- start 2024-03-12T05:25:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369621380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x56336961f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369620560 con 0x563369617940 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369680cc0 con 0x563369619fc0 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563369680e00 con 0x5633696113e0 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27dd3e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x56336961f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27dd3e9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x56336961f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38010/0 (socket says 172.21.15.100:38010) 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27dd3e9640 1 -- 172.21.15.100:0/409617320 learned_addr learned my addr 172.21.15.100:0/409617320 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27ddbea640 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369621380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.851+0000 7f27e2a29740 10 client.0 _ll_register_callbacks cb 0x7f27d4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27dd3e9640 1 -- 172.21.15.100:0/409617320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 msgr2=0x563369621380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369621380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27dd3e9640 1 -- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 msgr2=0x56336961fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27dd3e9640 1 -- 172.21.15.100:0/409617320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56336956e610 con 0x563369617940 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27dd3e9640 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x56336961f720 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f27c8004c50 tx=0x7f27c8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6749e434ab5c6e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.852+0000 7f27c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.853+0000 7f27c17fa640 1 -- 172.21.15.100:0/409617320 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8003a00 con 0x563369617940 2024-03-12T05:25:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.853+0000 7f27c17fa640 1 -- 172.21.15.100:0/409617320 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f27c8048040 con 0x563369617940 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.853+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d4003610 con 0x563369617940 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27c17fa640 1 -- 172.21.15.100:0/409617320 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c8004f60 con 0x563369617940 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 10 client.18267 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5633695306a0 con 0x563369617940 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27c17fa640 1 -- 172.21.15.100:0/409617320 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27d4003610 con 0x563369617940 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27c17fa640 1 -- 172.21.15.100:0/409617320 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f27c8067020 con 0x563369617940 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 10 client.18267 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 20 client.18267 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563369682060 0x563369684480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 4 client.18267 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:25:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.854+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633692fb890 con 0x563369682060 2024-03-12T05:25:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.855+0000 7f27ddbea640 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563369682060 0x563369684480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.855+0000 7f27ddbea640 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563369682060 0x563369684480 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fadfa4baf7737d server_cookie=21d60ddb50f7b84f in_seq=0 out_seq=0 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.855+0000 7f27c17fa640 10 client.18267 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27c17fa640 1 -- 172.21.15.100:0/409617320 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5633692fb890 con 0x563369682060 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27c17fa640 10 client.18267 handle_command_reply: tid=0 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 client.18267 shutdown 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 msgr2=0x56336961f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x56336961f720 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f27c8004c50 tx=0x7f27c8005d50 comp rx=0 tx=0).stop 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 shutdown_connections 2024-03-12T05:25:23.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563369682060 0x563369684480 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563369619fc0 0x56336961fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563369617940 0x56336961f720 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 --2- 172.21.15.100:0/409617320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633696113e0 0x563369621380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 >> 172.21.15.100:0/409617320 conn(0x56336960b1a0 msgr2=0x5633696011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27c17fa640 0 client.18267 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 shutdown_connections 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 1 -- 172.21.15.100:0/409617320 wait complete. 2024-03-12T05:25:23.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.856+0000 7f27e2a29740 20 client.18267 trim_cache size 0 max 16384 2024-03-12T05:25:23.858 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:25:23.858 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:25:23.858 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:25:23.858 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:25:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27c40a1ca0 msgr2=0x7f27c40a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27c40a1ca0 0x7f27c40a4160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f27c8036040 tx=0x7f27c8005fd0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 msgr2=0x7f27d8199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 0x7f27d8199e80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f27d000b9e0 tx=0x7f27d000bea0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 shutdown_connections 2024-03-12T05:25:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27c40a1ca0 0x7f27c40a4160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81999d0 0x7f27d8199e80 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d810da70 0x7f27d819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 --2- 172.21.15.100:0/1522461837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27d8102100 0x7f27d819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 >> 172.21.15.100:0/1522461837 conn(0x7f27d8076e10 msgr2=0x7f27d810e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 shutdown_connections 2024-03-12T05:25:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.857+0000 7f27dfe4e640 1 -- 172.21.15.100:0/1522461837 wait complete. 2024-03-12T05:25:23.866 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:25:23.866 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:25:23.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 -- 172.21.15.100:0/2904432400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c102d10 msgr2=0x7fa31c10b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2904432400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c102d10 0x7fa31c10b1f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa304009900 tx=0x7fa30402bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 -- 172.21.15.100:0/2904432400 shutdown_connections 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2904432400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 0x7fa31c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2904432400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c102d10 0x7fa31c10b1f0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2904432400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31c1023f0 0x7fa31c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 -- 172.21.15.100:0/2904432400 >> 172.21.15.100:0/2904432400 conn(0x7fa31c077180 msgr2=0x7fa31c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.979+0000 7fa3211d0640 1 -- 172.21.15.100:0/2904432400 shutdown_connections 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.980+0000 7fa3211d0640 1 -- 172.21.15.100:0/2904432400 wait complete. 2024-03-12T05:25:23.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.980+0000 7fa3211d0640 1 Processor -- start 2024-03-12T05:25:23.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.980+0000 7fa3211d0640 1 -- start start 2024-03-12T05:25:23.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 0x7fa31c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31c102d10 0x7fa31c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 0x7fa31c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c197e10 con 0x7fa31c1023f0 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c197f80 con 0x7fa31c10b7c0 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c198140 con 0x7fa31c102d10 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 0x7fa31c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 0x7fa31c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38018/0 (socket says 172.21.15.100:38018) 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 0x7fa31c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 -- 172.21.15.100:0/2424246324 learned_addr learned my addr 172.21.15.100:0/2424246324 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31259c640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31c102d10 0x7fa31c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 -- 172.21.15.100:0/2424246324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31c102d10 msgr2=0x7fa31c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31c102d10 0x7fa31c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 -- 172.21.15.100:0/2424246324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 msgr2=0x7fa31c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 0x7fa31c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 -- 172.21.15.100:0/2424246324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009560 con 0x7fa31c1023f0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31b59e640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 0x7fa31c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa31ad9d640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 0x7fa31c19d5c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e9f0 tx=0x7fa30c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c009720 con 0x7fa31c1023f0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa30c00cd80 con 0x7fa31c1023f0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c198420 con 0x7fa31c1023f0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c010690 con 0x7fa31c1023f0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.981+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c1a4540 con 0x7fa31c1023f0 2024-03-12T05:25:23.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.982+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c068b20 con 0x7fa31c1023f0 2024-03-12T05:25:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.983+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fa30c0046c0 con 0x7fa31c1023f0 2024-03-12T05:25:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.984+0000 7fa318d99640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa3080a1ba0 0x7fa3080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.984+0000 7fa31259c640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa3080a1ba0 0x7fa3080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.984+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa30c014070 con 0x7fa31c1023f0 2024-03-12T05:25:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.985+0000 7fa31259c640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa3080a1ba0 0x7fa3080a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa304006010 tx=0x7fa304005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:23.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:23.988+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa30c096c20 con 0x7fa31c1023f0 2024-03-12T05:25:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4634 61.44 MB/sec warmup 7 sec latency 239.637 ms 2024-03-12T05:25:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.239+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa31c067f70 con 0x7fa31c1023f0 2024-03-12T05:25:24.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.240+0000 7fa318d99640 1 -- 172.21.15.100:0/2424246324 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7fa30c09a2e0 con 0x7fa31c1023f0 2024-03-12T05:25:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:25:24.235 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:25:21.243438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:25:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-03-12T05:25:24.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa3080a1ba0 msgr2=0x7fa3080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa3080a1ba0 0x7fa3080a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa304006010 tx=0x7fa304005e40 comp rx=0 tx=0).stop 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 msgr2=0x7fa31c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 0x7fa31c19d5c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e9f0 tx=0x7fa30c00edf0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 shutdown_connections 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa3080a1ba0 0x7fa3080a4060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31c10b7c0 0x7fa31c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31c102d10 0x7fa31c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 --2- 172.21.15.100:0/2424246324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c1023f0 0x7fa31c19d5c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 >> 172.21.15.100:0/2424246324 conn(0x7fa31c077180 msgr2=0x7fa31c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 shutdown_connections 2024-03-12T05:25:24.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.242+0000 7fa3211d0640 1 -- 172.21.15.100:0/2424246324 wait complete. 2024-03-12T05:25:24.248 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:25:24.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:25:24.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 -- 172.21.15.100:0/1643912047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0105330 msgr2=0x7f67c010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 --2- 172.21.15.100:0/1643912047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0105330 0x7f67c010cf00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009cd0 tx=0x7f67b802c0e0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 -- 172.21.15.100:0/1643912047 shutdown_connections 2024-03-12T05:25:24.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 --2- 172.21.15.100:0/1643912047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0105330 0x7f67c010cf00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 --2- 172.21.15.100:0/1643912047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c0100c10 0x7f67c0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 --2- 172.21.15.100:0/1643912047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0100260 0x7f67c0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 -- 172.21.15.100:0/1643912047 >> 172.21.15.100:0/1643912047 conn(0x7f67c0077180 msgr2=0x7f67c0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 -- 172.21.15.100:0/1643912047 shutdown_connections 2024-03-12T05:25:24.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.374+0000 7f67c7c15640 1 -- 172.21.15.100:0/1643912047 wait complete. 2024-03-12T05:25:24.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 Processor -- start 2024-03-12T05:25:24.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 -- start start 2024-03-12T05:25:24.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0100260 0x7f67c01976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 0x7f67c0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c019bf70 0x7f67c019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c019c960 con 0x7f67c0105330 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c019cad0 con 0x7f67c0100260 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c7c15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c019cc40 con 0x7f67c019bf70 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c51b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 0x7f67c0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c51b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 0x7f67c0197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38028/0 (socket says 172.21.15.100:38028) 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c51b0640 1 -- 172.21.15.100:0/859207509 learned_addr learned my addr 172.21.15.100:0/859207509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.375+0000 7f67c59b1640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0100260 0x7f67c01976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.376+0000 7f67c51b0640 1 -- 172.21.15.100:0/859207509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c019bf70 msgr2=0x7f67c019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.376+0000 7f67c51b0640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c019bf70 0x7f67c019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.376+0000 7f67c51b0640 1 -- 172.21.15.100:0/859207509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0100260 msgr2=0x7f67c01976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.376+0000 7f67c51b0640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0100260 0x7f67c01976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.376+0000 7f67c51b0640 1 -- 172.21.15.100:0/859207509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009900 con 0x7f67c0105330 2024-03-12T05:25:24.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.376+0000 7f67c51b0640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 0x7f67c0197bf0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9c0 tx=0x7f67b000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.377+0000 7f67b6ffd640 1 -- 172.21.15.100:0/859207509 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0009700 con 0x7f67c0105330 2024-03-12T05:25:24.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.377+0000 7f67b6ffd640 1 -- 172.21.15.100:0/859207509 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67b0005500 con 0x7f67c0105330 2024-03-12T05:25:24.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.377+0000 7f67b6ffd640 1 -- 172.21.15.100:0/859207509 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0010430 con 0x7f67c0105330 2024-03-12T05:25:24.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.377+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c0101c20 con 0x7f67c0105330 2024-03-12T05:25:24.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.378+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c0102080 con 0x7f67c0105330 2024-03-12T05:25:24.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.381+0000 7f67b6ffd640 1 -- 172.21.15.100:0/859207509 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f67b000ccc0 con 0x7f67c0105330 2024-03-12T05:25:24.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.381+0000 7f67ca7e1740 1 Processor -- start 2024-03-12T05:25:24.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.381+0000 7f67ca7e1740 1 -- start start 2024-03-12T05:25:24.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5250 0x56322e7b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5b70 0x56322e7be210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e770960 con 0x56322e7b5b70 2024-03-12T05:25:24.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67c61b2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5250 0x56322e7b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67c61b2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5250 0x56322e7b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37580/0 (socket says 172.21.15.100:37580) 2024-03-12T05:25:24.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67c61b2640 1 -- 172.21.15.100:0/471882019 learned_addr learned my addr 172.21.15.100:0/471882019 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67c59b1640 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5b70 0x56322e7be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e724310 con 0x56322e7b5250 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e721840 con 0x56322e7be750 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.382+0000 7f67b6ffd640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f67ac0a1d70 0x7f67ac0a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c51b0640 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c61b2640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f67ac0a1d70 0x7f67ac0a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c59b1640 1 -- 172.21.15.100:0/471882019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 msgr2=0x56322e7c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c59b1640 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67b6ffd640 1 -- 172.21.15.100:0/859207509 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f67b0014070 con 0x7f67c0105330 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c59b1640 1 -- 172.21.15.100:0/471882019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5250 msgr2=0x56322e7b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c59b1640 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5250 0x56322e7b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c59b1640 1 -- 172.21.15.100:0/471882019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56322e6df060 con 0x56322e7b5b70 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c59b1640 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5b70 0x56322e7be210 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00abf0 tx=0x7f67bc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfc04ca4d4fd00d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67c61b2640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f67ac0a1d70 0x7f67ac0a4230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f67b8033000 tx=0x7f67b80095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67abfff640 1 -- 172.21.15.100:0/471882019 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc005220 con 0x56322e7b5b70 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67abfff640 1 -- 172.21.15.100:0/471882019 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67bc0047a0 con 0x56322e7b5b70 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.383+0000 7f67abfff640 1 -- 172.21.15.100:0/471882019 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc018770 con 0x56322e7b5b70 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5b70 msgr2=0x56322e7be210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5b70 0x56322e7be210 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00abf0 tx=0x7f67bc00f790 comp rx=0 tx=0).stop 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 shutdown_connections 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5b70 0x56322e7be210 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/471882019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5250 0x56322e7b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 >> 172.21.15.100:0/471882019 conn(0x56322e7af980 msgr2=0x56322e7afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 shutdown_connections 2024-03-12T05:25:24.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.384+0000 7f67ca7e1740 1 -- 172.21.15.100:0/471882019 wait complete. 2024-03-12T05:25:24.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.385+0000 7f67ca7e1740 1 -- start start 2024-03-12T05:25:24.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.385+0000 7f67ca7e1740 1 Processor -- start 2024-03-12T05:25:24.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.385+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67c59b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67c59b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38050/0 (socket says 172.21.15.100:38050) 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67c59b1640 1 -- 172.21.15.100:0/390130367 learned_addr learned my addr 172.21.15.100:0/390130367 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e71d020 con 0x56322e7b5250 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e3f0430 con 0x56322e7b5b70 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e3ace00 con 0x56322e7be750 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67ca7e1740 10 client.0 _ll_register_callbacks cb 0x56322e6c4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67c61b2640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.386+0000 7f67c51b0640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67c59b1640 1 -- 172.21.15.100:0/390130367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 msgr2=0x56322e7bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67c59b1640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67c59b1640 1 -- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 msgr2=0x56322e7bbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67c59b1640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7bbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67c59b1640 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc00a820 con 0x56322e7b5250 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67c59b1640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bb730 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f67bc01a040 tx=0x7f67bc0052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fbcc7fea6ef42d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.387+0000 7f67a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.388+0000 7f67a97fa640 1 -- 172.21.15.100:0/390130367 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0056e0 con 0x56322e7b5250 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.388+0000 7f67a97fa640 1 -- 172.21.15.100:0/390130367 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f67bc0184b0 con 0x56322e7b5250 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.388+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56322e41c7e0 con 0x56322e7b5250 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.388+0000 7f67a97fa640 1 -- 172.21.15.100:0/390130367 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67bc0186b0 con 0x56322e7b5250 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.388+0000 7f67ca7e1740 10 client.18282 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:24.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.388+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56322e71b6a0 con 0x56322e7b5250 2024-03-12T05:25:24.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.389+0000 7f67a97fa640 1 -- 172.21.15.100:0/390130367 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67bc018cf0 con 0x56322e7b5250 2024-03-12T05:25:24.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.389+0000 7f67a97fa640 1 -- 172.21.15.100:0/390130367 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f67bc023a80 con 0x56322e7b5250 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.389+0000 7f67ca7e1740 10 client.18282 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.389+0000 7f67ca7e1740 10 client.18282 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.390+0000 7f67ca7e1740 20 client.18282 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.390+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e7d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.390+0000 7f67ca7e1740 4 client.18282 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.390+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56322e41c7e0 con 0x56322e7d7220 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.390+0000 7f67c51b0640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e7d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.391+0000 7f67c51b0640 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e7d96c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=489ad10c35bbe93 server_cookie=209bf51abb820de8 in_seq=0 out_seq=0 2024-03-12T05:25:24.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.391+0000 7f67a97fa640 10 client.18282 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:24.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.397+0000 7f67a97fa640 1 -- 172.21.15.100:0/390130367 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56322e41c7e0 con 0x56322e7d7220 2024-03-12T05:25:24.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.397+0000 7f67a97fa640 10 client.18282 handle_command_reply: tid=0 2024-03-12T05:25:24.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.397+0000 7f67ca7e1740 1 client.18282 shutdown 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 msgr2=0x56322e7bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bb730 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f67bc01a040 tx=0x7f67bc0052b0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 shutdown_connections 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e7d96c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67a97fa640 0 client.18282 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/390130367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bb730 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 >> 172.21.15.100:0/390130367 conn(0x56322e7af980 msgr2=0x56322e7b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 shutdown_connections 2024-03-12T05:25:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 1 -- 172.21.15.100:0/390130367 wait complete. 2024-03-12T05:25:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.398+0000 7f67ca7e1740 20 client.18282 trim_cache size 0 max 16384 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 Processor -- start 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 -- start start 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e6a2130 con 0x56322e7b5250 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e7a1a40 con 0x56322e7b5b70 2024-03-12T05:25:24.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e7ab5c0 con 0x56322e7be750 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c59b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c59b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38054/0 (socket says 172.21.15.100:38054) 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c59b1640 1 -- 172.21.15.100:0/164555333 learned_addr learned my addr 172.21.15.100:0/164555333 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c51b0640 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c61b2640 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c51b0640 1 -- 172.21.15.100:0/164555333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 msgr2=0x56322e7b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c51b0640 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c51b0640 1 -- 172.21.15.100:0/164555333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 msgr2=0x56322e7bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c51b0640 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c51b0640 1 -- 172.21.15.100:0/164555333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56322e6b7c20 con 0x56322e7b5b70 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.416+0000 7f67c59b1640 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.417+0000 7f67c51b0640 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7b8990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67b00cae90 tx=0x7f67b006d6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eddd7613158f110 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.417+0000 7f67ab7fe640 1 -- 172.21.15.100:0/164555333 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0065b90 con 0x56322e7b5b70 2024-03-12T05:25:24.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.417+0000 7f67ab7fe640 1 -- 172.21.15.100:0/164555333 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67b009aa70 con 0x56322e7b5b70 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.417+0000 7f67ab7fe640 1 -- 172.21.15.100:0/164555333 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0098700 con 0x56322e7b5b70 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- 172.21.15.100:0/164555333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 msgr2=0x56322e7b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7b8990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67b00cae90 tx=0x7f67b006d6a0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- 172.21.15.100:0/164555333 shutdown_connections 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7be750 0x56322e7b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7b8990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/164555333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7b5250 0x56322e7bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- 172.21.15.100:0/164555333 >> 172.21.15.100:0/164555333 conn(0x56322e7af980 msgr2=0x56322e7ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- 172.21.15.100:0/164555333 shutdown_connections 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- 172.21.15.100:0/164555333 wait complete. 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- start start 2024-03-12T05:25:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 Processor -- start 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 0x56322e7c4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7be750 0x56322e82d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e82dd30 con 0x56322e7be750 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e82de70 con 0x56322e7b5b70 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322e82e000 con 0x56322e7b5250 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.418+0000 7f67ca7e1740 10 client.0 _ll_register_callbacks cb 0x7f67bc00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c59b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 0x56322e7c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c59b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 0x56322e7c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41366/0 (socket says 172.21.15.100:41366) 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37604/0 (socket says 172.21.15.100:37604) 2024-03-12T05:25:24.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c59b1640 1 -- 172.21.15.100:0/3553592943 learned_addr learned my addr 172.21.15.100:0/3553592943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c61b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7be750 0x56322e82d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 -- 172.21.15.100:0/3553592943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 msgr2=0x56322e7c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 0x56322e7c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 -- 172.21.15.100:0/3553592943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7be750 msgr2=0x56322e82d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7be750 0x56322e82d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 -- 172.21.15.100:0/3553592943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56322e71b6a0 con 0x56322e7b5b70 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c59b1640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 0x56322e7c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c61b2640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7be750 0x56322e82d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67c51b0640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7c4d50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67b009a300 tx=0x7f67b0005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5edac0795bbb3784 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67a97fa640 1 -- 172.21.15.100:0/3553592943 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b009e4c0 con 0x56322e7b5b70 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67a97fa640 1 -- 172.21.15.100:0/3553592943 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f67b00a2b10 con 0x56322e7b5b70 2024-03-12T05:25:24.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67bc0047a0 con 0x56322e7b5b70 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67a97fa640 1 -- 172.21.15.100:0/3553592943 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b00cb110 con 0x56322e7b5b70 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67ca7e1740 10 client.27650 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.419+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f67c0102080 con 0x56322e7b5b70 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67a97fa640 1 -- 172.21.15.100:0/3553592943 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67bc0047a0 con 0x56322e7b5b70 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67a97fa640 1 -- 172.21.15.100:0/3553592943 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f67b00997c0 con 0x56322e7b5b70 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67ca7e1740 10 client.27650 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67ca7e1740 10 client.27650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67ca7e1740 20 client.27650 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e8313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67ca7e1740 4 client.27650 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56322e459720 con 0x56322e7d7220 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67c59b1640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e8313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67a97fa640 10 client.27650 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:24.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.420+0000 7f67c59b1640 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e8313d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a84d0dd41472c63 server_cookie=cc2d811f4e3d91e2 in_seq=0 out_seq=0 2024-03-12T05:25:24.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67a97fa640 1 -- 172.21.15.100:0/3553592943 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56322e459720 con 0x56322e7d7220 2024-03-12T05:25:24.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67a97fa640 10 client.27650 handle_command_reply: tid=0 2024-03-12T05:25:24.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 client.27650 shutdown 2024-03-12T05:25:24.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 msgr2=0x56322e7c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7c4d50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67b009a300 tx=0x7f67b0005670 comp rx=0 tx=0).stop 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 shutdown_connections 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56322e7d7220 0x56322e8313d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56322e7be750 0x56322e82d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67a97fa640 0 client.27650 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56322e7b5b70 0x56322e7c4d50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 --2- 172.21.15.100:0/3553592943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56322e7b5250 0x56322e7c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 >> 172.21.15.100:0/3553592943 conn(0x56322e7af980 msgr2=0x56322e7ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 shutdown_connections 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 1 -- 172.21.15.100:0/3553592943 wait complete. 2024-03-12T05:25:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.423+0000 7f67ca7e1740 20 client.27650 trim_cache size 0 max 16384 2024-03-12T05:25:24.424 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:25:24.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f67ac0a1d70 msgr2=0x7f67ac0a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f67ac0a1d70 0x7f67ac0a4230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f67b8033000 tx=0x7f67b80095c0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 msgr2=0x7f67c0197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 0x7f67c0197bf0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9c0 tx=0x7f67b000edc0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 shutdown_connections 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f67ac0a1d70 0x7f67ac0a4230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c019bf70 0x7f67c019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c0105330 0x7f67c0197bf0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 --2- 172.21.15.100:0/859207509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67c0100260 0x7f67c01976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 >> 172.21.15.100:0/859207509 conn(0x7f67c0077180 msgr2=0x7f67c00685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 shutdown_connections 2024-03-12T05:25:24.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.424+0000 7f67c7c15640 1 -- 172.21.15.100:0/859207509 wait complete. 2024-03-12T05:25:24.429 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:25:24.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:25:24.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 -- 172.21.15.100:0/3747406000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8840069aa0 msgr2=0x7f8840069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 --2- 172.21.15.100:0/3747406000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8840069aa0 0x7f8840069e80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8830009930 tx=0x7f883002bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 -- 172.21.15.100:0/3747406000 shutdown_connections 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 --2- 172.21.15.100:0/3747406000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88401064e0 0x7f884010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 --2- 172.21.15.100:0/3747406000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f884006a450 0x7f8840105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 --2- 172.21.15.100:0/3747406000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8840069aa0 0x7f8840069e80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 -- 172.21.15.100:0/3747406000 >> 172.21.15.100:0/3747406000 conn(0x7f8840077150 msgr2=0x7f8840077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 -- 172.21.15.100:0/3747406000 shutdown_connections 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.543+0000 7f88462df640 1 -- 172.21.15.100:0/3747406000 wait complete. 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 Processor -- start 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 -- start start 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8840069aa0 0x7f884019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 0x7f884019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8840197610 0x7f8840197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840198000 con 0x7f88401064e0 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840198170 con 0x7f8840197610 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88462df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840198330 con 0x7f8840069aa0 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 0x7f884019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 0x7f884019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38068/0 (socket says 172.21.15.100:38068) 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 -- 172.21.15.100:0/3776961426 learned_addr learned my addr 172.21.15.100:0/3776961426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f884487c640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8840197610 0x7f8840197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f883ffff640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8840069aa0 0x7f884019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 -- 172.21.15.100:0/3776961426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8840069aa0 msgr2=0x7f884019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8840069aa0 0x7f884019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 -- 172.21.15.100:0/3776961426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8840197610 msgr2=0x7f8840197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8840197610 0x7f8840197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 -- 172.21.15.100:0/3776961426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009560 con 0x7f88401064e0 2024-03-12T05:25:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.544+0000 7f88377fe640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 0x7f884019d9a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f882800db20 tx=0x7f882800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.545+0000 7f883dffb640 1 -- 172.21.15.100:0/3776961426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800bea0 con 0x7f88401064e0 2024-03-12T05:25:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.545+0000 7f883dffb640 1 -- 172.21.15.100:0/3776961426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8828003e50 con 0x7f88401064e0 2024-03-12T05:25:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.545+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88401a4320 con 0x7f88401064e0 2024-03-12T05:25:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.545+0000 7f883dffb640 1 -- 172.21.15.100:0/3776961426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280106c0 con 0x7f88401064e0 2024-03-12T05:25:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.545+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88401a4880 con 0x7f88401064e0 2024-03-12T05:25:24.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.547+0000 7f883dffb640 1 -- 172.21.15.100:0/3776961426 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8828003970 con 0x7f88401064e0 2024-03-12T05:25:24.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.547+0000 7f883dffb640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f882c0a1c20 0x7f882c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.547+0000 7f8848f90740 1 Processor -- start 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.547+0000 7f8848f90740 1 -- start start 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.547+0000 7f883ffff640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f882c0a1c20 0x7f882c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.547+0000 7f883dffb640 1 -- 172.21.15.100:0/3776961426 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f88280cd4e0 con 0x7f88401064e0 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f8848f90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 0x55a814e85360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f8848f90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e858a0 0x55a814e92620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f8848f90740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e92b60 0x55a814e94f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f8848f90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814e40960 con 0x55a814e84f80 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f8848f90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814df4310 con 0x55a814e858a0 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f8848f90740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814df1840 con 0x55a814e92b60 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f88377fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e858a0 0x55a814e92620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f883ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 0x55a814e85360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e92b60 0x55a814e94f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f88377fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e858a0 0x55a814e92620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37624/0 (socket says 172.21.15.100:37624) 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f883ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 0x55a814e85360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38078/0 (socket says 172.21.15.100:38078) 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f88377fe640 1 -- 172.21.15.100:0/4169242094 learned_addr learned my addr 172.21.15.100:0/4169242094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f883ffff640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f882c0a1c20 0x7f882c0a40e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8830005ab0 tx=0x7f8830004720 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 -- 172.21.15.100:0/4169242094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e858a0 msgr2=0x55a814e92620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e858a0 0x55a814e92620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 -- 172.21.15.100:0/4169242094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 msgr2=0x55a814e85360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 0x55a814e85360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 -- 172.21.15.100:0/4169242094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a814daf060 con 0x55a814e92b60 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f883ffff640 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 0x55a814e85360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f884487c640 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e92b60 0x55a814e94f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8838009b80 tx=0x7f8838011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83e7e365ca97b7f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f88367fc640 1 -- 172.21.15.100:0/4169242094 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838004050 con 0x55a814e92b60 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f88367fc640 1 -- 172.21.15.100:0/4169242094 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8838005500 con 0x55a814e92b60 2024-03-12T05:25:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.548+0000 7f88367fc640 1 -- 172.21.15.100:0/4169242094 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838004c60 con 0x55a814e92b60 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 -- 172.21.15.100:0/4169242094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e92b60 msgr2=0x55a814e94f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e92b60 0x55a814e94f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8838009b80 tx=0x7f8838011810 comp rx=0 tx=0).stop 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 -- 172.21.15.100:0/4169242094 shutdown_connections 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e92b60 0x55a814e94f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e858a0 0x55a814e92620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 --2- 172.21.15.100:0/4169242094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e84f80 0x55a814e85360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 -- 172.21.15.100:0/4169242094 >> 172.21.15.100:0/4169242094 conn(0x55a814e7f980 msgr2=0x55a814e7fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 -- 172.21.15.100:0/4169242094 shutdown_connections 2024-03-12T05:25:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.549+0000 7f8848f90740 1 -- 172.21.15.100:0/4169242094 wait complete. 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 -- start start 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 Processor -- start 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e90070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814ded020 con 0x55a814e858a0 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814ac0430 con 0x55a814e92b60 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814a7ce00 con 0x55a814e84f80 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f8848f90740 10 client.0 _ll_register_callbacks cb 0x55a814d94cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e90070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e90070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38088/0 (socket says 172.21.15.100:38088) 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 -- 172.21.15.100:0/1505648957 learned_addr learned my addr 172.21.15.100:0/1505648957 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f883ffff640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f884487c640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 -- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 msgr2=0x55a814e8fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e8fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 -- 172.21.15.100:0/1505648957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 msgr2=0x55a814e905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.550+0000 7f88377fe640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e905b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f88377fe640 1 -- 172.21.15.100:0/1505648957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8838009830 con 0x55a814e858a0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f88377fe640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e90070 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f88280caca0 tx=0x7f882809e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a42989eeb5a3a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f881f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f881f7fe640 1 -- 172.21.15.100:0/1505648957 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828098540 con 0x55a814e858a0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f881f7fe640 1 -- 172.21.15.100:0/1505648957 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f88280042b0 con 0x55a814e858a0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a814aec7e0 con 0x55a814e858a0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f881f7fe640 1 -- 172.21.15.100:0/1505648957 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88280cae70 con 0x55a814e858a0 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 10 client.18300 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:24.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a814deb6a0 con 0x55a814e858a0 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f881f7fe640 1 -- 172.21.15.100:0/1505648957 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a814aec7e0 con 0x55a814e858a0 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f881f7fe640 1 -- 172.21.15.100:0/1505648957 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f882807c070 con 0x55a814e858a0 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 10 client.18300 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 10 client.18300 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 20 client.18300 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814ea30b0 0x55a814ea54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 4 client.18300 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.551+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a814b29720 con 0x55a814ea30b0 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.552+0000 7f883ffff640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814ea30b0 0x55a814ea54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.552+0000 7f883ffff640 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814ea30b0 0x55a814ea54d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2036edcfeb9290d2 server_cookie=900a5ebff17e02cb in_seq=0 out_seq=0 2024-03-12T05:25:24.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.552+0000 7f881f7fe640 10 client.18300 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:24.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f881f7fe640 1 -- 172.21.15.100:0/1505648957 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a814b29720 con 0x55a814ea30b0 2024-03-12T05:25:24.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f881f7fe640 10 client.18300 handle_command_reply: tid=0 2024-03-12T05:25:24.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 client.18300 shutdown 2024-03-12T05:25:24.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 msgr2=0x55a814e90070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e90070 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f88280caca0 tx=0x7f882809e2a0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 shutdown_connections 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814ea30b0 0x55a814ea54d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e90070 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f881f7fe640 0 client.18300 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 --2- 172.21.15.100:0/1505648957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e8fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 >> 172.21.15.100:0/1505648957 conn(0x55a814e7f980 msgr2=0x55a814e816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 shutdown_connections 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 1 -- 172.21.15.100:0/1505648957 wait complete. 2024-03-12T05:25:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.555+0000 7f8848f90740 20 client.18300 trim_cache size 0 max 16384 2024-03-12T05:25:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 Processor -- start 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 -- start start 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e8cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814d72130 con 0x55a814e858a0 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814e80bb0 con 0x55a814e92b60 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f8848f90740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814e845c0 con 0x55a814e84f80 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e903d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38104/0 (socket says 172.21.15.100:38104) 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 -- 172.21.15.100:0/3769671512 learned_addr learned my addr 172.21.15.100:0/3769671512 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f883ffff640 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f884487c640 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e8cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 -- 172.21.15.100:0/3769671512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 msgr2=0x55a814e92260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e92260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 -- 172.21.15.100:0/3769671512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 msgr2=0x55a814e8cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.567+0000 7f88377fe640 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e8cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.568+0000 7f88377fe640 1 -- 172.21.15.100:0/3769671512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a814d87c20 con 0x55a814e858a0 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.568+0000 7f88377fe640 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e903d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8828098430 tx=0x7f8828004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a812d15c369a733 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.568+0000 7f8835ffb640 1 -- 172.21.15.100:0/3769671512 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280995d0 con 0x55a814e858a0 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.568+0000 7f8835ffb640 1 -- 172.21.15.100:0/3769671512 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8828098540 con 0x55a814e858a0 2024-03-12T05:25:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.568+0000 7f8835ffb640 1 -- 172.21.15.100:0/3769671512 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280b2e90 con 0x55a814e858a0 2024-03-12T05:25:24.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 -- 172.21.15.100:0/3769671512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 msgr2=0x55a814e903d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e903d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8828098430 tx=0x7f8828004680 comp rx=0 tx=0).stop 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 -- 172.21.15.100:0/3769671512 shutdown_connections 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e8cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e903d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 --2- 172.21.15.100:0/3769671512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 -- 172.21.15.100:0/3769671512 >> 172.21.15.100:0/3769671512 conn(0x55a814e7f980 msgr2=0x55a814e8ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 -- 172.21.15.100:0/3769671512 shutdown_connections 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 -- 172.21.15.100:0/3769671512 wait complete. 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.569+0000 7f8848f90740 1 -- start start 2024-03-12T05:25:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 Processor -- start 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e98460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814ef95c0 con 0x55a814e858a0 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814ef9700 con 0x55a814e92b60 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a814ef9840 con 0x55a814e84f80 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f8848f90740 10 client.0 _ll_register_callbacks cb 0x7f8838009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38110/0 (socket says 172.21.15.100:38110) 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 -- 172.21.15.100:0/3305310850 learned_addr learned my addr 172.21.15.100:0/3305310850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f883ffff640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e98460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f884487c640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 -- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 msgr2=0x55a814e98460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e98460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 -- 172.21.15.100:0/3305310850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 msgr2=0x55a814e98ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e98ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.570+0000 7f88377fe640 1 -- 172.21.15.100:0/3305310850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a814deb6a0 con 0x55a814e858a0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f88377fe640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e989a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f88280965e0 tx=0x7f88280b84e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdf3e43f5682e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f881f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f881f7fe640 1 -- 172.21.15.100:0/3305310850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280b2e70 con 0x55a814e858a0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f881f7fe640 1 -- 172.21.15.100:0/3305310850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f88280a2870 con 0x55a814e858a0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8838005500 con 0x55a814e858a0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f881f7fe640 1 -- 172.21.15.100:0/3305310850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88280995d0 con 0x55a814e858a0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f8848f90740 10 client.18306 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a814dad8c0 con 0x55a814e858a0 2024-03-12T05:25:24.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.571+0000 7f881f7fe640 1 -- 172.21.15.100:0/3305310850 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8838005500 con 0x55a814e858a0 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f881f7fe640 1 -- 172.21.15.100:0/3305310850 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f882807c050 con 0x55a814e858a0 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f8848f90740 10 client.18306 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f8848f90740 10 client.18306 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f8848f90740 20 client.18306 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f8848f90740 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814efaab0 0x55a814efcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f8848f90740 4 client.18306 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a814b78890 con 0x55a814efaab0 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f883ffff640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814efaab0 0x55a814efcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f881f7fe640 10 client.18306 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:24.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.572+0000 7f883ffff640 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814efaab0 0x55a814efcea0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21d521055781c072 server_cookie=ed2ad491792babf8 in_seq=0 out_seq=0 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f881f7fe640 1 -- 172.21.15.100:0/3305310850 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a814b78890 con 0x55a814efaab0 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f881f7fe640 10 client.18306 handle_command_reply: tid=0 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 client.18306 shutdown 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 msgr2=0x55a814e989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e989a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f88280965e0 tx=0x7f88280b84e0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 shutdown_connections 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55a814efaab0 0x55a814efcea0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a814e92b60 0x55a814e98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f881f7fe640 0 client.18306 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a814e858a0 0x55a814e989a0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 --2- 172.21.15.100:0/3305310850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a814e84f80 0x55a814e98460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 >> 172.21.15.100:0/3305310850 conn(0x55a814e7f980 msgr2=0x55a814e80db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 shutdown_connections 2024-03-12T05:25:24.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 1 -- 172.21.15.100:0/3305310850 wait complete. 2024-03-12T05:25:24.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.576+0000 7f8848f90740 20 client.18306 trim_cache size 0 max 16384 2024-03-12T05:25:24.574 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f882c0a1c20 msgr2=0x7f882c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f882c0a1c20 0x7f882c0a40e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8830005ab0 tx=0x7f8830004720 comp rx=0 tx=0).stop 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 msgr2=0x7f884019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 0x7f884019d9a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f882800db20 tx=0x7f882800b260 comp rx=0 tx=0).stop 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 shutdown_connections 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f882c0a1c20 0x7f882c0a40e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8840197610 0x7f8840197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88401064e0 0x7f884019d9a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 --2- 172.21.15.100:0/3776961426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8840069aa0 0x7f884019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 >> 172.21.15.100:0/3776961426 conn(0x7f8840077150 msgr2=0x7f88400fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 shutdown_connections 2024-03-12T05:25:24.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.577+0000 7f88462df640 1 -- 172.21.15.100:0/3776961426 wait complete. 2024-03-12T05:25:24.582 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:25:24.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:25:24.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/2734791058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e580685d0 msgr2=0x7f0e580689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/2734791058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e580685d0 0x7f0e580689b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009cd0 tx=0x7f0e4802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/2734791058 shutdown_connections 2024-03-12T05:25:24.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/2734791058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e58106430 0x7f0e5810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/2734791058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e58068f80 0x7f0e58105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/2734791058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e580685d0 0x7f0e580689b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/2734791058 >> 172.21.15.100:0/2734791058 conn(0x7f0e58077180 msgr2=0x7f0e58077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/2734791058 shutdown_connections 2024-03-12T05:25:24.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.699+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/2734791058 wait complete. 2024-03-12T05:25:24.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 Processor -- start 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 -- start start 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e580685d0 0x7f0e5819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 0x7f0e5819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e581975f0 0x7f0e58197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58197fe0 con 0x7f0e58106430 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58198150 con 0x7f0e580685d0 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5d1c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58198310 con 0x7f0e581975f0 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 0x7f0e5819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 0x7f0e5819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38120/0 (socket says 172.21.15.100:38120) 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5659c640 1 -- 172.21.15.100:0/3312474353 learned_addr learned my addr 172.21.15.100:0/3312474353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e580685d0 0x7f0e5819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.700+0000 7f0e5759e640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e581975f0 0x7f0e58197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5659c640 1 -- 172.21.15.100:0/3312474353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e581975f0 msgr2=0x7f0e58197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5659c640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e581975f0 0x7f0e58197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5659c640 1 -- 172.21.15.100:0/3312474353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e580685d0 msgr2=0x7f0e5819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5659c640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e580685d0 0x7f0e5819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5659c640 1 -- 172.21.15.100:0/3312474353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48009900 con 0x7f0e58106430 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5659c640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 0x7f0e5819d980 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e9f0 tx=0x7f0e4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e3ffff640 1 -- 172.21.15.100:0/3312474353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0096c0 con 0x7f0e58106430 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e3ffff640 1 -- 172.21.15.100:0/3312474353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e4c005500 con 0x7f0e58106430 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e3ffff640 1 -- 172.21.15.100:0/3312474353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c010430 con 0x7f0e58106430 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e581a4300 con 0x7f0e58106430 2024-03-12T05:25:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.701+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e581a4810 con 0x7f0e58106430 2024-03-12T05:25:24.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.703+0000 7f0e3ffff640 1 -- 172.21.15.100:0/3312474353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0e4c005020 con 0x7f0e58106430 2024-03-12T05:25:24.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 Processor -- start 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 -- start start 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed0589960 con 0x55aed05ce250 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed053d310 con 0x55aed05ceb70 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed053a840 con 0x55aed05d7750 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e3ffff640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e400a1c20 0x7f0e400a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38126/0 (socket says 172.21.15.100:38126) 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5759e640 1 -- 172.21.15.100:0/1816336499 learned_addr learned my addr 172.21.15.100:0/1816336499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5659c640 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e5759e640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e400a1c20 0x7f0e400a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e3ffff640 1 -- 172.21.15.100:0/3312474353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0e4c014070 con 0x7f0e58106430 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e56d9d640 1 -- 172.21.15.100:0/1816336499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 msgr2=0x55aed05d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e56d9d640 1 -- 172.21.15.100:0/1816336499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 msgr2=0x55aed05ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.704+0000 7f0e56d9d640 1 -- 172.21.15.100:0/1816336499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed04f8060 con 0x55aed05ceb70 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.705+0000 7f0e5759e640 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.705+0000 7f0e5759e640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e400a1c20 0x7f0e400a40e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e50011980 tx=0x7f0e50005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.705+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d7210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e4802c700 tx=0x7f0e48005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abae152235253f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.705+0000 7f0e3d7fa640 1 -- 172.21.15.100:0/1816336499 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48034440 con 0x55aed05ceb70 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.705+0000 7f0e3d7fa640 1 -- 172.21.15.100:0/1816336499 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e480345a0 con 0x55aed05ceb70 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.705+0000 7f0e3d7fa640 1 -- 172.21.15.100:0/1816336499 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48003bd0 con 0x55aed05ceb70 2024-03-12T05:25:24.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/1816336499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 msgr2=0x55aed05d7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d7210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e4802c700 tx=0x7f0e48005f70 comp rx=0 tx=0).stop 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/1816336499 shutdown_connections 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d7210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/1816336499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/1816336499 >> 172.21.15.100:0/1816336499 conn(0x55aed05c8980 msgr2=0x55aed05c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/1816336499 shutdown_connections 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/1816336499 wait complete. 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- start start 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 Processor -- start 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05ce250 0x55aed05d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 0x55aed05d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05d7750 0x55aed05d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed0536020 con 0x55aed05ceb70 2024-03-12T05:25:24.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed0209430 con 0x55aed05d7750 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed01c5e00 con 0x55aed05ce250 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5fd9e740 10 client.0 _ll_register_callbacks cb 0x55aed04ddcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 0x55aed05d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 0x55aed05d4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38140/0 (socket says 172.21.15.100:38140) 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5659c640 1 -- 172.21.15.100:0/115428545 learned_addr learned my addr 172.21.15.100:0/115428545 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05ce250 0x55aed05d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.706+0000 7f0e5759e640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05d7750 0x55aed05d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5659c640 1 -- 172.21.15.100:0/115428545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05ce250 msgr2=0x55aed05d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5659c640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05ce250 0x55aed05d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5659c640 1 -- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05d7750 msgr2=0x55aed05d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5659c640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05d7750 0x55aed05d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5659c640 1 -- 172.21.15.100:0/115428545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5810e810 con 0x55aed05ceb70 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5659c640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 0x55aed05d4c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0970f0 tx=0x7f0e4c069b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6667c33ab314b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:24.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e3aff5640 1 -- 172.21.15.100:0/115428545 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c083ea0 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e3aff5640 1 -- 172.21.15.100:0/115428545 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0e4c005670 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aed02357e0 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e3aff5640 1 -- 172.21.15.100:0/115428545 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e4c09e870 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 10 client.18318 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aed05346a0 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e3aff5640 1 -- 172.21.15.100:0/115428545 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e4c069020 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e3aff5640 1 -- 172.21.15.100:0/115428545 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0e4c01f060 con 0x55aed05ceb70 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 10 client.18318 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 20 client.18318 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed05e7b90 0x55aed05e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 4 client.18318 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.707+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aed0272720 con 0x55aed05e7b90 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.708+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed05e7b90 0x55aed05e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.708+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed05e7b90 0x55aed05e9fb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19017ac27c3f5f06 server_cookie=1bbe2987d438f76c in_seq=0 out_seq=0 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.708+0000 7f0e3aff5640 10 client.18318 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e3aff5640 1 -- 172.21.15.100:0/115428545 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aed0272720 con 0x55aed05e7b90 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e3aff5640 10 client.18318 handle_command_reply: tid=0 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 client.18318 shutdown 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 msgr2=0x55aed05d4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 0x55aed05d4c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0970f0 tx=0x7f0e4c069b00 comp rx=0 tx=0).stop 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 shutdown_connections 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed05e7b90 0x55aed05e9fb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05d7750 0x55aed05d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e3aff5640 0 client.18318 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ceb70 0x55aed05d4c60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/115428545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05ce250 0x55aed05d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 >> 172.21.15.100:0/115428545 conn(0x55aed05c8980 msgr2=0x55aed05ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 shutdown_connections 2024-03-12T05:25:24.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/115428545 wait complete. 2024-03-12T05:25:24.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.709+0000 7f0e5fd9e740 20 client.18318 trim_cache size 0 max 16384 2024-03-12T05:25:24.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 Processor -- start 2024-03-12T05:25:24.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 -- start start 2024-03-12T05:25:24.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed04bb130 con 0x55aed05ce250 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed05c4080 con 0x55aed05ceb70 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.721+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed05c42b0 con 0x55aed05d7750 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05d6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38156/0 (socket says 172.21.15.100:38156) 2024-03-12T05:25:24.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 -- 172.21.15.100:0/814273564 learned_addr learned my addr 172.21.15.100:0/814273564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e5759e640 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e5659c640 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 -- 172.21.15.100:0/814273564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 msgr2=0x55aed05d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 -- 172.21.15.100:0/814273564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 msgr2=0x55aed05d4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 -- 172.21.15.100:0/814273564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed04d0c20 con 0x55aed05ce250 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e5659c640 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05d6e50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009430 tx=0x7f0e48041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bd9ffdb1b14c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e3cff9640 1 -- 172.21.15.100:0/814273564 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48056070 con 0x55aed05ce250 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e3cff9640 1 -- 172.21.15.100:0/814273564 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e48034790 con 0x55aed05ce250 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.722+0000 7f0e3cff9640 1 -- 172.21.15.100:0/814273564 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48032910 con 0x55aed05ce250 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/814273564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 msgr2=0x55aed05d6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05d6e50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009430 tx=0x7f0e48041040 comp rx=0 tx=0).stop 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/814273564 shutdown_connections 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed05d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/814273564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05d6e50 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/814273564 >> 172.21.15.100:0/814273564 conn(0x55aed05c8980 msgr2=0x55aed05ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/814273564 shutdown_connections 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/814273564 wait complete. 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 -- start start 2024-03-12T05:25:24.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 Processor -- start 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.723+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5fd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed063e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed063e600 con 0x55aed05ce250 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed063e740 con 0x55aed05ceb70 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5fd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed063e880 con 0x55aed05d7750 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5fd9e740 10 client.0 _ll_register_callbacks cb 0x7f0e50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed063e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05ddcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37740/0 (socket says 172.21.15.100:37740) 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5659c640 1 -- 172.21.15.100:0/220425244 learned_addr learned my addr 172.21.15.100:0/220425244 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 -- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 msgr2=0x55aed05ddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05ddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 -- 172.21.15.100:0/220425244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 msgr2=0x55aed05dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 -- 172.21.15.100:0/220425244 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aed04f8060 con 0x55aed05d7750 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e5759e640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed063e0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005f60 tx=0x7f0e50004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed10ef763175b4e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.724+0000 7f0e3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e3aff5640 1 -- 172.21.15.100:0/220425244 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50025040 con 0x55aed05d7750 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e480345a0 con 0x55aed05d7750 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e3aff5640 1 -- 172.21.15.100:0/220425244 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0e50003cc0 con 0x55aed05d7750 2024-03-12T05:25:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e3aff5640 1 -- 172.21.15.100:0/220425244 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e50003e80 con 0x55aed05d7750 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 10 client.27637 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aed04f68c0 con 0x55aed05d7750 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e3aff5640 1 -- 172.21.15.100:0/220425244 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e480345a0 con 0x55aed05d7750 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e3aff5640 1 -- 172.21.15.100:0/220425244 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0e50028030 con 0x55aed05d7750 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 10 client.27637 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 10 client.27637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 20 client.27637 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed06400b0 0x55aed06424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 4 client.27637 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.725+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aed02c1890 con 0x55aed06400b0 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.726+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed06400b0 0x55aed06424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.726+0000 7f0e3aff5640 10 client.27637 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.726+0000 7f0e56d9d640 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed06400b0 0x55aed06424d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b026e8dad470cd1 server_cookie=4a29f770e9c1ac56 in_seq=0 out_seq=0 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e3aff5640 1 -- 172.21.15.100:0/220425244 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aed02c1890 con 0x55aed06400b0 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e3aff5640 10 client.27637 handle_command_reply: tid=0 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 client.27637 shutdown 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 msgr2=0x55aed063e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed063e0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005f60 tx=0x7f0e50004730 comp rx=0 tx=0).stop 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 shutdown_connections 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55aed06400b0 0x55aed06424d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed05d7750 0x55aed063e0c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aed05ceb70 0x55aed05ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e3aff5640 0 client.27637 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 --2- 172.21.15.100:0/220425244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aed05ce250 0x55aed05dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 >> 172.21.15.100:0/220425244 conn(0x55aed05c8980 msgr2=0x55aed05c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 shutdown_connections 2024-03-12T05:25:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 1 -- 172.21.15.100:0/220425244 wait complete. 2024-03-12T05:25:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5fd9e740 20 client.27637 trim_cache size 0 max 16384 2024-03-12T05:25:24.729 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e400a1c20 msgr2=0x7f0e400a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e400a1c20 0x7f0e400a40e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e50011980 tx=0x7f0e50005f90 comp rx=0 tx=0).stop 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 msgr2=0x7f0e5819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.727+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 0x7f0e5819d980 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e9f0 tx=0x7f0e4c00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 shutdown_connections 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0e400a1c20 0x7f0e400a40e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e581975f0 0x7f0e58197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e58106430 0x7f0e5819d980 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 --2- 172.21.15.100:0/3312474353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e580685d0 0x7f0e5819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 >> 172.21.15.100:0/3312474353 conn(0x7f0e58077180 msgr2=0x7f0e580fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 shutdown_connections 2024-03-12T05:25:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:24.728+0000 7f0e5d1c9640 1 -- 172.21.15.100:0/3312474353 wait complete. 2024-03-12T05:25:25.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5130 55.94 MB/sec warmup 8 sec latency 292.257 ms 2024-03-12T05:25:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:24 smithi156 ceph-mon[36270]: pgmap v416: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2024-03-12T05:25:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:24 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2424246324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:25:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:24 smithi100 ceph-mon[32210]: pgmap v416: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2024-03-12T05:25:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:24 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2424246324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:25:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:24 smithi161 ceph-mon[36043]: pgmap v416: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2024-03-12T05:25:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:24 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2424246324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:25:25.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:25:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:25:25.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 -- 172.21.15.100:0/3540681152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101d30 msgr2=0x7f18b0105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3540681152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101d30 0x7f18b0105d70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009930 tx=0x7f18a402bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 -- 172.21.15.100:0/3540681152 shutdown_connections 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3540681152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b0106450 0x7f18b010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3540681152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101d30 0x7f18b0105d70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3540681152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b0101380 0x7f18b0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 -- 172.21.15.100:0/3540681152 >> 172.21.15.100:0/3540681152 conn(0x7f18b0078580 msgr2=0x7f18b0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:25.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 -- 172.21.15.100:0/3540681152 shutdown_connections 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.853+0000 7f18b4e40640 1 -- 172.21.15.100:0/3540681152 wait complete. 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 Processor -- start 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 -- start start 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 0x7f18b019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b0106450 0x7f18b019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b01998e0 0x7f18b0199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019a2d0 con 0x7f18b0101380 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019a440 con 0x7f18b0106450 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.854+0000 7f18b4e40640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019a600 con 0x7f18b01998e0 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 0x7f18b019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b01998e0 0x7f18b0199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 0x7f18b019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38172/0 (socket says 172.21.15.100:38172) 2024-03-12T05:25:25.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18ae59c640 1 -- 172.21.15.100:0/3392116958 learned_addr learned my addr 172.21.15.100:0/3392116958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18add9b640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b0106450 0x7f18b019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 -- 172.21.15.100:0/3392116958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b0106450 msgr2=0x7f18b019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b0106450 0x7f18b019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 -- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 msgr2=0x7f18b019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 0x7f18b019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 -- 172.21.15.100:0/3392116958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009840 con 0x7f18b01998e0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18ae59c640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 0x7f18b019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18aed9d640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b01998e0 0x7f18b0199d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f18a800ec50 tx=0x7f18a800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f189f7fe640 1 -- 172.21.15.100:0/3392116958 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8003b70 con 0x7f18b01998e0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a4009560 con 0x7f18b01998e0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f189f7fe640 1 -- 172.21.15.100:0/3392116958 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a800ce70 con 0x7f18b01998e0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f189f7fe640 1 -- 172.21.15.100:0/3392116958 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8010830 con 0x7f18b01998e0 2024-03-12T05:25:25.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.855+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b01a6940 con 0x7f18b01998e0 2024-03-12T05:25:25.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f189f7fe640 1 -- 172.21.15.100:0/3392116958 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f18a8004020 con 0x7f18b01998e0 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 Processor -- start 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 -- start start 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f189f7fe640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f18980a1ba0 0x7f18980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91200b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c9120142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 0x55c912016d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911f79f90 con 0x55c91200bd30 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911f774c0 con 0x55c91200b380 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.858+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911f72ca0 con 0x55c9120148b0 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f189f7fe640 1 -- 172.21.15.100:0/3392116958 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f18a8014070 con 0x7f18b01998e0 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18add9b640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f18980a1ba0 0x7f18980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91200b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 0x55c912016d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18add9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c9120142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 0x55c912016d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41508/0 (socket says 172.21.15.100:41508) 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 -- 172.21.15.100:0/174819942 learned_addr learned my addr 172.21.15.100:0/174819942 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 -- 172.21.15.100:0/174819942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 msgr2=0x55c91200b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91200b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 -- 172.21.15.100:0/174819942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 msgr2=0x55c9120142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c9120142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 -- 172.21.15.100:0/174819942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c911f350b0 con 0x55c9120148b0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18aed9d640 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 0x55c912016d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f18a8097470 tx=0x7f18a809a930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdace90fcf415e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f18add9b640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f18980a1ba0 0x7f18980a4060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000c00 tx=0x7f18a4004750 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f189cff9640 1 -- 172.21.15.100:0/174819942 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8003e60 con 0x55c9120148b0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f189cff9640 1 -- 172.21.15.100:0/174819942 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a809b1a0 con 0x55c9120148b0 2024-03-12T05:25:25.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.859+0000 7f189cff9640 1 -- 172.21.15.100:0/174819942 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80042d0 con 0x55c9120148b0 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 -- 172.21.15.100:0/174819942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 msgr2=0x55c912016d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 0x55c912016d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f18a8097470 tx=0x7f18a809a930 comp rx=0 tx=0).stop 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 -- 172.21.15.100:0/174819942 shutdown_connections 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9120148b0 0x55c912016d00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c9120142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/174819942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91200b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 -- 172.21.15.100:0/174819942 >> 172.21.15.100:0/174819942 conn(0x55c912005840 msgr2=0x55c912005c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 -- 172.21.15.100:0/174819942 shutdown_connections 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.860+0000 7f18b7a0d740 1 -- 172.21.15.100:0/174819942 wait complete. 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 -- start start 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 Processor -- start 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200b380 0x55c9120117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c912011d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9120148b0 0x55c912012270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911f17f40 con 0x55c91200bd30 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911c464b0 con 0x55c9120148b0 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911ef8670 con 0x55c91200b380 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18b7a0d740 10 client.0 _ll_register_callbacks cb 0x55c911f65f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c912011d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200b380 0x55c9120117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c912011d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38192/0 (socket says 172.21.15.100:38192) 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200b380 0x55c9120117f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41522/0 (socket says 172.21.15.100:41522) 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 -- 172.21.15.100:0/3963012924 learned_addr learned my addr 172.21.15.100:0/3963012924 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18aed9d640 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9120148b0 0x55c912012270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 -- 172.21.15.100:0/3963012924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200b380 msgr2=0x55c9120117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200b380 0x55c9120117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 -- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9120148b0 msgr2=0x55c912012270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9120148b0 0x55c912012270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.861+0000 7f18add9b640 1 -- 172.21.15.100:0/3963012924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b01a6940 con 0x55c91200bd30 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f18add9b640 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c912011d30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f18a4004aa0 tx=0x7f18a4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298a8508e41ebd83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f1895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f1895ffb640 1 -- 172.21.15.100:0/3963012924 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4048070 con 0x55c91200bd30 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f1895ffb640 1 -- 172.21.15.100:0/3963012924 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f18a4004df0 con 0x55c91200bd30 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c911f4cf40 con 0x55c91200bd30 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f1895ffb640 1 -- 172.21.15.100:0/3963012924 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a4004fb0 con 0x55c91200bd30 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f18b7a0d740 10 client.18333 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c911f71320 con 0x55c91200bd30 2024-03-12T05:25:25.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.862+0000 7f1895ffb640 1 -- 172.21.15.100:0/3963012924 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c911f4cf40 con 0x55c91200bd30 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f1895ffb640 1 -- 172.21.15.100:0/3963012924 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f18a4003c40 con 0x55c91200bd30 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18b7a0d740 10 client.18333 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18b7a0d740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18b7a0d740 20 client.18333 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c912024e90 0x55c9120272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18b7a0d740 4 client.18333 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c911f69310 con 0x55c912024e90 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18ae59c640 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c912024e90 0x55c9120272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f1895ffb640 10 client.18333 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:25.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.863+0000 7f18ae59c640 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c912024e90 0x55c9120272b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb34a1d80c352c0 server_cookie=cb12ea30e15225f5 in_seq=0 out_seq=0 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f1895ffb640 1 -- 172.21.15.100:0/3963012924 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c911f69310 con 0x55c912024e90 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f1895ffb640 10 client.18333 handle_command_reply: tid=0 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f18b7a0d740 1 client.18333 shutdown 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 msgr2=0x55c912011d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c912011d30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f18a4004aa0 tx=0x7f18a4005fd0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 shutdown_connections 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.864+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c912024e90 0x55c9120272b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9120148b0 0x55c912012270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f1895ffb640 0 client.18333 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200bd30 0x55c912011d30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/3963012924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200b380 0x55c9120117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 >> 172.21.15.100:0/3963012924 conn(0x55c912005840 msgr2=0x55c9120077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:25.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 shutdown_connections 2024-03-12T05:25:25.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 1 -- 172.21.15.100:0/3963012924 wait complete. 2024-03-12T05:25:25.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.865+0000 7f18b7a0d740 20 client.18333 trim_cache size 0 max 16384 2024-03-12T05:25:25.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 Processor -- start 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 -- start start 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c912005840 0x55c912013f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 0x55c912012090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200bd30 0x55c91200ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9120053a0 con 0x55c91200b380 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c911fffda0 con 0x55c91200bd30 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91202c4c0 con 0x55c912005840 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18add9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 0x55c912012090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18add9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 0x55c912012090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38200/0 (socket says 172.21.15.100:38200) 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18add9b640 1 -- 172.21.15.100:0/2223257109 learned_addr learned my addr 172.21.15.100:0/2223257109 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:25.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18ae59c640 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c912005840 0x55c912013f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.882+0000 7f18aed9d640 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200bd30 0x55c91200ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f18add9b640 1 -- 172.21.15.100:0/2223257109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c912005840 msgr2=0x55c912013f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f18add9b640 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c912005840 0x55c912013f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f18add9b640 1 -- 172.21.15.100:0/2223257109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200bd30 msgr2=0x55c91200ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f18add9b640 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200bd30 0x55c91200ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f18add9b640 1 -- 172.21.15.100:0/2223257109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c911f0dc00 con 0x55c91200b380 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f18add9b640 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 0x55c912012090 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f18a402c780 tx=0x7f18a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d6fa6650484744 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f1897fff640 1 -- 172.21.15.100:0/2223257109 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4048070 con 0x55c91200b380 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f1897fff640 1 -- 172.21.15.100:0/2223257109 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a40048e0 con 0x55c91200b380 2024-03-12T05:25:25.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.883+0000 7f1897fff640 1 -- 172.21.15.100:0/2223257109 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4050ab0 con 0x55c91200b380 2024-03-12T05:25:25.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- 172.21.15.100:0/2223257109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 msgr2=0x55c912012090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 0x55c912012090 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f18a402c780 tx=0x7f18a4009780 comp rx=0 tx=0).stop 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- 172.21.15.100:0/2223257109 shutdown_connections 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200bd30 0x55c91200ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91200b380 0x55c912012090 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/2223257109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c912005840 0x55c912013f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- 172.21.15.100:0/2223257109 >> 172.21.15.100:0/2223257109 conn(0x55c91200c480 msgr2=0x55c9120100c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- 172.21.15.100:0/2223257109 shutdown_connections 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- 172.21.15.100:0/2223257109 wait complete. 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- start start 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 Processor -- start 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 0x55c91201a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91201a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200bd30 0x55c91201ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91207b390 con 0x55c912005840 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91207b500 con 0x55c91200b380 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.884+0000 7f18b7a0d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91207b6c0 con 0x55c91200bd30 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18b7a0d740 10 client.0 _ll_register_callbacks cb 0x7f18a8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 0x55c91201a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 0x55c91201a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38210/0 (socket says 172.21.15.100:38210) 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 -- 172.21.15.100:0/1266733177 learned_addr learned my addr 172.21.15.100:0/1266733177 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18add9b640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91201a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18aed9d640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200bd30 0x55c91201ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 -- 172.21.15.100:0/1266733177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200bd30 msgr2=0x55c91201ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200bd30 0x55c91201ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 -- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 msgr2=0x55c91201a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91201a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 -- 172.21.15.100:0/1266733177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c911f71320 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18ae59c640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 0x55c91201a3d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000c00 tx=0x7f18a0013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d4acd82ded7ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f1895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f1895ffb640 1 -- 172.21.15.100:0/1266733177 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0017070 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f1895ffb640 1 -- 172.21.15.100:0/1266733177 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f18a0003b20 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c911f98900 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f1895ffb640 1 -- 172.21.15.100:0/1266733177 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a00122d0 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18b7a0d740 10 client.18339 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c911f33910 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.885+0000 7f1895ffb640 1 -- 172.21.15.100:0/1266733177 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c911f98900 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f1895ffb640 1 -- 172.21.15.100:0/1266733177 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f18a0010430 con 0x55c912005840 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18b7a0d740 10 client.18339 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18b7a0d740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18b7a0d740 20 client.18339 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c9120148b0 0x55c91207efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18b7a0d740 4 client.18339 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c911c72860 con 0x55c9120148b0 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18add9b640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c9120148b0 0x55c91207efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f1895ffb640 10 client.18339 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.886+0000 7f18add9b640 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c9120148b0 0x55c91207efa0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9842adae23b5e0 server_cookie=bd8e7634b1ae19f3 in_seq=0 out_seq=0 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f1895ffb640 1 -- 172.21.15.100:0/1266733177 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c911c72860 con 0x55c9120148b0 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f1895ffb640 10 client.18339 handle_command_reply: tid=0 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 client.18339 shutdown 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 msgr2=0x55c91201a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 0x55c91201a3d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000c00 tx=0x7f18a0013c70 comp rx=0 tx=0).stop 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 shutdown_connections 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c9120148b0 0x55c91207efa0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91200bd30 0x55c91201ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91200b380 0x55c91201a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f1895ffb640 0 client.18339 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 --2- 172.21.15.100:0/1266733177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c912005840 0x55c91201a3d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 >> 172.21.15.100:0/1266733177 conn(0x55c91200c480 msgr2=0x55c912006e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 shutdown_connections 2024-03-12T05:25:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 1 -- 172.21.15.100:0/1266733177 wait complete. 2024-03-12T05:25:25.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b7a0d740 20 client.18339 trim_cache size 0 max 16384 2024-03-12T05:25:25.889 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:25:25.889 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:25:25.889 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c220d4e9-f5ff-4fe3-bb5b-9800f7c4556b", 2024-03-12T05:25:25.889 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:25:25.889 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:25:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f18980a1ba0 msgr2=0x7f18980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f18980a1ba0 0x7f18980a4060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000c00 tx=0x7f18a4004750 comp rx=0 tx=0).stop 2024-03-12T05:25:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b01998e0 msgr2=0x7f18b0199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b01998e0 0x7f18b0199d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f18a800ec50 tx=0x7f18a800c6a0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 shutdown_connections 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f18980a1ba0 0x7f18980a4060 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b01998e0 0x7f18b0199d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b0106450 0x7f18b019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 --2- 172.21.15.100:0/3392116958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0101380 0x7f18b019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.888+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 >> 172.21.15.100:0/3392116958 conn(0x7f18b0078580 msgr2=0x7f18b00ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.889+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 shutdown_connections 2024-03-12T05:25:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:25.889+0000 7f18b4e40640 1 -- 172.21.15.100:0/3392116958 wait complete. 2024-03-12T05:25:25.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:25:25.894 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:25:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 -- 172.21.15.100:0/2352797891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc105340 msgr2=0x7fc1cc10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2352797891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc105340 0x7fc1cc10cf10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009cd0 tx=0x7fc1c402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 -- 172.21.15.100:0/2352797891 shutdown_connections 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2352797891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc105340 0x7fc1cc10cf10 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2352797891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100c20 0x7fc1cc104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2352797891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc100270 0x7fc1cc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 -- 172.21.15.100:0/2352797891 >> 172.21.15.100:0/2352797891 conn(0x7fc1cc077230 msgr2=0x7fc1cc077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 -- 172.21.15.100:0/2352797891 shutdown_connections 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.015+0000 7fc1d2281640 1 -- 172.21.15.100:0/2352797891 wait complete. 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 Processor -- start 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 -- start start 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100270 0x7fc1cc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc105340 0x7fc1cc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc199060 0x7fc1cc1a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1a3ce0 con 0x7fc1cc199060 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1a3e50 con 0x7fc1cc105340 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d2281640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1a4010 con 0x7fc1cc100270 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100270 0x7fc1cc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc105340 0x7fc1cc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc199060 0x7fc1cc1a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100270 0x7fc1cc198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41562/0 (socket says 172.21.15.100:41562) 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cbfff640 1 -- 172.21.15.100:0/2624297404 learned_addr learned my addr 172.21.15.100:0/2624297404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/2624297404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100270 msgr2=0x7fc1cc198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100270 0x7fc1cc198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc199060 msgr2=0x7fc1cc1a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc199060 0x7fc1cc1a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/2624297404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4009900 con 0x7fc1cc105340 2024-03-12T05:25:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.016+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc199060 0x7fc1cc1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.017+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc105340 0x7fc1cc198ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000e9c0 tx=0x7fc1c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.017+0000 7fc1c97fa640 1 -- 172.21.15.100:0/2624297404 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000ccc0 con 0x7fc1cc105340 2024-03-12T05:25:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.017+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc1a42f0 con 0x7fc1cc105340 2024-03-12T05:25:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.017+0000 7fc1c97fa640 1 -- 172.21.15.100:0/2624297404 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1c000ce20 con 0x7fc1cc105340 2024-03-12T05:25:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.017+0000 7fc1c97fa640 1 -- 172.21.15.100:0/2624297404 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0004a00 con 0x7fc1cc105340 2024-03-12T05:25:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.017+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc1a4800 con 0x7fc1cc105340 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.019+0000 7fc1c97fa640 1 -- 172.21.15.100:0/2624297404 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc1c0005020 con 0x7fc1cc105340 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.019+0000 7fc1d4e54740 1 Processor -- start 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.019+0000 7fc1d4e54740 1 -- start start 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd92690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1c97fa640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1b40a1ba0 0x7fc1b40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd92bd0 0x55b11bd9b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bd01280 con 0x55b11bd92bd0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bcfe7b0 con 0x55b11bd922b0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bcf9f90 con 0x55b11bd9b7b0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd92bd0 0x55b11bd9b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd92690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd92690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37820/0 (socket says 172.21.15.100:37820) 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 -- 172.21.15.100:0/2046074487 learned_addr learned my addr 172.21.15.100:0/2046074487 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd92bd0 0x55b11bd9b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38230/0 (socket says 172.21.15.100:38230) 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1b40a1ba0 0x7fc1b40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1c97fa640 1 -- 172.21.15.100:0/2624297404 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1c0014070 con 0x7fc1cc105340 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 -- 172.21.15.100:0/2046074487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd9b7b0 msgr2=0x55b11bd9dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 -- 172.21.15.100:0/2046074487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd92bd0 msgr2=0x55b11bd9b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd92bd0 0x55b11bd9b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 -- 172.21.15.100:0/2046074487 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b11bcb8570 con 0x55b11bd922b0 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.020+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd92690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009430 tx=0x7fc1c4009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e32e3672519a6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1b40a1ba0 0x7fc1b40a4060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009c90 tx=0x7fc1bc005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1b27fc640 1 -- 172.21.15.100:0/2046074487 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4034c90 con 0x55b11bd922b0 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1b27fc640 1 -- 172.21.15.100:0/2046074487 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1c4034df0 con 0x55b11bd922b0 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1b27fc640 1 -- 172.21.15.100:0/2046074487 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4004290 con 0x55b11bd922b0 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2046074487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 msgr2=0x55b11bd92690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd92690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009430 tx=0x7fc1c4009360 comp rx=0 tx=0).stop 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2046074487 shutdown_connections 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd92bd0 0x55b11bd9b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2046074487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd92690 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.021+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2046074487 >> 172.21.15.100:0/2046074487 conn(0x55b11bd8c990 msgr2=0x55b11bd8cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2046074487 shutdown_connections 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2046074487 wait complete. 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 -- start start 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 Processor -- start 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd9b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bd985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd98d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11b9cd430 con 0x55b11bd9b7b0 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11b989e00 con 0x55b11bd922b0 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.022+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bc7efd0 con 0x55b11bd92bd0 2024-03-12T05:25:26.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1d4e54740 10 client.0 _ll_register_callbacks cb 0x55b11bcd4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bd985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bd985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41586/0 (socket says 172.21.15.100:41586) 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/253557124 learned_addr learned my addr 172.21.15.100:0/253557124 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1d081e640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd98d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd9b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 msgr2=0x55b11bd9b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd9b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/253557124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 msgr2=0x55b11bd98d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd98d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 -- 172.21.15.100:0/253557124 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc10d720 con 0x55b11bd92bd0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1d081e640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd98d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bd985f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00cb480 tx=0x7fc1c006a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42abce25304f4e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:25:26.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/253557124 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0098b10 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/253557124 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1c009aa60 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.023+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b11b9f97e0 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/253557124 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c0098920 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 10 client.27667 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b11bcf8610 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/253557124 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b11b9f97e0 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/253557124 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc1c0098e40 con 0x55b11bd92bd0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 10 client.27667 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 10 client.27667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 20 client.27667 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11bdab9d0 0x55b11bdaddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 4 client.27667 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b11ba36720 con 0x55b11bdab9d0 2024-03-12T05:25:26.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.024+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11bdab9d0 0x55b11bdaddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.025+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11bdab9d0 0x55b11bdaddf0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdcb9a428cceb6d6 server_cookie=82c117081a9c11fa in_seq=0 out_seq=0 2024-03-12T05:25:26.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.025+0000 7fc1ab7fe640 10 client.27667 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/253557124 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b11ba36720 con 0x55b11bdab9d0 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1ab7fe640 10 client.27667 handle_command_reply: tid=0 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 client.27667 shutdown 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 msgr2=0x55b11bd985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bd985f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00cb480 tx=0x7fc1c006a400 comp rx=0 tx=0).stop 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 shutdown_connections 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11bdab9d0 0x55b11bdaddf0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd98d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1ab7fe640 0 client.27667 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:26.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bd985f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/253557124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bd9b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 >> 172.21.15.100:0/253557124 conn(0x55b11bd8c990 msgr2=0x55b11bd8e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:26.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 shutdown_connections 2024-03-12T05:25:26.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 1 -- 172.21.15.100:0/253557124 wait complete. 2024-03-12T05:25:26.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.026+0000 7fc1d4e54740 20 client.27667 trim_cache size 0 max 16384 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 Processor -- start 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 -- start start 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd922b0 0x55b11bda2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd92bd0 0x55b11bda2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bdb2fe0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bd93410 con 0x55b11bd92bd0 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bd96b40 con 0x55b11bd922b0 2024-03-12T05:25:26.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38256/0 (socket says 172.21.15.100:38256) 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd92bd0 0x55b11bda2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1d081e640 1 -- 172.21.15.100:0/1806536387 learned_addr learned my addr 172.21.15.100:0/1806536387 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.045+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd922b0 0x55b11bda2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1d081e640 1 -- 172.21.15.100:0/1806536387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd922b0 msgr2=0x55b11bda2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1d081e640 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd922b0 0x55b11bda2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1d081e640 1 -- 172.21.15.100:0/1806536387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd92bd0 msgr2=0x55b11bda2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1d081e640 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd92bd0 0x55b11bda2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1d081e640 1 -- 172.21.15.100:0/1806536387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b11bc94ac0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1d081e640 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9ad20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4034fd0 tx=0x7fc1c4003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba265efef1b6bed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1b1ffb640 1 -- 172.21.15.100:0/1806536387 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003b70 con 0x55b11bd9b7b0 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1b1ffb640 1 -- 172.21.15.100:0/1806536387 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1c4004b40 con 0x55b11bd9b7b0 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.046+0000 7fc1b1ffb640 1 -- 172.21.15.100:0/1806536387 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4042840 con 0x55b11bd9b7b0 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 -- 172.21.15.100:0/1806536387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 msgr2=0x55b11bd9ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9ad20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4034fd0 tx=0x7fc1c4003d20 comp rx=0 tx=0).stop 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 -- 172.21.15.100:0/1806536387 shutdown_connections 2024-03-12T05:25:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bd9ad20 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd92bd0 0x55b11bda2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/1806536387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd922b0 0x55b11bda2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 -- 172.21.15.100:0/1806536387 >> 172.21.15.100:0/1806536387 conn(0x55b11bd8c990 msgr2=0x55b11bd93910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 -- 172.21.15.100:0/1806536387 shutdown_connections 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 -- 172.21.15.100:0/1806536387 wait complete. 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.047+0000 7fc1d4e54740 1 -- start start 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 Processor -- start 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bda0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bda1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bda19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bda2ec0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11be062f0 con 0x55b11bd922b0 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11be06460 con 0x55b11bd92bd0 2024-03-12T05:25:26.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d4e54740 10 client.0 _ll_register_callbacks cb 0x7fc1bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bda19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bda19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38272/0 (socket says 172.21.15.100:38272) 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 -- 172.21.15.100:0/2673564733 learned_addr learned my addr 172.21.15.100:0/2673564733 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bda0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1cb7fe640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bda1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 -- 172.21.15.100:0/2673564733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 msgr2=0x55b11bda1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bda1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 -- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 msgr2=0x55b11bda0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bda0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 -- 172.21.15.100:0/2673564733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c4048040 con 0x55b11bd9b7b0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1d081e640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bda19a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4044040 tx=0x7fc1c4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b72516d6ecd811 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.048+0000 7fc1ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:26.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/2673564733 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40328b0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/2673564733 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc1c4032a10 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1c4034df0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/2673564733 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c40424c0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 10 client.18357 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b11bcba6a0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/2673564733 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1c4034df0 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/2673564733 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc1c404e640 con 0x55b11bd9b7b0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 10 client.18357 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 20 client.18357 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11be077d0 0x55b11be09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 4 client.18357 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.049+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b11ba85890 con 0x55b11be077d0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11be077d0 0x55b11be09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1cbfff640 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11be077d0 0x55b11be09bc0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06d1f19f6785fbd server_cookie=bd2d34a35103b100 in_seq=0 out_seq=0 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1ab7fe640 10 client.18357 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1ab7fe640 1 -- 172.21.15.100:0/2673564733 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b11ba85890 con 0x55b11be077d0 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1ab7fe640 10 client.18357 handle_command_reply: tid=0 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1d4e54740 1 client.18357 shutdown 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 msgr2=0x55b11bda19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bda19a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4044040 tx=0x7fc1c4009360 comp rx=0 tx=0).stop 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.050+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 shutdown_connections 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b11be077d0 0x55b11be09bc0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b11bd9b7b0 0x55b11bda19a0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1ab7fe640 0 client.18357 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b11bd92bd0 0x55b11bda1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 --2- 172.21.15.100:0/2673564733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b11bd922b0 0x55b11bda0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 >> 172.21.15.100:0/2673564733 conn(0x55b11bd8c990 msgr2=0x55b11bd97e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 shutdown_connections 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 1 -- 172.21.15.100:0/2673564733 wait complete. 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d4e54740 20 client.18357 trim_cache size 0 max 16384 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stdout: "c220d4e9-f5ff-4fe3-bb5b-9800f7c4556b": { 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:25:26.051 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c220d4e9-f5ff-4fe3-bb5b-9800f7c4556b", 2024-03-12T05:25:26.052 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:25:26.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:25:26.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:25:26.052 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:25:26.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1b40a1ba0 msgr2=0x7fc1b40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1b40a1ba0 0x7fc1b40a4060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009c90 tx=0x7fc1bc005f90 comp rx=0 tx=0).stop 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc105340 msgr2=0x7fc1cc198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc105340 0x7fc1cc198ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000e9c0 tx=0x7fc1c000ee80 comp rx=0 tx=0).stop 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 shutdown_connections 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc1b40a1ba0 0x7fc1b40a4060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1cc199060 0x7fc1cc1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc105340 0x7fc1cc198ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.051+0000 7fc1d2281640 1 --2- 172.21.15.100:0/2624297404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1cc100270 0x7fc1cc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.052+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 >> 172.21.15.100:0/2624297404 conn(0x7fc1cc077230 msgr2=0x7fc1cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.052+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 shutdown_connections 2024-03-12T05:25:26.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:26.052+0000 7fc1d2281640 1 -- 172.21.15.100:0/2624297404 wait complete. 2024-03-12T05:25:26.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:26 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:25:26.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:26 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:26 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:26.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:26 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:26 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:26.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:26 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.058 INFO:tasks.cephfs.filesystem:scrub status for tag:c220d4e9-f5ff-4fe3-bb5b-9800f7c4556b - {'path': '/', 'tag': 'c220d4e9-f5ff-4fe3-bb5b-9800f7c4556b', 'options': 'recursive,force'} 2024-03-12T05:25:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5957 55.03 MB/sec warmup 9 sec latency 179.538 ms 2024-03-12T05:25:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:26 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:25:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:26 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:26 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:26 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:26 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:26 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:26 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:25:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:26 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:26 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:26 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:26 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:26 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 6709 52.40 MB/sec warmup 10 sec latency 204.755 ms 2024-03-12T05:25:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:27 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:27 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:27 smithi100 ceph-mon[32210]: pgmap v417: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.7 MiB/s wr, 324 op/s 2024-03-12T05:25:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:27 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:27 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:27 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:27 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:27 smithi156 ceph-mon[36270]: pgmap v417: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.7 MiB/s wr, 324 op/s 2024-03-12T05:25:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:27 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:27 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:27 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:27 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:27 smithi161 ceph-mon[36043]: pgmap v417: 97 pgs: 97 active+clean; 583 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.7 MiB/s wr, 324 op/s 2024-03-12T05:25:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:27 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:27.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:27 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7552 52.67 MB/sec warmup 11 sec latency 81.947 ms 2024-03-12T05:25:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:28 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:25:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:28 smithi100 ceph-mon[32210]: pgmap v418: 97 pgs: 97 active+clean; 605 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-03-12T05:25:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:28 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:25:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:28 smithi156 ceph-mon[36270]: pgmap v418: 97 pgs: 97 active+clean; 605 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-03-12T05:25:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:28 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:25:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:28 smithi161 ceph-mon[36043]: pgmap v418: 97 pgs: 97 active+clean; 605 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-03-12T05:25:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8469 53.32 MB/sec warmup 12 sec latency 78.938 ms 2024-03-12T05:25:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:29 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:25:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:29 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:25:29.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:29 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:25:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:29 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:25:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:29 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:25:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:29 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:25:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:29 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:25:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:29 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:25:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:29 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:25:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8906 51.18 MB/sec warmup 13 sec latency 129.245 ms 2024-03-12T05:25:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:30 smithi100 ceph-mon[32210]: pgmap v419: 97 pgs: 97 active+clean; 605 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 378 op/s 2024-03-12T05:25:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:30 smithi156 ceph-mon[36270]: pgmap v419: 97 pgs: 97 active+clean; 605 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 378 op/s 2024-03-12T05:25:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:30 smithi161 ceph-mon[36043]: pgmap v419: 97 pgs: 97 active+clean; 605 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 378 op/s 2024-03-12T05:25:31.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9466 48.79 MB/sec warmup 14 sec latency 104.033 ms 2024-03-12T05:25:32.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 10165 48.26 MB/sec warmup 15 sec latency 85.304 ms 2024-03-12T05:25:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:32 smithi161 ceph-mon[36043]: pgmap v420: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.9 MiB/s wr, 584 op/s 2024-03-12T05:25:33.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 11038 47.30 MB/sec warmup 16 sec latency 140.464 ms 2024-03-12T05:25:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:32 smithi100 ceph-mon[32210]: pgmap v420: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.9 MiB/s wr, 584 op/s 2024-03-12T05:25:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:32 smithi156 ceph-mon[36270]: pgmap v420: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.9 MiB/s wr, 584 op/s 2024-03-12T05:25:34.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12037 48.26 MB/sec warmup 17 sec latency 153.228 ms 2024-03-12T05:25:35.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12729 48.28 MB/sec warmup 18 sec latency 218.763 ms 2024-03-12T05:25:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:34 smithi100 ceph-mon[32210]: pgmap v421: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2024-03-12T05:25:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:34 smithi156 ceph-mon[36270]: pgmap v421: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2024-03-12T05:25:35.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:34 smithi161 ceph-mon[36043]: pgmap v421: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2024-03-12T05:25:36.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13243 46.78 MB/sec warmup 19 sec latency 143.275 ms 2024-03-12T05:25:37.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13916 45.99 MB/sec warmup 20 sec latency 150.415 ms 2024-03-12T05:25:37.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:36 smithi100 ceph-mon[32210]: pgmap v422: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2024-03-12T05:25:37.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:36 smithi156 ceph-mon[36270]: pgmap v422: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2024-03-12T05:25:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:36 smithi161 ceph-mon[36043]: pgmap v422: 97 pgs: 97 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2024-03-12T05:25:38.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 14642 46.18 MB/sec warmup 21 sec latency 66.072 ms 2024-03-12T05:25:39.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15405 45.27 MB/sec warmup 22 sec latency 84.306 ms 2024-03-12T05:25:39.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:38 smithi100 ceph-mon[32210]: pgmap v423: 97 pgs: 97 active+clean; 651 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-03-12T05:25:39.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:38 smithi156 ceph-mon[36270]: pgmap v423: 97 pgs: 97 active+clean; 651 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-03-12T05:25:39.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:38 smithi161 ceph-mon[36043]: pgmap v423: 97 pgs: 97 active+clean; 651 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-03-12T05:25:40.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16207 45.51 MB/sec warmup 23 sec latency 110.120 ms 2024-03-12T05:25:40.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:39 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:25:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:39 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:25:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:39 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:25:41.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16922 45.52 MB/sec warmup 24 sec latency 82.473 ms 2024-03-12T05:25:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:40 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:25:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:40 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:25:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:40 smithi100 ceph-mon[32210]: pgmap v424: 97 pgs: 97 active+clean; 651 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 388 op/s 2024-03-12T05:25:41.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:40 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:25:41.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:40 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:25:41.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:40 smithi161 ceph-mon[36043]: pgmap v424: 97 pgs: 97 active+clean; 651 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 388 op/s 2024-03-12T05:25:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:40 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:25:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:40 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:25:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:40 smithi156 ceph-mon[36270]: pgmap v424: 97 pgs: 97 active+clean; 651 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 388 op/s 2024-03-12T05:25:42.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17618 45.00 MB/sec warmup 25 sec latency 137.083 ms 2024-03-12T05:25:43.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18233 44.62 MB/sec warmup 26 sec latency 192.209 ms 2024-03-12T05:25:43.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:42 smithi100 ceph-mon[32210]: pgmap v425: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.7 MiB/s wr, 599 op/s 2024-03-12T05:25:43.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:42 smithi156 ceph-mon[36270]: pgmap v425: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.7 MiB/s wr, 599 op/s 2024-03-12T05:25:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:42 smithi161 ceph-mon[36043]: pgmap v425: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.7 MiB/s wr, 599 op/s 2024-03-12T05:25:44.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18792 44.08 MB/sec warmup 27 sec latency 192.800 ms 2024-03-12T05:25:45.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19554 43.49 MB/sec warmup 28 sec latency 313.122 ms 2024-03-12T05:25:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:44 smithi100 ceph-mon[32210]: pgmap v426: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2024-03-12T05:25:45.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:44 smithi156 ceph-mon[36270]: pgmap v426: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2024-03-12T05:25:45.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:44 smithi161 ceph-mon[36043]: pgmap v426: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2024-03-12T05:25:46.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 20343 43.72 MB/sec warmup 29 sec latency 205.990 ms 2024-03-12T05:25:47.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21099 43.83 MB/sec warmup 30 sec latency 173.773 ms 2024-03-12T05:25:47.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:46 smithi100 ceph-mon[32210]: pgmap v427: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2024-03-12T05:25:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:46 smithi156 ceph-mon[36270]: pgmap v427: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2024-03-12T05:25:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:46 smithi161 ceph-mon[36043]: pgmap v427: 97 pgs: 97 active+clean; 674 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2024-03-12T05:25:48.230 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21757 43.51 MB/sec warmup 31 sec latency 44.355 ms 2024-03-12T05:25:49.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22397 43.27 MB/sec warmup 32 sec latency 205.694 ms 2024-03-12T05:25:49.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:48 smithi100 ceph-mon[32210]: pgmap v428: 97 pgs: 97 active+clean; 696 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 598 op/s 2024-03-12T05:25:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:48 smithi156 ceph-mon[36270]: pgmap v428: 97 pgs: 97 active+clean; 696 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 598 op/s 2024-03-12T05:25:49.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:48 smithi161 ceph-mon[36043]: pgmap v428: 97 pgs: 97 active+clean; 696 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 598 op/s 2024-03-12T05:25:50.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22779 42.38 MB/sec warmup 33 sec latency 488.462 ms 2024-03-12T05:25:50.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:50 smithi100 ceph-mon[32210]: pgmap v429: 97 pgs: 97 active+clean; 696 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 416 op/s 2024-03-12T05:25:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:50 smithi156 ceph-mon[36270]: pgmap v429: 97 pgs: 97 active+clean; 696 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 416 op/s 2024-03-12T05:25:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:50 smithi161 ceph-mon[36043]: pgmap v429: 97 pgs: 97 active+clean; 696 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 416 op/s 2024-03-12T05:25:51.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23512 42.53 MB/sec warmup 34 sec latency 106.131 ms 2024-03-12T05:25:52.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24310 42.27 MB/sec warmup 35 sec latency 58.941 ms 2024-03-12T05:25:53.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24919 42.05 MB/sec warmup 36 sec latency 71.815 ms 2024-03-12T05:25:53.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:52 smithi100 ceph-mon[32210]: pgmap v430: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 23 MiB/s wr, 643 op/s 2024-03-12T05:25:53.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:52 smithi156 ceph-mon[36270]: pgmap v430: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 23 MiB/s wr, 643 op/s 2024-03-12T05:25:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:52 smithi161 ceph-mon[36043]: pgmap v430: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 23 MiB/s wr, 643 op/s 2024-03-12T05:25:54.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25809 42.37 MB/sec warmup 37 sec latency 85.026 ms 2024-03-12T05:25:55.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26202 41.80 MB/sec warmup 38 sec latency 247.624 ms 2024-03-12T05:25:55.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:54 smithi156 ceph-mon[36270]: pgmap v431: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 432 op/s 2024-03-12T05:25:55.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:54 smithi100 ceph-mon[32210]: pgmap v431: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 432 op/s 2024-03-12T05:25:55.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:54 smithi161 ceph-mon[36043]: pgmap v431: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 432 op/s 2024-03-12T05:25:56.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:25:56.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 -- 172.21.15.100:0/1754695220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c100c20 msgr2=0x7f967c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 --2- 172.21.15.100:0/1754695220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c100c20 0x7f967c104c60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9664009930 tx=0x7f966402bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 -- 172.21.15.100:0/1754695220 shutdown_connections 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 --2- 172.21.15.100:0/1754695220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c105340 0x7f967c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 --2- 172.21.15.100:0/1754695220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c100c20 0x7f967c104c60 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 --2- 172.21.15.100:0/1754695220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c100270 0x7f967c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 -- 172.21.15.100:0/1754695220 >> 172.21.15.100:0/1754695220 conn(0x7f967c077230 msgr2=0x7f967c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 -- 172.21.15.100:0/1754695220 shutdown_connections 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.199+0000 7f9680e88640 1 -- 172.21.15.100:0/1754695220 wait complete. 2024-03-12T05:25:56.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 Processor -- start 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 -- start start 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 0x7f967c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 0x7f967c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c1997b0 0x7f967c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19a260 con 0x7f967c105340 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19a3d0 con 0x7f967c1997b0 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9680e88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19a540 con 0x7f967c100270 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9673fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 0x7f967c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 0x7f967c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 0x7f967c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58036/0 (socket says 172.21.15.100:58036) 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.200+0000 7f9673fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 0x7f967c19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51886/0 (socket says 172.21.15.100:51886) 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f967a59c640 1 -- 172.21.15.100:0/1183207805 learned_addr learned my addr 172.21.15.100:0/1183207805 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c1997b0 0x7f967c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9673fff640 1 -- 172.21.15.100:0/1183207805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 msgr2=0x7f967c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9673fff640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 0x7f967c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9673fff640 1 -- 172.21.15.100:0/1183207805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c1997b0 msgr2=0x7f967c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9673fff640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c1997b0 0x7f967c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9673fff640 1 -- 172.21.15.100:0/1183207805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664009560 con 0x7f967c105340 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f967a59c640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 0x7f967c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9673fff640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 0x7f967c19fb80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f966402c170 tx=0x7f966402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f96737fe640 1 -- 172.21.15.100:0/1183207805 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96640042d0 con 0x7f967c105340 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f96737fe640 1 -- 172.21.15.100:0/1183207805 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9664004430 con 0x7f967c105340 2024-03-12T05:25:56.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c1a6490 con 0x7f967c105340 2024-03-12T05:25:56.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f96737fe640 1 -- 172.21.15.100:0/1183207805 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664003c90 con 0x7f967c105340 2024-03-12T05:25:56.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.201+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c1a6920 con 0x7f967c105340 2024-03-12T05:25:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.203+0000 7f96737fe640 1 -- 172.21.15.100:0/1183207805 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9664004df0 con 0x7f967c105340 2024-03-12T05:25:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 Processor -- start 2024-03-12T05:25:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 -- start start 2024-03-12T05:25:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f96737fe640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f96680a1ba0 0x7f96680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26ddbd0 0x55daa26e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f96680a1ba0 0x7f96680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26e67b0 0x55daa26e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa264c280 con 0x55daa26ddbd0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa26497b0 con 0x55daa26e67b0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9683a69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2644f90 con 0x55daa26dd2b0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f96737fe640 1 -- 172.21.15.100:0/1183207805 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f96640ef6a0 con 0x7f967c105340 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26dd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58052/0 (socket says 172.21.15.100:58052) 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f967a59c640 1 -- 172.21.15.100:0/2806644478 learned_addr learned my addr 172.21.15.100:0/2806644478 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f967ad9d640 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26e67b0 0x55daa26e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.204+0000 7f9673fff640 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26ddbd0 0x55daa26e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967a59c640 1 -- 172.21.15.100:0/2806644478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26e67b0 msgr2=0x55daa26e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967a59c640 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26e67b0 0x55daa26e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967a59c640 1 -- 172.21.15.100:0/2806644478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26ddbd0 msgr2=0x55daa26e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967a59c640 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26ddbd0 0x55daa26e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967a59c640 1 -- 172.21.15.100:0/2806644478 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daa2603570 con 0x55daa26dd2b0 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f96680a1ba0 0x7f96680a4060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9674009fd0 tx=0x7f9674009d80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f967a59c640 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26dd690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f966c011650 tx=0x7f966c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdce97ef957c988 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f9670ff9640 1 -- 172.21.15.100:0/2806644478 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c00f6c0 con 0x55daa26dd2b0 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f9670ff9640 1 -- 172.21.15.100:0/2806644478 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f966c00fd00 con 0x55daa26dd2b0 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.205+0000 7f9670ff9640 1 -- 172.21.15.100:0/2806644478 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c019db0 con 0x55daa26dd2b0 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 -- 172.21.15.100:0/2806644478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 msgr2=0x55daa26dd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26dd690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f966c011650 tx=0x7f966c009f90 comp rx=0 tx=0).stop 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 -- 172.21.15.100:0/2806644478 shutdown_connections 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26e67b0 0x55daa26e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26ddbd0 0x55daa26e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 --2- 172.21.15.100:0/2806644478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26dd690 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 -- 172.21.15.100:0/2806644478 >> 172.21.15.100:0/2806644478 conn(0x55daa26d7990 msgr2=0x55daa26d7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 -- 172.21.15.100:0/2806644478 shutdown_connections 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 -- 172.21.15.100:0/2806644478 wait complete. 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 -- start start 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.206+0000 7f9683a69740 1 Processor -- start 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa2318430 con 0x55daa26e67b0 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa22d4e00 con 0x55daa26ddbd0 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa25c9fd0 con 0x55daa26dd2b0 2024-03-12T05:25:56.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9683a69740 10 client.0 _ll_register_callbacks cb 0x55daa261ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9673fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51906/0 (socket says 172.21.15.100:51906) 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58066/0 (socket says 172.21.15.100:58066) 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 -- 172.21.15.100:0/1178910339 learned_addr learned my addr 172.21.15.100:0/1178910339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 -- 172.21.15.100:0/1178910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 msgr2=0x55daa26e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 -- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 msgr2=0x55daa26e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 -- 172.21.15.100:0/1178910339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c009e00 con 0x55daa26e67b0 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26e3cc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f967400fdf0 tx=0x7f967401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86f30ef5631d02b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.207+0000 7f9661ffb640 1 -- 172.21.15.100:0/1178910339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674013120 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9661ffb640 1 -- 172.21.15.100:0/1178910339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9674025030 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daa23447e0 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9661ffb640 1 -- 172.21.15.100:0/1178910339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9674013280 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 10 client.18369 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55daa2643610 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9661ffb640 1 -- 172.21.15.100:0/1178910339 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55daa23447e0 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9661ffb640 1 -- 172.21.15.100:0/1178910339 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9674030070 con 0x55daa26e67b0 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 10 client.18369 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 10 client.18369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 20 client.18369 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa26f6950 0x55daa26f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 4 client.18369 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:56.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daa2381720 con 0x55daa26f6950 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.208+0000 7f967a59c640 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa26f6950 0x55daa26f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9661ffb640 10 client.18369 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f967a59c640 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa26f6950 0x55daa26f8d70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1102f450ede06584 server_cookie=fd60098962a7f448 in_seq=0 out_seq=0 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9661ffb640 1 -- 172.21.15.100:0/1178910339 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55daa2381720 con 0x55daa26f6950 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9661ffb640 10 client.18369 handle_command_reply: tid=0 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9683a69740 1 client.18369 shutdown 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 msgr2=0x55daa26e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9683a69740 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26e3cc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f967400fdf0 tx=0x7f967401d000 comp rx=0 tx=0).stop 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 shutdown_connections 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9683a69740 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa26f6950 0x55daa26f8d70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.209+0000 7f9683a69740 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26e3cc0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9683a69740 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9683a69740 1 --2- 172.21.15.100:0/1178910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 >> 172.21.15.100:0/1178910339 conn(0x55daa26d7990 msgr2=0x55daa26d9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9661ffb640 0 client.18369 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 shutdown_connections 2024-03-12T05:25:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9683a69740 1 -- 172.21.15.100:0/1178910339 wait complete. 2024-03-12T05:25:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.210+0000 7f9683a69740 20 client.18369 trim_cache size 0 max 16384 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 Processor -- start 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 -- start start 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa26f6750 con 0x55daa26e67b0 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa26d71d0 con 0x55daa26ddbd0 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9683a69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa26d89e0 con 0x55daa26dd2b0 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f967ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f967a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f967ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ed660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51922/0 (socket says 172.21.15.100:51922) 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f967ad9d640 1 -- 172.21.15.100:0/3781985507 learned_addr learned my addr 172.21.15.100:0/3781985507 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.225+0000 7f9673fff640 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f967ad9d640 1 -- 172.21.15.100:0/3781985507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 msgr2=0x55daa26e5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f967ad9d640 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f967ad9d640 1 -- 172.21.15.100:0/3781985507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 msgr2=0x55daa26e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f967ad9d640 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f967ad9d640 1 -- 172.21.15.100:0/3781985507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daa25dfac0 con 0x55daa26e67b0 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f967ad9d640 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ed660 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9674010e20 tx=0x7f9674009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8829151de1b0cbe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f9663fff640 1 -- 172.21.15.100:0/3781985507 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674013120 con 0x55daa26e67b0 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f9663fff640 1 -- 172.21.15.100:0/3781985507 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9674010910 con 0x55daa26e67b0 2024-03-12T05:25:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.226+0000 7f9663fff640 1 -- 172.21.15.100:0/3781985507 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674021a70 con 0x55daa26e67b0 2024-03-12T05:25:56.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- 172.21.15.100:0/3781985507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 msgr2=0x55daa26ed660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ed660 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9674010e20 tx=0x7f9674009a90 comp rx=0 tx=0).stop 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- 172.21.15.100:0/3781985507 shutdown_connections 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ed660 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26ddbd0 0x55daa26e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- 172.21.15.100:0/3781985507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26dd2b0 0x55daa26e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- 172.21.15.100:0/3781985507 >> 172.21.15.100:0/3781985507 conn(0x55daa26d7990 msgr2=0x55daa26de8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- 172.21.15.100:0/3781985507 shutdown_connections 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- 172.21.15.100:0/3781985507 wait complete. 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- start start 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 Processor -- start 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26dd2b0 0x55daa26edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26ddbd0 0x55daa26ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa26ecb60 con 0x55daa26e67b0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa26eccd0 con 0x55daa26dd2b0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daa274cea0 con 0x55daa26ddbd0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.227+0000 7f9683a69740 10 client.0 _ll_register_callbacks cb 0x7f966c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ec4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51930/0 (socket says 172.21.15.100:51930) 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 -- 172.21.15.100:0/1044573052 learned_addr learned my addr 172.21.15.100:0/1044573052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967a59c640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26dd2b0 0x55daa26edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9673fff640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26ddbd0 0x55daa26ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 -- 172.21.15.100:0/1044573052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26ddbd0 msgr2=0x55daa26ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26ddbd0 0x55daa26ebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 -- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26dd2b0 msgr2=0x55daa26edaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26dd2b0 0x55daa26edaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 -- 172.21.15.100:0/1044573052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daa2643610 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f967ad9d640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ec4c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9674010660 tx=0x7f9674010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee7e30c7839653e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9661ffb640 1 -- 172.21.15.100:0/1044573052 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674013550 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9661ffb640 1 -- 172.21.15.100:0/1044573052 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9674025040 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f966c00fd00 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9661ffb640 1 -- 172.21.15.100:0/1044573052 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9674021490 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9683a69740 10 client.18375 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55daa26056a0 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.228+0000 7f9661ffb640 1 -- 172.21.15.100:0/1044573052 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9674048080 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9661ffb640 1 -- 172.21.15.100:0/1044573052 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9674030040 con 0x55daa26e67b0 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 10 client.18375 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 10 client.18375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 20 client.18375 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:56.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa274e350 0x55daa2750770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 4 client.18375 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55daa23d0890 con 0x55daa274e350 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f967a59c640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa274e350 0x55daa2750770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9661ffb640 10 client.18375 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f967a59c640 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa274e350 0x55daa2750770 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9d896af8dba290d server_cookie=4180cad5d7072c00 in_seq=0 out_seq=0 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9661ffb640 1 -- 172.21.15.100:0/1044573052 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55daa23d0890 con 0x55daa274e350 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9661ffb640 10 client.18375 handle_command_reply: tid=0 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.229+0000 7f9683a69740 1 client.18375 shutdown 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 msgr2=0x55daa26ec4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ec4c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9674010660 tx=0x7f9674010690 comp rx=0 tx=0).stop 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 shutdown_connections 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daa274e350 0x55daa2750770 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daa26e67b0 0x55daa26ec4c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daa26ddbd0 0x55daa26ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 --2- 172.21.15.100:0/1044573052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daa26dd2b0 0x55daa26edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 >> 172.21.15.100:0/1044573052 conn(0x55daa26d7990 msgr2=0x55daa26e2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9661ffb640 0 client.18375 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 shutdown_connections 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 1 -- 172.21.15.100:0/1044573052 wait complete. 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9683a69740 20 client.18375 trim_cache size 0 max 16384 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:25:56.231 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:25:56.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f96680a1ba0 msgr2=0x7f96680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f96680a1ba0 0x7f96680a4060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9674009fd0 tx=0x7f9674009d80 comp rx=0 tx=0).stop 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 msgr2=0x7f967c19fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 0x7f967c19fb80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f966402c170 tx=0x7f966402c5d0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 shutdown_connections 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f96680a1ba0 0x7f96680a4060 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c1997b0 0x7f967c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f967c105340 0x7f967c19fb80 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 --2- 172.21.15.100:0/1183207805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f967c100270 0x7f967c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.230+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 >> 172.21.15.100:0/1183207805 conn(0x7f967c077230 msgr2=0x7f967c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.231+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 shutdown_connections 2024-03-12T05:25:56.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.231+0000 7f9680e88640 1 -- 172.21.15.100:0/1183207805 wait complete. 2024-03-12T05:25:56.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26629 41.31 MB/sec warmup 39 sec latency 278.505 ms 2024-03-12T05:25:56.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:25:56.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:25:56.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 -- 172.21.15.100:0/3516779614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7301023f0 msgr2=0x7fd7301027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 --2- 172.21.15.100:0/3516779614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7301023f0 0x7fd7301027d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd720009900 tx=0x7fd72002bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 -- 172.21.15.100:0/3516779614 shutdown_connections 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 --2- 172.21.15.100:0/3516779614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd73010b7c0 0x7fd73010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 --2- 172.21.15.100:0/3516779614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd730102d10 0x7fd73010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 --2- 172.21.15.100:0/3516779614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7301023f0 0x7fd7301027d0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 -- 172.21.15.100:0/3516779614 >> 172.21.15.100:0/3516779614 conn(0x7fd730077180 msgr2=0x7fd730077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 -- 172.21.15.100:0/3516779614 shutdown_connections 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 -- 172.21.15.100:0/3516779614 wait complete. 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.358+0000 7fd736177640 1 Processor -- start 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 -- start start 2024-03-12T05:25:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7301023f0 0x7fd73019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 0x7fd73019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010b7c0 0x7fd730197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd730197dc0 con 0x7fd730102d10 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd730197f30 con 0x7fd7301023f0 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd736177640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7301980f0 con 0x7fd73010b7c0 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 0x7fd73019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010b7c0 0x7fd730197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 0x7fd73019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51942/0 (socket says 172.21.15.100:51942) 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72effd640 1 -- 172.21.15.100:0/650630214 learned_addr learned my addr 172.21.15.100:0/650630214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72f7fe640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7301023f0 0x7fd73019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72ffff640 1 -- 172.21.15.100:0/650630214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7301023f0 msgr2=0x7fd73019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72ffff640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7301023f0 0x7fd73019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72ffff640 1 -- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 msgr2=0x7fd73019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72ffff640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 0x7fd73019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72ffff640 1 -- 172.21.15.100:0/650630214 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd720009560 con 0x7fd73010b7c0 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.359+0000 7fd72effd640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 0x7fd73019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.360+0000 7fd72ffff640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010b7c0 0x7fd730197630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd728009fd0 tx=0x7fd72800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.360+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800ce20 con 0x7fd73010b7c0 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.360+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd728005020 con 0x7fd73010b7c0 2024-03-12T05:25:56.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.360+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7301983d0 con 0x7fd73010b7c0 2024-03-12T05:25:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.360+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728010690 con 0x7fd73010b7c0 2024-03-12T05:25:56.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.360+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7301a44f0 con 0x7fd73010b7c0 2024-03-12T05:25:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.361+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd730068b20 con 0x7fd73010b7c0 2024-03-12T05:25:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.362+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd728005500 con 0x7fd73010b7c0 2024-03-12T05:25:56.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.363+0000 7fd72cff9640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd7180a1ba0 0x7fd7180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.363+0000 7fd72f7fe640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd7180a1ba0 0x7fd7180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.363+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd728014070 con 0x7fd73010b7c0 2024-03-12T05:25:56.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.363+0000 7fd72f7fe640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd7180a1ba0 0x7fd7180a4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd720005ab0 tx=0x7fd720005a20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.370+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd728096da0 con 0x7fd73010b7c0 2024-03-12T05:25:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.617+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd730067f70 con 0x7fd73010b7c0 2024-03-12T05:25:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.618+0000 7fd72cff9640 1 -- 172.21.15.100:0/650630214 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7fd728005190 con 0x7fd73010b7c0 2024-03-12T05:25:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:25:56.613 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:25:21.243438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:25:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-03-12T05:25:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd7180a1ba0 msgr2=0x7fd7180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd7180a1ba0 0x7fd7180a4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd720005ab0 tx=0x7fd720005a20 comp rx=0 tx=0).stop 2024-03-12T05:25:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010b7c0 msgr2=0x7fd730197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010b7c0 0x7fd730197630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd728009fd0 tx=0x7fd72800ef10 comp rx=0 tx=0).stop 2024-03-12T05:25:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 shutdown_connections 2024-03-12T05:25:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd7180a1ba0 0x7fd7180a4060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010b7c0 0x7fd730197630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd730102d10 0x7fd73019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 --2- 172.21.15.100:0/650630214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7301023f0 0x7fd73019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 >> 172.21.15.100:0/650630214 conn(0x7fd730077180 msgr2=0x7fd7300fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 shutdown_connections 2024-03-12T05:25:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.620+0000 7fd736177640 1 -- 172.21.15.100:0/650630214 wait complete. 2024-03-12T05:25:56.626 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:25:56.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:25:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 -- 172.21.15.100:0/782083077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8106240 msgr2=0x7f20c8106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 --2- 172.21.15.100:0/782083077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8106240 0x7f20c8106620 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009900 tx=0x7f20c402bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 -- 172.21.15.100:0/782083077 shutdown_connections 2024-03-12T05:25:56.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 --2- 172.21.15.100:0/782083077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c8100d20 0x7f20c810ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 --2- 172.21.15.100:0/782083077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c8100360 0x7f20c81007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 --2- 172.21.15.100:0/782083077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8106240 0x7f20c8106620 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 -- 172.21.15.100:0/782083077 >> 172.21.15.100:0/782083077 conn(0x7f20c8077060 msgr2=0x7f20c8077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 -- 172.21.15.100:0/782083077 shutdown_connections 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.745+0000 7f20d07c9640 1 -- 172.21.15.100:0/782083077 wait complete. 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 Processor -- start 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 -- start start 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 0x7f20c8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c8100d20 0x7f20c8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c8106240 0x7f20c81a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8199320 con 0x7f20c8100360 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c81a3b40 con 0x7f20c8106240 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20d07c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c81a3c80 con 0x7f20c8100d20 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 0x7f20c8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 0x7f20c8198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51966/0 (socket says 172.21.15.100:51966) 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 -- 172.21.15.100:0/2245812947 learned_addr learned my addr 172.21.15.100:0/2245812947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20cdd64640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c8100d20 0x7f20c8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ced66640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c8106240 0x7f20c81a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 -- 172.21.15.100:0/2245812947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c8100d20 msgr2=0x7f20c8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c8100d20 0x7f20c8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 -- 172.21.15.100:0/2245812947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c8106240 msgr2=0x7f20c81a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c8106240 0x7f20c81a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.746+0000 7f20ce565640 1 -- 172.21.15.100:0/2245812947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4009560 con 0x7f20c8100360 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.747+0000 7f20ce565640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 0x7f20c8198580 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005940 tx=0x7f20c402c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.747+0000 7f20bf7fe640 1 -- 172.21.15.100:0/2245812947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40043d0 con 0x7f20c8100360 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.747+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c81a3e80 con 0x7f20c8100360 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.747+0000 7f20bf7fe640 1 -- 172.21.15.100:0/2245812947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c40052d0 con 0x7f20c8100360 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.747+0000 7f20bf7fe640 1 -- 172.21.15.100:0/2245812947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4003d30 con 0x7f20c8100360 2024-03-12T05:25:56.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.747+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c81a4300 con 0x7f20c8100360 2024-03-12T05:25:56.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.750+0000 7f20bf7fe640 1 -- 172.21.15.100:0/2245812947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f20c4005440 con 0x7f20c8100360 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.750+0000 7f20d339f740 1 Processor -- start 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.750+0000 7f20bf7fe640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f20b40a1ba0 0x7f20b40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.750+0000 7f20d339f740 1 -- start start 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ced66640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f20b40a1ba0 0x7f20b40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20bf7fe640 1 -- 172.21.15.100:0/2245812947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f20c40ef860 con 0x7f20c8100360 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20d339f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9411630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20d339f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf941a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20d339f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf941a750 0x556bf941cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20d339f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf93cc960 con 0x556bf9411250 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20d339f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf9380310 con 0x556bf941a750 2024-03-12T05:25:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20d339f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf937d840 con 0x556bf9411b70 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9411630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9411630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51970/0 (socket says 172.21.15.100:51970) 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 -- 172.21.15.100:0/56230827 learned_addr learned my addr 172.21.15.100:0/56230827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20cdd64640 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf941a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ced66640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f20b40a1ba0 0x7f20b40a4060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005fd0 tx=0x7f20c0009600 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ced66640 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf941a750 0x556bf941cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 -- 172.21.15.100:0/56230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 msgr2=0x556bf941a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf941a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 -- 172.21.15.100:0/56230827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf941a750 msgr2=0x556bf941cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.751+0000 7f20ce565640 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf941a750 0x556bf941cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.752+0000 7f20ce565640 1 -- 172.21.15.100:0/56230827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bf933b060 con 0x556bf9411250 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.752+0000 7f20ce565640 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9411630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f20c40b89c0 tx=0x7f20c40b88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79a6db218c894f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.752+0000 7f20bcff9640 1 -- 172.21.15.100:0/56230827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x556bf9411250 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.752+0000 7f20bcff9640 1 -- 172.21.15.100:0/56230827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c40bcc60 con 0x556bf9411250 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.752+0000 7f20bcff9640 1 -- 172.21.15.100:0/56230827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40ed1f0 con 0x556bf9411250 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 -- 172.21.15.100:0/56230827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 msgr2=0x556bf9411630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9411630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f20c40b89c0 tx=0x7f20c40b88a0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 -- 172.21.15.100:0/56230827 shutdown_connections 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf941a750 0x556bf941cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf941a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 --2- 172.21.15.100:0/56230827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9411630 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 -- 172.21.15.100:0/56230827 >> 172.21.15.100:0/56230827 conn(0x556bf940b980 msgr2=0x556bf940bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 -- 172.21.15.100:0/56230827 shutdown_connections 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.753+0000 7f20d339f740 1 -- 172.21.15.100:0/56230827 wait complete. 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 -- start start 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 Processor -- start 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9417840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411b70 0x556bf9417d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf941a750 0x556bf94182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf9379020 con 0x556bf9411250 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf904c430 con 0x556bf9411b70 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf9008e00 con 0x556bf941a750 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20d339f740 10 client.0 _ll_register_callbacks cb 0x556bf9320cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:56.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20ce565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9417840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20ce565640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9417840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51980/0 (socket says 172.21.15.100:51980) 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20ce565640 1 -- 172.21.15.100:0/3000083732 learned_addr learned my addr 172.21.15.100:0/3000083732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20ced66640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf941a750 0x556bf94182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20cdd64640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411b70 0x556bf9417d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20ce565640 1 -- 172.21.15.100:0/3000083732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf941a750 msgr2=0x556bf94182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.754+0000 7f20ce565640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf941a750 0x556bf94182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20ce565640 1 -- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411b70 msgr2=0x556bf9417d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20ce565640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411b70 0x556bf9417d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20ce565640 1 -- 172.21.15.100:0/3000083732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c81a4300 con 0x556bf9411250 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20cdd64640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411b70 0x556bf9417d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20ce565640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9417840 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f20c40ed130 tx=0x7f20c40a72c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948a4f3a36883a52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20b1ffb640 1 -- 172.21.15.100:0/3000083732 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20b1ffb640 1 -- 172.21.15.100:0/3000083732 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f20c40bc560 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bf90787e0 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20b1ffb640 1 -- 172.21.15.100:0/3000083732 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c40c4d00 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20d339f740 10 client.18393 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556bf93776a0 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.755+0000 7f20b1ffb640 1 -- 172.21.15.100:0/3000083732 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c40c4870 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20b1ffb640 1 -- 172.21.15.100:0/3000083732 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f20c403c070 con 0x556bf9411250 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20d339f740 10 client.18393 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20d339f740 10 client.18393 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20d339f740 20 client.18393 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20d339f740 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf942a4b0 0x556bf942c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20d339f740 4 client.18393 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bf90787e0 con 0x556bf942a4b0 2024-03-12T05:25:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20cdd64640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf942a4b0 0x556bf942c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20cdd64640 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf942a4b0 0x556bf942c8d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41dc28f0b298312c server_cookie=5059c523a9d57fc7 in_seq=0 out_seq=0 2024-03-12T05:25:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.756+0000 7f20b1ffb640 10 client.18393 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20b1ffb640 1 -- 172.21.15.100:0/3000083732 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bf90787e0 con 0x556bf942a4b0 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20b1ffb640 10 client.18393 handle_command_reply: tid=0 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 client.18393 shutdown 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 msgr2=0x556bf9417840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9417840 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f20c40ed130 tx=0x7f20c40a72c0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 shutdown_connections 2024-03-12T05:25:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf942a4b0 0x556bf942c8d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf941a750 0x556bf94182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411b70 0x556bf9417d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.760+0000 7f20d339f740 1 --2- 172.21.15.100:0/3000083732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf9411250 0x556bf9417840 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.761+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 >> 172.21.15.100:0/3000083732 conn(0x556bf940b980 msgr2=0x556bf940d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.761+0000 7f20b1ffb640 0 client.18393 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.761+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 shutdown_connections 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.761+0000 7f20d339f740 1 -- 172.21.15.100:0/3000083732 wait complete. 2024-03-12T05:25:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.761+0000 7f20d339f740 20 client.18393 trim_cache size 0 max 16384 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 Processor -- start 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 -- start start 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf94180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf940aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf94213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf92fe130 con 0x556bf941a750 2024-03-12T05:25:56.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf90787e0 con 0x556bf9411250 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20d339f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf940c980 con 0x556bf9411b70 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20ced66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf94213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.774+0000 7f20ced66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf94213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51994/0 (socket says 172.21.15.100:51994) 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 -- 172.21.15.100:0/3520379424 learned_addr learned my addr 172.21.15.100:0/3520379424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20cdd64640 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf940aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ce565640 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf94180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 -- 172.21.15.100:0/3520379424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 msgr2=0x556bf940aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf940aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 -- 172.21.15.100:0/3520379424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 msgr2=0x556bf94180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf94180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 -- 172.21.15.100:0/3520379424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bf9313c20 con 0x556bf941a750 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20ced66640 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf94213c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f20c0012970 tx=0x7f20c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab424ccbee7953d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20b3fff640 1 -- 172.21.15.100:0/3520379424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0025070 con 0x556bf941a750 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20b3fff640 1 -- 172.21.15.100:0/3520379424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c0012e30 con 0x556bf941a750 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.775+0000 7f20b3fff640 1 -- 172.21.15.100:0/3520379424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c001fde0 con 0x556bf941a750 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 -- 172.21.15.100:0/3520379424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 msgr2=0x556bf94213c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf94213c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f20c0012970 tx=0x7f20c0013000 comp rx=0 tx=0).stop 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 -- 172.21.15.100:0/3520379424 shutdown_connections 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf94213c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf940aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 --2- 172.21.15.100:0/3520379424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf94180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 -- 172.21.15.100:0/3520379424 >> 172.21.15.100:0/3520379424 conn(0x556bf940b980 msgr2=0x556bf940a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 -- 172.21.15.100:0/3520379424 shutdown_connections 2024-03-12T05:25:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 -- 172.21.15.100:0/3520379424 wait complete. 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 -- start start 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.776+0000 7f20d339f740 1 Processor -- start 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf9479d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf947a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf9420790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf9420e90 con 0x556bf941a750 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf9421000 con 0x556bf9411250 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf94211c0 con 0x556bf9411b70 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20d339f740 10 client.0 _ll_register_callbacks cb 0x7f20c4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf9420790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ce565640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf9479d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf9420790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51998/0 (socket says 172.21.15.100:51998) 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 -- 172.21.15.100:0/4075632336 learned_addr learned my addr 172.21.15.100:0/4075632336 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20cdd64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf947a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 -- 172.21.15.100:0/4075632336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 msgr2=0x556bf947a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf947a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 -- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 msgr2=0x556bf9479d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf9479d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 -- 172.21.15.100:0/4075632336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c0012620 con 0x556bf941a750 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20cdd64640 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf947a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20ced66640 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf9420790 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f20c001fc10 tx=0x7f20c001fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f394cf1a8f023926 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20b1ffb640 1 -- 172.21.15.100:0/4075632336 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0025030 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.777+0000 7f20b1ffb640 1 -- 172.21.15.100:0/4075632336 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f20c001f420 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20c40bcc60 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20b1ffb640 1 -- 172.21.15.100:0/4075632336 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c0003970 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 10 client.18399 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556bf93398c0 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20b1ffb640 1 -- 172.21.15.100:0/4075632336 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c40bcc60 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20b1ffb640 1 -- 172.21.15.100:0/4075632336 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f20c001a070 con 0x556bf941a750 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 10 client.18399 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 10 client.18399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 20 client.18399 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf9482d50 0x556bf9485170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 4 client.18399 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:25:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556bf90b5720 con 0x556bf9482d50 2024-03-12T05:25:56.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.778+0000 7f20ce565640 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf9482d50 0x556bf9485170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.779+0000 7f20b1ffb640 10 client.18399 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:56.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.779+0000 7f20ce565640 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf9482d50 0x556bf9485170 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e62a3d84a5b45407 server_cookie=e7126f77ba621e33 in_seq=0 out_seq=0 2024-03-12T05:25:56.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20b1ffb640 1 -- 172.21.15.100:0/4075632336 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556bf90b5720 con 0x556bf9482d50 2024-03-12T05:25:56.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20b1ffb640 10 client.18399 handle_command_reply: tid=0 2024-03-12T05:25:56.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 client.18399 shutdown 2024-03-12T05:25:56.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 msgr2=0x556bf9420790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf9420790 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f20c001fc10 tx=0x7f20c001fc40 comp rx=0 tx=0).stop 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 shutdown_connections 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x556bf9482d50 0x556bf9485170 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bf941a750 0x556bf9420790 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20b1ffb640 0 client.18399 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bf9411b70 0x556bf947a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 --2- 172.21.15.100:0/4075632336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556bf9411250 0x556bf9479d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 >> 172.21.15.100:0/4075632336 conn(0x556bf940b980 msgr2=0x556bf940d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 shutdown_connections 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 1 -- 172.21.15.100:0/4075632336 wait complete. 2024-03-12T05:25:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.785+0000 7f20d339f740 20 client.18399 trim_cache size 0 max 16384 2024-03-12T05:25:56.783 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:25:56.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f20b40a1ba0 msgr2=0x7f20b40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f20b40a1ba0 0x7f20b40a4060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005fd0 tx=0x7f20c0009600 comp rx=0 tx=0).stop 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 msgr2=0x7f20c8198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 0x7f20c8198580 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005940 tx=0x7f20c402c6e0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 shutdown_connections 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f20b40a1ba0 0x7f20b40a4060 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c8106240 0x7f20c81a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c8100d20 0x7f20c8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 --2- 172.21.15.100:0/2245812947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c8100360 0x7f20c8198580 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 >> 172.21.15.100:0/2245812947 conn(0x7f20c8077060 msgr2=0x7f20c80fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 shutdown_connections 2024-03-12T05:25:56.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.786+0000 7f20d07c9640 1 -- 172.21.15.100:0/2245812947 wait complete. 2024-03-12T05:25:56.791 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:25:56.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 -- 172.21.15.100:0/2862906235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b81064c0 msgr2=0x7f92b810e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 --2- 172.21.15.100:0/2862906235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b81064c0 0x7f92b810e070 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009cd0 tx=0x7f92b002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 -- 172.21.15.100:0/2862906235 shutdown_connections 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 --2- 172.21.15.100:0/2862906235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b81064c0 0x7f92b810e070 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 --2- 172.21.15.100:0/2862906235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8068fa0 0x7f92b8105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 --2- 172.21.15.100:0/2862906235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80685f0 0x7f92b80689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 -- 172.21.15.100:0/2862906235 >> 172.21.15.100:0/2862906235 conn(0x7f92b8077180 msgr2=0x7f92b8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 -- 172.21.15.100:0/2862906235 shutdown_connections 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.923+0000 7f92bfafb640 1 -- 172.21.15.100:0/2862906235 wait complete. 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 Processor -- start 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 -- start start 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 0x7f92b8100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b81064c0 0x7f92b8100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8100ac0 0x7f92b81a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81a5f80 con 0x7f92b80685f0 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81a60f0 con 0x7f92b81064c0 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bfafb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81a62b0 con 0x7f92b8100ac0 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 0x7f92b8100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 0x7f92b8100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52010/0 (socket says 172.21.15.100:52010) 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 -- 172.21.15.100:0/1717312544 learned_addr learned my addr 172.21.15.100:0/1717312544 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd096640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b81064c0 0x7f92b8100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92be098640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8100ac0 0x7f92b81a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 -- 172.21.15.100:0/1717312544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8100ac0 msgr2=0x7f92b81a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8100ac0 0x7f92b81a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 -- 172.21.15.100:0/1717312544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b81064c0 msgr2=0x7f92b8100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b81064c0 0x7f92b8100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 -- 172.21.15.100:0/1717312544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0009900 con 0x7f92b80685f0 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92be098640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8100ac0 0x7f92b81a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.924+0000 7f92bd897640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 0x7f92b8100040 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f92b400e9f0 tx=0x7f92b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.925+0000 7f92aeffd640 1 -- 172.21.15.100:0/1717312544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4009700 con 0x7f92b80685f0 2024-03-12T05:25:56.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.925+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b81a6590 con 0x7f92b80685f0 2024-03-12T05:25:56.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.925+0000 7f92aeffd640 1 -- 172.21.15.100:0/1717312544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92b400cd80 con 0x7f92b80685f0 2024-03-12T05:25:56.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.925+0000 7f92aeffd640 1 -- 172.21.15.100:0/1717312544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4018920 con 0x7f92b80685f0 2024-03-12T05:25:56.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.925+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b81a6af0 con 0x7f92b80685f0 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.927+0000 7f92aeffd640 1 -- 172.21.15.100:0/1717312544 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f92b4018a80 con 0x7f92b80685f0 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.927+0000 7f92c27ac740 1 Processor -- start 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 -- start start 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 0x55dfa47f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 0x55dfa47f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa47a6960 con 0x55dfa47ebb70 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa475a310 con 0x55dfa47eb250 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92c27ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa4757840 con 0x55dfa47f4750 2024-03-12T05:25:56.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd897640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 0x55dfa47f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92be098640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 0x55dfa47f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 0x55dfa47f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58198/0 (socket says 172.21.15.100:58198) 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd897640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 0x55dfa47f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52022/0 (socket says 172.21.15.100:52022) 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 -- 172.21.15.100:0/1662136568 learned_addr learned my addr 172.21.15.100:0/1662136568 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92be098640 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47eb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44756/0 (socket says 172.21.15.100:44756) 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92aeffd640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92a40a1bd0 0x7f92a40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 -- 172.21.15.100:0/1662136568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 msgr2=0x55dfa47eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 -- 172.21.15.100:0/1662136568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 msgr2=0x55dfa47f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92be098640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92a40a1bd0 0x7f92a40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 0x55dfa47f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.928+0000 7f92bd096640 1 -- 172.21.15.100:0/1662136568 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfa4715060 con 0x55dfa47f4750 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92bd897640 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 0x55dfa47f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92aeffd640 1 -- 172.21.15.100:0/1717312544 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f92b4014070 con 0x7f92b80685f0 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92bd096640 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 0x55dfa47f6b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009c00 tx=0x7f92a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d097aa55e9766f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92be098640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92a40a1bd0 0x7f92a40a4090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f92b0005cc0 tx=0x7f92b00095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92a3fff640 1 -- 172.21.15.100:0/1662136568 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8004050 con 0x55dfa47f4750 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92a3fff640 1 -- 172.21.15.100:0/1662136568 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92a80041b0 con 0x55dfa47f4750 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.929+0000 7f92a3fff640 1 -- 172.21.15.100:0/1662136568 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80197b0 con 0x55dfa47f4750 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 -- 172.21.15.100:0/1662136568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 msgr2=0x55dfa47f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 0x55dfa47f6b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009c00 tx=0x7f92a8011810 comp rx=0 tx=0).stop 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 -- 172.21.15.100:0/1662136568 shutdown_connections 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47f4750 0x55dfa47f6b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47ebb70 0x55dfa47f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 --2- 172.21.15.100:0/1662136568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 -- 172.21.15.100:0/1662136568 >> 172.21.15.100:0/1662136568 conn(0x55dfa47e5980 msgr2=0x55dfa47e5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 -- 172.21.15.100:0/1662136568 shutdown_connections 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 -- 172.21.15.100:0/1662136568 wait complete. 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.930+0000 7f92c27ac740 1 -- start start 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 Processor -- start 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa4753020 con 0x55dfa47f4750 2024-03-12T05:25:56.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa4426430 con 0x55dfa47eb250 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa43e2e00 con 0x55dfa47ebb70 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92c27ac740 10 client.0 _ll_register_callbacks cb 0x55dfa46facb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92be098640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92bd096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92bd897640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92be098640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52038/0 (socket says 172.21.15.100:52038) 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.931+0000 7f92be098640 1 -- 172.21.15.100:0/3091804233 learned_addr learned my addr 172.21.15.100:0/3091804233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92be098640 1 -- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 msgr2=0x55dfa47f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92be098640 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92be098640 1 -- 172.21.15.100:0/3091804233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 msgr2=0x55dfa47f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92be098640 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92be098640 1 -- 172.21.15.100:0/3091804233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a8009830 con 0x55dfa47f4750 2024-03-12T05:25:56.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92bd096640 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92be098640 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47f21f0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009cd0 tx=0x7f92b002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c825de9fbbe15665 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92a17fa640 1 -- 172.21.15.100:0/3091804233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0003d80 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92a17fa640 1 -- 172.21.15.100:0/3091804233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f92b00324c0 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfa44527e0 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92a17fa640 1 -- 172.21.15.100:0/3091804233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b0032af0 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92c27ac740 10 client.18411 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dfa47516a0 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.932+0000 7f92a17fa640 1 -- 172.21.15.100:0/3091804233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dfa44527e0 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92a17fa640 1 -- 172.21.15.100:0/3091804233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f92b0049050 con 0x55dfa47f4750 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92c27ac740 10 client.18411 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92c27ac740 10 client.18411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:25:56.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92c27ac740 20 client.18411 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa4804b20 0x55dfa4806f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92c27ac740 4 client.18411 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfa448f720 con 0x55dfa4804b20 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92bd897640 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa4804b20 0x55dfa4806f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92a17fa640 10 client.18411 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.933+0000 7f92bd897640 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa4804b20 0x55dfa4806f40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba896f1a3f983eb server_cookie=98f9c5040806679d in_seq=0 out_seq=0 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.934+0000 7f92a17fa640 1 -- 172.21.15.100:0/3091804233 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfa448f720 con 0x55dfa4804b20 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.934+0000 7f92a17fa640 10 client.18411 handle_command_reply: tid=0 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.934+0000 7f92c27ac740 1 client.18411 shutdown 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 msgr2=0x55dfa47f21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47f21f0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009cd0 tx=0x7f92b002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 shutdown_connections 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47f21f0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa4804b20 0x55dfa4806f40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3091804233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 >> 172.21.15.100:0/3091804233 conn(0x55dfa47e5980 msgr2=0x55dfa47e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92a17fa640 0 client.18411 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 shutdown_connections 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 1 -- 172.21.15.100:0/3091804233 wait complete. 2024-03-12T05:25:56.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.935+0000 7f92c27ac740 20 client.18411 trim_cache size 0 max 16384 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 Processor -- start 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 -- start start 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa46d8130 con 0x55dfa47f4750 2024-03-12T05:25:56.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa47e54e0 con 0x55dfa47eb250 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.948+0000 7f92c27ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa47e6c00 con 0x55dfa47ebb70 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47ee9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52040/0 (socket says 172.21.15.100:52040) 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 -- 172.21.15.100:0/656813782 learned_addr learned my addr 172.21.15.100:0/656813782 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92bd096640 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92bd897640 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 -- 172.21.15.100:0/656813782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 msgr2=0x55dfa47f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 -- 172.21.15.100:0/656813782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 msgr2=0x55dfa47f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 -- 172.21.15.100:0/656813782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfa46edc20 con 0x55dfa47f4750 2024-03-12T05:25:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92be098640 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47ee9d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f92b0037040 tx=0x7f92b0009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6cc4a9f4164530 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92a37fe640 1 -- 172.21.15.100:0/656813782 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b004c440 con 0x55dfa47f4750 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92a37fe640 1 -- 172.21.15.100:0/656813782 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92b0059070 con 0x55dfa47f4750 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.949+0000 7f92a37fe640 1 -- 172.21.15.100:0/656813782 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0055c10 con 0x55dfa47f4750 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 -- 172.21.15.100:0/656813782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 msgr2=0x55dfa47ee9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47ee9d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f92b0037040 tx=0x7f92b0009380 comp rx=0 tx=0).stop 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 -- 172.21.15.100:0/656813782 shutdown_connections 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47f4750 0x55dfa47ee9d0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 --2- 172.21.15.100:0/656813782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47eb250 0x55dfa47f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 -- 172.21.15.100:0/656813782 >> 172.21.15.100:0/656813782 conn(0x55dfa47e5980 msgr2=0x55dfa47eca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 -- 172.21.15.100:0/656813782 shutdown_connections 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 -- 172.21.15.100:0/656813782 wait complete. 2024-03-12T05:25:56.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 -- start start 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.950+0000 7f92c27ac740 1 Processor -- start 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 0x55dfa47fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47f4750 0x55dfa47fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa47fa5e0 con 0x55dfa47eb250 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa47fa750 con 0x55dfa47f4750 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa47fa910 con 0x55dfa47ebb70 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92c27ac740 10 client.0 _ll_register_callbacks cb 0x7f92a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 0x55dfa47fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 0x55dfa47fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52056/0 (socket says 172.21.15.100:52056) 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 -- 172.21.15.100:0/3992253000 learned_addr learned my addr 172.21.15.100:0/3992253000 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92be098640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47f4750 0x55dfa47fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 -- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 msgr2=0x55dfa47f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 -- 172.21.15.100:0/3992253000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47f4750 msgr2=0x55dfa47fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47f4750 0x55dfa47fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 -- 172.21.15.100:0/3992253000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfa47516a0 con 0x55dfa47eb250 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd096640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92bd897640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 0x55dfa47fb7c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f92b40cb890 tx=0x7f92b40ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce69a7803b444f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.951+0000 7f92a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:56.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92a17fa640 1 -- 172.21.15.100:0/3992253000 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b409a180 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92a17fa640 1 -- 172.21.15.100:0/3992253000 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f92b409a820 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92a80041b0 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92a17fa640 1 -- 172.21.15.100:0/3992253000 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b409e8c0 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 10 client.18417 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dfa47138c0 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92a17fa640 1 -- 172.21.15.100:0/3992253000 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92a80041b0 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92a17fa640 1 -- 172.21.15.100:0/3992253000 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f92b40ba0b0 con 0x55dfa47eb250 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 10 client.18417 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 10 client.18417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 20 client.18417 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa485c6b0 0x55dfa485eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 4 client.18417 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:25:56.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfa44de890 con 0x55dfa485c6b0 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.952+0000 7f92bd096640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa485c6b0 0x55dfa485eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.953+0000 7f92a17fa640 10 client.18417 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.953+0000 7f92bd096640 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa485c6b0 0x55dfa485eaa0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad0eff47b2fa1fdf server_cookie=980c64c7cb29e152 in_seq=0 out_seq=0 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92a17fa640 1 -- 172.21.15.100:0/3992253000 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfa44de890 con 0x55dfa485c6b0 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92a17fa640 10 client.18417 handle_command_reply: tid=0 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 client.18417 shutdown 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 msgr2=0x55dfa47fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 0x55dfa47fb7c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f92b40cb890 tx=0x7f92b40ba040 comp rx=0 tx=0).stop 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 shutdown_connections 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55dfa485c6b0 0x55dfa485eaa0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfa47f4750 0x55dfa47fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfa47ebb70 0x55dfa47f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92a17fa640 0 client.18417 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:25:56.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 --2- 172.21.15.100:0/3992253000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfa47eb250 0x55dfa47fb7c0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 >> 172.21.15.100:0/3992253000 conn(0x55dfa47e5980 msgr2=0x55dfa47e6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 shutdown_connections 2024-03-12T05:25:56.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 1 -- 172.21.15.100:0/3992253000 wait complete. 2024-03-12T05:25:56.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.954+0000 7f92c27ac740 20 client.18417 trim_cache size 0 max 16384 2024-03-12T05:25:56.957 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:25:56.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92a40a1bd0 msgr2=0x7f92a40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92a40a1bd0 0x7f92a40a4090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f92b0005cc0 tx=0x7f92b00095c0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 msgr2=0x7f92b8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 0x7f92b8100040 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f92b400e9f0 tx=0x7f92b400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 shutdown_connections 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f92a40a1bd0 0x7f92a40a4090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8100ac0 0x7f92b81a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b81064c0 0x7f92b8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 --2- 172.21.15.100:0/1717312544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b80685f0 0x7f92b8100040 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 >> 172.21.15.100:0/1717312544 conn(0x7f92b8077180 msgr2=0x7f92b80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 shutdown_connections 2024-03-12T05:25:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:56.955+0000 7f92bfafb640 1 -- 172.21.15.100:0/1717312544 wait complete. 2024-03-12T05:25:56.961 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:25:56.961 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:25:57.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 -- 172.21.15.100:0/2537315996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c100c10 msgr2=0x7f0b1c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2537315996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c100c10 0x7f0b1c104c50 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009930 tx=0x7f0b0802bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 -- 172.21.15.100:0/2537315996 shutdown_connections 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2537315996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c105330 0x7f0b1c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2537315996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c100c10 0x7f0b1c104c50 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2537315996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c100260 0x7f0b1c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 -- 172.21.15.100:0/2537315996 >> 172.21.15.100:0/2537315996 conn(0x7f0b1c077180 msgr2=0x7f0b1c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 -- 172.21.15.100:0/2537315996 shutdown_connections 2024-03-12T05:25:57.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 -- 172.21.15.100:0/2537315996 wait complete. 2024-03-12T05:25:57.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.083+0000 7f0b22e98640 1 Processor -- start 2024-03-12T05:25:57.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 -- start start 2024-03-12T05:25:57.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c100260 0x7f0b1c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c105330 0x7f0b1c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c19af90 0x7f0b1c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c19b980 con 0x7f0b1c105330 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c19baf0 con 0x7f0b1c19af90 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b22e98640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c19bc60 con 0x7f0b1c100260 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c105330 0x7f0b1c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c100260 0x7f0b1c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c105330 0x7f0b1c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52066/0 (socket says 172.21.15.100:52066) 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b1bfff640 1 -- 172.21.15.100:0/2961281991 learned_addr learned my addr 172.21.15.100:0/2961281991 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b21435640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c19af90 0x7f0b1c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b20c34640 1 -- 172.21.15.100:0/2961281991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c19af90 msgr2=0x7f0b1c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b20c34640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c19af90 0x7f0b1c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b20c34640 1 -- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c105330 msgr2=0x7f0b1c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b20c34640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c105330 0x7f0b1c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b20c34640 1 -- 172.21.15.100:0/2961281991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08009560 con 0x7f0b1c100260 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.084+0000 7f0b21435640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c19af90 0x7f0b1c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.085+0000 7f0b20c34640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c100260 0x7f0b1c1966d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000e9f0 tx=0x7f0b1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.085+0000 7f0b19ffb640 1 -- 172.21.15.100:0/2961281991 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000ccc0 con 0x7f0b1c100260 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.085+0000 7f0b19ffb640 1 -- 172.21.15.100:0/2961281991 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b1000ce20 con 0x7f0b1c100260 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.085+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c1a3240 con 0x7f0b1c100260 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.085+0000 7f0b19ffb640 1 -- 172.21.15.100:0/2961281991 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10018920 con 0x7f0b1c100260 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.086+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c0682e0 con 0x7f0b1c100260 2024-03-12T05:25:57.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.088+0000 7f0b19ffb640 1 -- 172.21.15.100:0/2961281991 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0b10018a80 con 0x7f0b1c100260 2024-03-12T05:25:57.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b19ffb640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0b0c0a3db0 0x7f0b0c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0b0c0a3db0 0x7f0b0c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b19ffb640 1 -- 172.21.15.100:0/2961281991 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0b10014070 con 0x7f0b1c100260 2024-03-12T05:25:57.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 Processor -- start 2024-03-12T05:25:57.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 -- start start 2024-03-12T05:25:57.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d72074c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720755210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d720757b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d720707960 con 0x55d72074cb70 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7206bb310 con 0x55d720755750 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7206b8840 con 0x55d72074c250 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720755210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d72074c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d72074c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58234/0 (socket says 172.21.15.100:58234) 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b21435640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d720757b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0b0c0a3db0 0x7f0b0c0a6270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0ffbc0 tx=0x7f0b0802c670 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.089+0000 7f0b20c34640 1 -- 172.21.15.100:0/3616495526 learned_addr learned my addr 172.21.15.100:0/3616495526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b21435640 1 -- 172.21.15.100:0/3616495526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 msgr2=0x55d72074c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b21435640 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d72074c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b21435640 1 -- 172.21.15.100:0/3616495526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 msgr2=0x55d720755210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b21435640 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720755210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b21435640 1 -- 172.21.15.100:0/3616495526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d720676060 con 0x55d720755750 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b21435640 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d720757b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005fd0 tx=0x7f0b14011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9a8152d4d84094 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b06ffd640 1 -- 172.21.15.100:0/3616495526 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004d20 con 0x55d720755750 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.090+0000 7f0b06ffd640 1 -- 172.21.15.100:0/3616495526 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b14004e80 con 0x55d720755750 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b06ffd640 1 -- 172.21.15.100:0/3616495526 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004440 con 0x55d720755750 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 -- 172.21.15.100:0/3616495526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 msgr2=0x55d720757b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d720757b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005fd0 tx=0x7f0b14011bf0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 -- 172.21.15.100:0/3616495526 shutdown_connections 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d720757b40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720755210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/3616495526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d72074c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.091+0000 7f0b25a5f740 1 -- 172.21.15.100:0/3616495526 >> 172.21.15.100:0/3616495526 conn(0x55d720746980 msgr2=0x55d720746d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.092+0000 7f0b25a5f740 1 -- 172.21.15.100:0/3616495526 shutdown_connections 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.092+0000 7f0b25a5f740 1 -- 172.21.15.100:0/3616495526 wait complete. 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.092+0000 7f0b25a5f740 1 -- start start 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.092+0000 7f0b25a5f740 1 Processor -- start 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d720752780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720752cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d720755750 0x55d720753200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7206b4020 con 0x55d72074cb70 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d720387430 con 0x55d72074c250 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d720343e00 con 0x55d720755750 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d720752780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d720752780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44824/0 (socket says 172.21.15.100:44824) 2024-03-12T05:25:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 -- 172.21.15.100:0/1206137759 learned_addr learned my addr 172.21.15.100:0/1206137759 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b25a5f740 10 client.0 _ll_register_callbacks cb 0x55d72065bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b21435640 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d720755750 0x55d720753200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 -- 172.21.15.100:0/1206137759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d720755750 msgr2=0x55d720753200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d720755750 0x55d720753200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 -- 172.21.15.100:0/1206137759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 msgr2=0x55d720752cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720752cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.093+0000 7f0b20c34640 1 -- 172.21.15.100:0/1206137759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b14011440 con 0x55d72074c250 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0b20c34640 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d720752780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0b100cb290 tx=0x7f0b1009a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346f9734c2a28d68 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0afbfff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0afbfff640 1 -- 172.21.15.100:0/1206137759 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100a2e60 con 0x55d72074c250 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7203f0720 con 0x55d72074c250 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0afbfff640 1 -- 172.21.15.100:0/1206137759 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0b10096de0 con 0x55d72074c250 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0afbfff640 1 -- 172.21.15.100:0/1206137759 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b10096fa0 con 0x55d72074c250 2024-03-12T05:25:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0b25a5f740 10 client.27794 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7206b26a0 con 0x55d72074c250 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.094+0000 7f0afbfff640 1 -- 172.21.15.100:0/1206137759 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d7203f0720 con 0x55d72074c250 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0afbfff640 1 -- 172.21.15.100:0/1206137759 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0b10085140 con 0x55d72074c250 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b25a5f740 10 client.27794 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b25a5f740 10 client.27794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b25a5f740 20 client.27794 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d72076c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b25a5f740 4 client.27794 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d72043f890 con 0x55d720769f10 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d72076c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0afbfff640 10 client.27794 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.095+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d72076c300 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654658ab45b6d28e server_cookie=cfffd15d081881ba in_seq=0 out_seq=0 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0afbfff640 1 -- 172.21.15.100:0/1206137759 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d72043f890 con 0x55d720769f10 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0afbfff640 10 client.27794 handle_command_reply: tid=0 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 client.27794 shutdown 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 msgr2=0x55d720752780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d720752780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0b100cb290 tx=0x7f0b1009a190 comp rx=0 tx=0).stop 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 shutdown_connections 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d72076c300 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0afbfff640 0 client.27794 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d720755750 0x55d720753200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720752cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1206137759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d720752780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.096+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 >> 172.21.15.100:0/1206137759 conn(0x55d720746980 msgr2=0x55d7207486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.097+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 shutdown_connections 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.097+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1206137759 wait complete. 2024-03-12T05:25:57.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.097+0000 7f0b25a5f740 20 client.27794 trim_cache size 0 max 16384 2024-03-12T05:25:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.110+0000 7f0b25a5f740 1 Processor -- start 2024-03-12T05:25:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.110+0000 7f0b25a5f740 1 -- start start 2024-03-12T05:25:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d720754eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720753020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d72074f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d720639130 con 0x55d72074cb70 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d720747980 con 0x55d720755750 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d72074bc90 con 0x55d72074c250 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720753020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d720754eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720753020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52088/0 (socket says 172.21.15.100:52088) 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 -- 172.21.15.100:0/1431749920 learned_addr learned my addr 172.21.15.100:0/1431749920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b21435640 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d72074f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 -- 172.21.15.100:0/1431749920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 msgr2=0x55d720754eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d720754eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 -- 172.21.15.100:0/1431749920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 msgr2=0x55d72074f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d72074f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 -- 172.21.15.100:0/1431749920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d72064ec20 con 0x55d72074cb70 2024-03-12T05:25:57.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720753020 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009230 tx=0x7f0b0802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15de10fbf4e234ed server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:57.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b067fc640 1 -- 172.21.15.100:0/1431749920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08053070 con 0x55d72074cb70 2024-03-12T05:25:57.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b067fc640 1 -- 172.21.15.100:0/1431749920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b0802ca60 con 0x55d72074cb70 2024-03-12T05:25:57.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.111+0000 7f0b067fc640 1 -- 172.21.15.100:0/1431749920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08003d50 con 0x55d72074cb70 2024-03-12T05:25:57.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1431749920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 msgr2=0x55d720753020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720753020 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009230 tx=0x7f0b0802bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:57.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1431749920 shutdown_connections 2024-03-12T05:25:57.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d720755750 0x55d72074f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d72074cb70 0x55d720753020 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/1431749920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074c250 0x55d720754eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1431749920 >> 172.21.15.100:0/1431749920 conn(0x55d720746980 msgr2=0x55d720751050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1431749920 shutdown_connections 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.112+0000 7f0b25a5f740 1 -- 172.21.15.100:0/1431749920 wait complete. 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 -- start start 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 Processor -- start 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d72075f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 0x55d72075fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 0x55d72075ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7207c0430 con 0x55d720755750 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7207c0570 con 0x55d72074c250 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7207c06b0 con 0x55d72074cb70 2024-03-12T05:25:57.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b25a5f740 10 client.0 _ll_register_callbacks cb 0x7f0b14009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b21435640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 0x55d72075ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 0x55d72075fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b20c34640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d72075f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 0x55d72075fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58264/0 (socket says 172.21.15.100:58264) 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b21435640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 0x55d72075ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52100/0 (socket says 172.21.15.100:52100) 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.113+0000 7f0b1bfff640 1 -- 172.21.15.100:0/182005620 learned_addr learned my addr 172.21.15.100:0/182005620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b1bfff640 1 -- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 msgr2=0x55d72075f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d72075f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b1bfff640 1 -- 172.21.15.100:0/182005620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 msgr2=0x55d72075ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 0x55d72075ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b1bfff640 1 -- 172.21.15.100:0/182005620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c10d710 con 0x55d72074cb70 2024-03-12T05:25:57.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b21435640 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 0x55d72075ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b1bfff640 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 0x55d72075fa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b08036040 tx=0x7f0b08003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a48f6bacf7db96 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0afbfff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0afbfff640 1 -- 172.21.15.100:0/182005620 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08053040 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0afbfff640 1 -- 172.21.15.100:0/182005620 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0b08047040 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7203b37e0 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0afbfff640 1 -- 172.21.15.100:0/182005620 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b0802cbd0 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b25a5f740 10 client.27751 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7206748c0 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.114+0000 7f0afbfff640 1 -- 172.21.15.100:0/182005620 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d7203b37e0 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0afbfff640 1 -- 172.21.15.100:0/182005620 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0b0805f070 con 0x55d72074cb70 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 10 client.27751 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 10 client.27751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:57.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 20 client.27751 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d7207c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 4 client.27751 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d72035f700 con 0x55d720769f10 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b20c34640 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d7207c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0afbfff640 10 client.27751 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b20c34640 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d7207c3b40 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc9d19cf89d50ef server_cookie=3abea9b90bf77a08 in_seq=0 out_seq=0 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0afbfff640 1 -- 172.21.15.100:0/182005620 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d72035f700 con 0x55d720769f10 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0afbfff640 10 client.27751 handle_command_reply: tid=0 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.115+0000 7f0b25a5f740 1 client.27751 shutdown 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 msgr2=0x55d72075fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 0x55d72075fa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b08036040 tx=0x7f0b08003a90 comp rx=0 tx=0).stop 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 shutdown_connections 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d720769f10 0x55d7207c3b40 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d720755750 0x55d72075ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d72074cb70 0x55d72075fa00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0afbfff640 0 client.27751 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 --2- 172.21.15.100:0/182005620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d72074c250 0x55d72075f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 >> 172.21.15.100:0/182005620 conn(0x55d720746980 msgr2=0x55d7207452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 shutdown_connections 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 1 -- 172.21.15.100:0/182005620 wait complete. 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b25a5f740 20 client.27751 trim_cache size 0 max 16384 2024-03-12T05:25:57.117 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:25:57.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0b0c0a3db0 msgr2=0x7f0b0c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0b0c0a3db0 0x7f0b0c0a6270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0ffbc0 tx=0x7f0b0802c670 comp rx=0 tx=0).stop 2024-03-12T05:25:57.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c100260 msgr2=0x7f0b1c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c100260 0x7f0b1c1966d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000e9f0 tx=0x7f0b1000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 shutdown_connections 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0b0c0a3db0 0x7f0b0c0a6270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1c19af90 0x7f0b1c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c105330 0x7f0b1c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 --2- 172.21.15.100:0/2961281991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b1c100260 0x7f0b1c1966d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 >> 172.21.15.100:0/2961281991 conn(0x7f0b1c077180 msgr2=0x7f0b1c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 shutdown_connections 2024-03-12T05:25:57.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:57.116+0000 7f0b22e98640 1 -- 172.21.15.100:0/2961281991 wait complete. 2024-03-12T05:25:57.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27222 40.96 MB/sec warmup 40 sec latency 212.630 ms 2024-03-12T05:25:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:56 smithi100 ceph-mon[32210]: pgmap v432: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 432 op/s 2024-03-12T05:25:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:56 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/650630214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:25:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:56 smithi156 ceph-mon[36270]: pgmap v432: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 432 op/s 2024-03-12T05:25:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:56 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/650630214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:25:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:56 smithi161 ceph-mon[36043]: pgmap v432: 97 pgs: 97 active+clean; 804 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 432 op/s 2024-03-12T05:25:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:56 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/650630214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:25:58.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:25:58.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:25:58.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:25:58.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27940 40.90 MB/sec warmup 41 sec latency 160.126 ms 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 -- 172.21.15.100:0/1222346855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8101d30 msgr2=0x7f36b8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 --2- 172.21.15.100:0/1222346855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8101d30 0x7f36b8105d70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009930 tx=0x7f36a802bc30 comp rx=0 tx=0).stop 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 -- 172.21.15.100:0/1222346855 shutdown_connections 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 --2- 172.21.15.100:0/1222346855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8106450 0x7f36b810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 --2- 172.21.15.100:0/1222346855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8101d30 0x7f36b8105d70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 --2- 172.21.15.100:0/1222346855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8101380 0x7f36b8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.243+0000 7f36bf283640 1 -- 172.21.15.100:0/1222346855 >> 172.21.15.100:0/1222346855 conn(0x7f36b8078540 msgr2=0x7f36b8078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 -- 172.21.15.100:0/1222346855 shutdown_connections 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 -- 172.21.15.100:0/1222346855 wait complete. 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 Processor -- start 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 -- start start 2024-03-12T05:25:58.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8101380 0x7f36b819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 0x7f36b819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 0x7f36b8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8198100 con 0x7f36b8106450 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8198270 con 0x7f36b8101380 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.244+0000 7f36bf283640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8198430 con 0x7f36b8197630 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 0x7f36b819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd01f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8101380 0x7f36b819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 0x7f36b8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 0x7f36b8197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58290/0 (socket says 172.21.15.100:58290) 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 0x7f36b819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52126/0 (socket says 172.21.15.100:52126) 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 -- 172.21.15.100:0/1444673732 learned_addr learned my addr 172.21.15.100:0/1444673732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd01f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8101380 0x7f36b819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44866/0 (socket says 172.21.15.100:44866) 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 -- 172.21.15.100:0/1444673732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8101380 msgr2=0x7f36b819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8101380 0x7f36b819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 -- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 msgr2=0x7f36b819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 0x7f36b819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 -- 172.21.15.100:0/1444673732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a8009560 con 0x7f36b8197630 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 0x7f36b819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bd820640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 0x7f36b8197ab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36b000eb10 tx=0x7f36b000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36ae7fc640 1 -- 172.21.15.100:0/1444673732 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004b20 con 0x7f36b8197630 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b81a43d0 con 0x7f36b8197630 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36ae7fc640 1 -- 172.21.15.100:0/1444673732 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36b0004c80 con 0x7f36b8197630 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36ae7fc640 1 -- 172.21.15.100:0/1444673732 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0010650 con 0x7f36b8197630 2024-03-12T05:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.245+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81a48b0 con 0x7f36b8197630 2024-03-12T05:25:58.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36ae7fc640 1 -- 172.21.15.100:0/1444673732 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f36b0005020 con 0x7f36b8197630 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 Processor -- start 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 -- start start 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a069760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 0x555c3a0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a074d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c39fd7f90 con 0x555c3a069d30 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c39fd54c0 con 0x555c3a0728b0 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c39fd0ca0 con 0x555c3a069380 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bd01f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 0x555c3a0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a069760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58294/0 (socket says 172.21.15.100:58294) 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bd01f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 0x555c3a0722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52132/0 (socket says 172.21.15.100:52132) 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 -- 172.21.15.100:0/1785366786 learned_addr learned my addr 172.21.15.100:0/1785366786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36ae7fc640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36a40a1ba0 0x7f36a40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 -- 172.21.15.100:0/1785366786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 msgr2=0x555c3a074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 -- 172.21.15.100:0/1785366786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 msgr2=0x555c3a0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.248+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 0x555c3a0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36bc81e640 1 -- 172.21.15.100:0/1785366786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c39f930b0 con 0x555c3a069380 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36bd820640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36a40a1ba0 0x7f36a40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36bd01f640 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 0x555c3a0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36ae7fc640 1 -- 172.21.15.100:0/1444673732 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f36b0014070 con 0x7f36b8197630 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a069760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36a8005ec0 tx=0x7f36a8005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77af5aa55414a75a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36a37fe640 1 -- 172.21.15.100:0/1785366786 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a802cbf0 con 0x555c3a069380 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36a37fe640 1 -- 172.21.15.100:0/1785366786 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36a8004510 con 0x555c3a069380 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.249+0000 7f36a37fe640 1 -- 172.21.15.100:0/1785366786 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004df0 con 0x555c3a069380 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1785366786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 msgr2=0x555c3a069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36bd820640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36a40a1ba0 0x7f36a40a4060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36b00cb530 tx=0x7f36b006cb60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a069760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36a8005ec0 tx=0x7f36a8005e50 comp rx=0 tx=0).stop 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1785366786 shutdown_connections 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069d30 0x555c3a0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1785366786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a069760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1785366786 >> 172.21.15.100:0/1785366786 conn(0x555c3a063840 msgr2=0x555c3a063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1785366786 shutdown_connections 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1785366786 wait complete. 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 -- start start 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.250+0000 7f36c1e4c740 1 Processor -- start 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a07c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a069d30 0x555c3a07ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a0728b0 0x555c3a076d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c39f75f40 con 0x555c3a0728b0 2024-03-12T05:25:58.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c39ca44b0 con 0x555c3a069d30 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c39f56670 con 0x555c3a069380 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 10 client.0 _ll_register_callbacks cb 0x555c39fc3f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a07c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a07c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58310/0 (socket says 172.21.15.100:58310) 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 -- 172.21.15.100:0/655036143 learned_addr learned my addr 172.21.15.100:0/655036143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bd01f640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a069d30 0x555c3a07ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a0728b0 0x555c3a076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 -- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a069d30 msgr2=0x555c3a07ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a069d30 0x555c3a07ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 -- 172.21.15.100:0/655036143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a0728b0 msgr2=0x555c3a076d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a0728b0 0x555c3a076d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 -- 172.21.15.100:0/655036143 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b810e830 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36bc81e640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a07c930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009900 tx=0x7f36a8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac1a1aff1b782a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36a0ff9640 1 -- 172.21.15.100:0/655036143 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004440 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36a0ff9640 1 -- 172.21.15.100:0/655036143 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f36a8057030 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.251+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c39faaf40 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36a0ff9640 1 -- 172.21.15.100:0/655036143 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a80035f0 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 10 client.27763 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555c39fcf320 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36a0ff9640 1 -- 172.21.15.100:0/655036143 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c39faaf40 con 0x555c3a069380 2024-03-12T05:25:58.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36a0ff9640 1 -- 172.21.15.100:0/655036143 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f36a80055a0 con 0x555c3a069380 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 10 client.27763 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 10 client.27763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 20 client.27763 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a086e40 0x555c3a089230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 4 client.27763 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c39fc7310 con 0x555c3a086e40 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.252+0000 7f36bd01f640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a086e40 0x555c3a089230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.253+0000 7f36bd01f640 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a086e40 0x555c3a089230 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0282305d589a827 server_cookie=260021df0d5f91bd in_seq=0 out_seq=0 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.253+0000 7f36a0ff9640 10 client.27763 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36a0ff9640 1 -- 172.21.15.100:0/655036143 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c39fc7310 con 0x555c3a086e40 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36a0ff9640 10 client.27763 handle_command_reply: tid=0 2024-03-12T05:25:58.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 client.27763 shutdown 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 msgr2=0x555c3a07c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a07c930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009900 tx=0x7f36a8004af0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 shutdown_connections 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a086e40 0x555c3a089230 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a0728b0 0x555c3a076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36a0ff9640 0 client.27763 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a069d30 0x555c3a07ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/655036143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069380 0x555c3a07c930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 >> 172.21.15.100:0/655036143 conn(0x555c3a063840 msgr2=0x555c3a0657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 shutdown_connections 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 1 -- 172.21.15.100:0/655036143 wait complete. 2024-03-12T05:25:58.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.254+0000 7f36c1e4c740 20 client.27763 trim_cache size 0 max 16384 2024-03-12T05:25:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 Processor -- start 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 -- start start 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a07c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a07c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3a068d00 con 0x555c3a069380 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3a083d60 con 0x555c3a0728b0 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3a083680 con 0x555c3a069d30 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52146/0 (socket says 172.21.15.100:52146) 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36bc81e640 1 -- 172.21.15.100:0/2368417308 learned_addr learned my addr 172.21.15.100:0/2368417308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.272+0000 7f36bd01f640 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a07c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a07c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 -- 172.21.15.100:0/2368417308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 msgr2=0x555c3a07c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a07c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 -- 172.21.15.100:0/2368417308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 msgr2=0x555c3a07c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a07c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 -- 172.21.15.100:0/2368417308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c39f6bc00 con 0x555c3a069380 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36bc81e640 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a076ce0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f36a802ca80 tx=0x7f36a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef354fd76d5b1c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36a2ffd640 1 -- 172.21.15.100:0/2368417308 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004440 con 0x555c3a069380 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36a2ffd640 1 -- 172.21.15.100:0/2368417308 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36a8056020 con 0x555c3a069380 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.273+0000 7f36a2ffd640 1 -- 172.21.15.100:0/2368417308 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8003870 con 0x555c3a069380 2024-03-12T05:25:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 -- 172.21.15.100:0/2368417308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 msgr2=0x555c3a076ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a076ce0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f36a802ca80 tx=0x7f36a8005fd0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 -- 172.21.15.100:0/2368417308 shutdown_connections 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a07c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a07c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/2368417308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a076ce0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 -- 172.21.15.100:0/2368417308 >> 172.21.15.100:0/2368417308 conn(0x555c3a063840 msgr2=0x555c3a0765d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 -- 172.21.15.100:0/2368417308 shutdown_connections 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 -- 172.21.15.100:0/2368417308 wait complete. 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 -- start start 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.274+0000 7f36c1e4c740 1 Processor -- start 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a0d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a0d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a0d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3a0d1ea0 con 0x555c3a069380 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3a0d2010 con 0x555c3a0728b0 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3a0d21d0 con 0x555c3a069d30 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36c1e4c740 10 client.0 _ll_register_callbacks cb 0x7f36b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a0d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a0d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52158/0 (socket says 172.21.15.100:52158) 2024-03-12T05:25:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 -- 172.21.15.100:0/1700752733 learned_addr learned my addr 172.21.15.100:0/1700752733 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bd01f640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a0d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a0d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 -- 172.21.15.100:0/1700752733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 msgr2=0x555c3a0d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a0d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 -- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 msgr2=0x555c3a0d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a0d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 -- 172.21.15.100:0/1700752733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c39fcf320 con 0x555c3a069380 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bd01f640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a0d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36bc81e640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a0d74d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f36a80059d0 tx=0x7f36a8041780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d1153ac2512150 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.275+0000 7f36a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:58.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36a0ff9640 1 -- 172.21.15.100:0/1700752733 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8041a10 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36a0ff9640 1 -- 172.21.15.100:0/1700752733 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f36a8057070 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c39ff6900 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36a0ff9640 1 -- 172.21.15.100:0/1700752733 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a8041b70 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 10 client.18450 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555c39f91910 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36a0ff9640 1 -- 172.21.15.100:0/1700752733 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c39ff6900 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36a0ff9640 1 -- 172.21.15.100:0/1700752733 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f36a8004fb0 con 0x555c3a069380 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 10 client.18450 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 20 client.18450 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a0def80 0x555c3a0e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 4 client.18450 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:25:58.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.276+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555c39cd0860 con 0x555c3a0def80 2024-03-12T05:25:58.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.277+0000 7f36bd01f640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a0def80 0x555c3a0e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.277+0000 7f36bd01f640 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a0def80 0x555c3a0e1370 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08c460bb067da4a server_cookie=a0b5d7c401fc5cb2 in_seq=0 out_seq=0 2024-03-12T05:25:58.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.277+0000 7f36a0ff9640 10 client.18450 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36a0ff9640 1 -- 172.21.15.100:0/1700752733 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c39cd0860 con 0x555c3a0def80 2024-03-12T05:25:58.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36a0ff9640 10 client.18450 handle_command_reply: tid=0 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 client.18450 shutdown 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 msgr2=0x555c3a0d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a0d74d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f36a80059d0 tx=0x7f36a8041780 comp rx=0 tx=0).stop 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 shutdown_connections 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555c3a0def80 0x555c3a0e1370 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c3a0728b0 0x555c3a0d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c3a069d30 0x555c3a0d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36a0ff9640 0 client.18450 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 --2- 172.21.15.100:0/1700752733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c3a069380 0x555c3a0d74d0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 >> 172.21.15.100:0/1700752733 conn(0x555c3a063840 msgr2=0x555c3a062050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 shutdown_connections 2024-03-12T05:25:58.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 1 -- 172.21.15.100:0/1700752733 wait complete. 2024-03-12T05:25:58.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.279+0000 7f36c1e4c740 20 client.18450 trim_cache size 0 max 16384 2024-03-12T05:25:58.281 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:25:58.281 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:25:58.281 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "25cd79a6-8a47-4a5a-8898-d58c97f1475a", 2024-03-12T05:25:58.281 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:25:58.281 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:25:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36a40a1ba0 msgr2=0x7f36a40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36a40a1ba0 0x7f36a40a4060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36b00cb530 tx=0x7f36b006cb60 comp rx=0 tx=0).stop 2024-03-12T05:25:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 msgr2=0x7f36b8197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 0x7f36b8197ab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36b000eb10 tx=0x7f36b000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 shutdown_connections 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f36a40a1ba0 0x7f36a40a4060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36b8197630 0x7f36b8197ab0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8106450 0x7f36b819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 --2- 172.21.15.100:0/1444673732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36b8101380 0x7f36b819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 >> 172.21.15.100:0/1444673732 conn(0x7f36b8078540 msgr2=0x7f36b80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 shutdown_connections 2024-03-12T05:25:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.280+0000 7f36bf283640 1 -- 172.21.15.100:0/1444673732 wait complete. 2024-03-12T05:25:58.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:25:58.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:25:58.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.405+0000 7f844c68f640 1 -- 172.21.15.100:0/250832254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8444106470 msgr2=0x7f844410e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.405+0000 7f844c68f640 1 --2- 172.21.15.100:0/250832254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8444106470 0x7f844410e040 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f843c009cd0 tx=0x7f843c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- 172.21.15.100:0/250832254 shutdown_connections 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 --2- 172.21.15.100:0/250832254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8444106470 0x7f844410e040 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 --2- 172.21.15.100:0/250832254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844406a370 0x7f8444105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 --2- 172.21.15.100:0/250832254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84440699c0 0x7f8444069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- 172.21.15.100:0/250832254 >> 172.21.15.100:0/250832254 conn(0x7f8444077060 msgr2=0x7f8444077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- 172.21.15.100:0/250832254 shutdown_connections 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- 172.21.15.100:0/250832254 wait complete. 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 Processor -- start 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- start start 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 0x7f844419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8444106470 0x7f844419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84441975d0 0x7f8444197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444197fc0 con 0x7f84440699c0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444198130 con 0x7f8444106470 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.406+0000 7f844c68f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84441982f0 con 0x7f84441975d0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 0x7f844419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 0x7f844419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52166/0 (socket says 172.21.15.100:52166) 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 -- 172.21.15.100:0/644521075 learned_addr learned my addr 172.21.15.100:0/644521075 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f8449c2a640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8444106470 0x7f844419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844ac2c640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84441975d0 0x7f8444197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 -- 172.21.15.100:0/644521075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84441975d0 msgr2=0x7f8444197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84441975d0 0x7f8444197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 -- 172.21.15.100:0/644521075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8444106470 msgr2=0x7f844419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8444106470 0x7f844419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 -- 172.21.15.100:0/644521075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009900 con 0x7f84440699c0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844a42b640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 0x7f844419d420 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9f0 tx=0x7f844000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f843b7fe640 1 -- 172.21.15.100:0/644521075 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440009700 con 0x7f84440699c0 2024-03-12T05:25:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f843b7fe640 1 -- 172.21.15.100:0/644521075 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f844000cd80 con 0x7f84440699c0 2024-03-12T05:25:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84441a42e0 con 0x7f84440699c0 2024-03-12T05:25:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f843b7fe640 1 -- 172.21.15.100:0/644521075 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440018920 con 0x7f84440699c0 2024-03-12T05:25:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.407+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84441a47f0 con 0x7f84440699c0 2024-03-12T05:25:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 Processor -- start 2024-03-12T05:25:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f843b7fe640 1 -- 172.21.15.100:0/644521075 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8440018a80 con 0x7f84440699c0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 -- start start 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab3a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab45ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9faaa9280 con 0x55e9fab437b0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9faaa67b0 con 0x55e9fab3a2b0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.410+0000 7f844f25d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9faaa1f90 con 0x55e9fab3abd0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844ac2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab3a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab45ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844ac2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab3a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44926/0 (socket says 172.21.15.100:44926) 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844ac2c640 1 -- 172.21.15.100:0/3134309231 learned_addr learned my addr 172.21.15.100:0/3134309231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f843b7fe640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84300a1ba0 0x7f84300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 -- 172.21.15.100:0/3134309231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 msgr2=0x55e9fab3a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab3a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 -- 172.21.15.100:0/3134309231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 msgr2=0x55e9fab45ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab45ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 -- 172.21.15.100:0/3134309231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9faa60570 con 0x55e9fab3abd0 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844ac2c640 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab3a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f8449c2a640 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab45ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844ac2c640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84300a1ba0 0x7f84300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f843b7fe640 1 -- 172.21.15.100:0/644521075 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8440014070 con 0x7f84440699c0 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f844a42b640 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab43270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f84400cb2d0 tx=0x7f84400c4e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dad620a4d4b1d39 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f8438ff9640 1 -- 172.21.15.100:0/3134309231 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84400d7040 con 0x55e9fab3abd0 2024-03-12T05:25:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f8438ff9640 1 -- 172.21.15.100:0/3134309231 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84400d1080 con 0x55e9fab3abd0 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.411+0000 7f8438ff9640 1 -- 172.21.15.100:0/3134309231 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844009a860 con 0x55e9fab3abd0 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 -- 172.21.15.100:0/3134309231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 msgr2=0x55e9fab43270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab43270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f84400cb2d0 tx=0x7f84400c4e70 comp rx=0 tx=0).stop 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844ac2c640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84300a1ba0 0x7f84300a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f843c004af0 tx=0x7f843c0095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 -- 172.21.15.100:0/3134309231 shutdown_connections 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab45ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab43270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 --2- 172.21.15.100:0/3134309231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab3a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.412+0000 7f844f25d740 1 -- 172.21.15.100:0/3134309231 >> 172.21.15.100:0/3134309231 conn(0x55e9fab34990 msgr2=0x55e9fab34d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.413+0000 7f844f25d740 1 -- 172.21.15.100:0/3134309231 shutdown_connections 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.413+0000 7f844f25d740 1 -- 172.21.15.100:0/3134309231 wait complete. 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.413+0000 7f844f25d740 1 -- start start 2024-03-12T05:25:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.413+0000 7f844f25d740 1 Processor -- start 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab40600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab40d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fa775430 con 0x55e9fab437b0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fa731e00 con 0x55e9fab3a2b0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9faa26fd0 con 0x55e9fab3abd0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844f25d740 10 client.0 _ll_register_callbacks cb 0x55e9faa7cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844a42b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab40600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab40600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58368/0 (socket says 172.21.15.100:58368) 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f8449c2a640 1 -- 172.21.15.100:0/1702028131 learned_addr learned my addr 172.21.15.100:0/1702028131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844ac2c640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab40d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844a42b640 1 -- 172.21.15.100:0/1702028131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 msgr2=0x55e9fab40600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844a42b640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab40600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844a42b640 1 -- 172.21.15.100:0/1702028131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 msgr2=0x55e9fab40d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844a42b640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab40d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f844a42b640 1 -- 172.21.15.100:0/1702028131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84441a47f0 con 0x55e9fab3a2b0 2024-03-12T05:25:58.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.414+0000 7f8449c2a640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab40600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:25:58.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844a42b640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab43270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8440009380 tx=0x7f844001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13c1307491a3f517 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f842dffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:25:58.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f842dffb640 1 -- 172.21.15.100:0/1702028131 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844006e140 con 0x55e9fab3a2b0 2024-03-12T05:25:58.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9fa7de720 con 0x55e9fab3a2b0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f842dffb640 1 -- 172.21.15.100:0/1702028131 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f844009e900 con 0x55e9fab3a2b0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f842dffb640 1 -- 172.21.15.100:0/1702028131 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f844009a3a0 con 0x55e9fab3a2b0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 10 client.27830 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e9faaa0610 con 0x55e9fab3a2b0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f842dffb640 1 -- 172.21.15.100:0/1702028131 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e9fa7de720 con 0x55e9fab3a2b0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f842dffb640 1 -- 172.21.15.100:0/1702028131 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f84400cb550 con 0x55e9fab3a2b0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 10 client.27830 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 20 client.27830 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fab535f0 0x55e9fab5e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 4 client.27830 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.415+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9fa82d890 con 0x55e9fab535f0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.416+0000 7f8449c2a640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fab535f0 0x55e9fab5e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.416+0000 7f842dffb640 10 client.27830 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.416+0000 7f8449c2a640 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fab535f0 0x55e9fab5e280 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4ae506b6dc3a74 server_cookie=5c7e923401684481 in_seq=0 out_seq=0 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f842dffb640 1 -- 172.21.15.100:0/1702028131 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9fa82d890 con 0x55e9fab535f0 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f842dffb640 10 client.27830 handle_command_reply: tid=0 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 client.27830 shutdown 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 msgr2=0x55e9fab43270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab43270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8440009380 tx=0x7f844001f000 comp rx=0 tx=0).stop 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 shutdown_connections 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fab535f0 0x55e9fab5e280 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab437b0 0x55e9fab40d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f842dffb640 0 client.27830 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab3abd0 0x55e9fab40600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 --2- 172.21.15.100:0/1702028131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab43270 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 >> 172.21.15.100:0/1702028131 conn(0x55e9fab34990 msgr2=0x55e9fab36730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 shutdown_connections 2024-03-12T05:25:58.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 1 -- 172.21.15.100:0/1702028131 wait complete. 2024-03-12T05:25:58.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.417+0000 7f844f25d740 20 client.27830 trim_cache size 0 max 16384 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 Processor -- start 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 -- start start 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fab30120 con 0x55e9fab3abd0 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fab633d0 con 0x55e9fab3a2b0 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f844f25d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fab35790 con 0x55e9fab437b0 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab42d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52186/0 (socket says 172.21.15.100:52186) 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.437+0000 7f8449c2a640 1 -- 172.21.15.100:0/1569361114 learned_addr learned my addr 172.21.15.100:0/1569361114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f844a42b640 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f844ac2c640 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f8449c2a640 1 -- 172.21.15.100:0/1569361114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 msgr2=0x55e9fab43270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f8449c2a640 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab43270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f8449c2a640 1 -- 172.21.15.100:0/1569361114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 msgr2=0x55e9fab405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f8449c2a640 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f8449c2a640 1 -- 172.21.15.100:0/1569361114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9faa3cac0 con 0x55e9fab3abd0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f8449c2a640 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab42d30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55e9fab3fe60 tx=0x7f8434005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f764e3fc269ec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f842ffff640 1 -- 172.21.15.100:0/1569361114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434017070 con 0x55e9fab3abd0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f842ffff640 1 -- 172.21.15.100:0/1569361114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8434018040 con 0x55e9fab3abd0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.438+0000 7f842ffff640 1 -- 172.21.15.100:0/1569361114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340035f0 con 0x55e9fab3abd0 2024-03-12T05:25:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 -- 172.21.15.100:0/1569361114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 msgr2=0x55e9fab42d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab42d30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55e9fab3fe60 tx=0x7f8434005f90 comp rx=0 tx=0).stop 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 -- 172.21.15.100:0/1569361114 shutdown_connections 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab42d30 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 --2- 172.21.15.100:0/1569361114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 -- 172.21.15.100:0/1569361114 >> 172.21.15.100:0/1569361114 conn(0x55e9fab34990 msgr2=0x55e9fab3fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 -- 172.21.15.100:0/1569361114 shutdown_connections 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 -- 172.21.15.100:0/1569361114 wait complete. 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 -- start start 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.439+0000 7f844f25d740 1 Processor -- start 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab4a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fab498b0 con 0x55e9fab3abd0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fab49a20 con 0x55e9fab3a2b0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9fabb22b0 con 0x55e9fab437b0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 10 client.0 _ll_register_callbacks cb 0x7f84400d13e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab48c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52198/0 (socket says 172.21.15.100:52198) 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 -- 172.21.15.100:0/3448540409 learned_addr learned my addr 172.21.15.100:0/3448540409 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844a42b640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab4a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844ac2c640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 -- 172.21.15.100:0/3448540409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 msgr2=0x55e9fab491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 -- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 msgr2=0x55e9fab4a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:25:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab4a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 -- 172.21.15.100:0/3448540409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434009e50 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f8449c2a640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab48c90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x55e9fab41bd0 tx=0x7f8434012250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbfe2de56901ffe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f842dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f842dffb640 1 -- 172.21.15.100:0/3448540409 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434017070 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f842dffb640 1 -- 172.21.15.100:0/3448540409 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8434005000 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9fa7a17e0 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.440+0000 7f842dffb640 1 -- 172.21.15.100:0/3448540409 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8434005200 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 10 client.18468 fetch_fsmap learned FSMap version 27 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e9faa626a0 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f842dffb640 1 -- 172.21.15.100:0/3448540409 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e9fa7a17e0 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f842dffb640 1 -- 172.21.15.100:0/3448540409 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8434039070 con 0x55e9fab3abd0 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 10 client.18468 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:25:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 20 client.18468 populate_metadata read hostname 'smithi100' 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fabb3ac0 0x55e9fabb5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 4 client.18468 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e9fa7904e0 con 0x55e9fabb3ac0 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844a42b640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fabb3ac0 0x55e9fabb5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f842dffb640 10 client.18468 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.441+0000 7f844a42b640 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fabb3ac0 0x55e9fabb5ee0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7def7e3774d2abf7 server_cookie=bc03df455461e8a3 in_seq=0 out_seq=0 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f842dffb640 1 -- 172.21.15.100:0/3448540409 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e9fa7904e0 con 0x55e9fabb3ac0 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f842dffb640 10 client.18468 handle_command_reply: tid=0 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 client.18468 shutdown 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 msgr2=0x55e9fab48c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab48c90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x55e9fab41bd0 tx=0x7f8434012250 comp rx=0 tx=0).stop 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 shutdown_connections 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e9fabb3ac0 0x55e9fabb5ee0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9fab437b0 0x55e9fab491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9fab3abd0 0x55e9fab48c90 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f842dffb640 0 client.18468 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 --2- 172.21.15.100:0/3448540409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9fab3a2b0 0x55e9fab4a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 >> 172.21.15.100:0/3448540409 conn(0x55e9fab34990 msgr2=0x55e9fab3eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 shutdown_connections 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 1 -- 172.21.15.100:0/3448540409 wait complete. 2024-03-12T05:25:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.442+0000 7f844f25d740 20 client.18468 trim_cache size 0 max 16384 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: "25cd79a6-8a47-4a5a-8898-d58c97f1475a": { 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "25cd79a6-8a47-4a5a-8898-d58c97f1475a", 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:25:58.444 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:25:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84300a1ba0 msgr2=0x7f84300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84300a1ba0 0x7f84300a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f843c004af0 tx=0x7f843c0095c0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 msgr2=0x7f844419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:25:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 0x7f844419d420 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9f0 tx=0x7f844000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 shutdown_connections 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84300a1ba0 0x7f84300a4060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84441975d0 0x7f8444197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8444106470 0x7f844419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 --2- 172.21.15.100:0/644521075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84440699c0 0x7f844419d420 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 >> 172.21.15.100:0/644521075 conn(0x7f8444077060 msgr2=0x7f84440fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 shutdown_connections 2024-03-12T05:25:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:25:58.443+0000 7f844c68f640 1 -- 172.21.15.100:0/644521075 wait complete. 2024-03-12T05:25:58.451 INFO:tasks.cephfs.filesystem:scrub status for tag:25cd79a6-8a47-4a5a-8898-d58c97f1475a - {'path': '/', 'tag': '25cd79a6-8a47-4a5a-8898-d58c97f1475a', 'options': 'recursive,force'} 2024-03-12T05:25:59.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28523 40.53 MB/sec warmup 42 sec latency 224.725 ms 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: pgmap v433: 97 pgs: 97 active+clean; 831 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 25 MiB/s wr, 710 op/s 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:25:59.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:58 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: pgmap v433: 97 pgs: 97 active+clean; 831 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 25 MiB/s wr, 710 op/s 2024-03-12T05:25:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:25:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:59.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:59.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:59.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:25:59.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:58 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: pgmap v433: 97 pgs: 97 active+clean; 831 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 25 MiB/s wr, 710 op/s 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:25:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:58 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:00.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29129 40.33 MB/sec warmup 43 sec latency 262.388 ms 2024-03-12T05:26:00.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:59 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:00.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:25:59 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:00.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:59 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:00.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:25:59 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:59 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:25:59 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:01.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29533 40.14 MB/sec warmup 44 sec latency 412.484 ms 2024-03-12T05:26:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:00 smithi100 ceph-mon[32210]: pgmap v434: 97 pgs: 97 active+clean; 831 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 23 MiB/s wr, 505 op/s 2024-03-12T05:26:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:00 smithi156 ceph-mon[36270]: pgmap v434: 97 pgs: 97 active+clean; 831 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 23 MiB/s wr, 505 op/s 2024-03-12T05:26:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:00 smithi161 ceph-mon[36043]: pgmap v434: 97 pgs: 97 active+clean; 831 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 23 MiB/s wr, 505 op/s 2024-03-12T05:26:02.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29911 39.76 MB/sec warmup 45 sec latency 430.546 ms 2024-03-12T05:26:03.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30372 39.57 MB/sec warmup 46 sec latency 77.113 ms 2024-03-12T05:26:03.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:03 smithi100 ceph-mon[32210]: pgmap v435: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 28 MiB/s wr, 841 op/s 2024-03-12T05:26:03.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:03 smithi156 ceph-mon[36270]: pgmap v435: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 28 MiB/s wr, 841 op/s 2024-03-12T05:26:03.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:03 smithi161 ceph-mon[36043]: pgmap v435: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 28 MiB/s wr, 841 op/s 2024-03-12T05:26:04.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30782 39.04 MB/sec warmup 47 sec latency 65.465 ms 2024-03-12T05:26:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:04 smithi100 ceph-mon[32210]: pgmap v436: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 613 op/s 2024-03-12T05:26:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:04 smithi156 ceph-mon[36270]: pgmap v436: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 613 op/s 2024-03-12T05:26:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:04 smithi161 ceph-mon[36043]: pgmap v436: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 613 op/s 2024-03-12T05:26:05.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31268 38.72 MB/sec warmup 48 sec latency 207.412 ms 2024-03-12T05:26:06.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31789 38.38 MB/sec warmup 49 sec latency 337.729 ms 2024-03-12T05:26:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:06 smithi100 ceph-mon[32210]: osdmap e104: 12 total, 12 up, 12 in 2024-03-12T05:26:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:06 smithi100 ceph-mon[32210]: pgmap v438: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 736 op/s 2024-03-12T05:26:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:06 smithi156 ceph-mon[36270]: osdmap e104: 12 total, 12 up, 12 in 2024-03-12T05:26:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:06 smithi156 ceph-mon[36270]: pgmap v438: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 736 op/s 2024-03-12T05:26:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:06 smithi161 ceph-mon[36043]: osdmap e104: 12 total, 12 up, 12 in 2024-03-12T05:26:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:06 smithi161 ceph-mon[36043]: pgmap v438: 97 pgs: 97 active+clean; 846 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 736 op/s 2024-03-12T05:26:07.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32430 38.43 MB/sec warmup 50 sec latency 111.503 ms 2024-03-12T05:26:08.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33231 38.43 MB/sec warmup 51 sec latency 166.165 ms 2024-03-12T05:26:09.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33902 38.44 MB/sec warmup 52 sec latency 180.343 ms 2024-03-12T05:26:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:08 smithi100 ceph-mon[32210]: pgmap v439: 97 pgs: 97 active+clean; 867 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 881 op/s 2024-03-12T05:26:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:08 smithi156 ceph-mon[36270]: pgmap v439: 97 pgs: 97 active+clean; 867 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 881 op/s 2024-03-12T05:26:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:08 smithi161 ceph-mon[36043]: pgmap v439: 97 pgs: 97 active+clean; 867 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 881 op/s 2024-03-12T05:26:10.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34531 38.51 MB/sec warmup 53 sec latency 173.045 ms 2024-03-12T05:26:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:09 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:26:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:09 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:26:10.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:09 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:26:11.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34973 38.09 MB/sec warmup 54 sec latency 335.909 ms 2024-03-12T05:26:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:10 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:26:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:10 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:26:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:10 smithi100 ceph-mon[32210]: pgmap v440: 97 pgs: 97 active+clean; 867 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 882 op/s 2024-03-12T05:26:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:10 smithi100 ceph-mon[32210]: osdmap e105: 12 total, 12 up, 12 in 2024-03-12T05:26:11.370 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:10 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:26:11.370 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:10 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:26:11.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:10 smithi161 ceph-mon[36043]: pgmap v440: 97 pgs: 97 active+clean; 867 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 882 op/s 2024-03-12T05:26:11.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:10 smithi161 ceph-mon[36043]: osdmap e105: 12 total, 12 up, 12 in 2024-03-12T05:26:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:10 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:26:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:10 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:26:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:10 smithi156 ceph-mon[36270]: pgmap v440: 97 pgs: 97 active+clean; 867 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 882 op/s 2024-03-12T05:26:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:10 smithi156 ceph-mon[36270]: osdmap e105: 12 total, 12 up, 12 in 2024-03-12T05:26:12.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35661 38.15 MB/sec warmup 55 sec latency 139.877 ms 2024-03-12T05:26:13.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36250 38.03 MB/sec warmup 56 sec latency 77.442 ms 2024-03-12T05:26:13.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:12 smithi100 ceph-mon[32210]: pgmap v442: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 996 op/s 2024-03-12T05:26:13.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:12 smithi156 ceph-mon[36270]: pgmap v442: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 996 op/s 2024-03-12T05:26:13.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:12 smithi161 ceph-mon[36043]: pgmap v442: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 996 op/s 2024-03-12T05:26:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36818 37.78 MB/sec warmup 57 sec latency 84.764 ms 2024-03-12T05:26:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37779 37.96 MB/sec warmup 58 sec latency 67.625 ms 2024-03-12T05:26:15.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:14 smithi100 ceph-mon[32210]: pgmap v443: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 898 op/s 2024-03-12T05:26:15.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:14 smithi156 ceph-mon[36270]: pgmap v443: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 898 op/s 2024-03-12T05:26:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:14 smithi161 ceph-mon[36043]: pgmap v443: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 898 op/s 2024-03-12T05:26:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38208 37.84 MB/sec warmup 59 sec latency 263.225 ms 2024-03-12T05:26:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38749 37.77 MB/sec warmup 60 sec latency 126.863 ms 2024-03-12T05:26:17.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:16 smithi100 ceph-mon[32210]: pgmap v444: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 797 op/s 2024-03-12T05:26:17.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:16 smithi156 ceph-mon[36270]: pgmap v444: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 797 op/s 2024-03-12T05:26:17.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:16 smithi161 ceph-mon[36043]: pgmap v444: 97 pgs: 97 active+clean; 885 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 797 op/s 2024-03-12T05:26:18.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39370 37.71 MB/sec warmup 61 sec latency 45.113 ms 2024-03-12T05:26:19.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39889 37.54 MB/sec warmup 62 sec latency 326.797 ms 2024-03-12T05:26:19.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:19 smithi100 ceph-mon[32210]: pgmap v445: 97 pgs: 97 active+clean; 913 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-03-12T05:26:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:19 smithi156 ceph-mon[36270]: pgmap v445: 97 pgs: 97 active+clean; 913 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-03-12T05:26:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:19 smithi161 ceph-mon[36043]: pgmap v445: 97 pgs: 97 active+clean; 913 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-03-12T05:26:20.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40400 37.25 MB/sec warmup 63 sec latency 180.251 ms 2024-03-12T05:26:21.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40870 37.04 MB/sec warmup 64 sec latency 416.280 ms 2024-03-12T05:26:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:21 smithi100 ceph-mon[32210]: pgmap v446: 97 pgs: 97 active+clean; 913 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-03-12T05:26:21.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:21 smithi156 ceph-mon[36270]: pgmap v446: 97 pgs: 97 active+clean; 913 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-03-12T05:26:21.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:21 smithi161 ceph-mon[36043]: pgmap v446: 97 pgs: 97 active+clean; 913 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-03-12T05:26:22.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41629 37.16 MB/sec warmup 65 sec latency 165.311 ms 2024-03-12T05:26:22.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:22 smithi100 ceph-mon[32210]: pgmap v447: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 884 op/s 2024-03-12T05:26:22.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:22 smithi156 ceph-mon[36270]: pgmap v447: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 884 op/s 2024-03-12T05:26:22.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:22 smithi161 ceph-mon[36043]: pgmap v447: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 884 op/s 2024-03-12T05:26:23.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42259 37.13 MB/sec warmup 66 sec latency 192.395 ms 2024-03-12T05:26:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42913 37.14 MB/sec warmup 67 sec latency 67.193 ms 2024-03-12T05:26:25.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43698 37.24 MB/sec warmup 68 sec latency 73.525 ms 2024-03-12T05:26:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:24 smithi100 ceph-mon[32210]: pgmap v448: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:26:25.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:24 smithi156 ceph-mon[36270]: pgmap v448: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:26:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:24 smithi161 ceph-mon[36043]: pgmap v448: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:26:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44064 37.10 MB/sec warmup 69 sec latency 302.433 ms 2024-03-12T05:26:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44452 36.74 MB/sec warmup 70 sec latency 111.473 ms 2024-03-12T05:26:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:26 smithi100 ceph-mon[32210]: pgmap v449: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:26:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:26 smithi156 ceph-mon[36270]: pgmap v449: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:26:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:26 smithi161 ceph-mon[36043]: pgmap v449: 97 pgs: 97 active+clean; 955 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:26:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45093 36.59 MB/sec warmup 71 sec latency 82.691 ms 2024-03-12T05:26:28.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:26:28.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:26:28.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.598+0000 7f2ca1285640 1 -- 172.21.15.100:0/3325861898 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90005220 con 0x7f2c9c108390 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.598+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3325861898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108390 msgr2=0x7f2c9c108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3325861898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108390 0x7f2c9c108770 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009900 tx=0x7f2c9002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3325861898 shutdown_connections 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3325861898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c1142d0 0x7f2c9c1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3325861898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c108cb0 0x7f2c9c113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3325861898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108390 0x7f2c9c108770 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3325861898 >> 172.21.15.100:0/3325861898 conn(0x7f2c9c06c270 msgr2=0x7f2c9c06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3325861898 shutdown_connections 2024-03-12T05:26:28.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.599+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3325861898 wait complete. 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 Processor -- start 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 -- start start 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 0x7f2c9c1135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108cb0 0x7f2c9c113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1142d0 0x7f2c9c10b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c10b6a0 con 0x7f2c9c108cb0 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c10b810 con 0x7f2c9c108390 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca44eb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c10b9d0 con 0x7f2c9c1142d0 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 0x7f2c9c1135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 0x7f2c9c1135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48134/0 (socket says 172.21.15.100:48134) 2024-03-12T05:26:28.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca2287640 1 -- 172.21.15.100:0/3608620577 learned_addr learned my addr 172.21.15.100:0/3608620577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.600+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108cb0 0x7f2c9c113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca2287640 1 -- 172.21.15.100:0/3608620577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1142d0 msgr2=0x7f2c9c10b000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca2287640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1142d0 0x7f2c9c10b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca2287640 1 -- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108cb0 msgr2=0x7f2c9c113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca2287640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108cb0 0x7f2c9c113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca2287640 1 -- 172.21.15.100:0/3608620577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94009840 con 0x7f2c9c108390 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108cb0 0x7f2c9c113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca2287640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 0x7f2c9c1135a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c90000c00 tx=0x7f2c90004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2c8f7fe640 1 -- 172.21.15.100:0/3608620577 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9002c8e0 con 0x7f2c9c108390 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2c8f7fe640 1 -- 172.21.15.100:0/3608620577 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c90005000 con 0x7f2c9c108390 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c90009560 con 0x7f2c9c108390 2024-03-12T05:26:28.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2c8f7fe640 1 -- 172.21.15.100:0/3608620577 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90003ab0 con 0x7f2c9c108390 2024-03-12T05:26:28.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.601+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c10f0d0 con 0x7f2c9c108390 2024-03-12T05:26:28.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.604+0000 7f2c8f7fe640 1 -- 172.21.15.100:0/3608620577 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2c90005170 con 0x7f2c9c108390 2024-03-12T05:26:28.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.605+0000 7f2ca719a740 1 Processor -- start 2024-03-12T05:26:28.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.605+0000 7f2ca719a740 1 -- start start 2024-03-12T05:26:28.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2c8f7fe640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1cf0 0x7f2c880a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0bd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1cf0 0x7f2c880a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2c8f7fe640 1 -- 172.21.15.100:0/3608620577 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c900efa00 con 0x7f2c9c108390 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0bd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0c67b0 0x560adb0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca719a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb02c280 con 0x560adb0bd2b0 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca719a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb0297b0 con 0x560adb0c67b0 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca719a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb024f90 con 0x560adb0bdbd0 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0bd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43380/0 (socket says 172.21.15.100:43380) 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca2287640 1 -- 172.21.15.100:0/1769693778 learned_addr learned my addr 172.21.15.100:0/1769693778 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:28.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.606+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0c67b0 0x560adb0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1cf0 0x7f2c880a41b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005fd0 tx=0x7f2c94009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2287640 1 -- 172.21.15.100:0/1769693778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 msgr2=0x560adb0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2287640 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2287640 1 -- 172.21.15.100:0/1769693778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0c67b0 msgr2=0x560adb0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2287640 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0c67b0 0x560adb0c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2287640 1 -- 172.21.15.100:0/1769693778 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560adafe3570 con 0x560adb0bd2b0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0c67b0 0x560adb0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2ca2287640 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0bd690 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2c900c4910 tx=0x7f2c9003d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd234ab8fb95917 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2c8cbfe640 1 -- 172.21.15.100:0/1769693778 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900c07f0 con 0x560adb0bd2b0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2c8cbfe640 1 -- 172.21.15.100:0/1769693778 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c900c4e40 con 0x560adb0bd2b0 2024-03-12T05:26:28.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.607+0000 7f2c8cbfe640 1 -- 172.21.15.100:0/1769693778 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900056e0 con 0x560adb0bd2b0 2024-03-12T05:26:28.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 -- 172.21.15.100:0/1769693778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 msgr2=0x560adb0bd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0bd690 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2c900c4910 tx=0x7f2c9003d3b0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 -- 172.21.15.100:0/1769693778 shutdown_connections 2024-03-12T05:26:28.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0c67b0 0x560adb0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 --2- 172.21.15.100:0/1769693778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0bd690 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.608+0000 7f2ca719a740 1 -- 172.21.15.100:0/1769693778 >> 172.21.15.100:0/1769693778 conn(0x560adb0b7990 msgr2=0x560adb0b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.609+0000 7f2ca719a740 1 -- 172.21.15.100:0/1769693778 shutdown_connections 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.609+0000 7f2ca719a740 1 -- 172.21.15.100:0/1769693778 wait complete. 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.609+0000 7f2ca719a740 1 -- start start 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.609+0000 7f2ca719a740 1 Processor -- start 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bd2b0 0x560adb0c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adacf8430 con 0x560adb0c67b0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adacb4e00 con 0x560adb0bd2b0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adafa9fd0 con 0x560adb0bdbd0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bd2b0 0x560adb0c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca719a740 10 client.0 _ll_register_callbacks cb 0x560adaffffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bd2b0 0x560adb0c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48142/0 (socket says 172.21.15.100:48142) 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca2287640 1 -- 172.21.15.100:0/3723223600 learned_addr learned my addr 172.21.15.100:0/3723223600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca1a86640 1 -- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bd2b0 msgr2=0x560adb0c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bd2b0 0x560adb0c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca1a86640 1 -- 172.21.15.100:0/3723223600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 msgr2=0x560adb0c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca1a86640 1 -- 172.21.15.100:0/3723223600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c10f0d0 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.610+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c3b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c1a7120 tx=0x7f2c98009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c49058a2af2ef35 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2c85ffb640 1 -- 172.21.15.100:0/3723223600 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98011c40 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560adad247e0 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2c85ffb640 1 -- 172.21.15.100:0/3723223600 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c98011da0 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2c85ffb640 1 -- 172.21.15.100:0/3723223600 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c98005020 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2ca719a740 10 client.27796 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560adb023610 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2c85ffb640 1 -- 172.21.15.100:0/3723223600 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c98028080 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.611+0000 7f2c85ffb640 1 -- 172.21.15.100:0/3723223600 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2c9801d400 con 0x560adb0bdbd0 2024-03-12T05:26:28.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca719a740 10 client.27796 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca719a740 10 client.27796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca719a740 20 client.27796 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb0d6840 0x560adb0d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca719a740 4 client.27796 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560adad61720 con 0x560adb0d6840 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca2287640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb0d6840 0x560adb0d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2c85ffb640 10 client.27796 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.612+0000 7f2ca2287640 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb0d6840 0x560adb0d8c60 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3dc27c2462073d server_cookie=a5d18375a4d7d7be in_seq=0 out_seq=0 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.613+0000 7f2c85ffb640 1 -- 172.21.15.100:0/3723223600 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560adad61720 con 0x560adb0d6840 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.613+0000 7f2c85ffb640 10 client.27796 handle_command_reply: tid=0 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.613+0000 7f2ca719a740 1 client.27796 shutdown 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 msgr2=0x560adb0c3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c3b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c1a7120 tx=0x7f2c98009f90 comp rx=0 tx=0).stop 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 shutdown_connections 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb0d6840 0x560adb0d8c60 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2c85ffb640 0 client.27796 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bdbd0 0x560adb0c3b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3723223600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bd2b0 0x560adb0c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 >> 172.21.15.100:0/3723223600 conn(0x560adb0b7990 msgr2=0x560adb0b9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 shutdown_connections 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 1 -- 172.21.15.100:0/3723223600 wait complete. 2024-03-12T05:26:28.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.614+0000 7f2ca719a740 20 client.27796 trim_cache size 0 max 16384 2024-03-12T05:26:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.634+0000 7f2ca719a740 1 Processor -- start 2024-03-12T05:26:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.634+0000 7f2ca719a740 1 -- start start 2024-03-12T05:26:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0c67b0 0x560adb0c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca719a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb0d2bb0 con 0x560adb0bd2b0 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca719a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb0b8790 con 0x560adb0bdbd0 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca719a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb0c8f20 con 0x560adb0c67b0 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca2a88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0c67b0 0x560adb0c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca2a88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0c67b0 0x560adb0c08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40984/0 (socket says 172.21.15.100:40984) 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48144/0 (socket says 172.21.15.100:48144) 2024-03-12T05:26:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca2a88640 1 -- 172.21.15.100:0/3468906685 learned_addr learned my addr 172.21.15.100:0/3468906685 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 -- 172.21.15.100:0/3468906685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0c67b0 msgr2=0x560adb0c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0c67b0 0x560adb0c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 -- 172.21.15.100:0/3468906685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 msgr2=0x560adb0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.635+0000 7f2ca1a86640 1 -- 172.21.15.100:0/3468906685 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560adafbfac0 con 0x560adb0bdbd0 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.636+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0c3ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c9801d830 tx=0x7f2c98009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d6c4b82cc00055 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.636+0000 7f2c87fff640 1 -- 172.21.15.100:0/3468906685 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9801d400 con 0x560adb0bdbd0 2024-03-12T05:26:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.636+0000 7f2c87fff640 1 -- 172.21.15.100:0/3468906685 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560adad247e0 con 0x560adb0bdbd0 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.636+0000 7f2c87fff640 1 -- 172.21.15.100:0/3468906685 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004660 con 0x560adb0bdbd0 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 -- 172.21.15.100:0/3468906685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 msgr2=0x560adb0c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0c3ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c9801d830 tx=0x7f2c98009eb0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 -- 172.21.15.100:0/3468906685 shutdown_connections 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0c67b0 0x560adb0c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0c3ee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 --2- 172.21.15.100:0/3468906685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0bd2b0 0x560adb0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 -- 172.21.15.100:0/3468906685 >> 172.21.15.100:0/3468906685 conn(0x560adb0b7990 msgr2=0x560adb0b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 -- 172.21.15.100:0/3468906685 shutdown_connections 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.637+0000 7f2ca719a740 1 -- 172.21.15.100:0/3468906685 wait complete. 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 -- start start 2024-03-12T05:26:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 Processor -- start 2024-03-12T05:26:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 0x560adb0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb0cc8e0 con 0x560adb0c67b0 2024-03-12T05:26:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb0cca50 con 0x560adb0bdbd0 2024-03-12T05:26:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560adb12cd50 con 0x560adb0bd2b0 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca719a740 10 client.0 _ll_register_callbacks cb 0x7f2c900b8220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 0x560adb0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 0x560adb0cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40996/0 (socket says 172.21.15.100:40996) 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 -- 172.21.15.100:0/680034567 learned_addr learned my addr 172.21.15.100:0/680034567 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2a88640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 -- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 msgr2=0x560adb0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 -- 172.21.15.100:0/680034567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 msgr2=0x560adb0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.638+0000 7f2ca2287640 1 -- 172.21.15.100:0/680034567 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560adb023610 con 0x560adb0bd2b0 2024-03-12T05:26:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca2287640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 0x560adb0cd870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c900bc020 tx=0x7f2c900bc820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e29bf93151a3bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2c85ffb640 1 -- 172.21.15.100:0/680034567 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900c07f0 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c900c4e40 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2c85ffb640 1 -- 172.21.15.100:0/680034567 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c900b8d10 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2c85ffb640 1 -- 172.21.15.100:0/680034567 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c900ed330 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 10 client.27802 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560adafe56a0 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2c85ffb640 1 -- 172.21.15.100:0/680034567 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c900c4e40 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2c85ffb640 1 -- 172.21.15.100:0/680034567 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2c9003c070 con 0x560adb0bd2b0 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 10 client.27802 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 10 client.27802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 20 client.27802 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb12e1e0 0x560adb130600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 4 client.27802 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.639+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560adadb0890 con 0x560adb12e1e0 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.640+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb12e1e0 0x560adb130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.640+0000 7f2ca1a86640 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb12e1e0 0x560adb130600 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d00e0ec3593ca2 server_cookie=dc13a8072caddae7 in_seq=0 out_seq=0 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.640+0000 7f2c85ffb640 10 client.27802 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2c85ffb640 1 -- 172.21.15.100:0/680034567 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560adadb0890 con 0x560adb12e1e0 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2c85ffb640 10 client.27802 handle_command_reply: tid=0 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 client.27802 shutdown 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 msgr2=0x560adb0cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 0x560adb0cd870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c900bc020 tx=0x7f2c900bc820 comp rx=0 tx=0).stop 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 shutdown_connections 2024-03-12T05:26:28.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560adb12e1e0 0x560adb130600 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560adb0c67b0 0x560adb0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560adb0bdbd0 0x560adb0cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 --2- 172.21.15.100:0/680034567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560adb0bd2b0 0x560adb0cd870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2c85ffb640 0 client.27802 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 >> 172.21.15.100:0/680034567 conn(0x560adb0b7990 msgr2=0x560adb0c1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 shutdown_connections 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 1 -- 172.21.15.100:0/680034567 wait complete. 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.641+0000 7f2ca719a740 20 client.27802 trim_cache size 0 max 16384 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:26:28.640 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:26:28.641 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:26:28.641 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:26:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.642+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1cf0 msgr2=0x7f2c880a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.642+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1cf0 0x7f2c880a41b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005fd0 tx=0x7f2c94009500 comp rx=0 tx=0).stop 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.642+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 msgr2=0x7f2c9c1135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.642+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 0x7f2c9c1135a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c90000c00 tx=0x7f2c90004270 comp rx=0 tx=0).stop 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 shutdown_connections 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1cf0 0x7f2c880a41b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1142d0 0x7f2c9c10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c108cb0 0x7f2c9c113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 --2- 172.21.15.100:0/3608620577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c108390 0x7f2c9c1135a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 >> 172.21.15.100:0/3608620577 conn(0x7f2c9c06c270 msgr2=0x7f2c9c109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 shutdown_connections 2024-03-12T05:26:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.643+0000 7f2ca44eb640 1 -- 172.21.15.100:0/3608620577 wait complete. 2024-03-12T05:26:28.651 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:26:28.651 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/1471230637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08101990 msgr2=0x7f4c08105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/1471230637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08101990 0x7f4c08105e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009900 tx=0x7f4bf802bc30 comp rx=0 tx=0).stop 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/1471230637 shutdown_connections 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/1471230637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c08106510 0x7f4c0810cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/1471230637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08101990 0x7f4c08105e30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/1471230637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c08100fe0 0x7f4c081013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/1471230637 >> 172.21.15.100:0/1471230637 conn(0x7f4c080753a0 msgr2=0x7f4c080757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.802+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/1471230637 shutdown_connections 2024-03-12T05:26:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.803+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/1471230637 wait complete. 2024-03-12T05:26:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 Processor -- start 2024-03-12T05:26:28.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 -- start start 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 0x7f4c0810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c08106510 0x7f4c0810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0810e4d0 0x7f4c081a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081a5f60 con 0x7f4c08106510 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081a60d0 con 0x7f4c0810e4d0 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0fbbc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081a6290 con 0x7f4c08100fe0 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0d958640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 0x7f4c0810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0e159640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0810e4d0 0x7f4c081a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0d958640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 0x7f4c0810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41002/0 (socket says 172.21.15.100:41002) 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0d958640 1 -- 172.21.15.100:0/2109641407 learned_addr learned my addr 172.21.15.100:0/2109641407 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:28.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.804+0000 7f4c0d157640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c08106510 0x7f4c0810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.805+0000 7f4c0d958640 1 -- 172.21.15.100:0/2109641407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0810e4d0 msgr2=0x7f4c081a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.805+0000 7f4c0d958640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0810e4d0 0x7f4c081a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.805+0000 7f4c0d958640 1 -- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c08106510 msgr2=0x7f4c0810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:28.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.805+0000 7f4c0d958640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c08106510 0x7f4c0810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:28.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.805+0000 7f4c0d958640 1 -- 172.21.15.100:0/2109641407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009560 con 0x7f4c08100fe0 2024-03-12T05:26:28.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.805+0000 7f4c0d958640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 0x7f4c0810da50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400e9f0 tx=0x7f4c0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.807+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400cd80 con 0x7f4c08100fe0 2024-03-12T05:26:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.807+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c04010910 con 0x7f4c08100fe0 2024-03-12T05:26:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.807+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04018950 con 0x7f4c08100fe0 2024-03-12T05:26:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.807+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c081a6570 con 0x7f4c08100fe0 2024-03-12T05:26:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.808+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c081a6a00 con 0x7f4c08100fe0 2024-03-12T05:26:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.810+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c08067cd0 con 0x7f4c08100fe0 2024-03-12T05:26:28.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.812+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4c04010a80 con 0x7f4c08100fe0 2024-03-12T05:26:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.812+0000 7f4bfeffd640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bf40a1da0 0x7f4bf40a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.813+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4c04014070 con 0x7f4c08100fe0 2024-03-12T05:26:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.813+0000 7f4c0d157640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bf40a1da0 0x7f4bf40a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.816+0000 7f4c0d157640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bf40a1da0 0x7f4bf40a4260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4bf802c240 tx=0x7f4bf8005a20 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:28.819+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c04096a50 con 0x7f4c08100fe0 2024-03-12T05:26:29.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.164+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c081084e0 con 0x7f4c08100fe0 2024-03-12T05:26:29.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.165+0000 7f4bfeffd640 1 -- 172.21.15.100:0/2109641407 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f4c0409a110 con 0x7f4c08100fe0 2024-03-12T05:26:29.159 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:26:29.160 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:25:21.243438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:26:29.160 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-03-12T05:26:29.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.166+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bf40a1da0 msgr2=0x7f4bf40a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.166+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bf40a1da0 0x7f4bf40a4260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4bf802c240 tx=0x7f4bf8005a20 comp rx=0 tx=0).stop 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 msgr2=0x7f4c0810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 0x7f4c0810da50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400e9f0 tx=0x7f4c0400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 shutdown_connections 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bf40a1da0 0x7f4bf40a4260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0810e4d0 0x7f4c081a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c08106510 0x7f4c0810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 --2- 172.21.15.100:0/2109641407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c08100fe0 0x7f4c0810da50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 >> 172.21.15.100:0/2109641407 conn(0x7f4c080753a0 msgr2=0x7f4c081041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 shutdown_connections 2024-03-12T05:26:29.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.167+0000 7f4c0fbbc640 1 -- 172.21.15.100:0/2109641407 wait complete. 2024-03-12T05:26:29.175 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:26:29.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:26:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45902 36.73 MB/sec warmup 72 sec latency 101.643 ms 2024-03-12T05:26:29.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 -- 172.21.15.100:0/2591598710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4102420 msgr2=0x7f5bd40ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/2591598710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4102420 0x7f5bd40ffb60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009900 tx=0x7f5bc402bcd0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 -- 172.21.15.100:0/2591598710 shutdown_connections 2024-03-12T05:26:29.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/2591598710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4100180 0x7f5bd410fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/2591598710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4102420 0x7f5bd40ffb60 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/2591598710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd4102040 0x7f5bd40ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.314+0000 7f5bdbc81640 1 -- 172.21.15.100:0/2591598710 >> 172.21.15.100:0/2591598710 conn(0x7f5bd40765a0 msgr2=0x7f5bd40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 -- 172.21.15.100:0/2591598710 shutdown_connections 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 -- 172.21.15.100:0/2591598710 wait complete. 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 Processor -- start 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 -- start start 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 0x7f5bd419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd4102040 0x7f5bd419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4102420 0x7f5bd4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4199ee0 con 0x7f5bd4100180 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd419a050 con 0x7f5bd4102040 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.315+0000 7f5bdbc81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd419a210 con 0x7f5bd4102420 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 0x7f5bd419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 0x7f5bd419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43470/0 (socket says 172.21.15.100:43470) 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1580871271 learned_addr learned my addr 172.21.15.100:0/1580871271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bda21e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4102420 0x7f5bd4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd921c640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd4102040 0x7f5bd419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1580871271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4102420 msgr2=0x7f5bd4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4102420 0x7f5bd4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1580871271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd4102040 msgr2=0x7f5bd419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd4102040 0x7f5bd419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1580871271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009560 con 0x7f5bd4100180 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bda21e640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4102420 0x7f5bd4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 0x7f5bd419f6d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000e9f0 tx=0x7f5bd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bcaffd640 1 -- 172.21.15.100:0/1580871271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0009700 con 0x7f5bd4100180 2024-03-12T05:26:29.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bcaffd640 1 -- 172.21.15.100:0/1580871271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bd000cd80 con 0x7f5bd4100180 2024-03-12T05:26:29.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd419a4f0 con 0x7f5bd4100180 2024-03-12T05:26:29.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bcaffd640 1 -- 172.21.15.100:0/1580871271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0018920 con 0x7f5bd4100180 2024-03-12T05:26:29.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.316+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd41a6700 con 0x7f5bd4100180 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.319+0000 7f5bcaffd640 1 -- 172.21.15.100:0/1580871271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5bd0018a80 con 0x7f5bd4100180 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.319+0000 7f5bde854740 1 Processor -- start 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 -- start start 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 0x559086c6a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6ab70 0x559086c73210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c75b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086c25960 con 0x559086c73750 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086bd9310 con 0x559086c6ab70 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bde854740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086bd6840 con 0x559086c6a250 2024-03-12T05:26:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bda21e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 0x559086c6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd921c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c75b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bcaffd640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5bc00a1be0 0x7f5bc00a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bda21e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 0x559086c6a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41028/0 (socket says 172.21.15.100:41028) 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bda21e640 1 -- 172.21.15.100:0/3947287873 learned_addr learned my addr 172.21.15.100:0/3947287873 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6ab70 0x559086c73210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bda21e640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5bc00a1be0 0x7f5bc00a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd921c640 1 -- 172.21.15.100:0/3947287873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 msgr2=0x559086c6a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd921c640 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 0x559086c6a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd921c640 1 -- 172.21.15.100:0/3947287873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6ab70 msgr2=0x559086c73210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd921c640 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6ab70 0x559086c73210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bd921c640 1 -- 172.21.15.100:0/3947287873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559086b94060 con 0x559086c73750 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bda21e640 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 0x559086c6a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.320+0000 7f5bcaffd640 1 -- 172.21.15.100:0/1580871271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5bd0014070 con 0x7f5bd4100180 2024-03-12T05:26:29.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:28 smithi100 ceph-mon[32210]: pgmap v450: 97 pgs: 97 active+clean; 891 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 802 op/s 2024-03-12T05:26:29.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:28 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.321+0000 7f5bd921c640 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c75b40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009850 tx=0x7f5bc402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c299c724f15dd14 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.321+0000 7f5bda21e640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5bc00a1be0 0x7f5bc00a40a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005c40 tx=0x7f5bcc004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.321+0000 7f5bbffff640 1 -- 172.21.15.100:0/3947287873 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4004800 con 0x559086c73750 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.321+0000 7f5bbffff640 1 -- 172.21.15.100:0/3947287873 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4004960 con 0x559086c73750 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.321+0000 7f5bbffff640 1 -- 172.21.15.100:0/3947287873 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4005050 con 0x559086c73750 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 -- 172.21.15.100:0/3947287873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 msgr2=0x559086c75b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c75b40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009850 tx=0x7f5bc402c7a0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 -- 172.21.15.100:0/3947287873 shutdown_connections 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c75b40 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6ab70 0x559086c73210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- 172.21.15.100:0/3947287873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6a250 0x559086c6a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 -- 172.21.15.100:0/3947287873 >> 172.21.15.100:0/3947287873 conn(0x559086c64980 msgr2=0x559086c64d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 -- 172.21.15.100:0/3947287873 shutdown_connections 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 -- 172.21.15.100:0/3947287873 wait complete. 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 -- start start 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 Processor -- start 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c70cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.322+0000 7f5bde854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086bd2020 con 0x559086c73750 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590868a5430 con 0x559086c6a250 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086861e00 con 0x559086c6ab70 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 10 client.0 _ll_register_callbacks cb 0x559086b79cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bda21e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c70cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd9a1d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c70cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41042/0 (socket says 172.21.15.100:41042) 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 -- 172.21.15.100:0/2163479258 learned_addr learned my addr 172.21.15.100:0/2163479258 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bda21e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43488/0 (socket says 172.21.15.100:43488) 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 -- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 msgr2=0x559086c70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 -- 172.21.15.100:0/2163479258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 msgr2=0x559086c711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 -- 172.21.15.100:0/2163479258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd4110a10 con 0x559086c6ab70 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bda21e640 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bd921c640 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c70cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40050a0 tx=0x7f5bc40050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63147b671f130ea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/2163479258 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc402cbd0 con 0x559086c6ab70 2024-03-12T05:26:29.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590868d17e0 con 0x559086c6ab70 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/2163479258 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5bc4004050 con 0x559086c6ab70 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/2163479258 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bc4004ad0 con 0x559086c6ab70 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 10 client.27820 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.323+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559086bd06a0 con 0x559086c6ab70 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/2163479258 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5590868d17e0 con 0x559086c6ab70 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/2163479258 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5bc4056070 con 0x559086c6ab70 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bde854740 10 client.27820 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bde854740 10 client.27820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bde854740 20 client.27820 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bde854740 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086c838c0 0x559086c85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bde854740 4 client.27820 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55908690e720 con 0x559086c838c0 2024-03-12T05:26:29.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086c838c0 0x559086c85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bbd7fa640 10 client.27820 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.324+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086c838c0 0x559086c85ce0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85e197fb23fdb46d server_cookie=427ce4f5a1451a43 in_seq=0 out_seq=0 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.325+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/2163479258 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55908690e720 con 0x559086c838c0 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bbd7fa640 10 client.27820 handle_command_reply: tid=0 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 client.27820 shutdown 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 msgr2=0x559086c70cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c70cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40050a0 tx=0x7f5bc40050d0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 shutdown_connections 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086c838c0 0x559086c85ce0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c73750 0x559086c711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c70cb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 --2- 172.21.15.100:0/2163479258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bbd7fa640 0 client.27820 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:26:29.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 >> 172.21.15.100:0/2163479258 conn(0x559086c64980 msgr2=0x559086c666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 shutdown_connections 2024-03-12T05:26:29.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 1 -- 172.21.15.100:0/2163479258 wait complete. 2024-03-12T05:26:29.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.326+0000 7f5bde854740 20 client.27820 trim_cache size 0 max 16384 2024-03-12T05:26:29.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.339+0000 7f5bde854740 1 Processor -- start 2024-03-12T05:26:29.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 -- start start 2024-03-12T05:26:29.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c72ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 0x559086c71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c73750 0x559086c6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086b57130 con 0x559086c6ab70 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086c805e0 con 0x559086c6a250 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bde854740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086c69eb0 con 0x559086c73750 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 0x559086c71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 0x559086c71010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43496/0 (socket says 172.21.15.100:43496) 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 -- 172.21.15.100:0/2678573964 learned_addr learned my addr 172.21.15.100:0/2678573964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c72ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bda21e640 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c73750 0x559086c6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 -- 172.21.15.100:0/2678573964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c73750 msgr2=0x559086c6d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c73750 0x559086c6d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 -- 172.21.15.100:0/2678573964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 msgr2=0x559086c72ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c72ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 -- 172.21.15.100:0/2678573964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559086b6cc20 con 0x559086c6ab70 2024-03-12T05:26:29.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bd921c640 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 0x559086c71010 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4003ed0 tx=0x7f5bc402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04bb57413bf8125 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.340+0000 7f5bbf7fe640 1 -- 172.21.15.100:0/2678573964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4004ad0 con 0x559086c6ab70 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bbf7fe640 1 -- 172.21.15.100:0/2678573964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4005140 con 0x559086c6ab70 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bbf7fe640 1 -- 172.21.15.100:0/2678573964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4048b50 con 0x559086c6ab70 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 -- 172.21.15.100:0/2678573964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 msgr2=0x559086c71010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 0x559086c71010 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4003ed0 tx=0x7f5bc402c7e0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 -- 172.21.15.100:0/2678573964 shutdown_connections 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c73750 0x559086c6d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6ab70 0x559086c71010 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 --2- 172.21.15.100:0/2678573964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c6a250 0x559086c72ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 -- 172.21.15.100:0/2678573964 >> 172.21.15.100:0/2678573964 conn(0x559086c64980 msgr2=0x559086c6bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 -- 172.21.15.100:0/2678573964 shutdown_connections 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.341+0000 7f5bde854740 1 -- 172.21.15.100:0/2678573964 wait complete. 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 -- start start 2024-03-12T05:26:29.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 Processor -- start 2024-03-12T05:26:29.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 0x559086c7a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c73750 0x559086c79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086c79830 con 0x559086c6a250 2024-03-12T05:26:29.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086c799a0 con 0x559086c73750 2024-03-12T05:26:29.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559086cd9df0 con 0x559086c6ab70 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bde854740 10 client.0 _ll_register_callbacks cb 0x7f5bcc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bd9a1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 0x559086c7a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bd9a1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 0x559086c7a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43502/0 (socket says 172.21.15.100:43502) 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1674583673 learned_addr learned my addr 172.21.15.100:0/1674583673 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bd921c640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.342+0000 7f5bda21e640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c73750 0x559086c79160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1674583673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 msgr2=0x559086c78c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c78c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c73750 msgr2=0x559086c79160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c73750 0x559086c79160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bd9a1d640 1 -- 172.21.15.100:0/1674583673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559086bd06a0 con 0x559086c6a250 2024-03-12T05:26:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bd9a1d640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 0x559086c7a880 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0096c30 tx=0x7f5bd0085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc3af59294ce119 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/1674583673 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00b3280 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/1674583673 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5bd009e870 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559086c600f0 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/1674583673 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd00cb4c0 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bde854740 10 client.18510 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.343+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559086b928c0 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/1674583673 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559086c600f0 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/1674583673 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5bd0099c20 con 0x559086c6a250 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bde854740 10 client.18510 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bde854740 10 client.18510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bde854740 20 client.18510 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bde854740 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086cdb510 0x559086cdd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bde854740 4 client.18510 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:26:29.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5bc4004960 con 0x559086cdb510 2024-03-12T05:26:29.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.344+0000 7f5bd921c640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086cdb510 0x559086cdd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.345+0000 7f5bbd7fa640 10 client.18510 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:26:29.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.345+0000 7f5bd921c640 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086cdb510 0x559086cdd930 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85a8f6ef193602bb server_cookie=34ddf3dbdbf2cb1f in_seq=0 out_seq=0 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.347+0000 7f5bbd7fa640 1 -- 172.21.15.100:0/1674583673 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5bc4004960 con 0x559086cdb510 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.347+0000 7f5bbd7fa640 10 client.18510 handle_command_reply: tid=0 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.347+0000 7f5bde854740 1 client.18510 shutdown 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 msgr2=0x559086c7a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 0x559086c7a880 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0096c30 tx=0x7f5bd0085040 comp rx=0 tx=0).stop 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 shutdown_connections 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x559086cdb510 0x559086cdd930 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559086c73750 0x559086c79160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559086c6ab70 0x559086c78c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 --2- 172.21.15.100:0/1674583673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559086c6a250 0x559086c7a880 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bbd7fa640 0 client.18510 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:26:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 >> 172.21.15.100:0/1674583673 conn(0x559086c64980 msgr2=0x559086c632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 shutdown_connections 2024-03-12T05:26:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 1 -- 172.21.15.100:0/1674583673 wait complete. 2024-03-12T05:26:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bde854740 20 client.18510 trim_cache size 0 max 16384 2024-03-12T05:26:29.346 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5bc00a1be0 msgr2=0x7f5bc00a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5bc00a1be0 0x7f5bc00a40a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005c40 tx=0x7f5bcc004ab0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 msgr2=0x7f5bd419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 0x7f5bd419f6d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000e9f0 tx=0x7f5bd000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 shutdown_connections 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5bc00a1be0 0x7f5bc00a40a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4102420 0x7f5bd4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.348+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd4102040 0x7f5bd419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.349+0000 7f5bdbc81640 1 --2- 172.21.15.100:0/1580871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd4100180 0x7f5bd419f6d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.349+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 >> 172.21.15.100:0/1580871271 conn(0x7f5bd40765a0 msgr2=0x7f5bd40fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.349+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 shutdown_connections 2024-03-12T05:26:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.349+0000 7f5bdbc81640 1 -- 172.21.15.100:0/1580871271 wait complete. 2024-03-12T05:26:29.356 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:26:29.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:26:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:28 smithi156 ceph-mon[36270]: pgmap v450: 97 pgs: 97 active+clean; 891 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 802 op/s 2024-03-12T05:26:29.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:28 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:26:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:28 smithi161 ceph-mon[36043]: pgmap v450: 97 pgs: 97 active+clean; 891 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 802 op/s 2024-03-12T05:26:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:28 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:26:29.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 -- 172.21.15.100:0/1875759672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6460100240 msgr2=0x7f6460105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 --2- 172.21.15.100:0/1875759672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6460100240 0x7f6460105d70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6448009930 tx=0x7f644802bc30 comp rx=0 tx=0).stop 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 -- 172.21.15.100:0/1875759672 shutdown_connections 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 --2- 172.21.15.100:0/1875759672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 0x7f646010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 --2- 172.21.15.100:0/1875759672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6460100240 0x7f6460105d70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 --2- 172.21.15.100:0/1875759672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64600ff890 0x7f64600ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 -- 172.21.15.100:0/1875759672 >> 172.21.15.100:0/1875759672 conn(0x7f64600765a0 msgr2=0x7f64600fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 -- 172.21.15.100:0/1875759672 shutdown_connections 2024-03-12T05:26:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.482+0000 7f64648fb640 1 -- 172.21.15.100:0/1875759672 wait complete. 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 Processor -- start 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 -- start start 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 0x7f646019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 0x7f646019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6460197630 0x7f6460197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460198100 con 0x7f64600ff890 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460198270 con 0x7f6460197630 2024-03-12T05:26:29.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f64648fb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460198430 con 0x7f6460106450 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 0x7f646019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f6457fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 0x7f646019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 0x7f646019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43518/0 (socket says 172.21.15.100:43518) 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 -- 172.21.15.100:0/3744213628 learned_addr learned my addr 172.21.15.100:0/3744213628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645ed9d640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6460197630 0x7f6460197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 -- 172.21.15.100:0/3744213628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 msgr2=0x7f646019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 0x7f646019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 -- 172.21.15.100:0/3744213628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6460197630 msgr2=0x7f6460197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.483+0000 7f645e59c640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6460197630 0x7f6460197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f645e59c640 1 -- 172.21.15.100:0/3744213628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448009560 con 0x7f64600ff890 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f6457fff640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 0x7f646019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f645e59c640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 0x7f646019d570 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f645000e9f0 tx=0x7f645000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f64577fe640 1 -- 172.21.15.100:0/3744213628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450009700 con 0x7f64600ff890 2024-03-12T05:26:29.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f64577fe640 1 -- 172.21.15.100:0/3744213628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f645000cd80 con 0x7f64600ff890 2024-03-12T05:26:29.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f64577fe640 1 -- 172.21.15.100:0/3744213628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450018920 con 0x7f64600ff890 2024-03-12T05:26:29.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64601a43d0 con 0x7f64600ff890 2024-03-12T05:26:29.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.484+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64601a4860 con 0x7f64600ff890 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 Processor -- start 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64577fe640 1 -- 172.21.15.100:0/3744213628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6450018a80 con 0x7f64600ff890 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 -- start start 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 0x5614aba25630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25b70 0x5614aba2e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba2e750 0x5614aba30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab9e0960 con 0x5614aba25b70 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab994310 con 0x5614aba25250 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.487+0000 7f64675af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab991840 con 0x5614aba2e750 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 0x5614aba25630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25b70 0x5614aba2e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 0x5614aba25630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:48240/0 (socket says 172.21.15.100:48240) 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645ed9d640 1 -- 172.21.15.100:0/1180250157 learned_addr learned my addr 172.21.15.100:0/1180250157 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba2e750 0x5614aba30b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f64577fe640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f644c0a1ba0 0x7f644c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 -- 172.21.15.100:0/1180250157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 msgr2=0x5614aba25630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 0x5614aba25630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 -- 172.21.15.100:0/1180250157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25b70 msgr2=0x5614aba2e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25b70 0x5614aba2e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 -- 172.21.15.100:0/1180250157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614ab94f060 con 0x5614aba2e750 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645ed9d640 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 0x5614aba25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645ed9d640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f644c0a1ba0 0x7f644c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f64577fe640 1 -- 172.21.15.100:0/3744213628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6450014070 con 0x7f64600ff890 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6457fff640 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba2e750 0x5614aba30b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f644802c170 tx=0x7f644802c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61e34335974dd82 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f6454ff9640 1 -- 172.21.15.100:0/1180250157 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644802c7e0 con 0x5614aba2e750 2024-03-12T05:26:29.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.488+0000 7f645ed9d640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f644c0a1ba0 0x7f644c0a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64580107d0 tx=0x7f6458004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f6454ff9640 1 -- 172.21.15.100:0/1180250157 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6448004510 con 0x5614aba2e750 2024-03-12T05:26:29.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f6454ff9640 1 -- 172.21.15.100:0/1180250157 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480053d0 con 0x5614aba2e750 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 -- 172.21.15.100:0/1180250157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba2e750 msgr2=0x5614aba30b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba2e750 0x5614aba30b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f644802c170 tx=0x7f644802c610 comp rx=0 tx=0).stop 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 -- 172.21.15.100:0/1180250157 shutdown_connections 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba2e750 0x5614aba30b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25b70 0x5614aba2e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 --2- 172.21.15.100:0/1180250157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25250 0x5614aba25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.489+0000 7f64675af740 1 -- 172.21.15.100:0/1180250157 >> 172.21.15.100:0/1180250157 conn(0x5614aba1f980 msgr2=0x5614aba1fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 -- 172.21.15.100:0/1180250157 shutdown_connections 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 -- 172.21.15.100:0/1180250157 wait complete. 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 -- start start 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 Processor -- start 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25250 0x5614aba2b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25b70 0x5614aba2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 0x5614aba2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab98d020 con 0x5614aba2e750 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab660430 con 0x5614aba25b70 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab61ce00 con 0x5614aba25250 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f64675af740 10 client.0 _ll_register_callbacks cb 0x5614ab934cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.490+0000 7f645e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25250 0x5614aba2b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 0x5614aba2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f6457fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25b70 0x5614aba2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 0x5614aba2c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43530/0 (socket says 172.21.15.100:43530) 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 -- 172.21.15.100:0/2995993592 learned_addr learned my addr 172.21.15.100:0/2995993592 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25250 0x5614aba2b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41106/0 (socket says 172.21.15.100:41106) 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 -- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25250 msgr2=0x5614aba2b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25250 0x5614aba2b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 -- 172.21.15.100:0/2995993592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25b70 msgr2=0x5614aba2bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25b70 0x5614aba2bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 -- 172.21.15.100:0/2995993592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646010e830 con 0x5614aba2e750 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f645ed9d640 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 0x5614aba2c1f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f645800fee0 tx=0x7f6458004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321dd6bec18486eb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f6445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f6445ffb640 1 -- 172.21.15.100:0/2995993592 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458004d20 con 0x5614aba2e750 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f6445ffb640 1 -- 172.21.15.100:0/2995993592 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6458005020 con 0x5614aba2e750 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.491+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614ab68c7e0 con 0x5614aba2e750 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f6445ffb640 1 -- 172.21.15.100:0/2995993592 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6458004e80 con 0x5614aba2e750 2024-03-12T05:26:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 10 client.18522 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614ab98b6a0 con 0x5614aba2e750 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f6445ffb640 1 -- 172.21.15.100:0/2995993592 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5614ab68c7e0 con 0x5614aba2e750 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f6445ffb640 1 -- 172.21.15.100:0/2995993592 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f645801c700 con 0x5614aba2e750 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 10 client.18522 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 10 client.18522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 20 client.18522 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba3ebf0 0x5614aba41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 4 client.18522 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614ab6c9720 con 0x5614aba3ebf0 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.492+0000 7f645e59c640 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba3ebf0 0x5614aba41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.493+0000 7f6445ffb640 10 client.18522 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:26:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.493+0000 7f645e59c640 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba3ebf0 0x5614aba41010 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3d5bdbcdbe72e1 server_cookie=69625233c67ed86 in_seq=0 out_seq=0 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f6445ffb640 1 -- 172.21.15.100:0/2995993592 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614ab6c9720 con 0x5614aba3ebf0 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f6445ffb640 10 client.18522 handle_command_reply: tid=0 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 client.18522 shutdown 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 msgr2=0x5614aba2c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 0x5614aba2c1f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f645800fee0 tx=0x7f6458004ab0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 shutdown_connections 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba2e750 0x5614aba2c1f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba3ebf0 0x5614aba41010 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba25b70 0x5614aba2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 --2- 172.21.15.100:0/2995993592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25250 0x5614aba2b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f6445ffb640 0 client.18522 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:26:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 >> 172.21.15.100:0/2995993592 conn(0x5614aba1f980 msgr2=0x5614aba216d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 shutdown_connections 2024-03-12T05:26:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 1 -- 172.21.15.100:0/2995993592 wait complete. 2024-03-12T05:26:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.496+0000 7f64675af740 20 client.18522 trim_cache size 0 max 16384 2024-03-12T05:26:29.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 Processor -- start 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 -- start start 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba2c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba214c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ab912130 con 0x5614aba25250 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aba24340 con 0x5614aba2e750 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f64675af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aba1b320 con 0x5614aba25b70 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba2c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba2c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43540/0 (socket says 172.21.15.100:43540) 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f645e59c640 1 -- 172.21.15.100:0/3911344475 learned_addr learned my addr 172.21.15.100:0/3911344475 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f6457fff640 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f645ed9d640 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f645e59c640 1 -- 172.21.15.100:0/3911344475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 msgr2=0x5614aba214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.513+0000 7f645e59c640 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f645e59c640 1 -- 172.21.15.100:0/3911344475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 msgr2=0x5614aba355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f645e59c640 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba355a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f645e59c640 1 -- 172.21.15.100:0/3911344475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614ab927c20 con 0x5614aba25250 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f645e59c640 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba2c010 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f64500cb8b0 tx=0x7f6450085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aac64774d49c63 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f6447fff640 1 -- 172.21.15.100:0/3911344475 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645009e8e0 con 0x5614aba25250 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f6447fff640 1 -- 172.21.15.100:0/3911344475 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64500a2f30 con 0x5614aba25250 2024-03-12T05:26:29.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.514+0000 7f6447fff640 1 -- 172.21.15.100:0/3911344475 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500cb530 con 0x5614aba25250 2024-03-12T05:26:29.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- 172.21.15.100:0/3911344475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 msgr2=0x5614aba2c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba2c010 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f64500cb8b0 tx=0x7f6450085040 comp rx=0 tx=0).stop 2024-03-12T05:26:29.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- 172.21.15.100:0/3911344475 shutdown_connections 2024-03-12T05:26:29.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- 172.21.15.100:0/3911344475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba2c010 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- 172.21.15.100:0/3911344475 >> 172.21.15.100:0/3911344475 conn(0x5614aba1f980 msgr2=0x5614aba20db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- 172.21.15.100:0/3911344475 shutdown_connections 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- 172.21.15.100:0/3911344475 wait complete. 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- start start 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 Processor -- start 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba34cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba35200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aba8e830 con 0x5614aba25250 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aba8e9a0 con 0x5614aba2e750 2024-03-12T05:26:29.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aba8eb60 con 0x5614aba25b70 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.515+0000 7f64675af740 10 client.0 _ll_register_callbacks cb 0x7f64580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba34cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba34cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43554/0 (socket says 172.21.15.100:43554) 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 -- 172.21.15.100:0/2059223858 learned_addr learned my addr 172.21.15.100:0/2059223858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f6457fff640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba35200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645ed9d640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 -- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 msgr2=0x5614aba35200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba35200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 -- 172.21.15.100:0/2059223858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 msgr2=0x5614aba8e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba8e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 -- 172.21.15.100:0/2059223858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64601a4860 con 0x5614aba25250 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f645e59c640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba34cc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f64500a2a00 tx=0x7f6450010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909f9c2f7b11d41f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f6445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:29.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f6445ffb640 1 -- 172.21.15.100:0/2059223858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b3170 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f6445ffb640 1 -- 172.21.15.100:0/2059223858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f64500caaf0 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6448004510 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f6445ffb640 1 -- 172.21.15.100:0/2059223858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64500cacb0 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f64675af740 10 client.18528 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614ab94d8c0 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.516+0000 7f6445ffb640 1 -- 172.21.15.100:0/2059223858 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64500b7020 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f6445ffb640 1 -- 172.21.15.100:0/2059223858 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6450065420 con 0x5614aba25250 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f64675af740 10 client.18528 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:29.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f64675af740 10 client.18528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f64675af740 20 client.18528 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f64675af740 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba97330 0x5614aba99750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f64675af740 4 client.18528 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614ab718890 con 0x5614aba97330 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f6457fff640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba97330 0x5614aba99750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f6445ffb640 10 client.18528 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:26:29.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.517+0000 7f6457fff640 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba97330 0x5614aba99750 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e937a502e389bba server_cookie=fa190f259e02a5a9 in_seq=0 out_seq=0 2024-03-12T05:26:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f6445ffb640 1 -- 172.21.15.100:0/2059223858 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614ab718890 con 0x5614aba97330 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f6445ffb640 10 client.18528 handle_command_reply: tid=0 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 client.18528 shutdown 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 msgr2=0x5614aba34cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba34cc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f64500a2a00 tx=0x7f6450010f50 comp rx=0 tx=0).stop 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 shutdown_connections 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5614aba97330 0x5614aba99750 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614aba2e750 0x5614aba8e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614aba25b70 0x5614aba35200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 --2- 172.21.15.100:0/2059223858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614aba25250 0x5614aba34cc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 >> 172.21.15.100:0/2059223858 conn(0x5614aba1f980 msgr2=0x5614aba1eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f6445ffb640 0 client.18528 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:26:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 shutdown_connections 2024-03-12T05:26:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.521+0000 7f64675af740 1 -- 172.21.15.100:0/2059223858 wait complete. 2024-03-12T05:26:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64675af740 20 client.18528 trim_cache size 0 max 16384 2024-03-12T05:26:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f644c0a1ba0 msgr2=0x7f644c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f644c0a1ba0 0x7f644c0a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64580107d0 tx=0x7f6458004a40 comp rx=0 tx=0).stop 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 msgr2=0x7f646019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 0x7f646019d570 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f645000e9f0 tx=0x7f645000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 shutdown_connections 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f644c0a1ba0 0x7f644c0a4060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6460197630 0x7f6460197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6460106450 0x7f646019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 --2- 172.21.15.100:0/3744213628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64600ff890 0x7f646019d570 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 >> 172.21.15.100:0/3744213628 conn(0x7f64600765a0 msgr2=0x7f6460104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 shutdown_connections 2024-03-12T05:26:29.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.522+0000 7f64648fb640 1 -- 172.21.15.100:0/3744213628 wait complete. 2024-03-12T05:26:29.528 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:26:29.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 -- 172.21.15.100:0/133052371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c100c10 msgr2=0x7f5c2c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/133052371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c100c10 0x7f5c2c104c50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009930 tx=0x7f5c1402bc30 comp rx=0 tx=0).stop 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 -- 172.21.15.100:0/133052371 shutdown_connections 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/133052371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c105330 0x7f5c2c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/133052371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c100c10 0x7f5c2c104c50 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/133052371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c2c100260 0x7f5c2c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 -- 172.21.15.100:0/133052371 >> 172.21.15.100:0/133052371 conn(0x7f5c2c077180 msgr2=0x7f5c2c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 -- 172.21.15.100:0/133052371 shutdown_connections 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.664+0000 7f5c30b6c640 1 -- 172.21.15.100:0/133052371 wait complete. 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 Processor -- start 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 -- start start 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 0x7f5c2c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c2c105330 0x7f5c2c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 0x7f5c2c1a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c1a3d60 con 0x7f5c2c199110 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c1a3ed0 con 0x7f5c2c105330 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.665+0000 7f5c30b6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c1a4090 con 0x7f5c2c100260 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 0x7f5c2c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 0x7f5c2c1a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 0x7f5c2c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41128/0 (socket says 172.21.15.100:41128) 2024-03-12T05:26:29.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c2c105330 0x7f5c2c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2a59c640 1 -- 172.21.15.100:0/2366988424 learned_addr learned my addr 172.21.15.100:0/2366988424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 0x7f5c2c1a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43572/0 (socket says 172.21.15.100:43572) 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 -- 172.21.15.100:0/2366988424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 msgr2=0x7f5c2c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 0x7f5c2c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 -- 172.21.15.100:0/2366988424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c2c105330 msgr2=0x7f5c2c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c2c105330 0x7f5c2c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 -- 172.21.15.100:0/2366988424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14009560 con 0x7f5c2c199110 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 0x7f5c2c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c2ad9d640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 0x7f5c2c1a3670 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400da10 tx=0x7f5c2400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c237fe640 1 -- 172.21.15.100:0/2366988424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400bd70 con 0x7f5c2c199110 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c1a4370 con 0x7f5c2c199110 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c237fe640 1 -- 172.21.15.100:0/2366988424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c24003e50 con 0x7f5c2c199110 2024-03-12T05:26:29.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c237fe640 1 -- 172.21.15.100:0/2366988424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240106c0 con 0x7f5c2c199110 2024-03-12T05:26:29.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.666+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c1a4850 con 0x7f5c2c199110 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.668+0000 7f5c237fe640 1 -- 172.21.15.100:0/2366988424 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5c24004000 con 0x7f5c2c199110 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 Processor -- start 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 -- start start 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dab6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40da71960 con 0x55e40dabf750 2024-03-12T05:26:29.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40da25310 con 0x55e40dab6250 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c33818740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40da22840 con 0x55e40dab6b70 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43580/0 (socket says 172.21.15.100:43580) 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c23fff640 1 -- 172.21.15.100:0/2720571164 learned_addr learned my addr 172.21.15.100:0/2720571164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c23fff640 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dab6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.669+0000 7f5c237fe640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5c180a1ba0 0x7f5c180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2a59c640 1 -- 172.21.15.100:0/2720571164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 msgr2=0x55e40dab6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dab6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2a59c640 1 -- 172.21.15.100:0/2720571164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 msgr2=0x55e40dac1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2a59c640 1 -- 172.21.15.100:0/2720571164 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40d9e0060 con 0x55e40dab6b70 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c237fe640 1 -- 172.21.15.100:0/2366988424 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c240cdb40 con 0x7f5c2c199110 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2ad9d640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5c180a1ba0 0x7f5c180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabf210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c010550 tx=0x7f5c1c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6876ebd30d101fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c20ff9640 1 -- 172.21.15.100:0/2720571164 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c011760 con 0x55e40dab6b70 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c20ff9640 1 -- 172.21.15.100:0/2720571164 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c015070 con 0x55e40dab6b70 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c20ff9640 1 -- 172.21.15.100:0/2720571164 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c019c60 con 0x55e40dab6b70 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.670+0000 7f5c2ad9d640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5c180a1ba0 0x7f5c180a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c24097100 tx=0x7f5c24065f10 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 -- 172.21.15.100:0/2720571164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 msgr2=0x55e40dabf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabf210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c010550 tx=0x7f5c1c00bf90 comp rx=0 tx=0).stop 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 -- 172.21.15.100:0/2720571164 shutdown_connections 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabf210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 --2- 172.21.15.100:0/2720571164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dab6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 -- 172.21.15.100:0/2720571164 >> 172.21.15.100:0/2720571164 conn(0x55e40dab0980 msgr2=0x55e40dab0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 -- 172.21.15.100:0/2720571164 shutdown_connections 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 -- 172.21.15.100:0/2720571164 wait complete. 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 -- start start 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.671+0000 7f5c33818740 1 Processor -- start 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dabd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40da1e020 con 0x55e40dabf750 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40d6f1430 con 0x55e40dab6250 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40d6ade00 con 0x55e40dab6b70 2024-03-12T05:26:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 10 client.0 _ll_register_callbacks cb 0x55e40d9c5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41152/0 (socket says 172.21.15.100:41152) 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c2a59c640 1 -- 172.21.15.100:0/4093780928 learned_addr learned my addr 172.21.15.100:0/4093780928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dabd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 -- 172.21.15.100:0/4093780928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 msgr2=0x55e40dabcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 -- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 msgr2=0x55e40dabc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 -- 172.21.15.100:0/4093780928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c00bd70 con 0x55e40dabf750 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c23fff640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dabd0e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5c1402c780 tx=0x7f5c14005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71cd120ed5bc3ae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c11ffb640 1 -- 172.21.15.100:0/4093780928 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14004480 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c11ffb640 1 -- 172.21.15.100:0/4093780928 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c14004df0 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.672+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e40d71d7e0 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c11ffb640 1 -- 172.21.15.100:0/4093780928 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c14004fb0 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 10 client.18540 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e40da1c6a0 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c11ffb640 1 -- 172.21.15.100:0/4093780928 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e40d71d7e0 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c11ffb640 1 -- 172.21.15.100:0/4093780928 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5c14003ab0 con 0x55e40dabf750 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 10 client.18540 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 10 client.18540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 20 client.18540 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40dacf9a0 0x55e40dad1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 4 client.18540 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e40d75a720 con 0x55e40dacf9a0 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.673+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40dacf9a0 0x55e40dad1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.674+0000 7f5c11ffb640 10 client.18540 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.674+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40dacf9a0 0x55e40dad1d90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2778baca30a5934 server_cookie=38cbe3eb40922222 in_seq=0 out_seq=0 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c11ffb640 1 -- 172.21.15.100:0/4093780928 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e40d75a720 con 0x55e40dacf9a0 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c11ffb640 10 client.18540 handle_command_reply: tid=0 2024-03-12T05:26:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 client.18540 shutdown 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 msgr2=0x55e40dabd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dabd0e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5c1402c780 tx=0x7f5c14005d90 comp rx=0 tx=0).stop 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 shutdown_connections 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40dacf9a0 0x55e40dad1d90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dabd0e0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6b70 0x55e40dabcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 --2- 172.21.15.100:0/4093780928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c11ffb640 0 client.18540 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 >> 172.21.15.100:0/4093780928 conn(0x55e40dab0980 msgr2=0x55e40dab26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 shutdown_connections 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 1 -- 172.21.15.100:0/4093780928 wait complete. 2024-03-12T05:26:29.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.675+0000 7f5c33818740 20 client.18540 trim_cache size 0 max 16384 2024-03-12T05:26:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 Processor -- start 2024-03-12T05:26:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 -- start start 2024-03-12T05:26:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 0x55e40dabbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dabf750 0x55e40dab98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40d9a3130 con 0x55e40dab6b70 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40daa2a40 con 0x55e40dab6250 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c33818740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40dabc480 con 0x55e40dabf750 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 0x55e40dabbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c2a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 0x55e40dabbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43594/0 (socket says 172.21.15.100:43594) 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c2a59c640 1 -- 172.21.15.100:0/3309849765 learned_addr learned my addr 172.21.15.100:0/3309849765 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.688+0000 7f5c23fff640 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dabf750 0x55e40dab98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c2a59c640 1 -- 172.21.15.100:0/3309849765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dabf750 msgr2=0x55e40dab98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dabf750 0x55e40dab98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c2a59c640 1 -- 172.21.15.100:0/3309849765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 msgr2=0x55e40dabed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c2a59c640 1 -- 172.21.15.100:0/3309849765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40d9b8c20 con 0x55e40dab6b70 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c23fff640 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 0x55e40dabbf40 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c000c00 tx=0x7f5c1c019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f48b74aa4b5f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c13fff640 1 -- 172.21.15.100:0/3309849765 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c019970 con 0x55e40dab6b70 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c13fff640 1 -- 172.21.15.100:0/3309849765 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c005520 con 0x55e40dab6b70 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.689+0000 7f5c13fff640 1 -- 172.21.15.100:0/3309849765 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c029420 con 0x55e40dab6b70 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 -- 172.21.15.100:0/3309849765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 msgr2=0x55e40dabbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 0x55e40dabbf40 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c000c00 tx=0x7f5c1c019530 comp rx=0 tx=0).stop 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 -- 172.21.15.100:0/3309849765 shutdown_connections 2024-03-12T05:26:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dabf750 0x55e40dab98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dab6b70 0x55e40dabbf40 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 --2- 172.21.15.100:0/3309849765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6250 0x55e40dabed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 -- 172.21.15.100:0/3309849765 >> 172.21.15.100:0/3309849765 conn(0x55e40dab0980 msgr2=0x55e40daafb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 -- 172.21.15.100:0/3309849765 shutdown_connections 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 -- 172.21.15.100:0/3309849765 wait complete. 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 -- start start 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.690+0000 7f5c33818740 1 Processor -- start 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6250 0x55e40dac50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6b70 0x55e40dac55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40db25e60 con 0x55e40dabf750 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40db25fa0 con 0x55e40dab6b70 2024-03-12T05:26:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40db260e0 con 0x55e40dab6250 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c33818740 10 client.0 _ll_register_callbacks cb 0x7f5c14005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43608/0 (socket says 172.21.15.100:43608) 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 -- 172.21.15.100:0/1741056812 learned_addr learned my addr 172.21.15.100:0/1741056812 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6b70 0x55e40dac55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6250 0x55e40dac50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 -- 172.21.15.100:0/1741056812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6250 msgr2=0x55e40dac50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6250 0x55e40dac50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 -- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6b70 msgr2=0x55e40dac55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6b70 0x55e40dac55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 -- 172.21.15.100:0/1741056812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e40da1c6a0 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c23fff640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac5b30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5c14005ab0 tx=0x7f5c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b9febdc922ad2a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.691+0000 7f5c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c11ffb640 1 -- 172.21.15.100:0/1741056812 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1402cb10 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c11ffb640 1 -- 172.21.15.100:0/1741056812 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5c14000e40 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c1c015070 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c11ffb640 1 -- 172.21.15.100:0/1741056812 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c14060820 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 10 client.18546 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f5c1c01e040 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c11ffb640 1 -- 172.21.15.100:0/1741056812 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c015070 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c11ffb640 1 -- 172.21.15.100:0/1741056812 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5c14067050 con 0x55e40dabf750 2024-03-12T05:26:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 10 client.18546 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 10 client.18546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 20 client.18546 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40db27720 0x55e40db29b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 4 client.18546 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.692+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e40d7a9890 con 0x55e40db27720 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40db27720 0x55e40db29b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c2a59c640 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40db27720 0x55e40db29b40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81d379e8beae8f3 server_cookie=2af1049b3f075526 in_seq=0 out_seq=0 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c11ffb640 10 client.18546 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c11ffb640 1 -- 172.21.15.100:0/1741056812 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e40d7a9890 con 0x55e40db27720 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c11ffb640 10 client.18546 handle_command_reply: tid=0 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 client.18546 shutdown 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 msgr2=0x55e40dac5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac5b30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5c14005ab0 tx=0x7f5c140046c0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 shutdown_connections 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e40db27720 0x55e40db29b40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e40dabf750 0x55e40dac5b30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c11ffb640 0 client.18546 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e40dab6b70 0x55e40dac55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 --2- 172.21.15.100:0/1741056812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e40dab6250 0x55e40dac50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 >> 172.21.15.100:0/1741056812 conn(0x55e40dab0980 msgr2=0x55e40daaf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.693+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 shutdown_connections 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c33818740 1 -- 172.21.15.100:0/1741056812 wait complete. 2024-03-12T05:26:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c33818740 20 client.18546 trim_cache size 0 max 16384 2024-03-12T05:26:29.697 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:26:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5c180a1ba0 msgr2=0x7f5c180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5c180a1ba0 0x7f5c180a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c24097100 tx=0x7f5c24065f10 comp rx=0 tx=0).stop 2024-03-12T05:26:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 msgr2=0x7f5c2c1a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 0x7f5c2c1a3670 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400da10 tx=0x7f5c2400ded0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 shutdown_connections 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5c180a1ba0 0x7f5c180a4060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c2c199110 0x7f5c2c1a3670 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c2c105330 0x7f5c2c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 --2- 172.21.15.100:0/2366988424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c2c100260 0x7f5c2c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 >> 172.21.15.100:0/2366988424 conn(0x7f5c2c077180 msgr2=0x7f5c2c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 shutdown_connections 2024-03-12T05:26:29.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:29.694+0000 7f5c30b6c640 1 -- 172.21.15.100:0/2366988424 wait complete. 2024-03-12T05:26:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46591 36.74 MB/sec warmup 73 sec latency 140.686 ms 2024-03-12T05:26:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:29 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2109641407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:26:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:29 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:26:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:29 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:26:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:29 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:26:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:29 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2109641407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:26:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:29 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:26:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:29 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:26:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:29 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:26:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:29 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2109641407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:26:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:29 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:26:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:29 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:26:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:29 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:26:30.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:26:30.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:26:30.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:26:30.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.831+0000 7f478b59e640 1 -- 172.21.15.100:0/1761110064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c105220 msgr2=0x7f478c10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.831+0000 7f478b59e640 1 --2- 172.21.15.100:0/1761110064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c105220 0x7f478c10cdf0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f477c009930 tx=0x7f477c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:26:30.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 -- 172.21.15.100:0/1761110064 shutdown_connections 2024-03-12T05:26:30.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 --2- 172.21.15.100:0/1761110064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c105220 0x7f478c10cdf0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 --2- 172.21.15.100:0/1761110064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100b00 0x7f478c104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 --2- 172.21.15.100:0/1761110064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c100150 0x7f478c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 -- 172.21.15.100:0/1761110064 >> 172.21.15.100:0/1761110064 conn(0x7f478c0fb9d0 msgr2=0x7f478c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 -- 172.21.15.100:0/1761110064 shutdown_connections 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 -- 172.21.15.100:0/1761110064 wait complete. 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 Processor -- start 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.832+0000 7f478b59e640 1 -- start start 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100150 0x7f478c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c105220 0x7f478c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 0x7f478c19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19c8d0 con 0x7f478c19bee0 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19ca40 con 0x7f478c100150 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c19cbb0 con 0x7f478c105220 2024-03-12T05:26:30.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 0x7f478c19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 0x7f478c19c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43556/0 (socket says 172.21.15.100:43556) 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 -- 172.21.15.100:0/38605416 learned_addr learned my addr 172.21.15.100:0/38605416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100150 0x7f478c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f4789d9b640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c105220 0x7f478c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 -- 172.21.15.100:0/38605416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c105220 msgr2=0x7f478c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c105220 0x7f478c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 -- 172.21.15.100:0/38605416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100150 msgr2=0x7f478c197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100150 0x7f478c197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 -- 172.21.15.100:0/38605416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c009560 con 0x7f478c19bee0 2024-03-12T05:26:30.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478a59c640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100150 0x7f478c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:30.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478ad9d640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 0x7f478c19c390 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f477c005940 tx=0x7f477c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:30.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f477b7fe640 1 -- 172.21.15.100:0/38605416 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c004350 con 0x7f478c19bee0 2024-03-12T05:26:30.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f477b7fe640 1 -- 172.21.15.100:0/38605416 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f477c0044b0 con 0x7f478c19bee0 2024-03-12T05:26:30.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f477b7fe640 1 -- 172.21.15.100:0/38605416 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c003cc0 con 0x7f478c19bee0 2024-03-12T05:26:30.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c1a4190 con 0x7f478c19bee0 2024-03-12T05:26:30.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.833+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c1a4610 con 0x7f478c19bee0 2024-03-12T05:26:30.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.836+0000 7f477b7fe640 1 -- 172.21.15.100:0/38605416 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f477c005580 con 0x7f478c19bee0 2024-03-12T05:26:30.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.836+0000 7f47939d0740 1 Processor -- start 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 -- start start 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be982f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be98388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be9839340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be979df90 con 0x556be982f3b0 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be979b4c0 con 0x556be982fd60 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f47939d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9796ca0 con 0x556be9838ec0 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be982f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be98388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be982f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43566/0 (socket says 172.21.15.100:43566) 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f4789d9b640 1 -- 172.21.15.100:0/2594455592 learned_addr learned my addr 172.21.15.100:0/2594455592 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f4789d9b640 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be9839340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f477b7fe640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47740a1ba0 0x7f47740a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 -- 172.21.15.100:0/2594455592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 msgr2=0x556be9839340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be9839340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 -- 172.21.15.100:0/2594455592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 msgr2=0x556be982f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be982f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 -- 172.21.15.100:0/2594455592 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556be97590b0 con 0x556be982fd60 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478ad9d640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47740a1ba0 0x7f47740a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f477b7fe640 1 -- 172.21.15.100:0/38605416 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f477c0ef810 con 0x7f478c19bee0 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.837+0000 7f478a59c640 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be98388a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4784005fd0 tx=0x7f478400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a3152338f53e2f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.838+0000 7f4778ff9640 1 -- 172.21.15.100:0/2594455592 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784011700 con 0x556be982fd60 2024-03-12T05:26:30.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.838+0000 7f4778ff9640 1 -- 172.21.15.100:0/2594455592 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4784015070 con 0x556be982fd60 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.838+0000 7f4778ff9640 1 -- 172.21.15.100:0/2594455592 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478401cd30 con 0x556be982fd60 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.838+0000 7f478ad9d640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47740a1ba0 0x7f47740a4060 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f477c0ed0e0 tx=0x7f477c0e6bd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.838+0000 7f47939d0740 1 -- 172.21.15.100:0/2594455592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 msgr2=0x556be98388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.838+0000 7f47939d0740 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be98388a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4784005fd0 tx=0x7f478400bf90 comp rx=0 tx=0).stop 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- 172.21.15.100:0/2594455592 shutdown_connections 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be9839340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be98388a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 --2- 172.21.15.100:0/2594455592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be982f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- 172.21.15.100:0/2594455592 >> 172.21.15.100:0/2594455592 conn(0x556be9829a10 msgr2=0x556be9829df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- 172.21.15.100:0/2594455592 shutdown_connections 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- 172.21.15.100:0/2594455592 wait complete. 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- start start 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 Processor -- start 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be983c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be983d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be973bf40 con 0x556be982f3b0 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be946a4b0 con 0x556be982fd60 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be971c670 con 0x556be9838ec0 2024-03-12T05:26:30.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f47939d0740 10 client.0 _ll_register_callbacks cb 0x556be9789f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be983c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be983c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43580/0 (socket says 172.21.15.100:43580) 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f478a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f4789d9b640 1 -- 172.21.15.100:0/482385855 learned_addr learned my addr 172.21.15.100:0/482385855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.839+0000 7f4789d9b640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be983d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4789d9b640 1 -- 172.21.15.100:0/482385855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 msgr2=0x556be983d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4789d9b640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be983d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4789d9b640 1 -- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 msgr2=0x556be983cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4789d9b640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4789d9b640 1 -- 172.21.15.100:0/482385855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478400bc70 con 0x556be982f3b0 2024-03-12T05:26:30.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f478a59c640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4789d9b640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be983c9b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4780013c90 tx=0x7f4780015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b6a2d4921f7e7a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4771ffb640 1 -- 172.21.15.100:0/482385855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780004b20 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4771ffb640 1 -- 172.21.15.100:0/482385855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4780013e40 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556be9770f40 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4771ffb640 1 -- 172.21.15.100:0/482385855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4780004c80 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f47939d0740 10 client.18558 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556be9795320 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4771ffb640 1 -- 172.21.15.100:0/482385855 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556be9770f40 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f4771ffb640 1 -- 172.21.15.100:0/482385855 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f47800105b0 con 0x556be982f3b0 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f47939d0740 10 client.18558 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:30.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f47939d0740 10 client.18558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.840+0000 7f47939d0740 20 client.18558 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.841+0000 7f47939d0740 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be984d3d0 0x556be984f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.841+0000 7f47939d0740 4 client.18558 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.841+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556be978d310 con 0x556be984d3d0 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.841+0000 7f478a59c640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be984d3d0 0x556be984f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.841+0000 7f4771ffb640 10 client.18558 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.841+0000 7f478a59c640 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be984d3d0 0x556be984f7f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68f8b5199cbb219 server_cookie=c15c1a3c060fa02c in_seq=0 out_seq=0 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f4771ffb640 1 -- 172.21.15.100:0/482385855 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556be978d310 con 0x556be984d3d0 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f4771ffb640 10 client.18558 handle_command_reply: tid=0 2024-03-12T05:26:30.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 client.18558 shutdown 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 msgr2=0x556be983c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be983c9b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4780013c90 tx=0x7f4780015040 comp rx=0 tx=0).stop 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 shutdown_connections 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be984d3d0 0x556be984f7f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be983d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f4771ffb640 0 client.18558 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 --2- 172.21.15.100:0/482385855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be983c9b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 >> 172.21.15.100:0/482385855 conn(0x556be9829a10 msgr2=0x556be982b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 shutdown_connections 2024-03-12T05:26:30.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 1 -- 172.21.15.100:0/482385855 wait complete. 2024-03-12T05:26:30.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.842+0000 7f47939d0740 20 client.18558 trim_cache size 0 max 16384 2024-03-12T05:26:30.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 Processor -- start 2024-03-12T05:26:30.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 -- start start 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be982f3b0 0x556be983f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 0x556be9836870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be978d310 con 0x556be9838ec0 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9770f40 con 0x556be982fd60 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f47939d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9825350 con 0x556be982f3b0 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 0x556be9836870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f478a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 0x556be9836870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43592/0 (socket says 172.21.15.100:43592) 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.863+0000 7f4789d9b640 1 -- 172.21.15.100:0/3985664470 learned_addr learned my addr 172.21.15.100:0/3985664470 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be982f3b0 0x556be983f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 -- 172.21.15.100:0/3985664470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be982f3b0 msgr2=0x556be983f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be982f3b0 0x556be983f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 -- 172.21.15.100:0/3985664470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 msgr2=0x556be983a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 -- 172.21.15.100:0/3985664470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556be9731c00 con 0x556be9838ec0 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f478a59c640 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4789d9b640 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 0x556be9836870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4780009540 tx=0x7f47800021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343124953bedcc8d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4773fff640 1 -- 172.21.15.100:0/3985664470 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780010420 con 0x556be9838ec0 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4773fff640 1 -- 172.21.15.100:0/3985664470 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4780034070 con 0x556be9838ec0 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.864+0000 7f4773fff640 1 -- 172.21.15.100:0/3985664470 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000e9c0 con 0x556be9838ec0 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 -- 172.21.15.100:0/3985664470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 msgr2=0x556be9836870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 0x556be9836870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4780009540 tx=0x7f47800021d0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 -- 172.21.15.100:0/3985664470 shutdown_connections 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be9838ec0 0x556be9836870 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be983a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 --2- 172.21.15.100:0/3985664470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be982f3b0 0x556be983f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 -- 172.21.15.100:0/3985664470 >> 172.21.15.100:0/3985664470 conn(0x556be9829a10 msgr2=0x556be98376c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 -- 172.21.15.100:0/3985664470 shutdown_connections 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 -- 172.21.15.100:0/3985664470 wait complete. 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.865+0000 7f47939d0740 1 -- start start 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 Processor -- start 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be9842ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be9843220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be98437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be98a3ac0 con 0x556be982f3b0 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be98a3c30 con 0x556be982fd60 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be98a3df0 con 0x556be9838ec0 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f47939d0740 10 client.0 _ll_register_callbacks cb 0x7f478400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:30.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be9842ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f4789d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be9842ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43596/0 (socket says 172.21.15.100:43596) 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f4789d9b640 1 -- 172.21.15.100:0/2132934566 learned_addr learned my addr 172.21.15.100:0/2132934566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f478a59c640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be9843220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.866+0000 7f4789d9b640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be98437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4789d9b640 1 -- 172.21.15.100:0/2132934566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 msgr2=0x556be98437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4789d9b640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be98437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4789d9b640 1 -- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 msgr2=0x556be9843220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4789d9b640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be9843220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4789d9b640 1 -- 172.21.15.100:0/2132934566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556be97590b0 con 0x556be982f3b0 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4789d9b640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be9842ce0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f47800139b0 tx=0x7f4780004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a8065775dc27b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:30.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4771ffb640 1 -- 172.21.15.100:0/2132934566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478003e070 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4771ffb640 1 -- 172.21.15.100:0/2132934566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4780013e40 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556be97bc900 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4771ffb640 1 -- 172.21.15.100:0/2132934566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4780010630 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f47939d0740 10 client.18564 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556be9795320 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.867+0000 7f4771ffb640 1 -- 172.21.15.100:0/2132934566 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556be97bc900 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f4771ffb640 1 -- 172.21.15.100:0/2132934566 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4780041030 con 0x556be982f3b0 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f47939d0740 10 client.18564 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f47939d0740 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f47939d0740 20 client.18564 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f47939d0740 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be98a5120 0x556be98a7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f47939d0740 4 client.18564 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556be9496860 con 0x556be98a5120 2024-03-12T05:26:30.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f478a59c640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be98a5120 0x556be98a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f478a59c640 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be98a5120 0x556be98a7510 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc3b18ffe6fccdf server_cookie=3f4777925cd94c0e in_seq=0 out_seq=0 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.868+0000 7f4771ffb640 10 client.18564 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f4771ffb640 1 -- 172.21.15.100:0/2132934566 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556be9496860 con 0x556be98a5120 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f4771ffb640 10 client.18564 handle_command_reply: tid=0 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 client.18564 shutdown 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 msgr2=0x556be9842ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be9842ce0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f47800139b0 tx=0x7f4780004950 comp rx=0 tx=0).stop 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 shutdown_connections 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556be98a5120 0x556be98a7510 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556be9838ec0 0x556be98437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f4771ffb640 0 client.18564 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556be982fd60 0x556be9843220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 --2- 172.21.15.100:0/2132934566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556be982f3b0 0x556be9842ce0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.869+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 >> 172.21.15.100:0/2132934566 conn(0x556be9829a10 msgr2=0x556be9836870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 shutdown_connections 2024-03-12T05:26:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f47939d0740 1 -- 172.21.15.100:0/2132934566 wait complete. 2024-03-12T05:26:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f47939d0740 20 client.18564 trim_cache size 0 max 16384 2024-03-12T05:26:30.872 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:26:30.872 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:26:30.872 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "290e41ea-f988-4648-ac5a-de8acc66a5cf", 2024-03-12T05:26:30.872 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:26:30.872 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47740a1ba0 msgr2=0x7f47740a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f478b59e640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47740a1ba0 0x7f47740a4060 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f477c0ed0e0 tx=0x7f477c0e6bd0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 msgr2=0x7f478c19c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.870+0000 7f478b59e640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 0x7f478c19c390 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f477c005940 tx=0x7f477c02c650 comp rx=0 tx=0).stop 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 shutdown_connections 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47740a1ba0 0x7f47740a4060 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f478c19bee0 0x7f478c19c390 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c105220 0x7f478c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 --2- 172.21.15.100:0/38605416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f478c100150 0x7f478c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 >> 172.21.15.100:0/38605416 conn(0x7f478c0fb9d0 msgr2=0x7f478c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 shutdown_connections 2024-03-12T05:26:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:30.871+0000 7f478b59e640 1 -- 172.21.15.100:0/38605416 wait complete. 2024-03-12T05:26:30.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:26:30.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 -- 172.21.15.100:0/2540586702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41002d0 msgr2=0x7f01e4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/2540586702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41002d0 0x7f01e4100730 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009900 tx=0x7f01d802bc30 comp rx=0 tx=0).stop 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 -- 172.21.15.100:0/2540586702 shutdown_connections 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/2540586702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e4100c70 0x7f01e410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/2540586702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41002d0 0x7f01e4100730 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/2540586702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e41060a0 0x7f01e4106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 -- 172.21.15.100:0/2540586702 >> 172.21.15.100:0/2540586702 conn(0x7f01e4076e10 msgr2=0x7f01e4077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:31.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.011+0000 7f01e8ae1640 1 -- 172.21.15.100:0/2540586702 shutdown_connections 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.012+0000 7f01e8ae1640 1 -- 172.21.15.100:0/2540586702 wait complete. 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.012+0000 7f01e8ae1640 1 Processor -- start 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.012+0000 7f01e8ae1640 1 -- start start 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.012+0000 7f01e8ae1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 0x7f01e4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.012+0000 7f01e8ae1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e4100c70 0x7f01e4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e8ae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41060a0 0x7f01e419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e8ae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e419c620 con 0x7f01e41060a0 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e8ae1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e419c790 con 0x7f01e4100c70 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e8ae1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e419c950 con 0x7f01e41002d0 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 0x7f01e4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e4100c70 0x7f01e4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41060a0 0x7f01e419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 0x7f01e4197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43724/0 (socket says 172.21.15.100:43724) 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 -- 172.21.15.100:0/4285292743 learned_addr learned my addr 172.21.15.100:0/4285292743 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 -- 172.21.15.100:0/4285292743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e4100c70 msgr2=0x7f01e4197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e4100c70 0x7f01e4197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 -- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41060a0 msgr2=0x7f01e419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41060a0 0x7f01e419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 -- 172.21.15.100:0/4285292743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8009560 con 0x7f01e41002d0 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01e259c640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 0x7f01e4197760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01d400e9f0 tx=0x7f01d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.013+0000 7f01d37fe640 1 -- 172.21.15.100:0/4285292743 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d4009700 con 0x7f01e41002d0 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.014+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e419cc30 con 0x7f01e41002d0 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.014+0000 7f01d37fe640 1 -- 172.21.15.100:0/4285292743 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01d400cd80 con 0x7f01e41002d0 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.014+0000 7f01d37fe640 1 -- 172.21.15.100:0/4285292743 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d4018920 con 0x7f01e41002d0 2024-03-12T05:26:31.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.014+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e41a44b0 con 0x7f01e41002d0 2024-03-12T05:26:31.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.016+0000 7f01d37fe640 1 -- 172.21.15.100:0/4285292743 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f01d4018a80 con 0x7f01e41002d0 2024-03-12T05:26:31.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01d37fe640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01cc0a1ba0 0x7f01cc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 Processor -- start 2024-03-12T05:26:31.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01d37fe640 1 -- 172.21.15.100:0/4285292743 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f01d4014070 con 0x7f01e41002d0 2024-03-12T05:26:31.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01cc0a1ba0 0x7f01cc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 -- start start 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x56317749f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563177414280 con 0x5631774b29e0 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631774117b0 con 0x56317749f440 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.017+0000 7f01eb78a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317740cf90 con 0x56317749fe00 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43630/0 (socket says 172.21.15.100:43630) 2024-03-12T05:26:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 -- 172.21.15.100:0/4255718004 learned_addr learned my addr 172.21.15.100:0/4255718004 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01cc0a1ba0 0x7f01cc0a4060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01d8006010 tx=0x7f01d8005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x56317749f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 -- 172.21.15.100:0/4255718004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 msgr2=0x5631774b97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 -- 172.21.15.100:0/4255718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 msgr2=0x56317749f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x56317749f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 -- 172.21.15.100:0/4255718004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631773cb570 con 0x5631774b29e0 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01e259c640 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b2dc0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cb290 tx=0x7f01d409a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ed62f39043fcb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01d0ff9640 1 -- 172.21.15.100:0/4255718004 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40a2eb0 con 0x5631774b29e0 2024-03-12T05:26:31.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01d0ff9640 1 -- 172.21.15.100:0/4255718004 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01d4010b80 con 0x5631774b29e0 2024-03-12T05:26:31.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.018+0000 7f01d0ff9640 1 -- 172.21.15.100:0/4255718004 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40ba320 con 0x5631774b29e0 2024-03-12T05:26:31.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 -- 172.21.15.100:0/4255718004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 msgr2=0x5631774b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b2dc0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cb290 tx=0x7f01d409a120 comp rx=0 tx=0).stop 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 -- 172.21.15.100:0/4255718004 shutdown_connections 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x56317749f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 --2- 172.21.15.100:0/4255718004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b2dc0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 -- 172.21.15.100:0/4255718004 >> 172.21.15.100:0/4255718004 conn(0x5631774a1fb0 msgr2=0x5631774a2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 -- 172.21.15.100:0/4255718004 shutdown_connections 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.019+0000 7f01eb78a740 1 -- 172.21.15.100:0/4255718004 wait complete. 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 -- start start 2024-03-12T05:26:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 Processor -- start 2024-03-12T05:26:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 0x5631774b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631770e0430 con 0x56317749fe00 2024-03-12T05:26:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317709ce00 con 0x56317749f440 2024-03-12T05:26:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563177391fd0 con 0x5631774b29e0 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01eb78a740 10 client.0 _ll_register_callbacks cb 0x5631773e7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 0x5631774b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 0x5631774b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43644/0 (socket says 172.21.15.100:43644) 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e1d9b640 1 -- 172.21.15.100:0/1108874311 learned_addr learned my addr 172.21.15.100:0/1108874311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e1d9b640 1 -- 172.21.15.100:0/1108874311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 msgr2=0x5631774b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.020+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01e1d9b640 1 -- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 msgr2=0x5631774b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01e1d9b640 1 -- 172.21.15.100:0/1108874311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e41a44b0 con 0x56317749fe00 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 0x5631774b69d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009900 tx=0x7f01d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d846941d2235575d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01c9ffb640 1 -- 172.21.15.100:0/1108874311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80043d0 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01c9ffb640 1 -- 172.21.15.100:0/1108874311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f01d8047040 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56317710c7e0 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01c9ffb640 1 -- 172.21.15.100:0/1108874311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d8004df0 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01eb78a740 10 client.18576 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56317740b610 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.021+0000 7f01c9ffb640 1 -- 172.21.15.100:0/1108874311 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56317710c7e0 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01c9ffb640 1 -- 172.21.15.100:0/1108874311 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f01d805c070 con 0x56317749fe00 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01eb78a740 10 client.18576 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01eb78a740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01eb78a740 20 client.18576 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01eb78a740 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631774c7210 0x5631774c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01eb78a740 4 client.18576 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563177149720 con 0x5631774c7210 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01e259c640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631774c7210 0x5631774c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01c9ffb640 10 client.18576 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.022+0000 7f01e259c640 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631774c7210 0x5631774c9630 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935ec01315afa133 server_cookie=6bb734fe631543cd in_seq=0 out_seq=0 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01c9ffb640 1 -- 172.21.15.100:0/1108874311 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563177149720 con 0x5631774c7210 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01c9ffb640 10 client.18576 handle_command_reply: tid=0 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 client.18576 shutdown 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 msgr2=0x5631774b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 0x5631774b69d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009900 tx=0x7f01d8009260 comp rx=0 tx=0).stop 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 shutdown_connections 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631774c7210 0x5631774c9630 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749fe00 0x5631774b69d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01c9ffb640 0 client.18576 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 --2- 172.21.15.100:0/1108874311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 >> 172.21.15.100:0/1108874311 conn(0x5631774a1fb0 msgr2=0x5631774a16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 shutdown_connections 2024-03-12T05:26:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 1 -- 172.21.15.100:0/1108874311 wait complete. 2024-03-12T05:26:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.023+0000 7f01eb78a740 20 client.18576 trim_cache size 0 max 16384 2024-03-12T05:26:31.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 Processor -- start 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 -- start start 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317749b120 con 0x5631774b29e0 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631774c3540 con 0x56317749f440 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01eb78a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317749b320 con 0x56317749fe00 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01e2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43746/0 (socket says 172.21.15.100:43746) 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.040+0000 7f01e2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43658/0 (socket says 172.21.15.100:43658) 2024-03-12T05:26:31.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 -- 172.21.15.100:0/3801334927 learned_addr learned my addr 172.21.15.100:0/3801334927 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e259c640 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 -- 172.21.15.100:0/3801334927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 msgr2=0x5631774b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 -- 172.21.15.100:0/3801334927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 msgr2=0x5631774b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 -- 172.21.15.100:0/3801334927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631773a7ac0 con 0x5631774b29e0 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b9640 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f01dc011aa0 tx=0x7f01dc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c80a7e5f31d9626 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01cbfff640 1 -- 172.21.15.100:0/3801334927 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc016050 con 0x5631774b29e0 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01cbfff640 1 -- 172.21.15.100:0/3801334927 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01dc017040 con 0x5631774b29e0 2024-03-12T05:26:31.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.041+0000 7f01cbfff640 1 -- 172.21.15.100:0/3801334927 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc004290 con 0x5631774b29e0 2024-03-12T05:26:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 -- 172.21.15.100:0/3801334927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 msgr2=0x5631774b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b9640 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f01dc011aa0 tx=0x7f01dc009500 comp rx=0 tx=0).stop 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 -- 172.21.15.100:0/3801334927 shutdown_connections 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631774b29e0 0x5631774b9640 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56317749fe00 0x5631774b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 --2- 172.21.15.100:0/3801334927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749f440 0x5631774b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 -- 172.21.15.100:0/3801334927 >> 172.21.15.100:0/3801334927 conn(0x5631774a1fb0 msgr2=0x5631774b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 -- 172.21.15.100:0/3801334927 shutdown_connections 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.042+0000 7f01eb78a740 1 -- 172.21.15.100:0/3801334927 wait complete. 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 -- start start 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 Processor -- start 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 0x563177517da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749fe00 0x5631775182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631774bd9f0 con 0x56317749f440 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631774bdb60 con 0x56317749fe00 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631774bdd20 con 0x5631774b29e0 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01eb78a740 10 client.0 _ll_register_callbacks cb 0x7f01d4096340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01e259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 0x563177517da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01e259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 0x563177517da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43670/0 (socket says 172.21.15.100:43670) 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01e259c640 1 -- 172.21.15.100:0/472026027 learned_addr learned my addr 172.21.15.100:0/472026027 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749fe00 0x5631775182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.043+0000 7f01e2d9d640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01e259c640 1 -- 172.21.15.100:0/472026027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 msgr2=0x5631774bd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01e259c640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774bd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01e259c640 1 -- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749fe00 msgr2=0x5631775182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:26:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01e259c640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749fe00 0x5631775182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01e259c640 1 -- 172.21.15.100:0/472026027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56317740b610 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01e259c640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 0x563177517da0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cb2e0 tx=0x7f01d4010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f878cba3a33e878 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01c9ffb640 1 -- 172.21.15.100:0/472026027 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40a2eb0 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01c9ffb640 1 -- 172.21.15.100:0/472026027 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f01d409a7c0 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01d4010b80 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01c9ffb640 1 -- 172.21.15.100:0/472026027 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d40cb4b0 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 10 client.18582 fetch_fsmap learned FSMap version 27 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f01dc011750 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01c9ffb640 1 -- 172.21.15.100:0/472026027 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01d4010b80 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01c9ffb640 1 -- 172.21.15.100:0/472026027 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f01d40a8070 con 0x56317749f440 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 10 client.18582 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:26:31.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 20 client.18582 populate_metadata read hostname 'smithi100' 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56317751f9b0 0x563177521da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 4 client.18582 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.044+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563177198890 con 0x56317751f9b0 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56317751f9b0 0x563177521da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01c9ffb640 10 client.18582 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01e1d9b640 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56317751f9b0 0x563177521da0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba44a32277549bba server_cookie=7b0392f9633fef0c in_seq=0 out_seq=0 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01c9ffb640 1 -- 172.21.15.100:0/472026027 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563177198890 con 0x56317751f9b0 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01c9ffb640 10 client.18582 handle_command_reply: tid=0 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 client.18582 shutdown 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 msgr2=0x563177517da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 0x563177517da0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cb2e0 tx=0x7f01d4010f50 comp rx=0 tx=0).stop 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 shutdown_connections 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56317751f9b0 0x563177521da0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631774b29e0 0x5631774bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56317749fe00 0x5631775182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01c9ffb640 0 client.18582 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 --2- 172.21.15.100:0/472026027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56317749f440 0x563177517da0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 >> 172.21.15.100:0/472026027 conn(0x5631774a1fb0 msgr2=0x56317749e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 shutdown_connections 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.045+0000 7f01eb78a740 1 -- 172.21.15.100:0/472026027 wait complete. 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01eb78a740 20 client.18582 trim_cache size 0 max 16384 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:26:31.046 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: "290e41ea-f988-4648-ac5a-de8acc66a5cf": { 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "290e41ea-f988-4648-ac5a-de8acc66a5cf", 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:26:31.047 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01cc0a1ba0 msgr2=0x7f01cc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01cc0a1ba0 0x7f01cc0a4060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01d8006010 tx=0x7f01d8005e40 comp rx=0 tx=0).stop 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 msgr2=0x7f01e4197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 0x7f01e4197760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01d400e9f0 tx=0x7f01d400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 shutdown_connections 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f01cc0a1ba0 0x7f01cc0a4060 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e41060a0 0x7f01e419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e4100c70 0x7f01e4197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 --2- 172.21.15.100:0/4285292743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e41002d0 0x7f01e4197760 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:26:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 >> 172.21.15.100:0/4285292743 conn(0x7f01e4076e10 msgr2=0x7f01e40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:26:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 shutdown_connections 2024-03-12T05:26:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:26:31.046+0000 7f01e8ae1640 1 -- 172.21.15.100:0/4285292743 wait complete. 2024-03-12T05:26:31.051 INFO:tasks.cephfs.filesystem:scrub status for tag:290e41ea-f988-4648-ac5a-de8acc66a5cf - {'path': '/', 'tag': '290e41ea-f988-4648-ac5a-de8acc66a5cf', 'options': 'recursive,force'} 2024-03-12T05:26:31.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47165 36.70 MB/sec warmup 74 sec latency 250.956 ms 2024-03-12T05:26:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: pgmap v451: 97 pgs: 97 active+clean; 891 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 530 op/s 2024-03-12T05:26:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:26:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:26:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:26:31.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: pgmap v451: 97 pgs: 97 active+clean; 891 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 530 op/s 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:26:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: pgmap v451: 97 pgs: 97 active+clean; 891 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 530 op/s 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:26:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47849 36.79 MB/sec warmup 75 sec latency 172.922 ms 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:26:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:32 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:26:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:32 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:26:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:32 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:33.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48286 36.60 MB/sec warmup 76 sec latency 182.711 ms 2024-03-12T05:26:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:33 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:33 smithi100 ceph-mon[32210]: pgmap v452: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 793 op/s 2024-03-12T05:26:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:33 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:33 smithi156 ceph-mon[36270]: pgmap v452: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 793 op/s 2024-03-12T05:26:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:33 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:26:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:33 smithi161 ceph-mon[36043]: pgmap v452: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 793 op/s 2024-03-12T05:26:34.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48825 36.38 MB/sec warmup 77 sec latency 237.430 ms 2024-03-12T05:26:34.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:34 smithi100 ceph-mon[32210]: pgmap v453: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-03-12T05:26:34.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:34 smithi156 ceph-mon[36270]: pgmap v453: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-03-12T05:26:34.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:34 smithi161 ceph-mon[36043]: pgmap v453: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-03-12T05:26:35.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49614 36.47 MB/sec warmup 78 sec latency 62.394 ms 2024-03-12T05:26:36.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50318 36.55 MB/sec warmup 79 sec latency 145.797 ms 2024-03-12T05:26:37.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50886 36.45 MB/sec warmup 80 sec latency 92.205 ms 2024-03-12T05:26:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:36 smithi100 ceph-mon[32210]: pgmap v454: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-03-12T05:26:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:36 smithi156 ceph-mon[36270]: pgmap v454: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-03-12T05:26:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:36 smithi161 ceph-mon[36043]: pgmap v454: 97 pgs: 97 active+clean; 906 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-03-12T05:26:38.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 51617 36.48 MB/sec warmup 81 sec latency 81.885 ms 2024-03-12T05:26:39.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52250 36.57 MB/sec warmup 82 sec latency 221.481 ms 2024-03-12T05:26:39.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:38 smithi100 ceph-mon[32210]: pgmap v455: 97 pgs: 97 active+clean; 933 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 742 op/s 2024-03-12T05:26:39.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:38 smithi156 ceph-mon[36270]: pgmap v455: 97 pgs: 97 active+clean; 933 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 742 op/s 2024-03-12T05:26:39.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:38 smithi161 ceph-mon[36043]: pgmap v455: 97 pgs: 97 active+clean; 933 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 742 op/s 2024-03-12T05:26:40.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52704 36.39 MB/sec warmup 83 sec latency 55.398 ms 2024-03-12T05:26:41.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53086 36.15 MB/sec warmup 84 sec latency 299.181 ms 2024-03-12T05:26:41.272 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:40 smithi161 ceph-mon[36043]: pgmap v456: 97 pgs: 97 active+clean; 933 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.6 MiB/s wr, 489 op/s 2024-03-12T05:26:41.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:40 smithi100 ceph-mon[32210]: pgmap v456: 97 pgs: 97 active+clean; 933 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.6 MiB/s wr, 489 op/s 2024-03-12T05:26:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:40 smithi156 ceph-mon[36270]: pgmap v456: 97 pgs: 97 active+clean; 933 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.6 MiB/s wr, 489 op/s 2024-03-12T05:26:42.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53725 36.04 MB/sec warmup 85 sec latency 99.165 ms 2024-03-12T05:26:43.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54540 36.18 MB/sec warmup 86 sec latency 97.988 ms 2024-03-12T05:26:43.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:43 smithi100 ceph-mon[32210]: pgmap v457: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 740 op/s 2024-03-12T05:26:43.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:43 smithi156 ceph-mon[36270]: pgmap v457: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 740 op/s 2024-03-12T05:26:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:43 smithi161 ceph-mon[36043]: pgmap v457: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 740 op/s 2024-03-12T05:26:44.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55221 36.07 MB/sec warmup 87 sec latency 120.126 ms 2024-03-12T05:26:45.232 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56087 36.37 MB/sec warmup 88 sec latency 76.451 ms 2024-03-12T05:26:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:45 smithi100 ceph-mon[32210]: pgmap v458: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 476 op/s 2024-03-12T05:26:45.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:45 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:26:45.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:45 smithi156 ceph-mon[36270]: pgmap v458: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 476 op/s 2024-03-12T05:26:45.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:45 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:26:45.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:45 smithi161 ceph-mon[36043]: pgmap v458: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 476 op/s 2024-03-12T05:26:45.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:45 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:26:46.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56602 36.33 MB/sec warmup 89 sec latency 194.850 ms 2024-03-12T05:26:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:46 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:26:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:46 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:26:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:46 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:26:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:46 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:26:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:46 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:26:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:46 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:26:47.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56943 36.11 MB/sec warmup 90 sec latency 272.020 ms 2024-03-12T05:26:47.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:47 smithi100 ceph-mon[32210]: pgmap v459: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 476 op/s 2024-03-12T05:26:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:47 smithi156 ceph-mon[36270]: pgmap v459: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 476 op/s 2024-03-12T05:26:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:47 smithi161 ceph-mon[36043]: pgmap v459: 97 pgs: 97 active+clean; 969 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 476 op/s 2024-03-12T05:26:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57396 35.91 MB/sec warmup 91 sec latency 111.537 ms 2024-03-12T05:26:48.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:48 smithi100 ceph-mon[32210]: pgmap v460: 97 pgs: 97 active+clean; 1007 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 719 op/s 2024-03-12T05:26:48.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:48 smithi156 ceph-mon[36270]: pgmap v460: 97 pgs: 97 active+clean; 1007 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 719 op/s 2024-03-12T05:26:48.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:48 smithi161 ceph-mon[36043]: pgmap v460: 97 pgs: 97 active+clean; 1007 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 719 op/s 2024-03-12T05:26:49.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58123 35.91 MB/sec warmup 92 sec latency 73.823 ms 2024-03-12T05:26:50.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58916 35.97 MB/sec warmup 93 sec latency 72.137 ms 2024-03-12T05:26:51.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59501 35.94 MB/sec warmup 94 sec latency 136.917 ms 2024-03-12T05:26:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:50 smithi100 ceph-mon[32210]: pgmap v461: 97 pgs: 97 active+clean; 1007 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.6 MiB/s wr, 493 op/s 2024-03-12T05:26:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:50 smithi156 ceph-mon[36270]: pgmap v461: 97 pgs: 97 active+clean; 1007 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.6 MiB/s wr, 493 op/s 2024-03-12T05:26:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:50 smithi161 ceph-mon[36043]: pgmap v461: 97 pgs: 97 active+clean; 1007 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.6 MiB/s wr, 493 op/s 2024-03-12T05:26:52.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60101 35.93 MB/sec warmup 95 sec latency 212.880 ms 2024-03-12T05:26:53.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60790 35.98 MB/sec warmup 96 sec latency 107.664 ms 2024-03-12T05:26:53.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:52 smithi100 ceph-mon[32210]: pgmap v462: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 765 op/s 2024-03-12T05:26:53.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:52 smithi156 ceph-mon[36270]: pgmap v462: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 765 op/s 2024-03-12T05:26:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:52 smithi161 ceph-mon[36043]: pgmap v462: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 765 op/s 2024-03-12T05:26:54.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61263 35.88 MB/sec warmup 97 sec latency 167.021 ms 2024-03-12T05:26:55.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61754 35.73 MB/sec warmup 98 sec latency 89.946 ms 2024-03-12T05:26:55.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:54 smithi156 ceph-mon[36270]: pgmap v463: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 514 op/s 2024-03-12T05:26:55.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:54 smithi100 ceph-mon[32210]: pgmap v463: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 514 op/s 2024-03-12T05:26:55.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:55 smithi161 ceph-mon[36043]: pgmap v463: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 514 op/s 2024-03-12T05:26:56.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62346 35.69 MB/sec warmup 99 sec latency 52.982 ms 2024-03-12T05:26:57.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62795 35.54 MB/sec warmup 100 sec latency 256.265 ms 2024-03-12T05:26:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:57 smithi100 ceph-mon[32210]: pgmap v464: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 514 op/s 2024-03-12T05:26:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:57 smithi156 ceph-mon[36270]: pgmap v464: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 514 op/s 2024-03-12T05:26:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:57 smithi161 ceph-mon[36043]: pgmap v464: 97 pgs: 97 active+clean; 1006 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 514 op/s 2024-03-12T05:26:58.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63631 35.59 MB/sec warmup 101 sec latency 81.763 ms 2024-03-12T05:26:59.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64406 35.72 MB/sec warmup 102 sec latency 84.150 ms 2024-03-12T05:26:59.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:26:59 smithi100 ceph-mon[32210]: pgmap v465: 97 pgs: 97 active+clean; 940 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 762 op/s 2024-03-12T05:26:59.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:26:59 smithi156 ceph-mon[36270]: pgmap v465: 97 pgs: 97 active+clean; 940 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 762 op/s 2024-03-12T05:26:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:26:59 smithi161 ceph-mon[36043]: pgmap v465: 97 pgs: 97 active+clean; 940 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 762 op/s 2024-03-12T05:27:00.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65036 35.80 MB/sec warmup 103 sec latency 58.250 ms 2024-03-12T05:27:01.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:27:01.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:27:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 -- 172.21.15.100:0/3669993447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5664100180 msgr2=0x7f566410c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 --2- 172.21.15.100:0/3669993447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5664100180 0x7f566410c480 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f565c009cd0 tx=0x7f565c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 -- 172.21.15.100:0/3669993447 shutdown_connections 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 --2- 172.21.15.100:0/3669993447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5664100180 0x7f566410c480 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 --2- 172.21.15.100:0/3669993447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56640ff7c0 0x7f56640ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 --2- 172.21.15.100:0/3669993447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56641045a0 0x7f5664104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 -- 172.21.15.100:0/3669993447 >> 172.21.15.100:0/3669993447 conn(0x7f5664075170 msgr2=0x7f5664075580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 -- 172.21.15.100:0/3669993447 shutdown_connections 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.179+0000 7f56697a0640 1 -- 172.21.15.100:0/3669993447 wait complete. 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 Processor -- start 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 -- start start 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 0x7f56641975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5664100180 0x7f5664197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56641045a0 0x7f566419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566419c440 con 0x7f56640ff7c0 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566419c5b0 con 0x7f56641045a0 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f56697a0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566419c770 con 0x7f5664100180 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 0x7f56641975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 0x7f56641975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38852/0 (socket says 172.21.15.100:38852) 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.180+0000 7f5663fff640 1 -- 172.21.15.100:0/4094889980 learned_addr learned my addr 172.21.15.100:0/4094889980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5668f9f640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56641045a0 0x7f566419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f56637fe640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5664100180 0x7f5664197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5663fff640 1 -- 172.21.15.100:0/4094889980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5664100180 msgr2=0x7f5664197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5663fff640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5664100180 0x7f5664197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5663fff640 1 -- 172.21.15.100:0/4094889980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56641045a0 msgr2=0x7f566419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5663fff640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56641045a0 0x7f566419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5663fff640 1 -- 172.21.15.100:0/4094889980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c009900 con 0x7f56640ff7c0 2024-03-12T05:27:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f5663fff640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 0x7f56641975d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f565400d770 tx=0x7f565400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f56617fa640 1 -- 172.21.15.100:0/4094889980 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654009c30 con 0x7f56640ff7c0 2024-03-12T05:27:01.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566419ca50 con 0x7f56640ff7c0 2024-03-12T05:27:01.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f56617fa640 1 -- 172.21.15.100:0/4094889980 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f565400de80 con 0x7f56640ff7c0 2024-03-12T05:27:01.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f56617fa640 1 -- 172.21.15.100:0/4094889980 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654010460 con 0x7f56640ff7c0 2024-03-12T05:27:01.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.181+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56641a42d0 con 0x7f56640ff7c0 2024-03-12T05:27:01.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.183+0000 7f56617fa640 1 -- 172.21.15.100:0/4094889980 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5654003f60 con 0x7f56640ff7c0 2024-03-12T05:27:01.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.183+0000 7f566d5d5740 1 Processor -- start 2024-03-12T05:27:01.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 -- start start 2024-03-12T05:27:01.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f56617fa640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f564c0a1ba0 0x7f564c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e4f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e500dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e464280 con 0x557e3e4f54d0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e4617b0 con 0x557e3e4f5df0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f566d5d5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e45cf90 con 0x557e3e4fe9d0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5668f9f640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f564c0a1ba0 0x7f564c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f56617fa640 1 -- 172.21.15.100:0/4094889980 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f56540cd550 con 0x7f56640ff7c0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e4f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e4f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38854/0 (socket says 172.21.15.100:38854) 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 -- 172.21.15.100:0/2532943299 learned_addr learned my addr 172.21.15.100:0/2532943299 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f56637fe640 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 -- 172.21.15.100:0/2532943299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 msgr2=0x557e3e500dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e500dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 -- 172.21.15.100:0/2532943299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 msgr2=0x557e3e4fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 -- 172.21.15.100:0/2532943299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e3e41b570 con 0x557e3e4f54d0 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f56637fe640 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5668f9f640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f564c0a1ba0 0x7f564c0a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f565c005cc0 tx=0x7f565c032ee0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f5663fff640 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e4f58b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5654009fd0 tx=0x7f5654099b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413c9c9cdfab694 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.184+0000 7f564a7fc640 1 -- 172.21.15.100:0/2532943299 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565409e290 con 0x557e3e4f54d0 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f564a7fc640 1 -- 172.21.15.100:0/2532943299 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56540a28e0 con 0x557e3e4f54d0 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f564a7fc640 1 -- 172.21.15.100:0/2532943299 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540967b0 con 0x557e3e4f54d0 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 -- 172.21.15.100:0/2532943299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 msgr2=0x557e3e4f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e4f58b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5654009fd0 tx=0x7f5654099b60 comp rx=0 tx=0).stop 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 -- 172.21.15.100:0/2532943299 shutdown_connections 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e500dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2532943299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e4f58b0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 -- 172.21.15.100:0/2532943299 >> 172.21.15.100:0/2532943299 conn(0x557e3e4efc00 msgr2=0x557e3e4effe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 -- 172.21.15.100:0/2532943299 shutdown_connections 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 -- 172.21.15.100:0/2532943299 wait complete. 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.185+0000 7f566d5d5740 1 -- start start 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 Processor -- start 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e4fe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 0x557e3e4fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e130430 con 0x557e3e4f5df0 2024-03-12T05:27:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e0ece00 con 0x557e3e4f54d0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e3e1fd0 con 0x557e3e4fe9d0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f566d5d5740 10 client.0 _ll_register_callbacks cb 0x557e3e437fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 0x557e3e4fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 0x557e3e4fb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38862/0 (socket says 172.21.15.100:38862) 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 -- 172.21.15.100:0/19356252 learned_addr learned my addr 172.21.15.100:0/19356252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f5663fff640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e4fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f5668f9f640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 -- 172.21.15.100:0/19356252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 msgr2=0x557e3e4fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 -- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 msgr2=0x557e3e4fe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e4fe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f56637fe640 1 -- 172.21.15.100:0/19356252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56641a42d0 con 0x557e3e4f5df0 2024-03-12T05:27:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.186+0000 7f5663fff640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e4fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56637fe640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 0x557e3e4fb7a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f565800e700 tx=0x7f565800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7fc529ded54417 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56437fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56437fe640 1 -- 172.21.15.100:0/19356252 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800dbc0 con 0x557e3e4f5df0 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56437fe640 1 -- 172.21.15.100:0/19356252 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f565800dd20 con 0x557e3e4f5df0 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e3e15c7e0 con 0x557e3e4f5df0 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56437fe640 1 -- 172.21.15.100:0/19356252 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56580203f0 con 0x557e3e4f5df0 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 10 client.18594 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e3e45b610 con 0x557e3e4f5df0 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56437fe640 1 -- 172.21.15.100:0/19356252 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557e3e15c7e0 con 0x557e3e4f5df0 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f56437fe640 1 -- 172.21.15.100:0/19356252 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f56580185b0 con 0x557e3e4f5df0 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 10 client.18594 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 20 client.18594 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e50ec40 0x557e3e511060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 4 client.18594 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e3e199720 con 0x557e3e50ec40 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.187+0000 7f5663fff640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e50ec40 0x557e3e511060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.188+0000 7f56437fe640 10 client.18594 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.188+0000 7f5663fff640 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e50ec40 0x557e3e511060 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cec35d8bbd5fb0c server_cookie=1be01b123ca2a918 in_seq=0 out_seq=0 2024-03-12T05:27:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.188+0000 7f56437fe640 1 -- 172.21.15.100:0/19356252 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e3e199720 con 0x557e3e50ec40 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.188+0000 7f56437fe640 10 client.18594 handle_command_reply: tid=0 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.188+0000 7f566d5d5740 1 client.18594 shutdown 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 msgr2=0x557e3e4fb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 0x557e3e4fb7a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f565800e700 tx=0x7f565800ebc0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 shutdown_connections 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e50ec40 0x557e3e511060 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f5df0 0x557e3e4fb7a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f56437fe640 0 client.18594 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 --2- 172.21.15.100:0/19356252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e4fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 >> 172.21.15.100:0/19356252 conn(0x557e3e4efc00 msgr2=0x557e3e4f1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 shutdown_connections 2024-03-12T05:27:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 1 -- 172.21.15.100:0/19356252 wait complete. 2024-03-12T05:27:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.189+0000 7f566d5d5740 20 client.18594 trim_cache size 0 max 16384 2024-03-12T05:27:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.203+0000 7f566d5d5740 1 Processor -- start 2024-03-12T05:27:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.203+0000 7f566d5d5740 1 -- start start 2024-03-12T05:27:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e50a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f566d5d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e4eb300 con 0x557e3e4f54d0 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f566d5d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e4f0c00 con 0x557e3e4f5df0 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f566d5d5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e4f5130 con 0x557e3e4fe9d0 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e50a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e50a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38876/0 (socket says 172.21.15.100:38876) 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 -- 172.21.15.100:0/967385003 learned_addr learned my addr 172.21.15.100:0/967385003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f56637fe640 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 -- 172.21.15.100:0/967385003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 msgr2=0x557e3e4f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 -- 172.21.15.100:0/967385003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 msgr2=0x557e3e4f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 -- 172.21.15.100:0/967385003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e3e3f7ac0 con 0x557e3e4f54d0 2024-03-12T05:27:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5663fff640 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e50a330 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5654096b10 tx=0x7f5654000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d3812ee241cc4e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5649ffb640 1 -- 172.21.15.100:0/967385003 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654098440 con 0x557e3e4f54d0 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5649ffb640 1 -- 172.21.15.100:0/967385003 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56540035f0 con 0x557e3e4f54d0 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.204+0000 7f5649ffb640 1 -- 172.21.15.100:0/967385003 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654004470 con 0x557e3e4f54d0 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 -- 172.21.15.100:0/967385003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 msgr2=0x557e3e50a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e50a330 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5654096b10 tx=0x7f5654000bc0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 -- 172.21.15.100:0/967385003 shutdown_connections 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4fe9d0 0x557e3e4f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f5df0 0x557e3e4f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 --2- 172.21.15.100:0/967385003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4f54d0 0x557e3e50a330 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 -- 172.21.15.100:0/967385003 >> 172.21.15.100:0/967385003 conn(0x557e3e4efc00 msgr2=0x557e3e4eabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 -- 172.21.15.100:0/967385003 shutdown_connections 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 -- 172.21.15.100:0/967385003 wait complete. 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 -- start start 2024-03-12T05:27:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.205+0000 7f566d5d5740 1 Processor -- start 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e505920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4f5df0 0x557e3e501dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 0x557e3e502310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e5061d0 con 0x557e3e4fe9d0 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e502880 con 0x557e3e4f54d0 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3e502a40 con 0x557e3e4f5df0 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 10 client.0 _ll_register_callbacks cb 0x7f56540038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 0x557e3e502310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 0x557e3e502310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38886/0 (socket says 172.21.15.100:38886) 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 -- 172.21.15.100:0/2411308171 learned_addr learned my addr 172.21.15.100:0/2411308171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f56637fe640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4f5df0 0x557e3e501dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5663fff640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e505920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 -- 172.21.15.100:0/2411308171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4f5df0 msgr2=0x557e3e501dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4f5df0 0x557e3e501dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 -- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 msgr2=0x557e3e505920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e505920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 -- 172.21.15.100:0/2411308171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e3e41b570 con 0x557e3e4fe9d0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f5668f9f640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 0x557e3e502310 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f565c009cd0 tx=0x7f565c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0819c2df865c39a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f56437fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f56437fe640 1 -- 172.21.15.100:0/2411308171 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c003b70 con 0x557e3e4fe9d0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f56437fe640 1 -- 172.21.15.100:0/2411308171 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f565c046070 con 0x557e3e4fe9d0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.206+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56540a28e0 con 0x557e3e4fe9d0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f56437fe640 1 -- 172.21.15.100:0/2411308171 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f565c0324c0 con 0x557e3e4fe9d0 2024-03-12T05:27:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 10 client.18600 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e3e41d6a0 con 0x557e3e4fe9d0 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f56437fe640 1 -- 172.21.15.100:0/2411308171 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56540a28e0 con 0x557e3e4fe9d0 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f56437fe640 1 -- 172.21.15.100:0/2411308171 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f565c04c440 con 0x557e3e4fe9d0 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 10 client.18600 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 20 client.18600 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e5666c0 0x557e3e568ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 4 client.18600 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e3e1e8890 con 0x557e3e5666c0 2024-03-12T05:27:01.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f5663fff640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e5666c0 0x557e3e568ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f56437fe640 10 client.18600 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.207+0000 7f5663fff640 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e5666c0 0x557e3e568ae0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95a9528ee85e42b server_cookie=82cccb701b4f7016 in_seq=0 out_seq=0 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56437fe640 1 -- 172.21.15.100:0/2411308171 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557e3e1e8890 con 0x557e3e5666c0 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56437fe640 10 client.18600 handle_command_reply: tid=0 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 client.18600 shutdown 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 msgr2=0x557e3e502310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 0x557e3e502310 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f565c009cd0 tx=0x7f565c0095c0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 shutdown_connections 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557e3e5666c0 0x557e3e568ae0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e3e4fe9d0 0x557e3e502310 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e3e4f5df0 0x557e3e501dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56437fe640 0 client.18600 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 --2- 172.21.15.100:0/2411308171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557e3e4f54d0 0x557e3e505920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 >> 172.21.15.100:0/2411308171 conn(0x557e3e4efc00 msgr2=0x557e3e4f6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 shutdown_connections 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 1 -- 172.21.15.100:0/2411308171 wait complete. 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f566d5d5740 20 client.18600 trim_cache size 0 max 16384 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:27:01.210 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f564c0a1ba0 msgr2=0x7f564c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f564c0a1ba0 0x7f564c0a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f565c005cc0 tx=0x7f565c032ee0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 msgr2=0x7f56641975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 0x7f56641975d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f565400d770 tx=0x7f565400dc30 comp rx=0 tx=0).stop 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 shutdown_connections 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f564c0a1ba0 0x7f564c0a4060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56641045a0 0x7f566419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5664100180 0x7f5664197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 --2- 172.21.15.100:0/4094889980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640ff7c0 0x7f56641975d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.208+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 >> 172.21.15.100:0/4094889980 conn(0x7f5664075170 msgr2=0x7f56640fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.209+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 shutdown_connections 2024-03-12T05:27:01.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.209+0000 7f56697a0640 1 -- 172.21.15.100:0/4094889980 wait complete. 2024-03-12T05:27:01.214 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:27:01.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:27:01.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65422 35.68 MB/sec warmup 104 sec latency 264.522 ms 2024-03-12T05:27:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:01 smithi100 ceph-mon[32210]: pgmap v466: 97 pgs: 97 active+clean; 940 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.6 MiB/s wr, 519 op/s 2024-03-12T05:27:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.336+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/2061747553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d4105380 msgr2=0x7fb2d410cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.336+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/2061747553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d4105380 0x7fb2d410cf50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009cd0 tx=0x7fb2c402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/2061747553 shutdown_connections 2024-03-12T05:27:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/2061747553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d4105380 0x7fb2d410cf50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/2061747553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4100c60 0x7fb2d4104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/2061747553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d41002b0 0x7fb2d4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/2061747553 >> 172.21.15.100:0/2061747553 conn(0x7fb2d4077120 msgr2=0x7fb2d4077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/2061747553 shutdown_connections 2024-03-12T05:27:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.337+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/2061747553 wait complete. 2024-03-12T05:27:01.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 Processor -- start 2024-03-12T05:27:01.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 -- start start 2024-03-12T05:27:01.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d41002b0 0x7fb2d4101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 0x7fb2d4102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d41027c0 0x7fb2d41a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41a5ec0 con 0x7fb2d4105380 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41a6030 con 0x7fb2d41002b0 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d8f0c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41a61f0 con 0x7fb2d41027c0 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 0x7fb2d4102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 0x7fb2d4102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38912/0 (socket says 172.21.15.100:38912) 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 -- 172.21.15.100:0/3705348970 learned_addr learned my addr 172.21.15.100:0/3705348970 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d259c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d41002b0 0x7fb2d4101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d2d9d640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d41027c0 0x7fb2d41a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 -- 172.21.15.100:0/3705348970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d41027c0 msgr2=0x7fb2d41a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d41027c0 0x7fb2d41a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 -- 172.21.15.100:0/3705348970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d41002b0 msgr2=0x7fb2d4101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d41002b0 0x7fb2d4101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.338+0000 7fb2d1d9b640 1 -- 172.21.15.100:0/3705348970 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4009900 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.339+0000 7fb2d1d9b640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 0x7fb2d4102280 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c970 tx=0x7fb2c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.339+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8004440 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.339+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d41a64d0 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.339+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2c8007d40 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.339+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800f450 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.339+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41a6a30 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.340+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b4005290 con 0x7fb2d4105380 2024-03-12T05:27:01.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.342+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb2c8007860 con 0x7fb2d4105380 2024-03-12T05:27:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.342+0000 7fb2c37fe640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2bc0a1c50 0x7fb2bc0a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.343+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb2c80cd680 con 0x7fb2d4105380 2024-03-12T05:27:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.343+0000 7fb2d259c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2bc0a1c50 0x7fb2bc0a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.343+0000 7fb2d259c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2bc0a1c50 0x7fb2bc0a4110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009bd0 tx=0x7fb2cc005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.349+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2c8096510 con 0x7fb2d4105380 2024-03-12T05:27:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:01 smithi156 ceph-mon[36270]: pgmap v466: 97 pgs: 97 active+clean; 940 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.6 MiB/s wr, 519 op/s 2024-03-12T05:27:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:01 smithi161 ceph-mon[36043]: pgmap v466: 97 pgs: 97 active+clean; 940 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.6 MiB/s wr, 519 op/s 2024-03-12T05:27:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.693+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2b4005810 con 0x7fb2d4105380 2024-03-12T05:27:01.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.693+0000 7fb2c37fe640 1 -- 172.21.15.100:0/3705348970 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7fb2c8099bd0 con 0x7fb2d4105380 2024-03-12T05:27:01.688 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:27:01.688 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:25:21.243438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:27:01.688 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2bc0a1c50 msgr2=0x7fb2bc0a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2bc0a1c50 0x7fb2bc0a4110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009bd0 tx=0x7fb2cc005f90 comp rx=0 tx=0).stop 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 msgr2=0x7fb2d4102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 0x7fb2d4102280 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c970 tx=0x7fb2c800ce30 comp rx=0 tx=0).stop 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 shutdown_connections 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb2bc0a1c50 0x7fb2bc0a4110 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d41027c0 0x7fb2d41a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2d4105380 0x7fb2d4102280 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 --2- 172.21.15.100:0/3705348970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d41002b0 0x7fb2d4101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.695+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 >> 172.21.15.100:0/3705348970 conn(0x7fb2d4077120 msgr2=0x7fb2d4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.696+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 shutdown_connections 2024-03-12T05:27:01.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.696+0000 7fb2d8f0c640 1 -- 172.21.15.100:0/3705348970 wait complete. 2024-03-12T05:27:01.702 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:27:01.702 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:27:01.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.885+0000 7feedc8d4640 1 -- 172.21.15.100:0/1217499339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed81023c0 msgr2=0x7feed81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.885+0000 7feedc8d4640 1 --2- 172.21.15.100:0/1217499339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed81023c0 0x7feed81027a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec802bcd0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 -- 172.21.15.100:0/1217499339 shutdown_connections 2024-03-12T05:27:01.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 --2- 172.21.15.100:0/1217499339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed810b790 0x7feed810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 --2- 172.21.15.100:0/1217499339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8102ce0 0x7feed810b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 --2- 172.21.15.100:0/1217499339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed81023c0 0x7feed81027a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 -- 172.21.15.100:0/1217499339 >> 172.21.15.100:0/1217499339 conn(0x7feed8076400 msgr2=0x7feed8076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 -- 172.21.15.100:0/1217499339 shutdown_connections 2024-03-12T05:27:01.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.886+0000 7feedc8d4640 1 -- 172.21.15.100:0/1217499339 wait complete. 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 Processor -- start 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 -- start start 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed81023c0 0x7feed819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed8102ce0 0x7feed819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed810b790 0x7feed8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8197c80 con 0x7feed810b790 2024-03-12T05:27:01.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8197df0 con 0x7feed8102ce0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feedc8d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8197fb0 con 0x7feed81023c0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed81023c0 0x7feed819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed5d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed8102ce0 0x7feed819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed5d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed8102ce0 0x7feed819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:36500/0 (socket says 172.21.15.100:36500) 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed5d9b640 1 -- 172.21.15.100:0/3353788920 learned_addr learned my addr 172.21.15.100:0/3353788920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed6d9d640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed810b790 0x7feed8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed659c640 1 -- 172.21.15.100:0/3353788920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed8102ce0 msgr2=0x7feed819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed659c640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed8102ce0 0x7feed819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed659c640 1 -- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed810b790 msgr2=0x7feed8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed659c640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed810b790 0x7feed8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed659c640 1 -- 172.21.15.100:0/3353788920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8009560 con 0x7feed81023c0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.887+0000 7feed6d9d640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed810b790 0x7feed8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.888+0000 7feed659c640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed81023c0 0x7feed819d430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feec8036040 tx=0x7feec802c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.888+0000 7feec77fe640 1 -- 172.21.15.100:0/3353788920 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8004350 con 0x7feed81023c0 2024-03-12T05:27:01.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.888+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed8198230 con 0x7feed81023c0 2024-03-12T05:27:01.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.888+0000 7feec77fe640 1 -- 172.21.15.100:0/3353788920 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feec80044b0 con 0x7feed81023c0 2024-03-12T05:27:01.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.889+0000 7feec77fe640 1 -- 172.21.15.100:0/3353788920 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8003d30 con 0x7feed81023c0 2024-03-12T05:27:01.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.889+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed81a44d0 con 0x7feed81023c0 2024-03-12T05:27:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.890+0000 7feec77fe640 1 -- 172.21.15.100:0/3353788920 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7feec8004df0 con 0x7feed81023c0 2024-03-12T05:27:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.891+0000 7feec77fe640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feec00a1ba0 0x7feec00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 Processor -- start 2024-03-12T05:27:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feec77fe640 1 -- 172.21.15.100:0/3353788920 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feec80efe00 con 0x7feed81023c0 2024-03-12T05:27:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feed5d9b640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feec00a1ba0 0x7feec00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 -- start start 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c14620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c16f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feed5d9b640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feec00a1ba0 0x7feec00a4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed80ff890 tx=0x7feecc005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8bc2960 con 0x55afe8c06f00 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8b76310 con 0x55afe8c07820 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feedf586740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8b73840 con 0x55afe8c14b60 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.892+0000 7feed5d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c14620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed5d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c14620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:36514/0 (socket says 172.21.15.100:36514) 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed5d9b640 1 -- 172.21.15.100:0/1169670056 learned_addr learned my addr 172.21.15.100:0/1169670056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed6d9d640 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c16f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 -- 172.21.15.100:0/1169670056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 msgr2=0x55afe8c16f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c16f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 -- 172.21.15.100:0/1169670056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 msgr2=0x55afe8c14620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c14620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 -- 172.21.15.100:0/1169670056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe8b31060 con 0x55afe8c06f00 2024-03-12T05:27:01.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed6d9d640 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed5d9b640 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feed659c640 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c072e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7feec80ed620 tx=0x7feec80037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2e48594a6b0d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feec4ff9640 1 -- 172.21.15.100:0/1169670056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80bcb00 con 0x55afe8c06f00 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feec4ff9640 1 -- 172.21.15.100:0/1169670056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feec80c0c50 con 0x55afe8c06f00 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.893+0000 7feec4ff9640 1 -- 172.21.15.100:0/1169670056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80c4e10 con 0x55afe8c06f00 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.894+0000 7feedf586740 1 -- 172.21.15.100:0/1169670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 msgr2=0x55afe8c072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.894+0000 7feedf586740 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c072e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7feec80ed620 tx=0x7feec80037d0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 -- 172.21.15.100:0/1169670056 shutdown_connections 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 --2- 172.21.15.100:0/1169670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c072e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 -- 172.21.15.100:0/1169670056 >> 172.21.15.100:0/1169670056 conn(0x55afe8c01980 msgr2=0x55afe8c01d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 -- 172.21.15.100:0/1169670056 shutdown_connections 2024-03-12T05:27:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 -- 172.21.15.100:0/1169670056 wait complete. 2024-03-12T05:27:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 -- start start 2024-03-12T05:27:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.895+0000 7feedf586740 1 Processor -- start 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c11a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c11a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38960/0 (socket says 172.21.15.100:38960) 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 -- 172.21.15.100:0/2607268521 learned_addr learned my addr 172.21.15.100:0/2607268521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8b6f020 con 0x55afe8c06f00 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8842430 con 0x55afe8c07820 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe87fee00 con 0x55afe8c14b60 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed5d9b640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed6d9d640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feedf586740 10 client.0 _ll_register_callbacks cb 0x55afe8b16cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 -- 172.21.15.100:0/2607268521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 msgr2=0x55afe8c124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 -- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 msgr2=0x55afe8c11f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c11f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed81a44d0 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feed659c640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c11a40 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feec80f00e0 tx=0x7feec8003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580dbbc1fb0decff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.896+0000 7feebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afe886e7e0 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feebdffb640 1 -- 172.21.15.100:0/2607268521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80c4e10 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feebdffb640 1 -- 172.21.15.100:0/2607268521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feec80c4f70 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feebdffb640 1 -- 172.21.15.100:0/2607268521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec80882b0 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feedf586740 10 client.18615 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55afe8b6d6a0 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feebdffb640 1 -- 172.21.15.100:0/2607268521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feec803c070 con 0x55afe8c06f00 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feedf586740 10 client.18615 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.897+0000 7feedf586740 10 client.18615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feedf586740 20 client.18615 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feedf586740 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c26ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feedf586740 4 client.18615 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afe886e7e0 con 0x55afe8c24610 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feed5d9b640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c26ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feed5d9b640 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c26ab0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c8e31a40483b189 server_cookie=ff8043827da32370 in_seq=0 out_seq=0 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.898+0000 7feebdffb640 10 client.18615 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.899+0000 7feebdffb640 1 -- 172.21.15.100:0/2607268521 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afe886e7e0 con 0x55afe8c24610 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.899+0000 7feebdffb640 10 client.18615 handle_command_reply: tid=0 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 client.18615 shutdown 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 msgr2=0x55afe8c11a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c11a40 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feec80f00e0 tx=0x7feec8003e90 comp rx=0 tx=0).stop 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 shutdown_connections 2024-03-12T05:27:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c26ab0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feebdffb640 0 client.18615 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c14b60 0x55afe8c124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 --2- 172.21.15.100:0/2607268521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c06f00 0x55afe8c11a40 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 >> 172.21.15.100:0/2607268521 conn(0x55afe8c01980 msgr2=0x55afe8c036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 shutdown_connections 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 1 -- 172.21.15.100:0/2607268521 wait complete. 2024-03-12T05:27:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.900+0000 7feedf586740 20 client.18615 trim_cache size 0 max 16384 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.914+0000 7feedf586740 1 Processor -- start 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.914+0000 7feedf586740 1 -- start start 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.914+0000 7feedf586740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c14100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.914+0000 7feedf586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 0x55afe8c0eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feedf586740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c14b60 0x55afe8c0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feedf586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8af4130 con 0x55afe8c07820 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feedf586740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8bfd5c0 con 0x55afe8c14b60 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feedf586740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8c2bcf0 con 0x55afe8c06f00 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 0x55afe8c0eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 0x55afe8c0eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38968/0 (socket says 172.21.15.100:38968) 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 -- 172.21.15.100:0/2846577251 learned_addr learned my addr 172.21.15.100:0/2846577251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed6d9d640 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c14b60 0x55afe8c0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed659c640 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 -- 172.21.15.100:0/2846577251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 msgr2=0x55afe8c14100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c14100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 -- 172.21.15.100:0/2846577251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c14b60 msgr2=0x55afe8c0f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c14b60 0x55afe8c0f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 -- 172.21.15.100:0/2846577251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe8b09c20 con 0x55afe8c07820 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feed5d9b640 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 0x55afe8c0eca0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ae0 tx=0x7feecc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef153a68f932244 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.915+0000 7feebffff640 1 -- 172.21.15.100:0/2846577251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc025040 con 0x55afe8c07820 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feebffff640 1 -- 172.21.15.100:0/2846577251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feecc016070 con 0x55afe8c07820 2024-03-12T05:27:01.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feebffff640 1 -- 172.21.15.100:0/2846577251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00f3d0 con 0x55afe8c07820 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 -- 172.21.15.100:0/2846577251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 msgr2=0x55afe8c0eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 0x55afe8c0eca0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ae0 tx=0x7feecc014000 comp rx=0 tx=0).stop 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 -- 172.21.15.100:0/2846577251 shutdown_connections 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c14b60 0x55afe8c0f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c07820 0x55afe8c0eca0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 --2- 172.21.15.100:0/2846577251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c14100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 -- 172.21.15.100:0/2846577251 >> 172.21.15.100:0/2846577251 conn(0x55afe8c01980 msgr2=0x55afe8c03660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 -- 172.21.15.100:0/2846577251 shutdown_connections 2024-03-12T05:27:01.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.916+0000 7feedf586740 1 -- 172.21.15.100:0/2846577251 wait complete. 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 -- start start 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 Processor -- start 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c70b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c71070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 0x55afe8c715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8c7bd20 con 0x55afe8c14b60 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8c7be90 con 0x55afe8c07820 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8c7c050 con 0x55afe8c06f00 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feedf586740 10 client.0 _ll_register_callbacks cb 0x7feec80b8da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 0x55afe8c715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c70b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c70b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58240/0 (socket says 172.21.15.100:58240) 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 0x55afe8c715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38980/0 (socket says 172.21.15.100:38980) 2024-03-12T05:27:01.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed659c640 1 -- 172.21.15.100:0/504996678 learned_addr learned my addr 172.21.15.100:0/504996678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed5d9b640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c71070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 -- 172.21.15.100:0/504996678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 msgr2=0x55afe8c70b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c70b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 -- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 msgr2=0x55afe8c71070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c71070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed6d9d640 1 -- 172.21.15.100:0/504996678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc011d50 con 0x55afe8c14b60 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.917+0000 7feed659c640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c70b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feed6d9d640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 0x55afe8c715b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x55afe8c06bf0 tx=0x7feed0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f431f79ea82b7bef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feebdffb640 1 -- 172.21.15.100:0/504996678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0003d80 con 0x55afe8c14b60 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feebdffb640 1 -- 172.21.15.100:0/504996678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feed0004050 con 0x55afe8c14b60 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec80c0c50 con 0x55afe8c14b60 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feebdffb640 1 -- 172.21.15.100:0/504996678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feed0004210 con 0x55afe8c14b60 2024-03-12T05:27:01.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feedf586740 10 client.18621 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55afe8b2f8c0 con 0x55afe8c14b60 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.918+0000 7feebdffb640 1 -- 172.21.15.100:0/504996678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feec80c0c50 con 0x55afe8c14b60 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feebdffb640 1 -- 172.21.15.100:0/504996678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feed001daf0 con 0x55afe8c14b60 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feedf586740 10 client.18621 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feedf586740 10 client.18621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feedf586740 20 client.18621 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feedf586740 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feedf586740 4 client.18621 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:27:01.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55afe88ab720 con 0x55afe8c24610 2024-03-12T05:27:01.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.919+0000 7feed659c640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:01.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.920+0000 7feed659c640 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c7f8c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e354fa0ab28761db server_cookie=c85f734f439c1fdb in_seq=0 out_seq=0 2024-03-12T05:27:01.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.920+0000 7feebdffb640 10 client.18621 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feebdffb640 1 -- 172.21.15.100:0/504996678 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55afe88ab720 con 0x55afe8c24610 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feebdffb640 10 client.18621 handle_command_reply: tid=0 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 client.18621 shutdown 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 msgr2=0x55afe8c715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 0x55afe8c715b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x55afe8c06bf0 tx=0x7feed0010800 comp rx=0 tx=0).stop 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 shutdown_connections 2024-03-12T05:27:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55afe8c24610 0x55afe8c7f8c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55afe8c14b60 0x55afe8c715b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afe8c07820 0x55afe8c71070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 --2- 172.21.15.100:0/504996678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afe8c06f00 0x55afe8c70b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 >> 172.21.15.100:0/504996678 conn(0x55afe8c01980 msgr2=0x55afe8c002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feebdffb640 0 client.18621 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 shutdown_connections 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 1 -- 172.21.15.100:0/504996678 wait complete. 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.923+0000 7feedf586740 20 client.18621 trim_cache size 0 max 16384 2024-03-12T05:27:01.921 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:27:01.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feec00a1ba0 msgr2=0x7feec00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feec00a1ba0 0x7feec00a4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed80ff890 tx=0x7feecc005f90 comp rx=0 tx=0).stop 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed81023c0 msgr2=0x7feed819d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed81023c0 0x7feed819d430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feec8036040 tx=0x7feec802c720 comp rx=0 tx=0).stop 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 shutdown_connections 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7feec00a1ba0 0x7feec00a4060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed810b790 0x7feed8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feed8102ce0 0x7feed819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 --2- 172.21.15.100:0/3353788920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed81023c0 0x7feed819d430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 >> 172.21.15.100:0/3353788920 conn(0x7feed8076400 msgr2=0x7feed80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 shutdown_connections 2024-03-12T05:27:01.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:01.924+0000 7feedc8d4640 1 -- 172.21.15.100:0/3353788920 wait complete. 2024-03-12T05:27:01.931 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:27:01.931 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:27:02.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 -- 172.21.15.100:0/4149029071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 msgr2=0x7f2fec100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 --2- 172.21.15.100:0/4149029071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec100640 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009930 tx=0x7f2fe002bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 -- 172.21.15.100:0/4149029071 shutdown_connections 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 --2- 172.21.15.100:0/4149029071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fec105330 0x7f2fec10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 --2- 172.21.15.100:0/4149029071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fec100c10 0x7f2fec104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 --2- 172.21.15.100:0/4149029071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec100640 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 -- 172.21.15.100:0/4149029071 >> 172.21.15.100:0/4149029071 conn(0x7f2fec077180 msgr2=0x7f2fec077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 -- 172.21.15.100:0/4149029071 shutdown_connections 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 -- 172.21.15.100:0/4149029071 wait complete. 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 Processor -- start 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.053+0000 7f2ff1577640 1 -- start start 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fec105330 0x7f2fec19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fec199830 0x7f2fec199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec19a2e0 con 0x7f2fec100260 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec19a450 con 0x7f2fec199830 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec19a5c0 con 0x7f2fec105330 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39012/0 (socket says 172.21.15.100:39012) 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 -- 172.21.15.100:0/3699466703 learned_addr learned my addr 172.21.15.100:0/3699466703 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fec105330 0x7f2fec19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fec199830 0x7f2fec199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 -- 172.21.15.100:0/3699466703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fec105330 msgr2=0x7f2fec19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fec105330 0x7f2fec19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 -- 172.21.15.100:0/3699466703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fec199830 msgr2=0x7f2fec199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fec199830 0x7f2fec199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 -- 172.21.15.100:0/3699466703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009560 con 0x7f2fec100260 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2feaffd640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec19f6c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009230 tx=0x7f2fe002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2fd3fff640 1 -- 172.21.15.100:0/3699466703 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004350 con 0x7f2fec100260 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec1a6510 con 0x7f2fec100260 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.054+0000 7f2fd3fff640 1 -- 172.21.15.100:0/3699466703 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe00044b0 con 0x7f2fec100260 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.055+0000 7f2fd3fff640 1 -- 172.21.15.100:0/3699466703 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003d00 con 0x7f2fec100260 2024-03-12T05:27:02.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.055+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec1a6a20 con 0x7f2fec100260 2024-03-12T05:27:02.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.057+0000 7f2fd3fff640 1 -- 172.21.15.100:0/3699466703 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2fe0004df0 con 0x7f2fec100260 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.057+0000 7f2ff4221740 1 Processor -- start 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.057+0000 7f2ff4221740 1 -- start start 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a80210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a82b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2ff4221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065a32960 con 0x562065a77b70 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2ff4221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620659e6310 con 0x562065a77250 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2ff4221740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620659e3840 con 0x562065a80750 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2fd3fff640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fd40a1ba0 0x7f2fd40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2feb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a80210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2feb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a77630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:36588/0 (socket says 172.21.15.100:36588) 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2feb7fe640 1 -- 172.21.15.100:0/3816772616 learned_addr learned my addr 172.21.15.100:0/3816772616 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fd40a1ba0 0x7f2fd40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a82b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.058+0000 7f2fd3fff640 1 -- 172.21.15.100:0/3699466703 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2fe00ef880 con 0x7f2fec100260 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feaffd640 1 -- 172.21.15.100:0/3816772616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 msgr2=0x562065a82b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feaffd640 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a82b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feaffd640 1 -- 172.21.15.100:0/3816772616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 msgr2=0x562065a77630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feaffd640 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a77630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feaffd640 1 -- 172.21.15.100:0/3816772616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620659a1060 con 0x562065a77b70 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feaffd640 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a80210 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00ed0a0 tx=0x7f2fe008f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ac010e3643c0fb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fd40a1ba0 0x7f2fd40a4060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009fd0 tx=0x7f2fe4009b90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2fd17fa640 1 -- 172.21.15.100:0/3816772616 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003e60 con 0x562065a77b70 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2fd17fa640 1 -- 172.21.15.100:0/3816772616 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe00035f0 con 0x562065a77b70 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.059+0000 7f2fd17fa640 1 -- 172.21.15.100:0/3816772616 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00dc0c0 con 0x562065a77b70 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 -- 172.21.15.100:0/3816772616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 msgr2=0x562065a80210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a80210 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00ed0a0 tx=0x7f2fe008f830 comp rx=0 tx=0).stop 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 -- 172.21.15.100:0/3816772616 shutdown_connections 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a82b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a80210 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 --2- 172.21.15.100:0/3816772616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 -- 172.21.15.100:0/3816772616 >> 172.21.15.100:0/3816772616 conn(0x562065a71980 msgr2=0x562065a71d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 -- 172.21.15.100:0/3816772616 shutdown_connections 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 -- 172.21.15.100:0/3816772616 wait complete. 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 -- start start 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.060+0000 7f2ff4221740 1 Processor -- start 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a77250 0x562065a7d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a7dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620659df020 con 0x562065a77b70 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620656b2430 con 0x562065a80750 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206566ee00 con 0x562065a77250 2024-03-12T05:27:02.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a7dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2ff4221740 10 client.0 _ll_register_callbacks cb 0x562065986cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a7dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39026/0 (socket says 172.21.15.100:39026) 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 -- 172.21.15.100:0/2080101147 learned_addr learned my addr 172.21.15.100:0/2080101147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2feaffd640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a77250 0x562065a7d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 -- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a77250 msgr2=0x562065a7d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a77250 0x562065a7d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 -- 172.21.15.100:0/2080101147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 msgr2=0x562065a7e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 -- 172.21.15.100:0/2080101147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec1a6a20 con 0x562065a77b70 2024-03-12T05:27:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a7dc40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc012730 tx=0x7f2fdc012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e54bc86c11f417 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.061+0000 7f2fceff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2fceff5640 1 -- 172.21.15.100:0/2080101147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc017070 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2fceff5640 1 -- 172.21.15.100:0/2080101147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2fdc004320 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620656de7e0 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2fceff5640 1 -- 172.21.15.100:0/2080101147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fdc0044e0 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 10 client.18633 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5620659dd6a0 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2fceff5640 1 -- 172.21.15.100:0/2080101147 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fdc028080 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2fceff5640 1 -- 172.21.15.100:0/2080101147 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2fdc003c90 con 0x562065a77b70 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 10 client.18633 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 10 client.18633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:27:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 20 client.18633 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.062+0000 7f2ff4221740 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065a90a10 0x562065a92e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.063+0000 7f2ff4221740 4 client.18633 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.063+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56206571b720 con 0x562065a90a10 2024-03-12T05:27:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.063+0000 7f2feaffd640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065a90a10 0x562065a92e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.063+0000 7f2feaffd640 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065a90a10 0x562065a92e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93d18b50f4df1b95 server_cookie=ba8142c98e8d373e in_seq=0 out_seq=0 2024-03-12T05:27:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.063+0000 7f2fceff5640 10 client.18633 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2fceff5640 1 -- 172.21.15.100:0/2080101147 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56206571b720 con 0x562065a90a10 2024-03-12T05:27:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2fceff5640 10 client.18633 handle_command_reply: tid=0 2024-03-12T05:27:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 client.18633 shutdown 2024-03-12T05:27:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 msgr2=0x562065a7dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a7dc40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc012730 tx=0x7f2fdc012b30 comp rx=0 tx=0).stop 2024-03-12T05:27:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 shutdown_connections 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065a90a10 0x562065a92e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77b70 0x562065a7dc40 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 --2- 172.21.15.100:0/2080101147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a77250 0x562065a7d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 >> 172.21.15.100:0/2080101147 conn(0x562065a71980 msgr2=0x562065a736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2fceff5640 0 client.18633 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 shutdown_connections 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 1 -- 172.21.15.100:0/2080101147 wait complete. 2024-03-12T05:27:02.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.064+0000 7f2ff4221740 20 client.18633 trim_cache size 0 max 16384 2024-03-12T05:27:02.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 Processor -- start 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 -- start start 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 0x562065a7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a7a980 0x562065a7ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065964130 con 0x562065a77250 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065a8cdb0 con 0x562065a80750 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2ff4221740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065a76f00 con 0x562065a7a980 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 0x562065a7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 0x562065a7fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39040/0 (socket says 172.21.15.100:39040) 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2feaffd640 1 -- 172.21.15.100:0/1859683331 learned_addr learned my addr 172.21.15.100:0/1859683331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a7a980 0x562065a7ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.083+0000 7f2feaffd640 1 -- 172.21.15.100:0/1859683331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a7a980 msgr2=0x562065a7ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2feaffd640 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a7a980 0x562065a7ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2feaffd640 1 -- 172.21.15.100:0/1859683331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 msgr2=0x562065a7dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2feaffd640 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2feaffd640 1 -- 172.21.15.100:0/1859683331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562065979c20 con 0x562065a77250 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2feaffd640 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 0x562065a7fe30 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00b89e0 tx=0x7f2fe00dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d000735f5b2e9df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2fd0ff9640 1 -- 172.21.15.100:0/1859683331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00bc600 con 0x562065a77250 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2fd0ff9640 1 -- 172.21.15.100:0/1859683331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe0003e60 con 0x562065a77250 2024-03-12T05:27:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.084+0000 7f2fd0ff9640 1 -- 172.21.15.100:0/1859683331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00b8bf0 con 0x562065a77250 2024-03-12T05:27:02.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 -- 172.21.15.100:0/1859683331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 msgr2=0x562065a7fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 0x562065a7fe30 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00b89e0 tx=0x7f2fe00dc040 comp rx=0 tx=0).stop 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 -- 172.21.15.100:0/1859683331 shutdown_connections 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a7a980 0x562065a7ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a80750 0x562065a7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 --2- 172.21.15.100:0/1859683331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a77250 0x562065a7fe30 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 -- 172.21.15.100:0/1859683331 >> 172.21.15.100:0/1859683331 conn(0x562065a71980 msgr2=0x562065a78a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 -- 172.21.15.100:0/1859683331 shutdown_connections 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 -- 172.21.15.100:0/1859683331 wait complete. 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.085+0000 7f2ff4221740 1 -- start start 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 Processor -- start 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a85ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a7a980 0x562065a86420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a86960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065ae6f20 con 0x562065a7a980 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065ae7060 con 0x562065a77250 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562065ae71a0 con 0x562065a80750 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2ff4221740 10 client.0 _ll_register_callbacks cb 0x7f2fdc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a85ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a85ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:36604/0 (socket says 172.21.15.100:36604) 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 -- 172.21.15.100:0/406581104 learned_addr learned my addr 172.21.15.100:0/406581104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a86960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a7a980 0x562065a86420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 -- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 msgr2=0x562065a86960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a86960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 -- 172.21.15.100:0/406581104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a7a980 msgr2=0x562065a86420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a7a980 0x562065a86420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.086+0000 7f2feaffd640 1 -- 172.21.15.100:0/406581104 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620659a1060 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2feb7fe640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a86960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2feaffd640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a85ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005c40 tx=0x7f2fe0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c283f84742357189 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2fceff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2fceff5640 1 -- 172.21.15.100:0/406581104 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00bcc80 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fe00035f0 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2fceff5640 1 -- 172.21.15.100:0/406581104 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2fe00c06d0 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2fceff5640 1 -- 172.21.15.100:0/406581104 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe00bc580 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2ff4221740 10 client.28010 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.087+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56206599f8c0 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2fceff5640 1 -- 172.21.15.100:0/406581104 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe00035f0 con 0x562065a77250 2024-03-12T05:27:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2fceff5640 1 -- 172.21.15.100:0/406581104 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2fe00d6050 con 0x562065a77250 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2ff4221740 10 client.28010 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2ff4221740 10 client.28010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2ff4221740 20 client.28010 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2ff4221740 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065ae83f0 0x562065aea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2ff4221740 4 client.28010 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56206576a890 con 0x562065ae83f0 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.088+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065ae83f0 0x562065aea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2fceff5640 10 client.28010 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2fea7fc640 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065ae83f0 0x562065aea7e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b26a9bc4209fa7f server_cookie=8e46c4ef573dd937 in_seq=0 out_seq=0 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2fceff5640 1 -- 172.21.15.100:0/406581104 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56206576a890 con 0x562065ae83f0 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2fceff5640 10 client.28010 handle_command_reply: tid=0 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 client.28010 shutdown 2024-03-12T05:27:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 msgr2=0x562065a85ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a85ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005c40 tx=0x7f2fe0009260 comp rx=0 tx=0).stop 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 shutdown_connections 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562065a80750 0x562065a86960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562065a7a980 0x562065a86420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x562065ae83f0 0x562065aea7e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 --2- 172.21.15.100:0/406581104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562065a77250 0x562065a85ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2fceff5640 0 client.28010 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.089+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 >> 172.21.15.100:0/406581104 conn(0x562065a71980 msgr2=0x562065a7ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 shutdown_connections 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff4221740 1 -- 172.21.15.100:0/406581104 wait complete. 2024-03-12T05:27:02.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff4221740 20 client.28010 trim_cache size 0 max 16384 2024-03-12T05:27:02.090 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fd40a1ba0 msgr2=0x7f2fd40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fd40a1ba0 0x7f2fd40a4060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009fd0 tx=0x7f2fe4009b90 comp rx=0 tx=0).stop 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 msgr2=0x7f2fec19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec19f6c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009230 tx=0x7f2fe002c650 comp rx=0 tx=0).stop 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 shutdown_connections 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fd40a1ba0 0x7f2fd40a4060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fec199830 0x7f2fec199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fec105330 0x7f2fec19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 --2- 172.21.15.100:0/3699466703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fec100260 0x7f2fec19f6c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 >> 172.21.15.100:0/3699466703 conn(0x7f2fec077180 msgr2=0x7f2fec0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 shutdown_connections 2024-03-12T05:27:02.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.090+0000 7f2ff1577640 1 -- 172.21.15.100:0/3699466703 wait complete. 2024-03-12T05:27:02.096 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:27:02.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/646090389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d88100c30 msgr2=0x7f3d88104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/646090389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d88100c30 0x7f3d88104c70 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009930 tx=0x7f3d7002bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/646090389 shutdown_connections 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/646090389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88105350 0x7f3d8810cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/646090389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d88100c30 0x7f3d88104c70 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/646090389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d88100280 0x7f3d88100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/646090389 >> 172.21.15.100:0/646090389 conn(0x7f3d88077150 msgr2=0x7f3d88077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/646090389 shutdown_connections 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.228+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/646090389 wait complete. 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 Processor -- start 2024-03-12T05:27:02.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 -- start start 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88100280 0x7f3d8819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d88105350 0x7f3d8819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 0x7f3d88199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8819a2d0 con 0x7f3d881998e0 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8819a440 con 0x7f3d88105350 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8e8c0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8819a600 con 0x7f3d88100280 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 0x7f3d88199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d87fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88100280 0x7f3d8819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 0x7f3d88199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39062/0 (socket says 172.21.15.100:39062) 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.229+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/2177177813 learned_addr learned my addr 172.21.15.100:0/2177177813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d88105350 0x7f3d8819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/2177177813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88100280 msgr2=0x7f3d8819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88100280 0x7f3d8819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/2177177813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d88105350 msgr2=0x7f3d8819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d88105350 0x7f3d8819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/2177177813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009560 con 0x7f3d881998e0 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d87fff640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88100280 0x7f3d8819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 0x7f3d88199d90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000eb10 tx=0x7f3d8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d85ffb640 1 -- 172.21.15.100:0/2177177813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80004b20 con 0x7f3d881998e0 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d881a65f0 con 0x7f3d881998e0 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d85ffb640 1 -- 172.21.15.100:0/2177177813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d80004c80 con 0x7f3d881998e0 2024-03-12T05:27:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d85ffb640 1 -- 172.21.15.100:0/2177177813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80010670 con 0x7f3d881998e0 2024-03-12T05:27:02.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.230+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d881a6b20 con 0x7f3d881998e0 2024-03-12T05:27:02.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d85ffb640 1 -- 172.21.15.100:0/2177177813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3d80005020 con 0x7f3d881998e0 2024-03-12T05:27:02.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 Processor -- start 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 -- start start 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340dd9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 0x55d340deaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340ded350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340d94960 con 0x55d340dd9a10 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340d48310 con 0x55d340dd9060 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d91569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340d45840 con 0x55d340deaf60 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.233+0000 7f3d87fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 0x55d340deaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 0x55d340deaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39072/0 (socket says 172.21.15.100:39072) 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 -- 172.21.15.100:0/2785722504 learned_addr learned my addr 172.21.15.100:0/2785722504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340ded350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340dd9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 -- 172.21.15.100:0/2785722504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 msgr2=0x55d340ded350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340ded350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 -- 172.21.15.100:0/2785722504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 msgr2=0x55d340dd9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340dd9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 -- 172.21.15.100:0/2785722504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d340d03060 con 0x55d340dd9a10 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340ded350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d85ffb640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3d740a1ba0 0x7f3d740a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3d740a1ba0 0x7f3d740a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d85ffb640 1 -- 172.21.15.100:0/2177177813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d80014070 con 0x7f3d881998e0 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.234+0000 7f3d87fff640 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 0x55d340deaa20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3d780114f0 tx=0x7f3d780119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66f79c4a78c7ece server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d7e7fc640 1 -- 172.21.15.100:0/2785722504 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78011250 con 0x55d340dd9a10 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d7e7fc640 1 -- 172.21.15.100:0/2785722504 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d78004510 con 0x55d340dd9a10 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d7e7fc640 1 -- 172.21.15.100:0/2785722504 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800f5c0 con 0x55d340dd9a10 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3d740a1ba0 0x7f3d740a4060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000c00 tx=0x7f3d70045040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 -- 172.21.15.100:0/2785722504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 msgr2=0x55d340deaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 0x55d340deaa20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3d780114f0 tx=0x7f3d780119b0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 -- 172.21.15.100:0/2785722504 shutdown_connections 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340ded350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9a10 0x55d340deaa20 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 --2- 172.21.15.100:0/2785722504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340dd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.235+0000 7f3d91569740 1 -- 172.21.15.100:0/2785722504 >> 172.21.15.100:0/2785722504 conn(0x55d340dd3980 msgr2=0x55d340dd3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.236+0000 7f3d91569740 1 -- 172.21.15.100:0/2785722504 shutdown_connections 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.236+0000 7f3d91569740 1 -- 172.21.15.100:0/2785722504 wait complete. 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.236+0000 7f3d91569740 1 -- start start 2024-03-12T05:27:02.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.236+0000 7f3d91569740 1 Processor -- start 2024-03-12T05:27:02.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340de4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340deaf60 0x55d340de4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340ddfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340d41020 con 0x55d340dd9060 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340a14430 con 0x55d340deaf60 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3409d0e00 con 0x55d340dea750 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 10 client.0 _ll_register_callbacks cb 0x55d340ce8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340ddfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340ddfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58356/0 (socket says 172.21.15.100:58356) 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/1261552267 learned_addr learned my addr 172.21.15.100:0/1261552267 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d87fff640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340de4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340deaf60 0x55d340de4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340deaf60 msgr2=0x55d340de4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340deaf60 0x55d340de4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/1261552267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 msgr2=0x55d340de4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340de4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/1261552267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d78009cc0 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340deaf60 0x55d340de4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340ddfd90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d800972c0 tx=0x7f3d8009aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420fc5222a70ac30 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d677fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d677fe640 1 -- 172.21.15.100:0/1261552267 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800b2070 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d677fe640 1 -- 172.21.15.100:0/1261552267 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d800a3090 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.237+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d340a407e0 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d677fe640 1 -- 172.21.15.100:0/1261552267 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d80096f60 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 10 client.27961 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d340d3f6a0 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d677fe640 1 -- 172.21.15.100:0/1261552267 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d8007d020 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d677fe640 1 -- 172.21.15.100:0/1261552267 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3d8001f060 con 0x55d340dea750 2024-03-12T05:27:02.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 10 client.27961 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:02.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 10 client.27961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:02.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 20 client.27961 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:02.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e04050 0x55d340e06480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 4 client.27961 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:02.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d340a407e0 con 0x55d340e04050 2024-03-12T05:27:02.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.238+0000 7f3d87fff640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e04050 0x55d340e06480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65625 35.39 MB/sec warmup 105 sec latency 348.979 ms 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.239+0000 7f3d677fe640 10 client.27961 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.239+0000 7f3d87fff640 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e04050 0x55d340e06480 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8042ba6538ea22b server_cookie=9aafd8e5b2d4ceb3 in_seq=0 out_seq=0 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d677fe640 1 -- 172.21.15.100:0/1261552267 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d340a407e0 con 0x55d340e04050 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d677fe640 10 client.27961 handle_command_reply: tid=0 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 client.27961 shutdown 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 msgr2=0x55d340ddfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340ddfd90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d800972c0 tx=0x7f3d8009aff0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 shutdown_connections 2024-03-12T05:27:02.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e04050 0x55d340e06480 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340ddfd90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340deaf60 0x55d340de4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 --2- 172.21.15.100:0/1261552267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340de4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d677fe640 0 client.27961 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 >> 172.21.15.100:0/1261552267 conn(0x55d340dd3980 msgr2=0x55d340dd56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 shutdown_connections 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 1 -- 172.21.15.100:0/1261552267 wait complete. 2024-03-12T05:27:02.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.240+0000 7f3d91569740 20 client.27961 trim_cache size 0 max 16384 2024-03-12T05:27:02.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 Processor -- start 2024-03-12T05:27:02.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 -- start start 2024-03-12T05:27:02.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340ddfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dea750 0x55d340de45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340de4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340cc6130 con 0x55d340dd9060 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340ded6e0 con 0x55d340dea750 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d91569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340e0b660 con 0x55d340deaf60 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340de4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.258+0000 7f3d87fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340ddfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d7ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dea750 0x55d340de45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d87fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340ddfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39084/0 (socket says 172.21.15.100:39084) 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340de4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:58366/0 (socket says 172.21.15.100:58366) 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d87fff640 1 -- 172.21.15.100:0/3422949519 learned_addr learned my addr 172.21.15.100:0/3422949519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/3422949519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dea750 msgr2=0x55d340de45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dea750 0x55d340de45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/3422949519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 msgr2=0x55d340ddfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340ddfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/3422949519 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d340cdbc20 con 0x55d340deaf60 2024-03-12T05:27:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340de4b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d8009a880 tx=0x7f3d8009ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2781a8e644e9304e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d7dffb640 1 -- 172.21.15.100:0/3422949519 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800b2070 con 0x55d340deaf60 2024-03-12T05:27:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d7dffb640 1 -- 172.21.15.100:0/3422949519 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d8007d020 con 0x55d340deaf60 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.259+0000 7f3d7dffb640 1 -- 172.21.15.100:0/3422949519 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800997b0 con 0x55d340deaf60 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 -- 172.21.15.100:0/3422949519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 msgr2=0x55d340de4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340de4b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d8009a880 tx=0x7f3d8009ebc0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 -- 172.21.15.100:0/3422949519 shutdown_connections 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340deaf60 0x55d340de4b20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dea750 0x55d340de45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 --2- 172.21.15.100:0/3422949519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340dd9060 0x55d340ddfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.260+0000 7f3d91569740 1 -- 172.21.15.100:0/3422949519 >> 172.21.15.100:0/3422949519 conn(0x55d340dd3980 msgr2=0x55d340ddf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.261+0000 7f3d91569740 1 -- 172.21.15.100:0/3422949519 shutdown_connections 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.261+0000 7f3d91569740 1 -- 172.21.15.100:0/3422949519 wait complete. 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.261+0000 7f3d91569740 1 -- start start 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.261+0000 7f3d91569740 1 Processor -- start 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340de9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340de8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 0x55d340de8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340de8c80 con 0x55d340deaf60 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340de8df0 con 0x55d340dd9060 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d340de8fb0 con 0x55d340dea750 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d91569740 10 client.0 _ll_register_callbacks cb 0x7f3d78009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 0x55d340de8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d8ce5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 0x55d340de8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39086/0 (socket says 172.21.15.100:39086) 2024-03-12T05:27:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d8ce5d640 1 -- 172.21.15.100:0/3923987736 learned_addr learned my addr 172.21.15.100:0/3923987736 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d87fff640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340de9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340de8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d7ffff640 1 -- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 msgr2=0x55d340de9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340de9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d7ffff640 1 -- 172.21.15.100:0/3923987736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 msgr2=0x55d340de8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 0x55d340de8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d7ffff640 1 -- 172.21.15.100:0/3923987736 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d340d3f6a0 con 0x55d340dea750 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.262+0000 7f3d8ce5d640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 0x55d340de8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d7ffff640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340de8200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7002c170 tx=0x7f3d7002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42ea251f28eae4f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d677fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d677fe640 1 -- 172.21.15.100:0/3923987736 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700050c0 con 0x55d340dea750 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d677fe640 1 -- 172.21.15.100:0/3923987736 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d70005220 con 0x55d340dea750 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d78004510 con 0x55d340dea750 2024-03-12T05:27:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d677fe640 1 -- 172.21.15.100:0/3923987736 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d700053e0 con 0x55d340dea750 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d91569740 10 client.27967 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d340d018c0 con 0x55d340dea750 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d677fe640 1 -- 172.21.15.100:0/3923987736 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d78004510 con 0x55d340dea750 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.263+0000 7f3d677fe640 1 -- 172.21.15.100:0/3923987736 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3d700434c0 con 0x55d340dea750 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d91569740 10 client.27967 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d91569740 10 client.27967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d91569740 20 client.27967 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d91569740 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e5b9e0 0x55d340e5de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d91569740 4 client.27967 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d340a7d720 con 0x55d340e5b9e0 2024-03-12T05:27:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d87fff640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e5b9e0 0x55d340e5de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d87fff640 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e5b9e0 0x55d340e5de00 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae6f8d1b818226c server_cookie=de76115e3907d57f in_seq=0 out_seq=0 2024-03-12T05:27:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.264+0000 7f3d677fe640 10 client.27967 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d677fe640 1 -- 172.21.15.100:0/3923987736 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d340a7d720 con 0x55d340e5b9e0 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d677fe640 10 client.27967 handle_command_reply: tid=0 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 client.27967 shutdown 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 msgr2=0x55d340de8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340de8200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7002c170 tx=0x7f3d7002bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 shutdown_connections 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55d340e5b9e0 0x55d340e5de00 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d340deaf60 0x55d340de8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d340dea750 0x55d340de8200 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 --2- 172.21.15.100:0/3923987736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d340dd9060 0x55d340de9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d677fe640 0 client.27967 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 >> 172.21.15.100:0/3923987736 conn(0x55d340dd3980 msgr2=0x55d340dd22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 shutdown_connections 2024-03-12T05:27:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.265+0000 7f3d91569740 1 -- 172.21.15.100:0/3923987736 wait complete. 2024-03-12T05:27:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d91569740 20 client.27967 trim_cache size 0 max 16384 2024-03-12T05:27:02.264 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:27:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3d740a1ba0 msgr2=0x7f3d740a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3d740a1ba0 0x7f3d740a4060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000c00 tx=0x7f3d70045040 comp rx=0 tx=0).stop 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 msgr2=0x7f3d88199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 0x7f3d88199d90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000eb10 tx=0x7f3d8000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 shutdown_connections 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3d740a1ba0 0x7f3d740a4060 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d881998e0 0x7f3d88199d90 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d88105350 0x7f3d8819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 --2- 172.21.15.100:0/2177177813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d88100280 0x7f3d8819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.266+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 >> 172.21.15.100:0/2177177813 conn(0x7f3d88077150 msgr2=0x7f3d88069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.267+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 shutdown_connections 2024-03-12T05:27:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:02.267+0000 7f3d8e8c0640 1 -- 172.21.15.100:0/2177177813 wait complete. 2024-03-12T05:27:02.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:02 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3705348970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:27:02.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:02 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3705348970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:27:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:02 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3705348970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:27:03.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65918 35.15 MB/sec warmup 106 sec latency 93.971 ms 2024-03-12T05:27:03.277 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:27:03.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:27:03.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:27:03.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:03 smithi100 ceph-mon[32210]: pgmap v467: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 793 op/s 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 -- 172.21.15.100:0/3449048149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 msgr2=0x7fe8b4106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/3449048149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4106500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009890 tx=0x7fe8b002bbd0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 -- 172.21.15.100:0/3449048149 shutdown_connections 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/3449048149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b4069e40 0x7fe8b410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/3449048149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b40694a0 0x7fe8b4069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/3449048149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4106500 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 -- 172.21.15.100:0/3449048149 >> 172.21.15.100:0/3449048149 conn(0x7fe8b4077110 msgr2=0x7fe8b4077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 -- 172.21.15.100:0/3449048149 shutdown_connections 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.412+0000 7fe8bc59a640 1 -- 172.21.15.100:0/3449048149 wait complete. 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 Processor -- start 2024-03-12T05:27:03.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 -- start start 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b40694a0 0x7fe8b419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 0x7fe8b419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4197c70 con 0x7fe8b4069e40 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4197de0 con 0x7fe8b40694a0 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bc59a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4197fa0 con 0x7fe8b4106120 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 0x7fe8b419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 0x7fe8b419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39104/0 (socket says 172.21.15.100:39104) 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 -- 172.21.15.100:0/719997042 learned_addr learned my addr 172.21.15.100:0/719997042 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8ba336640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b40694a0 0x7fe8b419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bab37640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 -- 172.21.15.100:0/719997042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 msgr2=0x7fe8b4197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 -- 172.21.15.100:0/719997042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b40694a0 msgr2=0x7fe8b419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b40694a0 0x7fe8b419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8b9b35640 1 -- 172.21.15.100:0/719997042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0009490 con 0x7fe8b4069e40 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.413+0000 7fe8bab37640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.414+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 0x7fe8b419d9b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e9c0 tx=0x7fe8a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.414+0000 7fe8ab7fe640 1 -- 172.21.15.100:0/719997042 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400ccc0 con 0x7fe8b4069e40 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.414+0000 7fe8ab7fe640 1 -- 172.21.15.100:0/719997042 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8a400ce20 con 0x7fe8b4069e40 2024-03-12T05:27:03.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.414+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b4198280 con 0x7fe8b4069e40 2024-03-12T05:27:03.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.414+0000 7fe8ab7fe640 1 -- 172.21.15.100:0/719997042 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4010430 con 0x7fe8b4069e40 2024-03-12T05:27:03.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.414+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41a4510 con 0x7fe8b4069e40 2024-03-12T05:27:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.417+0000 7fe8ab7fe640 1 -- 172.21.15.100:0/719997042 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe8a4004f80 con 0x7fe8b4069e40 2024-03-12T05:27:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.417+0000 7fe8bf174740 1 Processor -- start 2024-03-12T05:27:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.417+0000 7fe8bf174740 1 -- start start 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdd81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bf174740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fd3cf90 con 0x55855fdce240 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8ab7fe640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8a00a1ba0 0x7fe8a00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bf174740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fd3a4c0 con 0x55855fdd7d50 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bf174740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fd35ca0 con 0x55855fdcebf0 2024-03-12T05:27:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39112/0 (socket says 172.21.15.100:39112) 2024-03-12T05:27:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 -- 172.21.15.100:0/4283841328 learned_addr learned my addr 172.21.15.100:0/4283841328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8ab7fe640 1 -- 172.21.15.100:0/719997042 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe8a4014070 con 0x7fe8b4069e40 2024-03-12T05:27:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8a00a1ba0 0x7fe8a00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8ba336640 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 -- 172.21.15.100:0/4283841328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 msgr2=0x55855fdd7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 -- 172.21.15.100:0/4283841328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 msgr2=0x55855fdd81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdd81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.418+0000 7fe8bab37640 1 -- 172.21.15.100:0/4283841328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855fcf80b0 con 0x55855fdce240 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.419+0000 7fe8ba336640 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.419+0000 7fe8bab37640 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdce620 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00d900 tx=0x7fe8ac00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5707aa8616d03b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.419+0000 7fe8bab37640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8a00a1ba0 0x7fe8a00a4060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c6a0 tx=0x7fe8ac025000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.419+0000 7fe8a8bfe640 1 -- 172.21.15.100:0/4283841328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac016dc0 con 0x55855fdce240 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.419+0000 7fe8a8bfe640 1 -- 172.21.15.100:0/4283841328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8ac009e40 con 0x55855fdce240 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.419+0000 7fe8a8bfe640 1 -- 172.21.15.100:0/4283841328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac024c30 con 0x55855fdce240 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 -- 172.21.15.100:0/4283841328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 msgr2=0x55855fdce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdce620 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00d900 tx=0x7fe8ac00ddc0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 -- 172.21.15.100:0/4283841328 shutdown_connections 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdd81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 --2- 172.21.15.100:0/4283841328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdce620 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 -- 172.21.15.100:0/4283841328 >> 172.21.15.100:0/4283841328 conn(0x55855fdc88a0 msgr2=0x55855fdc8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 -- 172.21.15.100:0/4283841328 shutdown_connections 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.420+0000 7fe8bf174740 1 -- 172.21.15.100:0/4283841328 wait complete. 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 -- start start 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 Processor -- start 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdcebf0 0x55855fdd11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdd7d50 0x55855fdd1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fcdaf40 con 0x55855fdce240 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fa094b0 con 0x55855fdcebf0 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fcbb670 con 0x55855fdd7d50 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bf174740 10 client.0 _ll_register_callbacks cb 0x55855fd28f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:03.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8bab37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdd7d50 0x55855fdd1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39126/0 (socket says 172.21.15.100:39126) 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 -- 172.21.15.100:0/1688252569 learned_addr learned my addr 172.21.15.100:0/1688252569 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdcebf0 0x55855fdd11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 -- 172.21.15.100:0/1688252569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdd7d50 msgr2=0x55855fdd1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdd7d50 0x55855fdd1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 -- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdcebf0 msgr2=0x55855fdd11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdcebf0 0x55855fdd11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.421+0000 7fe8ba336640 1 -- 172.21.15.100:0/1688252569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac020040 con 0x55855fdce240 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8ba336640 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd0cb0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009890 tx=0x7fe8b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b9c6f6ba86aceb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe89dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe89dffb640 1 -- 172.21.15.100:0/1688252569 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00323d0 con 0x55855fdce240 2024-03-12T05:27:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe89dffb640 1 -- 172.21.15.100:0/1688252569 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe8b0034440 con 0x55855fdce240 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55855fd0ff40 con 0x55855fdce240 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe89dffb640 1 -- 172.21.15.100:0/1688252569 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b0034600 con 0x55855fdce240 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 10 client.18669 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55855fd34320 con 0x55855fdce240 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe89dffb640 1 -- 172.21.15.100:0/1688252569 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55855fd0ff40 con 0x55855fdce240 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe89dffb640 1 -- 172.21.15.100:0/1688252569 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe8b004e070 con 0x55855fdce240 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 10 client.18669 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 10 client.18669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 20 client.18669 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fdea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 4 client.18669 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.422+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55855fd2c310 con 0x55855fde7ea0 2024-03-12T05:27:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.423+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fdea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.423+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fdea2c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be91ac7a21615b94 server_cookie=bfbdd975932e23d in_seq=0 out_seq=0 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.423+0000 7fe89dffb640 10 client.18669 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe89dffb640 1 -- 172.21.15.100:0/1688252569 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55855fd2c310 con 0x55855fde7ea0 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe89dffb640 10 client.18669 handle_command_reply: tid=0 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 client.18669 shutdown 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 msgr2=0x55855fdd0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd0cb0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009890 tx=0x7fe8b0004270 comp rx=0 tx=0).stop 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 shutdown_connections 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fdea2c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe89dffb640 0 client.18669 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdd7d50 0x55855fdd1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdcebf0 0x55855fdd11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 --2- 172.21.15.100:0/1688252569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd0cb0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 >> 172.21.15.100:0/1688252569 conn(0x55855fdc88a0 msgr2=0x55855fdca800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 shutdown_connections 2024-03-12T05:27:03.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 1 -- 172.21.15.100:0/1688252569 wait complete. 2024-03-12T05:27:03.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.424+0000 7fe8bf174740 20 client.18669 trim_cache size 0 max 16384 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.442+0000 7fe8bf174740 1 Processor -- start 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 -- start start 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdc78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fdcf140 con 0x55855fdce240 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fdef4d0 con 0x55855fdd7d50 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bf174740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fdc4160 con 0x55855fdcebf0 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39136/0 (socket says 172.21.15.100:39136) 2024-03-12T05:27:03.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 -- 172.21.15.100:0/2220882542 learned_addr learned my addr 172.21.15.100:0/2220882542 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8b9b35640 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8bab37640 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdc78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 -- 172.21.15.100:0/2220882542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 msgr2=0x55855fdd1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 -- 172.21.15.100:0/2220882542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 msgr2=0x55855fdc78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdc78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 -- 172.21.15.100:0/2220882542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855fcd0c00 con 0x55855fdce240 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.443+0000 7fe8ba336640 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd73c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe8b002c110 tx=0x7fe8b002c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62de00602e991fa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.444+0000 7fe89ffff640 1 -- 172.21.15.100:0/2220882542 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00042e0 con 0x55855fdce240 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.444+0000 7fe89ffff640 1 -- 172.21.15.100:0/2220882542 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8b00349b0 con 0x55855fdce240 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.444+0000 7fe89ffff640 1 -- 172.21.15.100:0/2220882542 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00035f0 con 0x55855fdce240 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.444+0000 7fe8bf174740 1 -- 172.21.15.100:0/2220882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 msgr2=0x55855fdd73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.444+0000 7fe8bf174740 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd73c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe8b002c110 tx=0x7fe8b002c2d0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- 172.21.15.100:0/2220882542 shutdown_connections 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdd7d50 0x55855fdc78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fdd1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 --2- 172.21.15.100:0/2220882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdce240 0x55855fdd73c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- 172.21.15.100:0/2220882542 >> 172.21.15.100:0/2220882542 conn(0x55855fdc88a0 msgr2=0x55855fdc71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- 172.21.15.100:0/2220882542 shutdown_connections 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- 172.21.15.100:0/2220882542 wait complete. 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- start start 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 Processor -- start 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 0x55855fde0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fde1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdd7d50 0x55855fde15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fe3e3a0 con 0x55855fdd7d50 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fe3e4e0 con 0x55855fdce240 2024-03-12T05:27:03.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855fe3e620 con 0x55855fdcebf0 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bf174740 10 client.0 _ll_register_callbacks cb 0x7fe8ac009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8bab37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdd7d50 0x55855fde15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8ba336640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 0x55855fde0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8b9b35640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fde1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8ba336640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 0x55855fde0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:36710/0 (socket says 172.21.15.100:36710) 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.445+0000 7fe8ba336640 1 -- 172.21.15.100:0/3040830747 learned_addr learned my addr 172.21.15.100:0/3040830747 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bab37640 1 -- 172.21.15.100:0/3040830747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 msgr2=0x55855fde1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bab37640 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fde1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bab37640 1 -- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 msgr2=0x55855fde0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bab37640 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 0x55855fde0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bab37640 1 -- 172.21.15.100:0/3040830747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55855fd34320 con 0x55855fdd7d50 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8ba336640 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 0x55855fde0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:03.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bab37640 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdd7d50 0x55855fde15c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac018f00 tx=0x7fe8ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9700ea7d1f9a7f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe89dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe89dffb640 1 -- 172.21.15.100:0/3040830747 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0099c0 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe89dffb640 1 -- 172.21.15.100:0/3040830747 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe8ac032050 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55855fd5b900 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe89dffb640 1 -- 172.21.15.100:0/3040830747 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8ac00d460 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bf174740 10 client.18675 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.446+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55855fcf6910 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe89dffb640 1 -- 172.21.15.100:0/3040830747 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55855fd5b900 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe89dffb640 1 -- 172.21.15.100:0/3040830747 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe8ac0052b0 con 0x55855fdd7d50 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8bf174740 10 client.18675 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8bf174740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8bf174740 20 client.18675 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8bf174740 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fe41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8bf174740 4 client.18675 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:27:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55855fa35860 con 0x55855fde7ea0 2024-03-12T05:27:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8ba336640 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fe41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe89dffb640 10 client.18675 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.447+0000 7fe8ba336640 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fe41ef0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f0e57995112e54 server_cookie=52170e4d06fce40f in_seq=0 out_seq=0 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe89dffb640 1 -- 172.21.15.100:0/3040830747 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55855fa35860 con 0x55855fde7ea0 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe89dffb640 10 client.18675 handle_command_reply: tid=0 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 client.18675 shutdown 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdd7d50 msgr2=0x55855fde15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdd7d50 0x55855fde15c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac018f00 tx=0x7fe8ac009bf0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 shutdown_connections 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55855fde7ea0 0x55855fe41ef0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55855fdd7d50 0x55855fde15c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55855fdcebf0 0x55855fde1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 --2- 172.21.15.100:0/3040830747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55855fdce240 0x55855fde0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe89dffb640 0 client.18675 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 >> 172.21.15.100:0/3040830747 conn(0x55855fdc88a0 msgr2=0x55855fdc7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 shutdown_connections 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 1 -- 172.21.15.100:0/3040830747 wait complete. 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.449+0000 7fe8bf174740 20 client.18675 trim_cache size 0 max 16384 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:27:03.449 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c38d4f15-615d-43b4-90eb-f91873d515f6", 2024-03-12T05:27:03.450 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:27:03.450 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:27:03.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8a00a1ba0 msgr2=0x7fe8a00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8a00a1ba0 0x7fe8a00a4060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c6a0 tx=0x7fe8ac025000 comp rx=0 tx=0).stop 2024-03-12T05:27:03.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 msgr2=0x7fe8b419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 0x7fe8b419d9b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e9c0 tx=0x7fe8a400ee80 comp rx=0 tx=0).stop 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 shutdown_connections 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe8a00a1ba0 0x7fe8a00a4060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b4106120 0x7fe8b4197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b4069e40 0x7fe8b419d9b0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 --2- 172.21.15.100:0/719997042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b40694a0 0x7fe8b419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 >> 172.21.15.100:0/719997042 conn(0x7fe8b4077110 msgr2=0x7fe8b40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 shutdown_connections 2024-03-12T05:27:03.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.450+0000 7fe8bc59a640 1 -- 172.21.15.100:0/719997042 wait complete. 2024-03-12T05:27:03.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:03 smithi156 ceph-mon[36270]: pgmap v467: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 793 op/s 2024-03-12T05:27:03.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:27:03.456 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:27:03.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:03 smithi161 ceph-mon[36043]: pgmap v467: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 793 op/s 2024-03-12T05:27:03.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 -- 172.21.15.100:0/698443524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c1061a0 msgr2=0x7fab1c106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 --2- 172.21.15.100:0/698443524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c1061a0 0x7fab1c106580 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009930 tx=0x7fab0c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 -- 172.21.15.100:0/698443524 shutdown_connections 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 --2- 172.21.15.100:0/698443524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c100c90 0x7fab1c10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 --2- 172.21.15.100:0/698443524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c1002d0 0x7fab1c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 --2- 172.21.15.100:0/698443524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c1061a0 0x7fab1c106580 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 -- 172.21.15.100:0/698443524 >> 172.21.15.100:0/698443524 conn(0x7fab1c076e10 msgr2=0x7fab1c077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 -- 172.21.15.100:0/698443524 shutdown_connections 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 -- 172.21.15.100:0/698443524 wait complete. 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 Processor -- start 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.615+0000 7fab226eb640 1 -- start start 2024-03-12T05:27:03.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab226eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c1002d0 0x7fab1c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab226eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 0x7fab1c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab226eb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c1061a0 0x7fab1c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab226eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c197c10 con 0x7fab1c100c90 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab226eb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c197d80 con 0x7fab1c1002d0 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab226eb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c197f40 con 0x7fab1c1061a0 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 0x7fab1c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 0x7fab1c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39160/0 (socket says 172.21.15.100:39160) 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 -- 172.21.15.100:0/400734462 learned_addr learned my addr 172.21.15.100:0/400734462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1bfff640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c1002d0 0x7fab1c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab20c88640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c1061a0 0x7fab1c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 -- 172.21.15.100:0/400734462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c1061a0 msgr2=0x7fab1c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c1061a0 0x7fab1c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 -- 172.21.15.100:0/400734462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c1002d0 msgr2=0x7fab1c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c1002d0 0x7fab1c19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 -- 172.21.15.100:0/400734462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c009560 con 0x7fab1c100c90 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 0x7fab1c19d950 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fab1000d950 tx=0x7fab1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.616+0000 7fab197fa640 1 -- 172.21.15.100:0/400734462 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10009a30 con 0x7fab1c100c90 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.617+0000 7fab197fa640 1 -- 172.21.15.100:0/400734462 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab10009b90 con 0x7fab1c100c90 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.617+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c198220 con 0x7fab1c100c90 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.617+0000 7fab197fa640 1 -- 172.21.15.100:0/400734462 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10004a00 con 0x7fab1c100c90 2024-03-12T05:27:03.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.617+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a44b0 con 0x7fab1c100c90 2024-03-12T05:27:03.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab197fa640 1 -- 172.21.15.100:0/400734462 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fab10009d00 con 0x7fab1c100c90 2024-03-12T05:27:03.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 Processor -- start 2024-03-12T05:27:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 -- start start 2024-03-12T05:27:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561787242280 con 0x5617872d3bd0 2024-03-12T05:27:03.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178723f7b0 con 0x5617872d32b0 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab25396740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178723af90 con 0x5617872dc7b0 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872dc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39162/0 (socket says 172.21.15.100:39162) 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab1bfff640 1 -- 172.21.15.100:0/2590965686 learned_addr learned my addr 172.21.15.100:0/2590965686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.619+0000 7fab197fa640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab040a1ba0 0x7fab040a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab20c88640 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1bfff640 1 -- 172.21.15.100:0/2590965686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 msgr2=0x5617872deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab20c88640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab040a1ba0 0x7fab040a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1bfff640 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872deba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1bfff640 1 -- 172.21.15.100:0/2590965686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 msgr2=0x5617872d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1bfff640 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1bfff640 1 -- 172.21.15.100:0/2590965686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617871f9570 con 0x5617872d3bd0 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab197fa640 1 -- 172.21.15.100:0/400734462 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fab100cd260 con 0x7fab1c100c90 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab1bfff640 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872dc270 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fab0c036040 tx=0x7fab0c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea3bbf7a486abf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab20c88640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab040a1ba0 0x7fab040a4060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab14009370 tx=0x7fab14009240 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab027fc640 1 -- 172.21.15.100:0/2590965686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c004900 con 0x5617872d3bd0 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab027fc640 1 -- 172.21.15.100:0/2590965686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab0c004a60 con 0x5617872d3bd0 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.620+0000 7fab027fc640 1 -- 172.21.15.100:0/2590965686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0051f0 con 0x5617872d3bd0 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 -- 172.21.15.100:0/2590965686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 msgr2=0x5617872dc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872dc270 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fab0c036040 tx=0x7fab0c005c00 comp rx=0 tx=0).stop 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 -- 172.21.15.100:0/2590965686 shutdown_connections 2024-03-12T05:27:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872dc270 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 --2- 172.21.15.100:0/2590965686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 -- 172.21.15.100:0/2590965686 >> 172.21.15.100:0/2590965686 conn(0x5617872cd990 msgr2=0x5617872cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 -- 172.21.15.100:0/2590965686 shutdown_connections 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 -- 172.21.15.100:0/2590965686 wait complete. 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 -- start start 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.621+0000 7fab25396740 1 Processor -- start 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872d9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872d9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561786f0e430 con 0x5617872d3bd0 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561786ecae00 con 0x5617872d32b0 2024-03-12T05:27:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617871bffd0 con 0x5617872dc7b0 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 10 client.0 _ll_register_callbacks cb 0x561787215fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872d9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872d9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39166/0 (socket says 172.21.15.100:39166) 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1357693848 learned_addr learned my addr 172.21.15.100:0/1357693848 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab20c88640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872d9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1357693848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 msgr2=0x5617872d9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872d9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 msgr2=0x5617872dc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872dc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1357693848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c069830 con 0x5617872d3bd0 2024-03-12T05:27:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872d9510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fab10000c00 tx=0x7fab10065580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4b2913cdc27eda server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7faafb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7faafb7fe640 1 -- 172.21.15.100:0/1357693848 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1009a550 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7faafb7fe640 1 -- 172.21.15.100:0/1357693848 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fab10099e50 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561786f3a7e0 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7faafb7fe640 1 -- 172.21.15.100:0/1357693848 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab10069070 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 10 client.18687 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.622+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561787239610 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7faafb7fe640 1 -- 172.21.15.100:0/1357693848 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561786f3a7e0 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7faafb7fe640 1 -- 172.21.15.100:0/1357693848 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fab100992a0 con 0x5617872d3bd0 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab25396740 10 client.18687 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab25396740 10 client.18687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab25396740 20 client.18687 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab25396740 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5617872ec7d0 0x5617872eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab25396740 4 client.18687 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561786f77720 con 0x5617872ec7d0 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5617872ec7d0 0x5617872eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7faafb7fe640 10 client.18687 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.623+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5617872ec7d0 0x5617872eebc0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38bbbaa5f6c7a37a server_cookie=eba84b7c3d4a09b4 in_seq=0 out_seq=0 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7faafb7fe640 1 -- 172.21.15.100:0/1357693848 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561786f77720 con 0x5617872ec7d0 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7faafb7fe640 10 client.18687 handle_command_reply: tid=0 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 client.18687 shutdown 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 msgr2=0x5617872d9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872d9510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fab10000c00 tx=0x7fab10065580 comp rx=0 tx=0).stop 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 shutdown_connections 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5617872ec7d0 0x5617872eebc0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872d9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872d9510 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7faafb7fe640 0 client.18687 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 --2- 172.21.15.100:0/1357693848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872dc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 >> 172.21.15.100:0/1357693848 conn(0x5617872cd990 msgr2=0x5617872cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 shutdown_connections 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 1 -- 172.21.15.100:0/1357693848 wait complete. 2024-03-12T05:27:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.624+0000 7fab25396740 20 client.18687 trim_cache size 0 max 16384 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 Processor -- start 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 -- start start 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 0x5617872d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872d3bd0 0x5617872dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872dc7b0 0x5617872dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617872d2f10 con 0x5617872d32b0 2024-03-12T05:27:03.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617872d4320 con 0x5617872dc7b0 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab25396740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617872c7e20 con 0x5617872d3bd0 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 0x5617872d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 0x5617872d94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39176/0 (socket says 172.21.15.100:39176) 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 -- 172.21.15.100:0/1415082276 learned_addr learned my addr 172.21.15.100:0/1415082276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab20c88640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872dc7b0 0x5617872dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872d3bd0 0x5617872dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 -- 172.21.15.100:0/1415082276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872d3bd0 msgr2=0x5617872dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872d3bd0 0x5617872dbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 -- 172.21.15.100:0/1415082276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872dc7b0 msgr2=0x5617872dc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872dc7b0 0x5617872dc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1bfff640 1 -- 172.21.15.100:0/1415082276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617871d5ac0 con 0x5617872d32b0 2024-03-12T05:27:03.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872d3bd0 0x5617872dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.639+0000 7fab20c88640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872dc7b0 0x5617872dc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 0x5617872d94b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fab0c004e60 tx=0x7fab0c004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb9b6618c04015e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab01ffb640 1 -- 172.21.15.100:0/1415082276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c051d00 con 0x5617872d32b0 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab01ffb640 1 -- 172.21.15.100:0/1415082276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab0c051390 con 0x5617872d32b0 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab01ffb640 1 -- 172.21.15.100:0/1415082276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0054f0 con 0x5617872d32b0 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 -- 172.21.15.100:0/1415082276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 msgr2=0x5617872d94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 0x5617872d94b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fab0c004e60 tx=0x7fab0c004df0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 -- 172.21.15.100:0/1415082276 shutdown_connections 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872dc7b0 0x5617872dc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872d3bd0 0x5617872dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 --2- 172.21.15.100:0/1415082276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d32b0 0x5617872d94b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 -- 172.21.15.100:0/1415082276 >> 172.21.15.100:0/1415082276 conn(0x5617872cd990 msgr2=0x5617872d8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 -- 172.21.15.100:0/1415082276 shutdown_connections 2024-03-12T05:27:03.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.640+0000 7fab25396740 1 -- 172.21.15.100:0/1415082276 wait complete. 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 -- start start 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 Processor -- start 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872e1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617872e1b50 con 0x5617872d3bd0 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617872e1cc0 con 0x5617872d32b0 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617872e1e80 con 0x5617872dc7b0 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab25396740 10 client.0 _ll_register_callbacks cb 0x7fab0c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39186/0 (socket says 172.21.15.100:39186) 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1145484351 learned_addr learned my addr 172.21.15.100:0/1145484351 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab20c88640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1145484351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 msgr2=0x5617872e1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872e1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 msgr2=0x5617872e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab1b7fe640 1 -- 172.21.15.100:0/1145484351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617871f9570 con 0x5617872d3bd0 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.641+0000 7fab20c88640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872e1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab1b7fe640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872e3b70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fab10099de0 tx=0x7fab10005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c340d4ea1db4a269 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7faafb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7faafb7fe640 1 -- 172.21.15.100:0/1145484351 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab100adea0 con 0x5617872d3bd0 2024-03-12T05:27:03.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7faafb7fe640 1 -- 172.21.15.100:0/1145484351 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fab1009a550 con 0x5617872d3bd0 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab0c004a60 con 0x5617872d3bd0 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7faafb7fe640 1 -- 172.21.15.100:0/1145484351 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab10096110 con 0x5617872d3bd0 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 10 client.18693 fetch_fsmap learned FSMap version 27 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5617871fb6a0 con 0x5617872d3bd0 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7faafb7fe640 1 -- 172.21.15.100:0/1145484351 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab0c004a60 con 0x5617872d3bd0 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7faafb7fe640 1 -- 172.21.15.100:0/1145484351 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fab100b80e0 con 0x5617872d3bd0 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 10 client.18693 fetch_fsmap finished waiting for FSMap version 27 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 10 client.18693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 20 client.18693 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561787344730 0x561787346b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 4 client.18693 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561786fc6890 con 0x561787344730 2024-03-12T05:27:03.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.642+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561787344730 0x561787346b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7faafb7fe640 10 client.18693 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab1bfff640 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561787344730 0x561787346b20 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad1c21976f781cf server_cookie=5005e43b257f82bc in_seq=0 out_seq=0 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7faafb7fe640 1 -- 172.21.15.100:0/1145484351 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561786fc6890 con 0x561787344730 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7faafb7fe640 10 client.18693 handle_command_reply: tid=0 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 client.18693 shutdown 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 msgr2=0x5617872e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872e3b70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fab10099de0 tx=0x7fab10005240 comp rx=0 tx=0).stop 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 shutdown_connections 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561787344730 0x561787346b20 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617872dc7b0 0x5617872e1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7faafb7fe640 0 client.18693 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:03.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617872d3bd0 0x5617872e3b70 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 --2- 172.21.15.100:0/1145484351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617872d32b0 0x5617872e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 >> 172.21.15.100:0/1145484351 conn(0x5617872cd990 msgr2=0x5617872cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 shutdown_connections 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 1 -- 172.21.15.100:0/1145484351 wait complete. 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.643+0000 7fab25396740 20 client.18693 trim_cache size 0 max 16384 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:27:03.646 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout: "c38d4f15-615d-43b4-90eb-f91873d515f6": { 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c38d4f15-615d-43b4-90eb-f91873d515f6", 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:27:03.647 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:27:03.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab040a1ba0 msgr2=0x7fab040a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab040a1ba0 0x7fab040a4060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab14009370 tx=0x7fab14009240 comp rx=0 tx=0).stop 2024-03-12T05:27:03.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 msgr2=0x7fab1c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 0x7fab1c19d950 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fab1000d950 tx=0x7fab1000de10 comp rx=0 tx=0).stop 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 shutdown_connections 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab040a1ba0 0x7fab040a4060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c1061a0 0x7fab1c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c100c90 0x7fab1c19d950 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 --2- 172.21.15.100:0/400734462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c1002d0 0x7fab1c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 >> 172.21.15.100:0/400734462 conn(0x7fab1c076e10 msgr2=0x7fab1c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 shutdown_connections 2024-03-12T05:27:03.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:03.644+0000 7fab226eb640 1 -- 172.21.15.100:0/400734462 wait complete. 2024-03-12T05:27:03.649 INFO:tasks.cephfs.filesystem:scrub status for tag:c38d4f15-615d-43b4-90eb-f91873d515f6 - {'path': '/', 'tag': 'c38d4f15-615d-43b4-90eb-f91873d515f6', 'options': 'recursive,force'} 2024-03-12T05:27:04.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66334 35.01 MB/sec warmup 107 sec latency 172.945 ms 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:04 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:04 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:04 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:05.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66766 34.89 MB/sec warmup 108 sec latency 142.213 ms 2024-03-12T05:27:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:05 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:05 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:05 smithi100 ceph-mon[32210]: pgmap v468: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.5 MiB/s wr, 521 op/s 2024-03-12T05:27:05.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:05 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:05 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:05 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:05 smithi156 ceph-mon[36270]: pgmap v468: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.5 MiB/s wr, 521 op/s 2024-03-12T05:27:05.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:05 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:05.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:05 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:05.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:05 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:05.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:05 smithi161 ceph-mon[36043]: pgmap v468: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.5 MiB/s wr, 521 op/s 2024-03-12T05:27:05.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:05 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:06.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67472 34.95 MB/sec warmup 109 sec latency 86.178 ms 2024-03-12T05:27:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:06 smithi100 ceph-mon[32210]: osdmap e106: 12 total, 12 up, 12 in 2024-03-12T05:27:06.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:06 smithi100 ceph-mon[32210]: pgmap v470: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-12T05:27:06.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:06 smithi156 ceph-mon[36270]: osdmap e106: 12 total, 12 up, 12 in 2024-03-12T05:27:06.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:06 smithi156 ceph-mon[36270]: pgmap v470: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-12T05:27:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:06 smithi161 ceph-mon[36043]: osdmap e106: 12 total, 12 up, 12 in 2024-03-12T05:27:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:06 smithi161 ceph-mon[36043]: pgmap v470: 97 pgs: 97 active+clean; 955 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-12T05:27:07.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68021 34.81 MB/sec warmup 110 sec latency 337.767 ms 2024-03-12T05:27:08.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68600 34.83 MB/sec warmup 111 sec latency 76.302 ms 2024-03-12T05:27:09.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69347 34.95 MB/sec warmup 112 sec latency 59.412 ms 2024-03-12T05:27:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:08 smithi100 ceph-mon[32210]: pgmap v471: 97 pgs: 97 active+clean; 987 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-03-12T05:27:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:08 smithi156 ceph-mon[36270]: pgmap v471: 97 pgs: 97 active+clean; 987 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-03-12T05:27:09.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:08 smithi161 ceph-mon[36043]: pgmap v471: 97 pgs: 97 active+clean; 987 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-03-12T05:27:10.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70007 35.03 MB/sec warmup 113 sec latency 95.846 ms 2024-03-12T05:27:11.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70422 34.89 MB/sec warmup 114 sec latency 76.920 ms 2024-03-12T05:27:11.271 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:11 smithi161 ceph-mon[36043]: pgmap v472: 97 pgs: 97 active+clean; 987 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-03-12T05:27:11.271 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:11 smithi161 ceph-mon[36043]: osdmap e107: 12 total, 12 up, 12 in 2024-03-12T05:27:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:11 smithi100 ceph-mon[32210]: pgmap v472: 97 pgs: 97 active+clean; 987 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-03-12T05:27:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:11 smithi100 ceph-mon[32210]: osdmap e107: 12 total, 12 up, 12 in 2024-03-12T05:27:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:11 smithi156 ceph-mon[36270]: pgmap v472: 97 pgs: 97 active+clean; 987 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-03-12T05:27:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:11 smithi156 ceph-mon[36270]: osdmap e107: 12 total, 12 up, 12 in 2024-03-12T05:27:12.233 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70742 34.69 MB/sec warmup 115 sec latency 212.291 ms 2024-03-12T05:27:13.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71363 34.58 MB/sec warmup 116 sec latency 119.844 ms 2024-03-12T05:27:13.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:13 smithi100 ceph-mon[32210]: pgmap v474: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 896 op/s 2024-03-12T05:27:13.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:13 smithi156 ceph-mon[36270]: pgmap v474: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 896 op/s 2024-03-12T05:27:13.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:13 smithi161 ceph-mon[36043]: pgmap v474: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 896 op/s 2024-03-12T05:27:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71990 34.64 MB/sec warmup 117 sec latency 179.591 ms 2024-03-12T05:27:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 72809 34.67 MB/sec warmup 118 sec latency 52.951 ms 2024-03-12T05:27:15.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:15 smithi100 ceph-mon[32210]: pgmap v475: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 803 op/s 2024-03-12T05:27:15.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:15 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:27:15.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:15 smithi156 ceph-mon[36270]: pgmap v475: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 803 op/s 2024-03-12T05:27:15.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:15 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:27:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:15 smithi161 ceph-mon[36043]: pgmap v475: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 803 op/s 2024-03-12T05:27:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:15 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:27:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73499 34.78 MB/sec warmup 119 sec latency 75.175 ms 2024-03-12T05:27:16.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:16 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:27:16.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:16 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:27:16.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:16 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:27:16.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:16 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:27:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:16 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:27:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:16 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:27:17.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:17 smithi100 ceph-mon[32210]: pgmap v476: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 717 op/s 2024-03-12T05:27:17.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:17 smithi156 ceph-mon[36270]: pgmap v476: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 717 op/s 2024-03-12T05:27:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:17 smithi161 ceph-mon[36043]: pgmap v476: 97 pgs: 97 active+clean; 1008 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 717 op/s 2024-03-12T05:27:18.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74490 25.48 MB/sec execute 1 sec latency 336.458 ms 2024-03-12T05:27:18.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:18 smithi100 ceph-mon[32210]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 629 op/s 2024-03-12T05:27:18.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:18 smithi156 ceph-mon[36270]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 629 op/s 2024-03-12T05:27:18.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:18 smithi161 ceph-mon[36043]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 629 op/s 2024-03-12T05:27:19.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74894 23.20 MB/sec execute 2 sec latency 228.793 ms 2024-03-12T05:27:20.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75441 24.28 MB/sec execute 3 sec latency 105.793 ms 2024-03-12T05:27:21.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76175 28.09 MB/sec execute 4 sec latency 87.366 ms 2024-03-12T05:27:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:21 smithi100 ceph-mon[32210]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 629 op/s 2024-03-12T05:27:21.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:21 smithi156 ceph-mon[36270]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 629 op/s 2024-03-12T05:27:21.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:21 smithi161 ceph-mon[36043]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 629 op/s 2024-03-12T05:27:22.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76703 27.50 MB/sec execute 5 sec latency 300.376 ms 2024-03-12T05:27:23.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77337 28.70 MB/sec execute 6 sec latency 69.397 ms 2024-03-12T05:27:23.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:23 smithi100 ceph-mon[32210]: pgmap v479: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 785 op/s 2024-03-12T05:27:23.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:23 smithi156 ceph-mon[36270]: pgmap v479: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 785 op/s 2024-03-12T05:27:23.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:23 smithi161 ceph-mon[36043]: pgmap v479: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 785 op/s 2024-03-12T05:27:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77977 30.53 MB/sec execute 7 sec latency 109.725 ms 2024-03-12T05:27:25.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 78603 31.51 MB/sec execute 8 sec latency 98.790 ms 2024-03-12T05:27:25.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:25 smithi156 ceph-mon[36270]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 527 op/s 2024-03-12T05:27:25.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:25 smithi100 ceph-mon[32210]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 527 op/s 2024-03-12T05:27:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:25 smithi161 ceph-mon[36043]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 527 op/s 2024-03-12T05:27:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79022 30.47 MB/sec execute 9 sec latency 98.441 ms 2024-03-12T05:27:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79322 28.25 MB/sec execute 10 sec latency 236.416 ms 2024-03-12T05:27:27.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:27 smithi100 ceph-mon[32210]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 527 op/s 2024-03-12T05:27:27.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:27 smithi156 ceph-mon[36270]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 527 op/s 2024-03-12T05:27:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:27 smithi161 ceph-mon[36043]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 527 op/s 2024-03-12T05:27:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79991 28.17 MB/sec execute 11 sec latency 299.242 ms 2024-03-12T05:27:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:28 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:27:28.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:28 smithi100 ceph-mon[32210]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-03-12T05:27:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:28 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:27:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:28 smithi156 ceph-mon[36270]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-03-12T05:27:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:28 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:27:28.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:28 smithi161 ceph-mon[36043]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-03-12T05:27:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80872 29.77 MB/sec execute 12 sec latency 79.810 ms 2024-03-12T05:27:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81563 31.10 MB/sec execute 13 sec latency 53.724 ms 2024-03-12T05:27:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:30 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:27:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:30 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:27:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:30 smithi100 ceph-mon[32210]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.1 MiB/s wr, 476 op/s 2024-03-12T05:27:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:30 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:27:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:30 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:27:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:30 smithi156 ceph-mon[36270]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.1 MiB/s wr, 476 op/s 2024-03-12T05:27:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:30 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:27:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:30 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:27:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:30 smithi161 ceph-mon[36043]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.1 MiB/s wr, 476 op/s 2024-03-12T05:27:31.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82145 31.52 MB/sec execute 14 sec latency 121.644 ms 2024-03-12T05:27:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:31 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:27:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:31 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:27:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:31 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:27:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:31 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:27:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:31 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:27:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:31 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:27:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:31 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:27:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:31 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:27:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:31 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:27:32.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82628 30.93 MB/sec execute 15 sec latency 144.706 ms 2024-03-12T05:27:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:32 smithi100 ceph-mon[32210]: pgmap v484: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-12T05:27:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:32 smithi156 ceph-mon[36270]: pgmap v484: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-12T05:27:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:32 smithi161 ceph-mon[36043]: pgmap v484: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-12T05:27:33.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83174 31.18 MB/sec execute 16 sec latency 164.294 ms 2024-03-12T05:27:33.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:27:33.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:27:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 -- 172.21.15.100:0/3329414654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281002d0 msgr2=0x7f4128100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 --2- 172.21.15.100:0/3329414654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281002d0 0x7f4128100730 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f411c009900 tx=0x7f411c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 -- 172.21.15.100:0/3329414654 shutdown_connections 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 --2- 172.21.15.100:0/3329414654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 0x7f412810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 --2- 172.21.15.100:0/3329414654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281002d0 0x7f4128100730 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 --2- 172.21.15.100:0/3329414654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281060b0 0x7f4128106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 -- 172.21.15.100:0/3329414654 >> 172.21.15.100:0/3329414654 conn(0x7f4128076e20 msgr2=0x7f4128077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 -- 172.21.15.100:0/3329414654 shutdown_connections 2024-03-12T05:27:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.775+0000 7f412db93640 1 -- 172.21.15.100:0/3329414654 wait complete. 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f412db93640 1 Processor -- start 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f412db93640 1 -- start start 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f412db93640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 0x7f4128195370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f412db93640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 0x7f41281958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f412db93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 0x7f4128195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f41277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 0x7f4128195370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f41277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 0x7f4128195370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:39376/0 (socket says 172.21.15.100:39376) 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.776+0000 7f4127fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 0x7f4128195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 0x7f4128195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53550/0 (socket says 172.21.15.100:53550) 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f41277fe640 1 -- 172.21.15.100:0/4192815629 learned_addr learned my addr 172.21.15.100:0/4192815629 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4126ffd640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 0x7f41281958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41281a9fa0 con 0x7f41281060b0 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41281aa0e0 con 0x7f41281002d0 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41281aa220 con 0x7f4128100c70 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 -- 172.21.15.100:0/4192815629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 msgr2=0x7f41281958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 0x7f41281958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 -- 172.21.15.100:0/4192815629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 msgr2=0x7f4128195370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 0x7f4128195370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 -- 172.21.15.100:0/4192815629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009560 con 0x7f41281060b0 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f41277fe640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 0x7f4128195370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4126ffd640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 0x7f41281958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.777+0000 7f4127fff640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 0x7f4128195df0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f412000ea80 tx=0x7f412000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.778+0000 7f4124ff9640 1 -- 172.21.15.100:0/4192815629 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000cea0 con 0x7f41281060b0 2024-03-12T05:27:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.778+0000 7f4124ff9640 1 -- 172.21.15.100:0/4192815629 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4120005020 con 0x7f41281060b0 2024-03-12T05:27:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.778+0000 7f4124ff9640 1 -- 172.21.15.100:0/4192815629 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120010690 con 0x7f41281060b0 2024-03-12T05:27:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.778+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41281aa4d0 con 0x7f41281060b0 2024-03-12T05:27:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.778+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41281aa960 con 0x7f41281060b0 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f4124ff9640 1 -- 172.21.15.100:0/4192815629 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4120005500 con 0x7f41281060b0 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 Processor -- start 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 -- start start 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f4124ff9640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f41100a1c10 0x7f41100a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c36d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36dbd0 0x55a97c376270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c378ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f4124ff9640 1 -- 172.21.15.100:0/4192815629 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4120014070 con 0x7f41281060b0 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c2dc280 con 0x55a97c36d2b0 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c2d97b0 con 0x55a97c36dbd0 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.781+0000 7f413076c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c2d4f90 con 0x55a97c3767b0 2024-03-12T05:27:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f4126ffd640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f41100a1c10 0x7f41100a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c36d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c36d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53552/0 (socket says 172.21.15.100:53552) 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 -- 172.21.15.100:0/3027170698 learned_addr learned my addr 172.21.15.100:0/3027170698 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f4126ffd640 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36dbd0 0x55a97c376270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f4127fff640 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c378ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 -- 172.21.15.100:0/3027170698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 msgr2=0x55a97c378ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c378ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 -- 172.21.15.100:0/3027170698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36dbd0 msgr2=0x55a97c376270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36dbd0 0x55a97c376270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 -- 172.21.15.100:0/3027170698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97c293570 con 0x55a97c36d2b0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f4126ffd640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f41100a1c10 0x7f41100a40d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f411c006010 tx=0x7f411c004b80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f41277fe640 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c36d690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f41180115c0 tx=0x7f4118011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22456d9f6f51ae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f410dffb640 1 -- 172.21.15.100:0/3027170698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800f5e0 con 0x55a97c36d2b0 2024-03-12T05:27:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f410dffb640 1 -- 172.21.15.100:0/3027170698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f411800fc20 con 0x55a97c36d2b0 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.782+0000 7f410dffb640 1 -- 172.21.15.100:0/3027170698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118019c90 con 0x55a97c36d2b0 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 -- 172.21.15.100:0/3027170698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 msgr2=0x55a97c36d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c36d690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f41180115c0 tx=0x7f4118011a80 comp rx=0 tx=0).stop 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 -- 172.21.15.100:0/3027170698 shutdown_connections 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c378ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36dbd0 0x55a97c376270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 --2- 172.21.15.100:0/3027170698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c36d690 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 -- 172.21.15.100:0/3027170698 >> 172.21.15.100:0/3027170698 conn(0x55a97c367990 msgr2=0x55a97c367d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 -- 172.21.15.100:0/3027170698 shutdown_connections 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.783+0000 7f413076c740 1 -- 172.21.15.100:0/3027170698 wait complete. 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 -- start start 2024-03-12T05:27:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 Processor -- start 2024-03-12T05:27:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c376240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c3735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c373cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97bfa8430 con 0x55a97c36dbd0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97bf64e00 con 0x55a97c36d2b0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c259fd0 con 0x55a97c3767b0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f413076c740 10 client.0 _ll_register_callbacks cb 0x55a97c2affd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f4126ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c3735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f41277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c376240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f4126ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c3735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53566/0 (socket says 172.21.15.100:53566) 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f4126ffd640 1 -- 172.21.15.100:0/2722094000 learned_addr learned my addr 172.21.15.100:0/2722094000 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f4127fff640 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c373cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f41277fe640 1 -- 172.21.15.100:0/2722094000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 msgr2=0x55a97c373cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f41277fe640 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c373cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f41277fe640 1 -- 172.21.15.100:0/2722094000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 msgr2=0x55a97c3735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f41277fe640 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c3735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.784+0000 7f41277fe640 1 -- 172.21.15.100:0/2722094000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4118009d80 con 0x55a97c36d2b0 2024-03-12T05:27:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f41277fe640 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c376240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4118000c00 tx=0x7f4118005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850e1ddfaf605521 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f4106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f4106ffd640 1 -- 172.21.15.100:0/2722094000 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118019410 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f4106ffd640 1 -- 172.21.15.100:0/2722094000 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f411800f7f0 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a97bfd47e0 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f4106ffd640 1 -- 172.21.15.100:0/2722094000 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f411800f9b0 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f413076c740 10 client.28073 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a97c2d3610 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f4106ffd640 1 -- 172.21.15.100:0/2722094000 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a97bfd47e0 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.785+0000 7f4106ffd640 1 -- 172.21.15.100:0/2722094000 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4118004a90 con 0x55a97c36d2b0 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f413076c740 10 client.28073 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f413076c740 10 client.28073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f413076c740 20 client.28073 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f413076c740 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3868f0 0x55a97c388d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f413076c740 4 client.28073 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a97c011720 con 0x55a97c3868f0 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f4126ffd640 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3868f0 0x55a97c388d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f4126ffd640 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3868f0 0x55a97c388d10 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d38045b9c6dbcd0 server_cookie=4e69b35cc1578544 in_seq=0 out_seq=0 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.786+0000 7f4106ffd640 10 client.28073 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f4106ffd640 1 -- 172.21.15.100:0/2722094000 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a97c011720 con 0x55a97c3868f0 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f4106ffd640 10 client.28073 handle_command_reply: tid=0 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 client.28073 shutdown 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 msgr2=0x55a97c376240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c376240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4118000c00 tx=0x7f4118005db0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 shutdown_connections 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3868f0 0x55a97c388d10 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c373cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c3735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 --2- 172.21.15.100:0/2722094000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c376240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 >> 172.21.15.100:0/2722094000 conn(0x55a97c367990 msgr2=0x55a97c369730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f4106ffd640 0 client.28073 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 shutdown_connections 2024-03-12T05:27:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 1 -- 172.21.15.100:0/2722094000 wait complete. 2024-03-12T05:27:33.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.787+0000 7f413076c740 20 client.28073 trim_cache size 0 max 16384 2024-03-12T05:27:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 Processor -- start 2024-03-12T05:27:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 -- start start 2024-03-12T05:27:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c375d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c376240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c372e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c3745c0 con 0x55a97c36dbd0 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c36c660 con 0x55a97c36d2b0 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f413076c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c368a10 con 0x55a97c3767b0 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.805+0000 7f4126ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c376240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c376240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53578/0 (socket says 172.21.15.100:53578) 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 -- 172.21.15.100:0/3491657072 learned_addr learned my addr 172.21.15.100:0/3491657072 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f41277fe640 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c375d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4127fff640 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c372e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 -- 172.21.15.100:0/3491657072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 msgr2=0x55a97c372e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c372e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 -- 172.21.15.100:0/3491657072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 msgr2=0x55a97c375d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c375d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 -- 172.21.15.100:0/3491657072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97c26fac0 con 0x55a97c36dbd0 2024-03-12T05:27:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f4126ffd640 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c376240 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x55a97c372e30 tx=0x7f411c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be18b454176e2e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f410d7fa640 1 -- 172.21.15.100:0/3491657072 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c053ea0 con 0x55a97c36dbd0 2024-03-12T05:27:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f410d7fa640 1 -- 172.21.15.100:0/3491657072 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f411c004440 con 0x55a97c36dbd0 2024-03-12T05:27:33.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.806+0000 7f410d7fa640 1 -- 172.21.15.100:0/3491657072 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c048070 con 0x55a97c36dbd0 2024-03-12T05:27:33.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 -- 172.21.15.100:0/3491657072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 msgr2=0x55a97c376240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c376240 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x55a97c372e30 tx=0x7f411c04c000 comp rx=0 tx=0).stop 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 -- 172.21.15.100:0/3491657072 shutdown_connections 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c3767b0 0x55a97c372e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36dbd0 0x55a97c376240 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 --2- 172.21.15.100:0/3491657072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c36d2b0 0x55a97c375d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 -- 172.21.15.100:0/3491657072 >> 172.21.15.100:0/3491657072 conn(0x55a97c367990 msgr2=0x55a97c371ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 -- 172.21.15.100:0/3491657072 shutdown_connections 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.807+0000 7f413076c740 1 -- 172.21.15.100:0/3491657072 wait complete. 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 -- start start 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 Processor -- start 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c37a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c36dbd0 0x55a97c37a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c3767b0 0x55a97c37ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c3dced0 con 0x55a97c36d2b0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c3dd040 con 0x55a97c3767b0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97c3dd200 con 0x55a97c36dbd0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f413076c740 10 client.0 _ll_register_callbacks cb 0x7f4118009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.808+0000 7f41277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c37a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c37a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53588/0 (socket says 172.21.15.100:53588) 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 -- 172.21.15.100:0/1752051316 learned_addr learned my addr 172.21.15.100:0/1752051316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4126ffd640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c36dbd0 0x55a97c37a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4127fff640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c3767b0 0x55a97c37ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 -- 172.21.15.100:0/1752051316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c36dbd0 msgr2=0x55a97c37a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c36dbd0 0x55a97c37a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 -- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c3767b0 msgr2=0x55a97c37ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c3767b0 0x55a97c37ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 -- 172.21.15.100:0/1752051316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412810d750 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4126ffd640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c36dbd0 0x55a97c37a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f41277fe640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c37a1e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f411800f470 tx=0x7f411800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a96b401cdd6b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4106ffd640 1 -- 172.21.15.100:0/1752051316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118016040 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4106ffd640 1 -- 172.21.15.100:0/1752051316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f411800f7f0 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f411800fc20 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f4106ffd640 1 -- 172.21.15.100:0/1752051316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f411800f9b0 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f413076c740 10 client.18711 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.809+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a97c2956a0 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f4106ffd640 1 -- 172.21.15.100:0/1752051316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f411800fc20 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f4106ffd640 1 -- 172.21.15.100:0/1752051316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4118019d20 con 0x55a97c36d2b0 2024-03-12T05:27:33.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f413076c740 10 client.18711 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f413076c740 10 client.18711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f413076c740 20 client.18711 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f413076c740 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3de810 0x55a97c3e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f413076c740 4 client.18711 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a97c060890 con 0x55a97c3de810 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f4126ffd640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3de810 0x55a97c3e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f4106ffd640 10 client.18711 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.810+0000 7f4126ffd640 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3de810 0x55a97c3e0c30 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7032446627c0d54 server_cookie=dfe9d49c616700e8 in_seq=0 out_seq=0 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f4106ffd640 1 -- 172.21.15.100:0/1752051316 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a97c060890 con 0x55a97c3de810 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f4106ffd640 10 client.18711 handle_command_reply: tid=0 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 client.18711 shutdown 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 msgr2=0x55a97c37a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c37a1e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f411800f470 tx=0x7f411800f4a0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 shutdown_connections 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a97c3de810 0x55a97c3e0c30 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97c3767b0 0x55a97c37ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f4106ffd640 0 client.18711 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a97c36dbd0 0x55a97c37a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 --2- 172.21.15.100:0/1752051316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a97c36d2b0 0x55a97c37a1e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 >> 172.21.15.100:0/1752051316 conn(0x55a97c367990 msgr2=0x55a97c372e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 shutdown_connections 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 1 -- 172.21.15.100:0/1752051316 wait complete. 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.811+0000 7f413076c740 20 client.18711 trim_cache size 0 max 16384 2024-03-12T05:27:33.811 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:27:33.812 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:27:33.812 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:27:33.812 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:27:33.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f41100a1c10 msgr2=0x7f41100a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f41100a1c10 0x7f41100a40d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f411c006010 tx=0x7f411c004b80 comp rx=0 tx=0).stop 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 msgr2=0x7f4128195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 0x7f4128195df0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f412000ea80 tx=0x7f412000ef40 comp rx=0 tx=0).stop 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 shutdown_connections 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f41100a1c10 0x7f41100a40d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41281060b0 0x7f4128195df0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4128100c70 0x7f41281958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 --2- 172.21.15.100:0/4192815629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41281002d0 0x7f4128195370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 >> 172.21.15.100:0/4192815629 conn(0x7f4128076e20 msgr2=0x7f41280fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 shutdown_connections 2024-03-12T05:27:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.812+0000 7f412db93640 1 -- 172.21.15.100:0/4192815629 wait complete. 2024-03-12T05:27:33.820 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:27:33.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:27:33.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 -- 172.21.15.100:0/843059217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c1002f0 msgr2=0x7f7b3c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 --2- 172.21.15.100:0/843059217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c1006d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009900 tx=0x7f7b2c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:33.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 -- 172.21.15.100:0/843059217 shutdown_connections 2024-03-12T05:27:33.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 --2- 172.21.15.100:0/843059217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b3c1053c0 0x7f7b3c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 --2- 172.21.15.100:0/843059217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c100ca0 0x7f7b3c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 --2- 172.21.15.100:0/843059217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c1006d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 -- 172.21.15.100:0/843059217 >> 172.21.15.100:0/843059217 conn(0x7f7b3c077180 msgr2=0x7f7b3c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 -- 172.21.15.100:0/843059217 shutdown_connections 2024-03-12T05:27:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.952+0000 7f7b41365640 1 -- 172.21.15.100:0/843059217 wait complete. 2024-03-12T05:27:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 Processor -- start 2024-03-12T05:27:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 -- start start 2024-03-12T05:27:33.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b3c1053c0 0x7f7b3c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c197690 0x7f7b3c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c198160 con 0x7f7b3c1002f0 2024-03-12T05:27:33.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1982d0 con 0x7f7b3c1053c0 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b41365640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c198490 con 0x7f7b3c197690 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b3affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b3affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53610/0 (socket says 172.21.15.100:53610) 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b3affd640 1 -- 172.21.15.100:0/1674106945 learned_addr learned my addr 172.21.15.100:0/1674106945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b3a7fc640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b3c1053c0 0x7f7b3c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.953+0000 7f7b3b7fe640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c197690 0x7f7b3c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b3affd640 1 -- 172.21.15.100:0/1674106945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c197690 msgr2=0x7f7b3c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b3affd640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c197690 0x7f7b3c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b3affd640 1 -- 172.21.15.100:0/1674106945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b3c1053c0 msgr2=0x7f7b3c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b3affd640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b3c1053c0 0x7f7b3c19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b3affd640 1 -- 172.21.15.100:0/1674106945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30009490 con 0x7f7b3c1002f0 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b3affd640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c19d580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c02c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c02dea0 con 0x7f7b3c1002f0 2024-03-12T05:27:33.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c009560 con 0x7f7b3c1002f0 2024-03-12T05:27:33.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c1a4720 con 0x7f7b3c1002f0 2024-03-12T05:27:33.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b2c004410 con 0x7f7b3c1002f0 2024-03-12T05:27:33.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.954+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c003b80 con 0x7f7b3c1002f0 2024-03-12T05:27:33.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.955+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b28005290 con 0x7f7b3c1002f0 2024-03-12T05:27:33.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.956+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7b2c003ce0 con 0x7f7b3c1002f0 2024-03-12T05:27:33.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.957+0000 7f7b23fff640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7b240a1c20 0x7f7b240a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:33.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.957+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7b2c0ef490 con 0x7f7b3c1002f0 2024-03-12T05:27:33.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.957+0000 7f7b3a7fc640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7b240a1c20 0x7f7b240a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:33.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.958+0000 7f7b3a7fc640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7b240a1c20 0x7f7b240a40e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c198e90 tx=0x7f7b30005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:33.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:33.962+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b2c0b83d0 con 0x7f7b3c1002f0 2024-03-12T05:27:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.226+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b28005810 con 0x7f7b3c1002f0 2024-03-12T05:27:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.227+0000 7f7b23fff640 1 -- 172.21.15.100:0/1674106945 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11359 (secure 0 0 0) 0x7f7b2c0bba90 con 0x7f7b3c1002f0 2024-03-12T05:27:34.221 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:27:34.221 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:27:28.572302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:27:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 29 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7b240a1c20 msgr2=0x7f7b240a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7b240a1c20 0x7f7b240a40e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c198e90 tx=0x7f7b30005f90 comp rx=0 tx=0).stop 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 msgr2=0x7f7b3c19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c19d580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c02c6b0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 shutdown_connections 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7b240a1c20 0x7f7b240a40e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b3c197690 0x7f7b3c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b3c1053c0 0x7f7b3c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 --2- 172.21.15.100:0/1674106945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b3c1002f0 0x7f7b3c19d580 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 >> 172.21.15.100:0/1674106945 conn(0x7f7b3c077180 msgr2=0x7f7b3c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.229+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 shutdown_connections 2024-03-12T05:27:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.230+0000 7f7b41365640 1 -- 172.21.15.100:0/1674106945 wait complete. 2024-03-12T05:27:34.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83598 30.29 MB/sec execute 17 sec latency 113.370 ms 2024-03-12T05:27:34.237 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:27:34.237 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:27:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 -- 172.21.15.100:0/3055904156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac100240 msgr2=0x7f0aac105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 --2- 172.21.15.100:0/3055904156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac100240 0x7f0aac105d70 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009930 tx=0x7f0a9c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 -- 172.21.15.100:0/3055904156 shutdown_connections 2024-03-12T05:27:34.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 --2- 172.21.15.100:0/3055904156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aac106450 0x7f0aac10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 --2- 172.21.15.100:0/3055904156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac100240 0x7f0aac105d70 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 --2- 172.21.15.100:0/3055904156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aac0ff890 0x7f0aac0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 -- 172.21.15.100:0/3055904156 >> 172.21.15.100:0/3055904156 conn(0x7f0aac0765a0 msgr2=0x7f0aac0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 -- 172.21.15.100:0/3055904156 shutdown_connections 2024-03-12T05:27:34.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.376+0000 7f0ab369a640 1 -- 172.21.15.100:0/3055904156 wait complete. 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.377+0000 7f0ab369a640 1 Processor -- start 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.377+0000 7f0ab369a640 1 -- start start 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.377+0000 7f0ab369a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 0x7f0aac19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab369a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aac106450 0x7f0aac19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 0x7f0aac19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab369a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aac197670 0x7f0aac197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab369a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac198140 con 0x7f0aac0ff890 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab369a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac1982b0 con 0x7f0aac106450 2024-03-12T05:27:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 0x7f0aac19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53634/0 (socket says 172.21.15.100:53634) 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab369a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac198470 con 0x7f0aac197670 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 -- 172.21.15.100:0/2415285844 learned_addr learned my addr 172.21.15.100:0/2415285844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1c37640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aac197670 0x7f0aac197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 -- 172.21.15.100:0/2415285844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aac197670 msgr2=0x7f0aac197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aac197670 0x7f0aac197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 -- 172.21.15.100:0/2415285844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aac106450 msgr2=0x7f0aac19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aac106450 0x7f0aac19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 -- 172.21.15.100:0/2415285844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009560 con 0x7f0aac0ff890 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0ab1436640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 0x7f0aac19d5b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800e9f0 tx=0x7f0aa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.378+0000 7f0aa27fc640 1 -- 172.21.15.100:0/2415285844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8009720 con 0x7f0aac0ff890 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.379+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac1a4410 con 0x7f0aac0ff890 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.379+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac1a4890 con 0x7f0aac0ff890 2024-03-12T05:27:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.379+0000 7f0aa27fc640 1 -- 172.21.15.100:0/2415285844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0aa800cd80 con 0x7f0aac0ff890 2024-03-12T05:27:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.379+0000 7f0aa27fc640 1 -- 172.21.15.100:0/2415285844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8010430 con 0x7f0aac0ff890 2024-03-12T05:27:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.382+0000 7f0ab626f740 1 Processor -- start 2024-03-12T05:27:34.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.382+0000 7f0aa27fc640 1 -- 172.21.15.100:0/2415285844 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0aa8010670 con 0x7f0aac0ff890 2024-03-12T05:27:34.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.383+0000 7f0ab626f740 1 -- start start 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0aa27fc640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a980a1c20 0x7f0a980a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0aa27fc640 1 -- 172.21.15.100:0/2415285844 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0aa8014070 con 0x7f0aac0ff890 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d88f0 0x55dc150d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab626f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc15086960 con 0x55dc150c5a50 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab626f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1503a310 con 0x55dc150d88f0 2024-03-12T05:27:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab626f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc15037840 con 0x55dc150d92a0 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab1c37640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a980a1c20 0x7f0a980a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d88f0 0x55dc150d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab1c37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab0c35640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d88f0 0x55dc150d8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:39452/0 (socket says 172.21.15.100:39452) 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab0c35640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50014/0 (socket says 172.21.15.100:50014) 2024-03-12T05:27:34.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab1436640 1 -- 172.21.15.100:0/3624326816 learned_addr learned my addr 172.21.15.100:0/3624326816 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.384+0000 7f0ab1c37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53642/0 (socket says 172.21.15.100:53642) 2024-03-12T05:27:34.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.385+0000 7f0ab0c35640 1 -- 172.21.15.100:0/3624326816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d88f0 msgr2=0x55dc150d8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.385+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d88f0 0x55dc150d8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.385+0000 7f0ab0c35640 1 -- 172.21.15.100:0/3624326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 msgr2=0x55dc150c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.385+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.385+0000 7f0ab0c35640 1 -- 172.21.15.100:0/3624326816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc14ff5060 con 0x55dc150d92a0 2024-03-12T05:27:34.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.386+0000 7f0ab1c37640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a980a1c20 0x7f0a980a40e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005fd0 tx=0x7f0aa4009330 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.386+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150c5430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c02c170 tx=0x7f0a9c005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567b5b4b1f6123fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.387+0000 7f0a977fe640 1 -- 172.21.15.100:0/3624326816 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0046c0 con 0x55dc150d92a0 2024-03-12T05:27:34.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.387+0000 7f0a977fe640 1 -- 172.21.15.100:0/3624326816 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c004820 con 0x55dc150d92a0 2024-03-12T05:27:34.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.387+0000 7f0a977fe640 1 -- 172.21.15.100:0/3624326816 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c005050 con 0x55dc150d92a0 2024-03-12T05:27:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.388+0000 7f0ab626f740 1 -- 172.21.15.100:0/3624326816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 msgr2=0x55dc150c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.388+0000 7f0ab626f740 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150c5430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c02c170 tx=0x7f0a9c005e60 comp rx=0 tx=0).stop 2024-03-12T05:27:34.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.389+0000 7f0ab626f740 1 -- 172.21.15.100:0/3624326816 shutdown_connections 2024-03-12T05:27:34.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.390+0000 7f0ab626f740 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.390+0000 7f0ab626f740 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150c5430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.390+0000 7f0ab626f740 1 --2- 172.21.15.100:0/3624326816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d88f0 0x55dc150d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.390+0000 7f0ab626f740 1 -- 172.21.15.100:0/3624326816 >> 172.21.15.100:0/3624326816 conn(0x55dc150c7ef0 msgr2=0x55dc150c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.390+0000 7f0ab626f740 1 -- 172.21.15.100:0/3624326816 shutdown_connections 2024-03-12T05:27:34.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.390+0000 7f0ab626f740 1 -- 172.21.15.100:0/3624326816 wait complete. 2024-03-12T05:27:34.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.391+0000 7f0ab626f740 1 -- start start 2024-03-12T05:27:34.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.391+0000 7f0ab626f740 1 Processor -- start 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d88f0 0x55dc150dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc15033020 con 0x55dc150c5a50 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc14d06430 con 0x55dc150d92a0 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc14cc2e00 con 0x55dc150d88f0 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150dc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53654/0 (socket says 172.21.15.100:53654) 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 -- 172.21.15.100:0/2679318264 learned_addr learned my addr 172.21.15.100:0/2679318264 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d88f0 0x55dc150dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab626f740 10 client.0 _ll_register_callbacks cb 0x55dc14fdacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1c37640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 -- 172.21.15.100:0/2679318264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d88f0 msgr2=0x55dc150dcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d88f0 0x55dc150dcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 -- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 msgr2=0x55dc150dd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150dd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1436640 1 -- 172.21.15.100:0/2679318264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac10e830 con 0x55dc150c5a50 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.392+0000 7f0ab1c37640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc14d327e0 con 0x55dc150c5a50 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab1436640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150dc870 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8097d30 tx=0x7f0aa809e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dad7cbcdafc2c52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0a94ff9640 10 client.18729 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0a94ff9640 1 -- 172.21.15.100:0/2679318264 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8099c90 con 0x55dc150c5a50 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0a94ff9640 1 -- 172.21.15.100:0/2679318264 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0aa80b3220 con 0x55dc150c5a50 2024-03-12T05:27:34.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0a94ff9640 1 -- 172.21.15.100:0/2679318264 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa8005500 con 0x55dc150c5a50 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab626f740 10 client.18729 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55dc150316a0 con 0x55dc150c5a50 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0a94ff9640 1 -- 172.21.15.100:0/2679318264 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0aa801f060 con 0x55dc150c5a50 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab626f740 10 client.18729 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab626f740 10 client.18729 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.393+0000 7f0ab626f740 20 client.18729 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.394+0000 7f0ab626f740 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc150ed050 0x55dc150ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.394+0000 7f0ab626f740 4 client.18729 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.394+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc14d6f720 con 0x55dc150ed050 2024-03-12T05:27:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.394+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc150ed050 0x55dc150ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.395+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc150ed050 0x55dc150ef470 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b129ff75b42df3 server_cookie=2d84fbe8355cd23 in_seq=0 out_seq=0 2024-03-12T05:27:34.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.395+0000 7f0a94ff9640 10 client.18729 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0a94ff9640 1 -- 172.21.15.100:0/2679318264 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc14d6f720 con 0x55dc150ed050 2024-03-12T05:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0a94ff9640 10 client.18729 handle_command_reply: tid=0 2024-03-12T05:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 client.18729 shutdown 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 msgr2=0x55dc150dc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150dc870 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8097d30 tx=0x7f0aa809e9f0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 shutdown_connections 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc150ed050 0x55dc150ef470 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d88f0 0x55dc150dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 --2- 172.21.15.100:0/2679318264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150c5a50 0x55dc150dc870 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 >> 172.21.15.100:0/2679318264 conn(0x55dc150c7ef0 msgr2=0x55dc150c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.399+0000 7f0a94ff9640 0 client.18729 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:34.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.400+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 shutdown_connections 2024-03-12T05:27:34.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.400+0000 7f0ab626f740 1 -- 172.21.15.100:0/2679318264 wait complete. 2024-03-12T05:27:34.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.400+0000 7f0ab626f740 20 client.18729 trim_cache size 0 max 16384 2024-03-12T05:27:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 Processor -- start 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 -- start start 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 0x55dc150defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc14fb8130 con 0x55dc150d88f0 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc150c12b0 con 0x55dc150c5a50 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.420+0000 7f0ab626f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc150c6b60 con 0x55dc150d92a0 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 0x55dc150defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1c37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 0x55dc150defa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:39462/0 (socket says 172.21.15.100:39462) 2024-03-12T05:27:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1c37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150e3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50022/0 (socket says 172.21.15.100:50022) 2024-03-12T05:27:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 -- 172.21.15.100:0/1286611492 learned_addr learned my addr 172.21.15.100:0/1286611492 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 -- 172.21.15.100:0/1286611492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 msgr2=0x55dc150e3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150e3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 -- 172.21.15.100:0/1286611492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 msgr2=0x55dc150dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150dd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 -- 172.21.15.100:0/1286611492 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc14fcdc20 con 0x55dc150c5a50 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.421+0000 7f0ab1436640 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 0x55dc150defa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800c6a0 tx=0x7f0aa809d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c112bd246881a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.422+0000 7f0a96ffd640 1 -- 172.21.15.100:0/1286611492 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8097b70 con 0x55dc150c5a50 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.422+0000 7f0a96ffd640 1 -- 172.21.15.100:0/1286611492 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dc14d327e0 con 0x55dc150c5a50 2024-03-12T05:27:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.422+0000 7f0a96ffd640 1 -- 172.21.15.100:0/1286611492 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80a2d50 con 0x55dc150c5a50 2024-03-12T05:27:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 -- 172.21.15.100:0/1286611492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 msgr2=0x55dc150defa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 0x55dc150defa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800c6a0 tx=0x7f0aa809d000 comp rx=0 tx=0).stop 2024-03-12T05:27:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 -- 172.21.15.100:0/1286611492 shutdown_connections 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150d92a0 0x55dc150e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1286611492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150c5a50 0x55dc150defa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 -- 172.21.15.100:0/1286611492 >> 172.21.15.100:0/1286611492 conn(0x55dc150c7ef0 msgr2=0x55dc150db160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 -- 172.21.15.100:0/1286611492 shutdown_connections 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.423+0000 7f0ab626f740 1 -- 172.21.15.100:0/1286611492 wait complete. 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 -- start start 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 Processor -- start 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150c5a50 0x55dc150e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc15143640 con 0x55dc150d88f0 2024-03-12T05:27:34.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.424+0000 7f0ab626f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc151437b0 con 0x55dc150d92a0 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab626f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc15143970 con 0x55dc150c5a50 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab626f740 10 client.0 _ll_register_callbacks cb 0x7f0aa8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150c5a50 0x55dc150e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab0c35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab0c35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53678/0 (socket says 172.21.15.100:53678) 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab0c35640 1 -- 172.21.15.100:0/1540075509 learned_addr learned my addr 172.21.15.100:0/1540075509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1c37640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 -- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 msgr2=0x55dc150e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 -- 172.21.15.100:0/1540075509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 msgr2=0x55dc150e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 -- 172.21.15.100:0/1540075509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc150316a0 con 0x55dc150c5a50 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0ab1436640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150c5a50 0x55dc150e29e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8004b30 tx=0x7f0aa809afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f038ff549b29eee server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0a94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.425+0000 7f0a94ff9640 1 -- 172.21.15.100:0/1540075509 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa809b6a0 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0a94ff9640 1 -- 172.21.15.100:0/1540075509 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0aa80b3220 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a9c004820 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0a94ff9640 1 -- 172.21.15.100:0/1540075509 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa8097b70 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 10 client.28036 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc14ff38c0 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0a94ff9640 1 -- 172.21.15.100:0/1540075509 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c004820 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0a94ff9640 1 -- 172.21.15.100:0/1540075509 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0aa808c070 con 0x55dc150c5a50 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 10 client.28036 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 10 client.28036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 20 client.28036 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc15144c40 0x55dc15147030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 4 client.28036 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:27:34.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc14dbe890 con 0x55dc15144c40 2024-03-12T05:27:34.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.426+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc15144c40 0x55dc15147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.427+0000 7f0a94ff9640 10 client.28036 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:34.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.427+0000 7f0ab0c35640 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc15144c40 0x55dc15147030 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4a9701a6d5bc442 server_cookie=229f3dce7ea93001 in_seq=0 out_seq=0 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0a94ff9640 1 -- 172.21.15.100:0/1540075509 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc14dbe890 con 0x55dc15144c40 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0a94ff9640 10 client.28036 handle_command_reply: tid=0 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 client.28036 shutdown 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150c5a50 msgr2=0x55dc150e29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150c5a50 0x55dc150e29e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8004b30 tx=0x7f0aa809afd0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 shutdown_connections 2024-03-12T05:27:34.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55dc15144c40 0x55dc15147030 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc150d92a0 0x55dc150e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc150d88f0 0x55dc150e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 --2- 172.21.15.100:0/1540075509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc150c5a50 0x55dc150e29e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 >> 172.21.15.100:0/1540075509 conn(0x55dc150c7ef0 msgr2=0x55dc150d7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0a94ff9640 0 client.28036 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.431+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 shutdown_connections 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab626f740 1 -- 172.21.15.100:0/1540075509 wait complete. 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab626f740 20 client.28036 trim_cache size 0 max 16384 2024-03-12T05:27:34.427 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:27:34.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a980a1c20 msgr2=0x7f0a980a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a980a1c20 0x7f0a980a40e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005fd0 tx=0x7f0aa4009330 comp rx=0 tx=0).stop 2024-03-12T05:27:34.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 msgr2=0x7f0aac19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 0x7f0aac19d5b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800e9f0 tx=0x7f0aa800edf0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 shutdown_connections 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0a980a1c20 0x7f0a980a40e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aac197670 0x7f0aac197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aac106450 0x7f0aac19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 --2- 172.21.15.100:0/2415285844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aac0ff890 0x7f0aac19d5b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 >> 172.21.15.100:0/2415285844 conn(0x7f0aac0765a0 msgr2=0x7f0aac1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 shutdown_connections 2024-03-12T05:27:34.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.432+0000 7f0ab369a640 1 -- 172.21.15.100:0/2415285844 wait complete. 2024-03-12T05:27:34.443 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:27:34.443 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:27:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 -- 172.21.15.100:0/1851101394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c068fa0 msgr2=0x7f174c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 --2- 172.21.15.100:0/1851101394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c068fa0 0x7f174c105de0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f173c009930 tx=0x7f173c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 -- 172.21.15.100:0/1851101394 shutdown_connections 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 --2- 172.21.15.100:0/1851101394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f174c1064c0 0x7f174c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 --2- 172.21.15.100:0/1851101394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c068fa0 0x7f174c105de0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 --2- 172.21.15.100:0/1851101394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f174c0685f0 0x7f174c0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 -- 172.21.15.100:0/1851101394 >> 172.21.15.100:0/1851101394 conn(0x7f174c077180 msgr2=0x7f174c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 -- 172.21.15.100:0/1851101394 shutdown_connections 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.583+0000 7f1753b27640 1 -- 172.21.15.100:0/1851101394 wait complete. 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 Processor -- start 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 -- start start 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f174c0685f0 0x7f174c100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 0x7f174c100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f174c100ac0 0x7f174c1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a5fa0 con 0x7f174c1064c0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a6110 con 0x7f174c0685f0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f1753b27640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a62d0 con 0x7f174c100ac0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 0x7f174c100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 0x7f174c100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53688/0 (socket says 172.21.15.100:53688) 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 -- 172.21.15.100:0/1921923378 learned_addr learned my addr 172.21.15.100:0/1921923378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17518c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f174c0685f0 0x7f174c100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17520c4640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f174c100ac0 0x7f174c1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 -- 172.21.15.100:0/1921923378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f174c100ac0 msgr2=0x7f174c1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f174c100ac0 0x7f174c1a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 -- 172.21.15.100:0/1921923378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f174c0685f0 msgr2=0x7f174c100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f174c0685f0 0x7f174c100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.584+0000 7f17510c2640 1 -- 172.21.15.100:0/1921923378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c009560 con 0x7f174c1064c0 2024-03-12T05:27:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.585+0000 7f17510c2640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 0x7f174c100580 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f173c02c170 tx=0x7f173c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.585+0000 7f1742ffd640 1 -- 172.21.15.100:0/1921923378 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c004250 con 0x7f174c1064c0 2024-03-12T05:27:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.585+0000 7f1742ffd640 1 -- 172.21.15.100:0/1921923378 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f173c0043b0 con 0x7f174c1064c0 2024-03-12T05:27:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.585+0000 7f1742ffd640 1 -- 172.21.15.100:0/1921923378 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c003b50 con 0x7f174c1064c0 2024-03-12T05:27:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.585+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c1a6550 con 0x7f174c1064c0 2024-03-12T05:27:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.585+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1a6ab0 con 0x7f174c1064c0 2024-03-12T05:27:34.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.587+0000 7f1742ffd640 1 -- 172.21.15.100:0/1921923378 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f173c003cb0 con 0x7f174c1064c0 2024-03-12T05:27:34.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f17567d0740 1 Processor -- start 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f17567d0740 1 -- start start 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f1742ffd640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17380a1c20 0x7f17380a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f17567d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 0x560c458a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f17567d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1b70 0x560c458aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f17567d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.588+0000 7f17567d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4585c960 con 0x560c458a1b70 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17567d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c45810310 con 0x560c458aa750 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17567d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4580d840 con 0x560c458a1250 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17510c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1b70 0x560c458aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 0x560c458a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f1742ffd640 1 -- 172.21.15.100:0/1921923378 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f173c0efd00 con 0x7f174c1064c0 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 0x560c458a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50054/0 (socket says 172.21.15.100:50054) 2024-03-12T05:27:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 -- 172.21.15.100:0/2883415943 learned_addr learned my addr 172.21.15.100:0/2883415943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17520c4640 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17520c4640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17380a1c20 0x7f17380a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 -- 172.21.15.100:0/2883415943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 msgr2=0x560c458acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 -- 172.21.15.100:0/2883415943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1b70 msgr2=0x560c458aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1b70 0x560c458aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17518c3640 1 -- 172.21.15.100:0/2883415943 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c457cb060 con 0x560c458a1250 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17510c2640 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1b70 0x560c458aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.589+0000 7f17520c4640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17380a1c20 0x7f17380a40e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x560c4589cb10 tx=0x7f17440096a0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.590+0000 7f17518c3640 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 0x560c458a1630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1748009ca0 tx=0x7f1748011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a01f429248c7f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.590+0000 7f1737fff640 1 -- 172.21.15.100:0/2883415943 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748019430 con 0x560c458a1250 2024-03-12T05:27:34.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.590+0000 7f1737fff640 1 -- 172.21.15.100:0/2883415943 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1748019a70 con 0x560c458a1250 2024-03-12T05:27:34.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.590+0000 7f1737fff640 1 -- 172.21.15.100:0/2883415943 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800eb20 con 0x560c458a1250 2024-03-12T05:27:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 -- 172.21.15.100:0/2883415943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 msgr2=0x560c458a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 0x560c458a1630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1748009ca0 tx=0x7f1748011810 comp rx=0 tx=0).stop 2024-03-12T05:27:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 -- 172.21.15.100:0/2883415943 shutdown_connections 2024-03-12T05:27:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1b70 0x560c458aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 --2- 172.21.15.100:0/2883415943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1250 0x560c458a1630 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 -- 172.21.15.100:0/2883415943 >> 172.21.15.100:0/2883415943 conn(0x560c4589b980 msgr2=0x560c4589bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 -- 172.21.15.100:0/2883415943 shutdown_connections 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 -- 172.21.15.100:0/2883415943 wait complete. 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 -- start start 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.591+0000 7f17567d0740 1 Processor -- start 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c45809020 con 0x560c458aa750 2024-03-12T05:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c454dc430 con 0x560c458a1250 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c45498e00 con 0x560c458a1b70 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17567d0740 10 client.0 _ll_register_callbacks cb 0x560c457b0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17518c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17518c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458a77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:39482/0 (socket says 172.21.15.100:39482) 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17518c3640 1 -- 172.21.15.100:0/2837152357 learned_addr learned my addr 172.21.15.100:0/2837152357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17510c2640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.592+0000 7f17520c4640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17518c3640 1 -- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 msgr2=0x560c458a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17518c3640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17518c3640 1 -- 172.21.15.100:0/2837152357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 msgr2=0x560c458a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17518c3640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17518c3640 1 -- 172.21.15.100:0/2837152357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748009950 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17518c3640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458a77c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1748009fc0 tx=0x7f1748005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d062a495b6bf79f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17357fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17357fa640 1 -- 172.21.15.100:0/2837152357 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480198c0 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17357fa640 1 -- 172.21.15.100:0/2837152357 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1748004030 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c455087e0 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.593+0000 7f17357fa640 1 -- 172.21.15.100:0/2837152357 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17480041f0 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 10 client.28109 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c458076a0 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17357fa640 1 -- 172.21.15.100:0/2837152357 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560c455087e0 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17357fa640 1 -- 172.21.15.100:0/2837152357 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f174800ece0 con 0x560c458a1250 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 10 client.28109 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 10 client.28109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:27:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 20 client.28109 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c458baba0 0x560c458bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 4 client.28109 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.594+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c45545720 con 0x560c458baba0 2024-03-12T05:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.595+0000 7f17510c2640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c458baba0 0x560c458bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.595+0000 7f17357fa640 10 client.28109 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.595+0000 7f17510c2640 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c458baba0 0x560c458bcf90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e11353b8f37155d6 server_cookie=bb8672fa81b63974 in_seq=0 out_seq=0 2024-03-12T05:27:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17357fa640 1 -- 172.21.15.100:0/2837152357 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c45545720 con 0x560c458baba0 2024-03-12T05:27:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17357fa640 10 client.28109 handle_command_reply: tid=0 2024-03-12T05:27:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 client.28109 shutdown 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 msgr2=0x560c458a77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458a77c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1748009fc0 tx=0x7f1748005d30 comp rx=0 tx=0).stop 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 shutdown_connections 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c458baba0 0x560c458bcf90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 --2- 172.21.15.100:0/2837152357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458a77c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 >> 172.21.15.100:0/2837152357 conn(0x560c4589b980 msgr2=0x560c4589d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17357fa640 0 client.28109 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 shutdown_connections 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.597+0000 7f17567d0740 1 -- 172.21.15.100:0/2837152357 wait complete. 2024-03-12T05:27:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.598+0000 7f17567d0740 20 client.28109 trim_cache size 0 max 16384 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 Processor -- start 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 -- start start 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 0x560c458a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458a3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4578e130 con 0x560c458a1250 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4589c9b0 con 0x560c458aa750 2024-03-12T05:27:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.613+0000 7f17567d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c458ba950 con 0x560c458a1b70 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 0x560c458a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 0x560c458a9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53706/0 (socket says 172.21.15.100:53706) 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 -- 172.21.15.100:0/678974831 learned_addr learned my addr 172.21.15.100:0/678974831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17520c4640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17510c2640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 -- 172.21.15.100:0/678974831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 msgr2=0x560c458a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 -- 172.21.15.100:0/678974831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 msgr2=0x560c458a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458a3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 -- 172.21.15.100:0/678974831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c457a3c20 con 0x560c458a1250 2024-03-12T05:27:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17520c4640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17510c2640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.614+0000 7f17518c3640 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 0x560c458a9ef0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1748001eb0 tx=0x7f1748009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7be67fe2c5e8ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.615+0000 7f17377fe640 1 -- 172.21.15.100:0/678974831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480196c0 con 0x560c458a1250 2024-03-12T05:27:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.615+0000 7f17377fe640 1 -- 172.21.15.100:0/678974831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1748004500 con 0x560c458a1250 2024-03-12T05:27:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.615+0000 7f17377fe640 1 -- 172.21.15.100:0/678974831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800ed30 con 0x560c458a1250 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.615+0000 7f17567d0740 1 -- 172.21.15.100:0/678974831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 msgr2=0x560c458a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.615+0000 7f17567d0740 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 0x560c458a9ef0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1748001eb0 tx=0x7f1748009b70 comp rx=0 tx=0).stop 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.615+0000 7f17567d0740 1 -- 172.21.15.100:0/678974831 shutdown_connections 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458aa750 0x560c458a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 --2- 172.21.15.100:0/678974831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458a1250 0x560c458a9ef0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- 172.21.15.100:0/678974831 >> 172.21.15.100:0/678974831 conn(0x560c4589b980 msgr2=0x560c458c1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- 172.21.15.100:0/678974831 shutdown_connections 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- 172.21.15.100:0/678974831 wait complete. 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- start start 2024-03-12T05:27:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 Processor -- start 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c458b0640 con 0x560c458aa750 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c458b07b0 con 0x560c458a1250 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c458b0970 con 0x560c458a1b70 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17567d0740 10 client.0 _ll_register_callbacks cb 0x7f1748009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.616+0000 7f17520c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17510c2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17518c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53708/0 (socket says 172.21.15.100:53708) 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 -- 172.21.15.100:0/299843132 learned_addr learned my addr 172.21.15.100:0/299843132 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 -- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 msgr2=0x560c458afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 -- 172.21.15.100:0/299843132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 msgr2=0x560c458b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 -- 172.21.15.100:0/299843132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1a6550 con 0x560c458aa750 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17520c4640 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458b0100 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1744013000 tx=0x7f1744004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a442b4999db5768d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17357fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17357fa640 1 -- 172.21.15.100:0/299843132 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400be90 con 0x560c458aa750 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17357fa640 1 -- 172.21.15.100:0/299843132 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1744004040 con 0x560c458aa750 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1748019a70 con 0x560c458aa750 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17357fa640 1 -- 172.21.15.100:0/299843132 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1744004240 con 0x560c458aa750 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17567d0740 10 client.18753 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.617+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f174401a040 con 0x560c458aa750 2024-03-12T05:27:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17357fa640 1 -- 172.21.15.100:0/299843132 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1748019a70 con 0x560c458aa750 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17357fa640 1 -- 172.21.15.100:0/299843132 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1744029070 con 0x560c458aa750 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17567d0740 10 client.18753 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17567d0740 10 client.18753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17567d0740 20 client.18753 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17567d0740 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c459125f0 0x560c45914a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17567d0740 4 client.18753 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560c45594890 con 0x560c459125f0 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17518c3640 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c459125f0 0x560c45914a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17357fa640 10 client.18753 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.618+0000 7f17518c3640 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c459125f0 0x560c45914a10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b67fbf281fd3f8b server_cookie=bbd9e567cd6355f1 in_seq=0 out_seq=0 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17357fa640 1 -- 172.21.15.100:0/299843132 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560c45594890 con 0x560c459125f0 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17357fa640 10 client.18753 handle_command_reply: tid=0 2024-03-12T05:27:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 client.18753 shutdown 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 msgr2=0x560c458b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458b0100 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1744013000 tx=0x7f1744004ab0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 shutdown_connections 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c458aa750 0x560c458b0100 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560c458a1b70 0x560c458afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x560c459125f0 0x560c45914a10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17357fa640 0 client.18753 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 --2- 172.21.15.100:0/299843132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c458a1250 0x560c458b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 >> 172.21.15.100:0/299843132 conn(0x560c4589b980 msgr2=0x560c458a2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 shutdown_connections 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 1 -- 172.21.15.100:0/299843132 wait complete. 2024-03-12T05:27:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.620+0000 7f17567d0740 20 client.18753 trim_cache size 0 max 16384 2024-03-12T05:27:34.621 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:27:34.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17380a1c20 msgr2=0x7f17380a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17380a1c20 0x7f17380a40e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x560c4589cb10 tx=0x7f17440096a0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 msgr2=0x7f174c100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 0x7f174c100580 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f173c02c170 tx=0x7f173c004970 comp rx=0 tx=0).stop 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 shutdown_connections 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f17380a1c20 0x7f17380a40e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f174c100ac0 0x7f174c1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174c1064c0 0x7f174c100580 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 --2- 172.21.15.100:0/1921923378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f174c0685f0 0x7f174c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 >> 172.21.15.100:0/1921923378 conn(0x7f174c077180 msgr2=0x7f174c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 shutdown_connections 2024-03-12T05:27:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.621+0000 7f1753b27640 1 -- 172.21.15.100:0/1921923378 wait complete. 2024-03-12T05:27:34.631 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:27:34.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:27:34.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 -- 172.21.15.100:0/1929822410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b80ff890 msgr2=0x7f70b80ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 --2- 172.21.15.100:0/1929822410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b80ff890 0x7f70b80ffc70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009930 tx=0x7f70a802bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 -- 172.21.15.100:0/1929822410 shutdown_connections 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 --2- 172.21.15.100:0/1929822410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b8106450 0x7f70b810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 --2- 172.21.15.100:0/1929822410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8100240 0x7f70b8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 --2- 172.21.15.100:0/1929822410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b80ff890 0x7f70b80ffc70 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 -- 172.21.15.100:0/1929822410 >> 172.21.15.100:0/1929822410 conn(0x7f70b80765a0 msgr2=0x7f70b80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 -- 172.21.15.100:0/1929822410 shutdown_connections 2024-03-12T05:27:34.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.769+0000 7f70be1b5640 1 -- 172.21.15.100:0/1929822410 wait complete. 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 Processor -- start 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 -- start start 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b80ff890 0x7f70b819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 0x7f70b819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8197610 0x7f70b8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8198000 con 0x7f70b8106450 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8198170 con 0x7f70b80ff890 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70be1b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8198330 con 0x7f70b8197610 2024-03-12T05:27:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 0x7f70b819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b80ff890 0x7f70b819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.770+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8197610 0x7f70b8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 0x7f70b819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53732/0 (socket says 172.21.15.100:53732) 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b80ff890 0x7f70b819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:39536/0 (socket says 172.21.15.100:39536) 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b6ffd640 1 -- 172.21.15.100:0/711230034 learned_addr learned my addr 172.21.15.100:0/711230034 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b7fff640 1 -- 172.21.15.100:0/711230034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b80ff890 msgr2=0x7f70b819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b7fff640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b80ff890 0x7f70b819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b7fff640 1 -- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 msgr2=0x7f70b819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b7fff640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 0x7f70b819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b7fff640 1 -- 172.21.15.100:0/711230034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009560 con 0x7f70b8197610 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b6ffd640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 0x7f70b819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b7fff640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8197610 0x7f70b8197ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009fd0 tx=0x7f70b000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b4ff9640 1 -- 172.21.15.100:0/711230034 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000ce20 con 0x7f70b8197610 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b4ff9640 1 -- 172.21.15.100:0/711230034 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70b0005020 con 0x7f70b8197610 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b8100730 con 0x7f70b8197610 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70b4ff9640 1 -- 172.21.15.100:0/711230034 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0010690 con 0x7f70b8197610 2024-03-12T05:27:34.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.771+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b8100bf0 con 0x7f70b8197610 2024-03-12T05:27:34.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.774+0000 7f70b4ff9640 1 -- 172.21.15.100:0/711230034 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f70b0005500 con 0x7f70b8197610 2024-03-12T05:27:34.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.774+0000 7f70b4ff9640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70a00a1ba0 0x7f70a00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.774+0000 7f70c0d88740 1 Processor -- start 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.774+0000 7f70b77fe640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70a00a1ba0 0x7f70a00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.774+0000 7f70c0d88740 1 -- start start 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.774+0000 7f70b4ff9640 1 -- 172.21.15.100:0/711230034 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70b0014070 con 0x7f70b8197610 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d253d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d2546210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2548b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70c0d88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d24f8960 con 0x55a0d2546750 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70c0d88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d24ac310 con 0x55a0d253db70 2024-03-12T05:27:34.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70c0d88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d24a9840 con 0x55a0d253d250 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2548b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2548b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53736/0 (socket says 172.21.15.100:53736) 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b77fe640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70a00a1ba0 0x7f70a00a4060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000c00 tx=0x7f70a8005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 -- 172.21.15.100:0/3813597532 learned_addr learned my addr 172.21.15.100:0/3813597532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b6ffd640 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d2546210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b77fe640 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d253d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 -- 172.21.15.100:0/3813597532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 msgr2=0x55a0d253d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d253d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 -- 172.21.15.100:0/3813597532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 msgr2=0x55a0d2546210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d2546210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 -- 172.21.15.100:0/3813597532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0d2467060 con 0x55a0d2546750 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.775+0000 7f70b7fff640 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2548b40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f70b00cb690 tx=0x7f70b009a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf1d1425bfd4d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f709dffb640 1 -- 172.21.15.100:0/3813597532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b2070 con 0x55a0d2546750 2024-03-12T05:27:34.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f709dffb640 1 -- 172.21.15.100:0/3813597532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70b009ec60 con 0x55a0d2546750 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f709dffb640 1 -- 172.21.15.100:0/3813597532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b008e760 con 0x55a0d2546750 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 -- 172.21.15.100:0/3813597532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 msgr2=0x55a0d2548b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2548b40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f70b00cb690 tx=0x7f70b009a780 comp rx=0 tx=0).stop 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 -- 172.21.15.100:0/3813597532 shutdown_connections 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2548b40 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d2546210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 --2- 172.21.15.100:0/3813597532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d253d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.776+0000 7f70c0d88740 1 -- 172.21.15.100:0/3813597532 >> 172.21.15.100:0/3813597532 conn(0x55a0d2537980 msgr2=0x55a0d2537d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 -- 172.21.15.100:0/3813597532 shutdown_connections 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 -- 172.21.15.100:0/3813597532 wait complete. 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 -- start start 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 Processor -- start 2024-03-12T05:27:34.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d25437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 0x55a0d2543d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d2546750 0x55a0d2544260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d24a5020 con 0x55a0d253db70 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d2178430 con 0x55a0d2546750 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d2134e00 con 0x55a0d253d250 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70c0d88740 10 client.0 _ll_register_callbacks cb 0x55a0d244ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:34.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 0x55a0d2543d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.777+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 0x55a0d2543d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53746/0 (socket says 172.21.15.100:53746) 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d2546750 0x55a0d2544260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d25437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b6ffd640 1 -- 172.21.15.100:0/995076731 learned_addr learned my addr 172.21.15.100:0/995076731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 -- 172.21.15.100:0/995076731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 msgr2=0x55a0d25437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d25437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 -- 172.21.15.100:0/995076731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 msgr2=0x55a0d2543d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 0x55a0d2543d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 -- 172.21.15.100:0/995076731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b8100bf0 con 0x55a0d2546750 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b6ffd640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 0x55a0d2543d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b77fe640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d25437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70b7fff640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d2546750 0x55a0d2544260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a0d2538b10 tx=0x7f70b00cb620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd41a70c8d829f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f7096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:27:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f7096ffd640 1 -- 172.21.15.100:0/995076731 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b2070 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a0d21a47e0 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f7096ffd640 1 -- 172.21.15.100:0/995076731 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70b009be40 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f7096ffd640 1 -- 172.21.15.100:0/995076731 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70b008e760 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70c0d88740 10 client.28127 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.778+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a0d24a36a0 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f7096ffd640 1 -- 172.21.15.100:0/995076731 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a0d21a47e0 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f7096ffd640 1 -- 172.21.15.100:0/995076731 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f70b001f070 con 0x55a0d2546750 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70c0d88740 10 client.28127 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70c0d88740 10 client.28127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70c0d88740 20 client.28127 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70c0d88740 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d2556880 0x55a0d2558ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70c0d88740 4 client.28127 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0d21e1720 con 0x55a0d2556880 2024-03-12T05:27:34.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.779+0000 7f70b77fe640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d2556880 0x55a0d2558ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f70b77fe640 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d2556880 0x55a0d2558ca0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d5621e14e3cf1e6 server_cookie=2bf21dff99571abb in_seq=0 out_seq=0 2024-03-12T05:27:34.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f7096ffd640 10 client.28127 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:34.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f7096ffd640 1 -- 172.21.15.100:0/995076731 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0d21e1720 con 0x55a0d2556880 2024-03-12T05:27:34.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f7096ffd640 10 client.28127 handle_command_reply: tid=0 2024-03-12T05:27:34.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f70c0d88740 1 client.28127 shutdown 2024-03-12T05:27:34.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d2546750 msgr2=0x55a0d2544260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.780+0000 7f70c0d88740 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d2546750 0x55a0d2544260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a0d2538b10 tx=0x7f70b00cb620 comp rx=0 tx=0).stop 2024-03-12T05:27:34.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 shutdown_connections 2024-03-12T05:27:34.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d2556880 0x55a0d2558ca0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d2546750 0x55a0d2544260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253db70 0x55a0d2543d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 --2- 172.21.15.100:0/995076731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253d250 0x55a0d25437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 >> 172.21.15.100:0/995076731 conn(0x55a0d2537980 msgr2=0x55a0d25396d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f7096ffd640 0 client.28127 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 shutdown_connections 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 1 -- 172.21.15.100:0/995076731 wait complete. 2024-03-12T05:27:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.781+0000 7f70c0d88740 20 client.28127 trim_cache size 0 max 16384 2024-03-12T05:27:34.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 Processor -- start 2024-03-12T05:27:34.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 -- start start 2024-03-12T05:27:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253d250 0x55a0d2545f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253db70 0x55a0d2544080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2540a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d242a130 con 0x55a0d2546750 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d25335c0 con 0x55a0d253d250 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.796+0000 7f70c0d88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d2541d50 con 0x55a0d253db70 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2540a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253db70 0x55a0d2544080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253db70 0x55a0d2544080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50122/0 (socket says 172.21.15.100:50122) 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2540a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53752/0 (socket says 172.21.15.100:53752) 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b6ffd640 1 -- 172.21.15.100:0/952224944 learned_addr learned my addr 172.21.15.100:0/952224944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b77fe640 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253d250 0x55a0d2545f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 -- 172.21.15.100:0/952224944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253db70 msgr2=0x55a0d2544080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253db70 0x55a0d2544080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 -- 172.21.15.100:0/952224944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253d250 msgr2=0x55a0d2545f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253d250 0x55a0d2545f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 -- 172.21.15.100:0/952224944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0d243fc20 con 0x55a0d2546750 2024-03-12T05:27:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f70b7fff640 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2540a80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f70b0096f70 tx=0x7f70b009a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c1ebe78fd2b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f709d7fa640 1 -- 172.21.15.100:0/952224944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b2070 con 0x55a0d2546750 2024-03-12T05:27:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f709d7fa640 1 -- 172.21.15.100:0/952224944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70b00a2e20 con 0x55a0d2546750 2024-03-12T05:27:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.797+0000 7f709d7fa640 1 -- 172.21.15.100:0/952224944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00aee50 con 0x55a0d2546750 2024-03-12T05:27:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.798+0000 7f70c0d88740 1 -- 172.21.15.100:0/952224944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 msgr2=0x55a0d2540a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.798+0000 7f70c0d88740 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2540a80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f70b0096f70 tx=0x7f70b009a780 comp rx=0 tx=0).stop 2024-03-12T05:27:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 -- 172.21.15.100:0/952224944 shutdown_connections 2024-03-12T05:27:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d2546750 0x55a0d2540a80 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d253db70 0x55a0d2544080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 --2- 172.21.15.100:0/952224944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253d250 0x55a0d2545f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 -- 172.21.15.100:0/952224944 >> 172.21.15.100:0/952224944 conn(0x55a0d2537980 msgr2=0x55a0d25420d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 -- 172.21.15.100:0/952224944 shutdown_connections 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 -- 172.21.15.100:0/952224944 wait complete. 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 -- start start 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.799+0000 7f70c0d88740 1 Processor -- start 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 0x55a0d254c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d254c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d2546750 0x55a0d254cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d25acd90 con 0x55a0d253d250 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d25aced0 con 0x55a0d253db70 2024-03-12T05:27:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d25ad010 con 0x55a0d2546750 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70c0d88740 10 client.0 _ll_register_callbacks cb 0x7f70b0096db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 0x55a0d254c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 0x55a0d254c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53760/0 (socket says 172.21.15.100:53760) 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 -- 172.21.15.100:0/2222200969 learned_addr learned my addr 172.21.15.100:0/2222200969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b7fff640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d2546750 0x55a0d254cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b6ffd640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d254c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 -- 172.21.15.100:0/2222200969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d2546750 msgr2=0x55a0d254cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d2546750 0x55a0d254cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 -- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 msgr2=0x55a0d254c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d254c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 -- 172.21.15.100:0/2222200969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0d24a36a0 con 0x55a0d253d250 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f70b77fe640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 0x55a0d254c090 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f70a802c780 tx=0x7f70a802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf6760cd3af28a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.800+0000 7f7096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.801+0000 7f7096ffd640 1 -- 172.21.15.100:0/2222200969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8053050 con 0x55a0d253d250 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.801+0000 7f7096ffd640 1 -- 172.21.15.100:0/2222200969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70a8047040 con 0x55a0d253d250 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.801+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70b009ec60 con 0x55a0d253d250 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.801+0000 7f7096ffd640 1 -- 172.21.15.100:0/2222200969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a802cbe0 con 0x55a0d253d250 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.801+0000 7f70c0d88740 10 client.18771 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.801+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a0d24658c0 con 0x55a0d253d250 2024-03-12T05:27:34.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f7096ffd640 1 -- 172.21.15.100:0/2222200969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70b009ec60 con 0x55a0d253d250 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f7096ffd640 1 -- 172.21.15.100:0/2222200969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f70a8064070 con 0x55a0d253d250 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70c0d88740 10 client.18771 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70c0d88740 10 client.18771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70c0d88740 20 client.18771 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70c0d88740 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d25ae860 0x55a0d25b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70c0d88740 4 client.18771 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a0d2230890 con 0x55a0d25ae860 2024-03-12T05:27:34.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.802+0000 7f70b6ffd640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d25ae860 0x55a0d25b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:34.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f7096ffd640 10 client.18771 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:34.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70b6ffd640 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d25ae860 0x55a0d25b0c80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708889efe60f6b0d server_cookie=aff3ea66ad76f6cd in_seq=0 out_seq=0 2024-03-12T05:27:34.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f7096ffd640 1 -- 172.21.15.100:0/2222200969 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a0d2230890 con 0x55a0d25ae860 2024-03-12T05:27:34.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f7096ffd640 10 client.18771 handle_command_reply: tid=0 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 client.18771 shutdown 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 msgr2=0x55a0d254c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 0x55a0d254c090 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f70a802c780 tx=0x7f70a802bc30 comp rx=0 tx=0).stop 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 shutdown_connections 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55a0d25ae860 0x55a0d25b0c80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a0d2546750 0x55a0d254cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f7096ffd640 0 client.18771 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a0d253db70 0x55a0d254c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 --2- 172.21.15.100:0/2222200969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0d253d250 0x55a0d254c090 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 >> 172.21.15.100:0/2222200969 conn(0x55a0d2537980 msgr2=0x55a0d25362f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 shutdown_connections 2024-03-12T05:27:34.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 1 -- 172.21.15.100:0/2222200969 wait complete. 2024-03-12T05:27:34.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.803+0000 7f70c0d88740 20 client.18771 trim_cache size 0 max 16384 2024-03-12T05:27:34.803 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:27:34.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70a00a1ba0 msgr2=0x7f70a00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70a00a1ba0 0x7f70a00a4060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000c00 tx=0x7f70a8005fd0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8197610 msgr2=0x7f70b8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:34.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8197610 0x7f70b8197ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009fd0 tx=0x7f70b000ef10 comp rx=0 tx=0).stop 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 shutdown_connections 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f70a00a1ba0 0x7f70a00a4060 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b8197610 0x7f70b8197ac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70b8106450 0x7f70b819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 --2- 172.21.15.100:0/711230034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b80ff890 0x7f70b819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 >> 172.21.15.100:0/711230034 conn(0x7f70b80765a0 msgr2=0x7f70b81046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 shutdown_connections 2024-03-12T05:27:34.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:34.804+0000 7f70be1b5640 1 -- 172.21.15.100:0/711230034 wait complete. 2024-03-12T05:27:35.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84183 30.14 MB/sec execute 18 sec latency 87.967 ms 2024-03-12T05:27:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:35 smithi100 ceph-mon[32210]: pgmap v485: 97 pgs: 97 active+clean; 1006 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 525 op/s 2024-03-12T05:27:35.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:35 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1674106945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:27:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:35 smithi156 ceph-mon[36270]: pgmap v485: 97 pgs: 97 active+clean; 1006 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 525 op/s 2024-03-12T05:27:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:35 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1674106945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:27:35.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:35 smithi161 ceph-mon[36043]: pgmap v485: 97 pgs: 97 active+clean; 1006 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 525 op/s 2024-03-12T05:27:35.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:35 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1674106945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:27:35.811 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:27:35.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:27:35.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:27:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.943+0000 7f23974d4640 1 -- 172.21.15.100:0/2678387239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2390100420 msgr2=0x7f2390110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- 172.21.15.100:0/2678387239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2390100420 0x7f2390110210 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2388009ca0 tx=0x7f238802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- 172.21.15.100:0/2678387239 shutdown_connections 2024-03-12T05:27:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- 172.21.15.100:0/2678387239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2390100420 0x7f2390110210 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- 172.21.15.100:0/2678387239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23900ffa80 0x7f23900ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- 172.21.15.100:0/2678387239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2390101f40 0x7f2390102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- 172.21.15.100:0/2678387239 >> 172.21.15.100:0/2678387239 conn(0x7f23900752e0 msgr2=0x7f23900756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- 172.21.15.100:0/2678387239 shutdown_connections 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- 172.21.15.100:0/2678387239 wait complete. 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 Processor -- start 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- start start 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 0x7f239019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2390100420 0x7f239019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2390101f40 0x7f2390199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2390199ee0 con 0x7f23900ffa80 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239019a050 con 0x7f2390101f40 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.944+0000 7f23974d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239019a210 con 0x7f2390100420 2024-03-12T05:27:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 0x7f239019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 0x7f239019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53784/0 (socket says 172.21.15.100:53784) 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 -- 172.21.15.100:0/1534087271 learned_addr learned my addr 172.21.15.100:0/1534087271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2394a6f640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2390100420 0x7f239019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395a71640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2390101f40 0x7f2390199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 -- 172.21.15.100:0/1534087271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2390100420 msgr2=0x7f239019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2390100420 0x7f239019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 -- 172.21.15.100:0/1534087271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2390101f40 msgr2=0x7f2390199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2390101f40 0x7f2390199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 -- 172.21.15.100:0/1534087271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388009900 con 0x7f23900ffa80 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f2395270640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 0x7f239019f6d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f238c00e9f0 tx=0x7f238c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f23867fc640 1 -- 172.21.15.100:0/1534087271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c009700 con 0x7f23900ffa80 2024-03-12T05:27:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239019a4f0 con 0x7f23900ffa80 2024-03-12T05:27:35.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f23867fc640 1 -- 172.21.15.100:0/1534087271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f238c00cd80 con 0x7f23900ffa80 2024-03-12T05:27:35.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f23867fc640 1 -- 172.21.15.100:0/1534087271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c018920 con 0x7f23900ffa80 2024-03-12T05:27:35.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.945+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23901a6700 con 0x7f23900ffa80 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f23867fc640 1 -- 172.21.15.100:0/1534087271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f238c018a80 con 0x7f23900ffa80 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 Processor -- start 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 -- start start 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7d99cf90 con 0x55ed7da2e380 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7d99a4c0 con 0x55ed7da2ed30 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f239a17e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7d995ca0 con 0x55ed7da378b0 2024-03-12T05:27:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2394a6f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2394a6f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da39d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50168/0 (socket says 172.21.15.100:50168) 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2394a6f640 1 -- 172.21.15.100:0/2192274471 learned_addr learned my addr 172.21.15.100:0/2192274471 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2395a71640 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2395270640 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2394a6f640 1 -- 172.21.15.100:0/2192274471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 msgr2=0x55ed7da372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.949+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f2394a6f640 1 -- 172.21.15.100:0/2192274471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 msgr2=0x55ed7da2e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da2e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f2394a6f640 1 -- 172.21.15.100:0/2192274471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed7d9580b0 con 0x55ed7da378b0 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f23867fc640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f237c0a1ba0 0x7f237c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f2395a71640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f237c0a1ba0 0x7f237c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da39d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2380009c60 tx=0x7f2380011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7648e30b80efeb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:35.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f23867fc640 1 -- 172.21.15.100:0/1534087271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f238c014070 con 0x7f23900ffa80 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f237b7fe640 1 -- 172.21.15.100:0/2192274471 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380004230 con 0x55ed7da378b0 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f237b7fe640 1 -- 172.21.15.100:0/2192274471 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2380004390 con 0x55ed7da378b0 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.950+0000 7f237b7fe640 1 -- 172.21.15.100:0/2192274471 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800197a0 con 0x55ed7da378b0 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 -- 172.21.15.100:0/2192274471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 msgr2=0x55ed7da39d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f2395a71640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f237c0a1ba0 0x7f237c0a4060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2388005e30 tx=0x7f2388005d80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da39d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2380009c60 tx=0x7f2380011810 comp rx=0 tx=0).stop 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 -- 172.21.15.100:0/2192274471 shutdown_connections 2024-03-12T05:27:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da39d00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 --2- 172.21.15.100:0/2192274471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da2e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 -- 172.21.15.100:0/2192274471 >> 172.21.15.100:0/2192274471 conn(0x55ed7da28840 msgr2=0x55ed7da28c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 -- 172.21.15.100:0/2192274471 shutdown_connections 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 -- 172.21.15.100:0/2192274471 wait complete. 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 -- start start 2024-03-12T05:27:35.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.951+0000 7f239a17e740 1 Processor -- start 2024-03-12T05:27:35.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2e380 0x55ed7da347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da378b0 0x55ed7da35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7d93af40 con 0x55ed7da378b0 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7d6694b0 con 0x55ed7da2ed30 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7d91b670 con 0x55ed7da2e380 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f2394a6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f2395270640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2e380 0x55ed7da347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f2394a6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da34d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:39608/0 (socket says 172.21.15.100:39608) 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f2394a6f640 1 -- 172.21.15.100:0/3100639554 learned_addr learned my addr 172.21.15.100:0/3100639554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:35.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f2395a71640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da378b0 0x55ed7da35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.952+0000 7f239a17e740 10 client.0 _ll_register_callbacks cb 0x55ed7d988f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2394a6f640 1 -- 172.21.15.100:0/3100639554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2e380 msgr2=0x55ed7da347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2394a6f640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2e380 0x55ed7da347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2394a6f640 1 -- 172.21.15.100:0/3100639554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da378b0 msgr2=0x55ed7da35260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2394a6f640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da378b0 0x55ed7da35260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2394a6f640 1 -- 172.21.15.100:0/3100639554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2380009890 con 0x55ed7da2ed30 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2395270640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2e380 0x55ed7da347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed7d96ff40 con 0x55ed7da2ed30 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2394a6f640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da34d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2380005890 tx=0x7f23800052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26faea37aa98e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2378ff9640 10 client.28145 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2378ff9640 1 -- 172.21.15.100:0/3100639554 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380009ed0 con 0x55ed7da2ed30 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2378ff9640 1 -- 172.21.15.100:0/3100639554 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23800044b0 con 0x55ed7da2ed30 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f2378ff9640 1 -- 172.21.15.100:0/3100639554 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23800035f0 con 0x55ed7da2ed30 2024-03-12T05:27:35.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f239a17e740 10 client.28145 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.953+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55ed7d994320 con 0x55ed7da2ed30 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f2378ff9640 1 -- 172.21.15.100:0/3100639554 <== mon.2 v2:172.21.15.156:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2380003750 con 0x55ed7da2ed30 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f239a17e740 10 client.28145 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f239a17e740 10 client.28145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f239a17e740 20 client.28145 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f239a17e740 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7da50480 0x55ed7da52870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f239a17e740 4 client.28145 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed7d98c310 con 0x55ed7da50480 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f2395270640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7da50480 0x55ed7da52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f2378ff9640 10 client.28145 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.954+0000 7f2395270640 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7da50480 0x55ed7da52870 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4594220eea446a7 server_cookie=8e562e663ea9f907 in_seq=0 out_seq=0 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.955+0000 7f2378ff9640 1 -- 172.21.15.100:0/3100639554 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed7d98c310 con 0x55ed7da50480 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.955+0000 7f2378ff9640 10 client.28145 handle_command_reply: tid=0 2024-03-12T05:27:35.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.955+0000 7f239a17e740 1 client.28145 shutdown 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.955+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 msgr2=0x55ed7da34d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.955+0000 7f239a17e740 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da34d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2380005890 tx=0x7f23800052f0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.955+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 shutdown_connections 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7da50480 0x55ed7da52870 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da378b0 0x55ed7da35260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2ed30 0x55ed7da34d20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f2378ff9640 0 client.28145 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:35.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 --2- 172.21.15.100:0/3100639554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2e380 0x55ed7da347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 >> 172.21.15.100:0/3100639554 conn(0x55ed7da28840 msgr2=0x55ed7da2a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:35.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 shutdown_connections 2024-03-12T05:27:35.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 1 -- 172.21.15.100:0/3100639554 wait complete. 2024-03-12T05:27:35.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.956+0000 7f239a17e740 20 client.28145 trim_cache size 0 max 16384 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.975+0000 7f239a17e740 1 Processor -- start 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 -- start start 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2e380 0x55ed7da36f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 0x55ed7da35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da31a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7da39ca0 con 0x55ed7da2ed30 2024-03-12T05:27:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7da52ba0 con 0x55ed7da2e380 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f239a17e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7da30350 con 0x55ed7da378b0 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 0x55ed7da35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 0x55ed7da35080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53820/0 (socket says 172.21.15.100:53820) 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 -- 172.21.15.100:0/2715303109 learned_addr learned my addr 172.21.15.100:0/2715303109 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2395a71640 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da31a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2395270640 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2e380 0x55ed7da36f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 -- 172.21.15.100:0/2715303109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 msgr2=0x55ed7da31a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da31a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 -- 172.21.15.100:0/2715303109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2e380 msgr2=0x55ed7da36f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2e380 0x55ed7da36f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 -- 172.21.15.100:0/2715303109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed7d930c00 con 0x55ed7da2ed30 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.976+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 0x55ed7da35080 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2380011d90 tx=0x7f2380024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7602a3c0f690da15 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.977+0000 7f237affd640 1 -- 172.21.15.100:0/2715303109 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800053e0 con 0x55ed7da2ed30 2024-03-12T05:27:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.977+0000 7f237affd640 1 -- 172.21.15.100:0/2715303109 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed7d96ff40 con 0x55ed7da2ed30 2024-03-12T05:27:35.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.977+0000 7f237affd640 1 -- 172.21.15.100:0/2715303109 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380036c80 con 0x55ed7da2ed30 2024-03-12T05:27:35.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.977+0000 7f239a17e740 1 -- 172.21.15.100:0/2715303109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 msgr2=0x55ed7da35080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.977+0000 7f239a17e740 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 0x55ed7da35080 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2380011d90 tx=0x7f2380024000 comp rx=0 tx=0).stop 2024-03-12T05:27:35.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.977+0000 7f239a17e740 1 -- 172.21.15.100:0/2715303109 shutdown_connections 2024-03-12T05:27:35.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da378b0 0x55ed7da31a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2ed30 0x55ed7da35080 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 --2- 172.21.15.100:0/2715303109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da2e380 0x55ed7da36f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 -- 172.21.15.100:0/2715303109 >> 172.21.15.100:0/2715303109 conn(0x55ed7da28840 msgr2=0x55ed7da330b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 -- 172.21.15.100:0/2715303109 shutdown_connections 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 -- 172.21.15.100:0/2715303109 wait complete. 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 -- start start 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.978+0000 7f239a17e740 1 Processor -- start 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2ed30 0x55ed7da46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da378b0 0x55ed7daa69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7daa6ee0 con 0x55ed7da2e380 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7daa7020 con 0x55ed7da378b0 2024-03-12T05:27:35.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7daa7190 con 0x55ed7da2ed30 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53826/0 (socket says 172.21.15.100:53826) 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 -- 172.21.15.100:0/2638008270 learned_addr learned my addr 172.21.15.100:0/2638008270 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f239a17e740 10 client.0 _ll_register_callbacks cb 0x7f2380009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395a71640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da378b0 0x55ed7daa69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2ed30 0x55ed7da46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 -- 172.21.15.100:0/2638008270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2ed30 msgr2=0x55ed7da46810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2ed30 0x55ed7da46810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 -- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da378b0 msgr2=0x55ed7daa69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da378b0 0x55ed7daa69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.979+0000 7f2395270640 1 -- 172.21.15.100:0/2638008270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed7d994320 con 0x55ed7da2e380 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f2395270640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da462d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f238c096cc0 tx=0x7f238c0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904130bd0ecb2d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:35.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f2378ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f2378ff9640 1 -- 172.21.15.100:0/2638008270 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c09a860 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f2378ff9640 1 -- 172.21.15.100:0/2638008270 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f238c09e900 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed7d9bb900 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f2378ff9640 1 -- 172.21.15.100:0/2638008270 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f238c0a2f50 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f239a17e740 10 client.18789 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.980+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ed7d956910 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f2378ff9640 1 -- 172.21.15.100:0/2638008270 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed7d9bb900 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f2378ff9640 1 -- 172.21.15.100:0/2638008270 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f238c0ba240 con 0x55ed7da2e380 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f239a17e740 10 client.18789 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f239a17e740 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f239a17e740 20 client.18789 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f239a17e740 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7daa8840 0x55ed7daaac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f239a17e740 4 client.18789 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:27:35.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ed7d695860 con 0x55ed7daa8840 2024-03-12T05:27:35.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7daa8840 0x55ed7daaac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:35.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f2378ff9640 10 client.18789 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:35.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.981+0000 7f2394a6f640 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7daa8840 0x55ed7daaac60 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882ffca7ec2b57cd server_cookie=96f78dc4511ceeb3 in_seq=0 out_seq=0 2024-03-12T05:27:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f2378ff9640 1 -- 172.21.15.100:0/2638008270 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ed7d695860 con 0x55ed7daa8840 2024-03-12T05:27:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f2378ff9640 10 client.18789 handle_command_reply: tid=0 2024-03-12T05:27:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 client.18789 shutdown 2024-03-12T05:27:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 msgr2=0x55ed7da462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da462d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f238c096cc0 tx=0x7f238c0ba040 comp rx=0 tx=0).stop 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 shutdown_connections 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55ed7daa8840 0x55ed7daaac60 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7da378b0 0x55ed7daa69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7da2ed30 0x55ed7da46810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 --2- 172.21.15.100:0/2638008270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed7da2e380 0x55ed7da462d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 >> 172.21.15.100:0/2638008270 conn(0x55ed7da28840 msgr2=0x55ed7da27050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f2378ff9640 0 client.18789 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 shutdown_connections 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 1 -- 172.21.15.100:0/2638008270 wait complete. 2024-03-12T05:27:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.983+0000 7f239a17e740 20 client.18789 trim_cache size 0 max 16384 2024-03-12T05:27:35.983 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:27:35.983 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:27:35.983 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "f54f5b35-1c3d-4457-b1b9-79c071cb1886", 2024-03-12T05:27:35.983 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:27:35.983 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f237c0a1ba0 msgr2=0x7f237c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f237c0a1ba0 0x7f237c0a4060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2388005e30 tx=0x7f2388005d80 comp rx=0 tx=0).stop 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 msgr2=0x7f239019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 0x7f239019f6d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f238c00e9f0 tx=0x7f238c00eeb0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 shutdown_connections 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f237c0a1ba0 0x7f237c0a4060 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2390101f40 0x7f2390199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2390100420 0x7f239019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 --2- 172.21.15.100:0/1534087271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23900ffa80 0x7f239019f6d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:35.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 >> 172.21.15.100:0/1534087271 conn(0x7f23900752e0 msgr2=0x7f23900fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:35.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 shutdown_connections 2024-03-12T05:27:35.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:35.984+0000 7f23974d4640 1 -- 172.21.15.100:0/1534087271 wait complete. 2024-03-12T05:27:35.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:27:35.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 -- 172.21.15.100:0/369998247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0105150 msgr2=0x7f23e010cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 --2- 172.21.15.100:0/369998247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0105150 0x7f23e010cd20 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009cd0 tx=0x7f23d802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 -- 172.21.15.100:0/369998247 shutdown_connections 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 --2- 172.21.15.100:0/369998247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0105150 0x7f23e010cd20 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 --2- 172.21.15.100:0/369998247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e0100a30 0x7f23e0104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 --2- 172.21.15.100:0/369998247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23e0100080 0x7f23e0100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 -- 172.21.15.100:0/369998247 >> 172.21.15.100:0/369998247 conn(0x7f23e0076c30 msgr2=0x7f23e00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 -- 172.21.15.100:0/369998247 shutdown_connections 2024-03-12T05:27:36.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.125+0000 7f23dffff640 1 -- 172.21.15.100:0/369998247 wait complete. 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 Processor -- start 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 -- start start 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 0x7f23e0196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e0105150 0x7f23e01969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23e019acf0 0x7f23e019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019b6e0 con 0x7f23e0100080 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019b850 con 0x7f23e0105150 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23dffff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019b9c0 con 0x7f23e019acf0 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 0x7f23e0196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 0x7f23e0196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53848/0 (socket says 172.21.15.100:53848) 2024-03-12T05:27:36.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 -- 172.21.15.100:0/3941887897 learned_addr learned my addr 172.21.15.100:0/3941887897 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23d67fc640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e0105150 0x7f23e01969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23df7fe640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23e019acf0 0x7f23e019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 -- 172.21.15.100:0/3941887897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23e019acf0 msgr2=0x7f23e019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23e019acf0 0x7f23e019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 -- 172.21.15.100:0/3941887897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e0105150 msgr2=0x7f23e01969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e0105150 0x7f23e01969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.126+0000 7f23deffd640 1 -- 172.21.15.100:0/3941887897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8009900 con 0x7f23e0100080 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.127+0000 7f23deffd640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 0x7f23e0196480 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f23d000e9f0 tx=0x7f23d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.127+0000 7f23e4949640 1 -- 172.21.15.100:0/3941887897 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0009720 con 0x7f23e0100080 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.127+0000 7f23e4949640 1 -- 172.21.15.100:0/3941887897 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23d000cd80 con 0x7f23e0100080 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.127+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e01a2fa0 con 0x7f23e0100080 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.127+0000 7f23e4949640 1 -- 172.21.15.100:0/3941887897 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0010690 con 0x7f23e0100080 2024-03-12T05:27:36.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.127+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e01a3370 con 0x7f23e0100080 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.129+0000 7f23e4949640 1 -- 172.21.15.100:0/3941887897 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f23d00046c0 con 0x7f23e0100080 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 Processor -- start 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 -- start start 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 0x55cc5da186c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e4949640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da25fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5d987280 con 0x55cc5da18c90 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5d9847b0 con 0x55cc5da23be0 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e85f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5d97ff90 con 0x55cc5da182e0 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 0x55cc5da186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 0x55cc5da186c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:50214/0 (socket says 172.21.15.100:50214) 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23df7fe640 1 -- 172.21.15.100:0/1845184155 learned_addr learned my addr 172.21.15.100:0/1845184155 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23d67fc640 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23df7fe640 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da25fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23e4949640 1 -- 172.21.15.100:0/3941887897 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23d0014070 con 0x7f23e0100080 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.130+0000 7f23deffd640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d67fc640 1 -- 172.21.15.100:0/1845184155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 msgr2=0x55cc5da186c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d67fc640 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 0x55cc5da186c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d67fc640 1 -- 172.21.15.100:0/1845184155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 msgr2=0x55cc5da25fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d67fc640 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da25fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d67fc640 1 -- 172.21.15.100:0/1845184155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc5d93e570 con 0x55cc5da18c90 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23df7fe640 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 0x55cc5da186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d67fc640 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da236a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f23e0073f60 tx=0x7f23c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e6bbc370cb2068 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23deffd640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23d0096c00 tx=0x7f23d0005720 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d5ffb640 1 -- 172.21.15.100:0/1845184155 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8004050 con 0x55cc5da18c90 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d5ffb640 1 -- 172.21.15.100:0/1845184155 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23c8005500 con 0x55cc5da18c90 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.131+0000 7f23d5ffb640 1 -- 172.21.15.100:0/1845184155 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8004c00 con 0x55cc5da18c90 2024-03-12T05:27:36.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 -- 172.21.15.100:0/1845184155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 msgr2=0x55cc5da236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da236a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f23e0073f60 tx=0x7f23c8011810 comp rx=0 tx=0).stop 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 -- 172.21.15.100:0/1845184155 shutdown_connections 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da25fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da236a0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 --2- 172.21.15.100:0/1845184155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da182e0 0x55cc5da186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 -- 172.21.15.100:0/1845184155 >> 172.21.15.100:0/1845184155 conn(0x55cc5da12c00 msgr2=0x55cc5da12fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 -- 172.21.15.100:0/1845184155 shutdown_connections 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.132+0000 7f23e85f1740 1 -- 172.21.15.100:0/1845184155 wait complete. 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.133+0000 7f23e85f1740 1 -- start start 2024-03-12T05:27:36.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.133+0000 7f23e85f1740 1 Processor -- start 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.133+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da1af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da1af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da1af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53862/0 (socket says 172.21.15.100:53862) 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 -- 172.21.15.100:0/3110384130 learned_addr learned my addr 172.21.15.100:0/3110384130 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da18c90 0x55cc5da19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5d653430 con 0x55cc5da182e0 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5d60fe00 con 0x55cc5da18c90 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5d904fd0 con 0x55cc5da23be0 2024-03-12T05:27:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23d67fc640 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da18c90 0x55cc5da19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 -- 172.21.15.100:0/3110384130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 msgr2=0x55cc5da19ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 -- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da18c90 msgr2=0x55cc5da19590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da18c90 0x55cc5da19590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c8009850 con 0x55cc5da182e0 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23df7fe640 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da1af10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005cc0 tx=0x7f23d8032ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48926b5967432569 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23beffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.134+0000 7f23e85f1740 10 client.0 _ll_register_callbacks cb 0x55cc5d95afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23beffd640 1 -- 172.21.15.100:0/3110384130 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8032970 con 0x55cc5da182e0 2024-03-12T05:27:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23beffd640 1 -- 172.21.15.100:0/3110384130 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23d8032ad0 con 0x55cc5da182e0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc5d67f7e0 con 0x55cc5da182e0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23beffd640 1 -- 172.21.15.100:0/3110384130 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d8003970 con 0x55cc5da182e0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 10 client.18801 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55cc5d97e610 con 0x55cc5da182e0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23beffd640 1 -- 172.21.15.100:0/3110384130 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f23d8003ad0 con 0x55cc5da182e0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 10 client.18801 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 20 client.18801 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da3c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 4 client.18801 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc5d6bc720 con 0x55cc5da3a0a0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.135+0000 7f23d67fc640 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da3c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.136+0000 7f23beffd640 10 client.18801 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.136+0000 7f23d67fc640 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da3c500 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f190e892feae32 server_cookie=5723daa589414452 in_seq=0 out_seq=0 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.136+0000 7f23beffd640 1 -- 172.21.15.100:0/3110384130 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc5d6bc720 con 0x55cc5da3a0a0 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.136+0000 7f23beffd640 10 client.18801 handle_command_reply: tid=0 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.136+0000 7f23e85f1740 1 client.18801 shutdown 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 msgr2=0x55cc5da1af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da1af10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005cc0 tx=0x7f23d8032ee0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 shutdown_connections 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da3c500 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23beffd640 0 client.18801 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da19ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da18c90 0x55cc5da19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 --2- 172.21.15.100:0/3110384130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da1af10 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 >> 172.21.15.100:0/3110384130 conn(0x55cc5da12c00 msgr2=0x55cc5da14950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 shutdown_connections 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 1 -- 172.21.15.100:0/3110384130 wait complete. 2024-03-12T05:27:36.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.137+0000 7f23e85f1740 20 client.18801 trim_cache size 0 max 16384 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 Processor -- start 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 -- start start 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da182e0 0x55cc5da4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da4a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da22ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5da41750 con 0x55cc5da18c90 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5da175c0 con 0x55cc5da182e0 2024-03-12T05:27:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23e85f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5da0d170 con 0x55cc5da23be0 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.154+0000 7f23d67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da4a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da182e0 0x55cc5da4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da4a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53872/0 (socket says 172.21.15.100:53872) 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 -- 172.21.15.100:0/2502540395 learned_addr learned my addr 172.21.15.100:0/2502540395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23df7fe640 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da22ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 -- 172.21.15.100:0/2502540395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 msgr2=0x55cc5da22ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da22ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 -- 172.21.15.100:0/2502540395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da182e0 msgr2=0x55cc5da4a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da182e0 0x55cc5da4a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 -- 172.21.15.100:0/2502540395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc5d91aac0 con 0x55cc5da18c90 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d67fc640 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da4a920 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009250 tx=0x7f23c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca13e22d9680096c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d57fa640 1 -- 172.21.15.100:0/2502540395 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c802a070 con 0x55cc5da18c90 2024-03-12T05:27:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d57fa640 1 -- 172.21.15.100:0/2502540395 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23c802d830 con 0x55cc5da18c90 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.155+0000 7f23d57fa640 1 -- 172.21.15.100:0/2502540395 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c80040d0 con 0x55cc5da18c90 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 -- 172.21.15.100:0/2502540395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 msgr2=0x55cc5da4a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da4a920 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009250 tx=0x7f23c8005f90 comp rx=0 tx=0).stop 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 -- 172.21.15.100:0/2502540395 shutdown_connections 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da23be0 0x55cc5da22ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da18c90 0x55cc5da4a920 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2502540395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da182e0 0x55cc5da4a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 -- 172.21.15.100:0/2502540395 >> 172.21.15.100:0/2502540395 conn(0x55cc5da12c00 msgr2=0x55cc5da1bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 -- 172.21.15.100:0/2502540395 shutdown_connections 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 -- 172.21.15.100:0/2502540395 wait complete. 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.156+0000 7f23e85f1740 1 -- start start 2024-03-12T05:27:36.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 Processor -- start 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da4a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da18c90 0x55cc5da467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da46d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5da4ac40 con 0x55cc5da182e0 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5da47250 con 0x55cc5da23be0 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5da47410 con 0x55cc5da18c90 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23e85f1740 10 client.0 _ll_register_callbacks cb 0x7f23c80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da4a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da4a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53884/0 (socket says 172.21.15.100:53884) 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 -- 172.21.15.100:0/2059114863 learned_addr learned my addr 172.21.15.100:0/2059114863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23d67fc640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da18c90 0x55cc5da467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da46d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 -- 172.21.15.100:0/2059114863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da18c90 msgr2=0x55cc5da467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da18c90 0x55cc5da467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 -- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 msgr2=0x55cc5da46d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da46d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 -- 172.21.15.100:0/2059114863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc5d93e570 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23df7fe640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da4a470 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f23d8032650 tx=0x7f23d803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1b7f2d500894b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.157+0000 7f23beffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23beffd640 1 -- 172.21.15.100:0/2059114863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003d80 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23beffd640 1 -- 172.21.15.100:0/2059114863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23d8003f70 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23c8005500 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23beffd640 1 -- 172.21.15.100:0/2059114863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d8032b20 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 10 client.18807 fetch_fsmap learned FSMap version 29 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cc5d9406a0 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23beffd640 1 -- 172.21.15.100:0/2059114863 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23c8005500 con 0x55cc5da182e0 2024-03-12T05:27:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23beffd640 1 -- 172.21.15.100:0/2059114863 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f23d804c450 con 0x55cc5da182e0 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 10 client.18807 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 10 client.18807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 20 client.18807 populate_metadata read hostname 'smithi100' 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da94070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 4 client.18807 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc5d70b890 con 0x55cc5da3a0a0 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.158+0000 7f23d67fc640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da94070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23beffd640 10 client.18807 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23d67fc640 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da94070 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca09a7e8d7b9e26f server_cookie=e7ff2a58a2714730 in_seq=0 out_seq=0 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23beffd640 1 -- 172.21.15.100:0/2059114863 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cc5d70b890 con 0x55cc5da3a0a0 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23beffd640 10 client.18807 handle_command_reply: tid=0 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 client.18807 shutdown 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 msgr2=0x55cc5da4a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da4a470 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f23d8032650 tx=0x7f23d803e000 comp rx=0 tx=0).stop 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 shutdown_connections 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cc5da3a0a0 0x55cc5da94070 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc5da23be0 0x55cc5da46d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc5da18c90 0x55cc5da467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23beffd640 0 client.18807 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 --2- 172.21.15.100:0/2059114863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc5da182e0 0x55cc5da4a470 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 >> 172.21.15.100:0/2059114863 conn(0x55cc5da12c00 msgr2=0x55cc5da19070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 shutdown_connections 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 1 -- 172.21.15.100:0/2059114863 wait complete. 2024-03-12T05:27:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.159+0000 7f23e85f1740 20 client.18807 trim_cache size 0 max 16384 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: "f54f5b35-1c3d-4457-b1b9-79c071cb1886": { 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "f54f5b35-1c3d-4457-b1b9-79c071cb1886", 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:27:36.163 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:27:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23cc0a1ba0 msgr2=0x7f23cc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23d0096c00 tx=0x7f23d0005720 comp rx=0 tx=0).stop 2024-03-12T05:27:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 msgr2=0x7f23e0196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:27:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 0x7f23e0196480 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f23d000e9f0 tx=0x7f23d000edf0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 shutdown_connections 2024-03-12T05:27:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23e019acf0 0x7f23e019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23e0105150 0x7f23e01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 --2- 172.21.15.100:0/3941887897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e0100080 0x7f23e0196480 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:27:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 >> 172.21.15.100:0/3941887897 conn(0x7f23e0076c30 msgr2=0x7f23e00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:27:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 shutdown_connections 2024-03-12T05:27:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:27:36.160+0000 7f23dffff640 1 -- 172.21.15.100:0/3941887897 wait complete. 2024-03-12T05:27:36.166 INFO:tasks.cephfs.filesystem:scrub status for tag:f54f5b35-1c3d-4457-b1b9-79c071cb1886 - {'path': '/', 'tag': 'f54f5b35-1c3d-4457-b1b9-79c071cb1886', 'options': 'recursive,force'} 2024-03-12T05:27:36.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84894 30.23 MB/sec execute 19 sec latency 73.987 ms 2024-03-12T05:27:37.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85608 31.07 MB/sec execute 20 sec latency 93.505 ms 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: pgmap v486: 97 pgs: 97 active+clean; 1006 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.4 MiB/s wr, 484 op/s 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:27:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:37 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: pgmap v486: 97 pgs: 97 active+clean; 1006 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.4 MiB/s wr, 484 op/s 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:27:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:37 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: pgmap v486: 97 pgs: 97 active+clean; 1006 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.4 MiB/s wr, 484 op/s 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:37.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:37 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:27:38.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86118 31.01 MB/sec execute 21 sec latency 246.722 ms 2024-03-12T05:27:39.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86712 30.89 MB/sec execute 22 sec latency 174.261 ms 2024-03-12T05:27:39.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:39 smithi100 ceph-mon[32210]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 715 op/s 2024-03-12T05:27:39.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:39 smithi156 ceph-mon[36270]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 715 op/s 2024-03-12T05:27:39.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:39 smithi161 ceph-mon[36043]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 715 op/s 2024-03-12T05:27:40.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87405 31.47 MB/sec execute 23 sec latency 52.034 ms 2024-03-12T05:27:41.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87902 31.24 MB/sec execute 24 sec latency 107.498 ms 2024-03-12T05:27:41.271 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:41 smithi161 ceph-mon[36043]: pgmap v488: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.3 MiB/s wr, 480 op/s 2024-03-12T05:27:41.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:41 smithi100 ceph-mon[32210]: pgmap v488: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.3 MiB/s wr, 480 op/s 2024-03-12T05:27:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:41 smithi156 ceph-mon[36270]: pgmap v488: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.3 MiB/s wr, 480 op/s 2024-03-12T05:27:42.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88513 30.97 MB/sec execute 25 sec latency 122.441 ms 2024-03-12T05:27:42.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:42 smithi100 ceph-mon[32210]: pgmap v489: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 695 op/s 2024-03-12T05:27:42.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:42 smithi156 ceph-mon[36270]: pgmap v489: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 695 op/s 2024-03-12T05:27:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:42 smithi161 ceph-mon[36043]: pgmap v489: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 695 op/s 2024-03-12T05:27:43.234 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89135 31.01 MB/sec execute 26 sec latency 256.935 ms 2024-03-12T05:27:44.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89841 31.30 MB/sec execute 27 sec latency 100.250 ms 2024-03-12T05:27:45.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90457 31.84 MB/sec execute 28 sec latency 53.105 ms 2024-03-12T05:27:45.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:45 smithi100 ceph-mon[32210]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 517 op/s 2024-03-12T05:27:45.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:45 smithi156 ceph-mon[36270]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 517 op/s 2024-03-12T05:27:45.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:45 smithi161 ceph-mon[36043]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 517 op/s 2024-03-12T05:27:46.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90959 31.42 MB/sec execute 29 sec latency 153.451 ms 2024-03-12T05:27:47.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91544 31.44 MB/sec execute 30 sec latency 68.995 ms 2024-03-12T05:27:47.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:47 smithi100 ceph-mon[32210]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 487 op/s 2024-03-12T05:27:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:47 smithi156 ceph-mon[36270]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 487 op/s 2024-03-12T05:27:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:47 smithi161 ceph-mon[36043]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 487 op/s 2024-03-12T05:27:48.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91937 31.05 MB/sec execute 31 sec latency 381.039 ms 2024-03-12T05:27:49.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92467 30.94 MB/sec execute 32 sec latency 186.756 ms 2024-03-12T05:27:49.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:49 smithi100 ceph-mon[32210]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 730 op/s 2024-03-12T05:27:49.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:49 smithi156 ceph-mon[36270]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 730 op/s 2024-03-12T05:27:49.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:49 smithi161 ceph-mon[36043]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 730 op/s 2024-03-12T05:27:50.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93256 31.10 MB/sec execute 33 sec latency 90.142 ms 2024-03-12T05:27:50.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:50 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:27:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:50 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:27:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:50 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:27:51.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93968 31.44 MB/sec execute 34 sec latency 43.253 ms 2024-03-12T05:27:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:51 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:27:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:51 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:27:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:51 smithi100 ceph-mon[32210]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 498 op/s 2024-03-12T05:27:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:51 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:27:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:51 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:27:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:51 smithi156 ceph-mon[36270]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 498 op/s 2024-03-12T05:27:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:51 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:27:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:51 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:27:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:51 smithi161 ceph-mon[36043]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 498 op/s 2024-03-12T05:27:52.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94648 31.80 MB/sec execute 35 sec latency 92.090 ms 2024-03-12T05:27:52.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:52 smithi100 ceph-mon[32210]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 718 op/s 2024-03-12T05:27:52.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:52 smithi156 ceph-mon[36270]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 718 op/s 2024-03-12T05:27:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:52 smithi161 ceph-mon[36043]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 718 op/s 2024-03-12T05:27:53.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95144 31.61 MB/sec execute 36 sec latency 229.860 ms 2024-03-12T05:27:54.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95642 31.60 MB/sec execute 37 sec latency 127.235 ms 2024-03-12T05:27:55.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96133 31.30 MB/sec execute 38 sec latency 95.435 ms 2024-03-12T05:27:55.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:55 smithi100 ceph-mon[32210]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-03-12T05:27:55.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:55 smithi156 ceph-mon[36270]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-03-12T05:27:55.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:55 smithi161 ceph-mon[36043]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-03-12T05:27:56.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96691 31.19 MB/sec execute 39 sec latency 63.266 ms 2024-03-12T05:27:57.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97472 31.29 MB/sec execute 40 sec latency 48.744 ms 2024-03-12T05:27:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:57 smithi100 ceph-mon[32210]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 508 op/s 2024-03-12T05:27:57.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:57 smithi156 ceph-mon[36270]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 508 op/s 2024-03-12T05:27:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:57 smithi161 ceph-mon[36043]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 508 op/s 2024-03-12T05:27:58.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98056 31.34 MB/sec execute 41 sec latency 245.909 ms 2024-03-12T05:27:58.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:27:58 smithi100 ceph-mon[32210]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 753 op/s 2024-03-12T05:27:58.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:27:58 smithi156 ceph-mon[36270]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 753 op/s 2024-03-12T05:27:58.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:27:58 smithi161 ceph-mon[36043]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 753 op/s 2024-03-12T05:27:59.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98787 31.73 MB/sec execute 42 sec latency 102.731 ms 2024-03-12T05:28:00.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99384 31.75 MB/sec execute 43 sec latency 95.460 ms 2024-03-12T05:28:01.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99881 31.75 MB/sec execute 44 sec latency 85.533 ms 2024-03-12T05:28:01.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:01 smithi100 ceph-mon[32210]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 510 op/s 2024-03-12T05:28:01.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:01 smithi156 ceph-mon[36270]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 510 op/s 2024-03-12T05:28:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:01 smithi161 ceph-mon[36043]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 510 op/s 2024-03-12T05:28:02.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100249 31.30 MB/sec execute 45 sec latency 85.006 ms 2024-03-12T05:28:03.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100496 30.78 MB/sec execute 46 sec latency 268.627 ms 2024-03-12T05:28:03.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:03 smithi100 ceph-mon[32210]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 784 op/s 2024-03-12T05:28:03.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:03 smithi156 ceph-mon[36270]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 784 op/s 2024-03-12T05:28:03.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:03 smithi161 ceph-mon[36043]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 784 op/s 2024-03-12T05:28:04.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100829 30.57 MB/sec execute 47 sec latency 262.882 ms 2024-03-12T05:28:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:04 smithi100 ceph-mon[32210]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 583 op/s 2024-03-12T05:28:04.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:04 smithi156 ceph-mon[36270]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 583 op/s 2024-03-12T05:28:04.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:04 smithi161 ceph-mon[36043]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 583 op/s 2024-03-12T05:28:05.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 101362 30.54 MB/sec execute 48 sec latency 108.042 ms 2024-03-12T05:28:06.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:28:06.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:28:06.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102049 30.66 MB/sec execute 49 sec latency 55.156 ms 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 -- 172.21.15.100:0/2545606704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8928100c20 msgr2=0x7f8928104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 --2- 172.21.15.100:0/2545606704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8928100c20 0x7f8928104c60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8918009930 tx=0x7f891802bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 -- 172.21.15.100:0/2545606704 shutdown_connections 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 --2- 172.21.15.100:0/2545606704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8928105340 0x7f892810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 --2- 172.21.15.100:0/2545606704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8928100c20 0x7f8928104c60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 --2- 172.21.15.100:0/2545606704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8928100270 0x7f8928100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 -- 172.21.15.100:0/2545606704 >> 172.21.15.100:0/2545606704 conn(0x7f8928077230 msgr2=0x7f8928077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 -- 172.21.15.100:0/2545606704 shutdown_connections 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.296+0000 7f893064e640 1 -- 172.21.15.100:0/2545606704 wait complete. 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 Processor -- start 2024-03-12T05:28:06.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 -- start start 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8928100270 0x7f8928197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8928105340 0x7f8928197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 0x7f892819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892819c9f0 con 0x7f892819c000 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892819cb60 con 0x7f8928100270 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f893064e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892819ccd0 con 0x7f8928105340 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 0x7f892819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 0x7f892819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40026/0 (socket says 172.21.15.100:40026) 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 -- 172.21.15.100:0/2655297008 learned_addr learned my addr 172.21.15.100:0/2655297008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892e3ea640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8928100270 0x7f8928197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892dbe9640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8928105340 0x7f8928197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 -- 172.21.15.100:0/2655297008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8928105340 msgr2=0x7f8928197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8928105340 0x7f8928197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 -- 172.21.15.100:0/2655297008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8928100270 msgr2=0x7f8928197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8928100270 0x7f8928197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.297+0000 7f892ebeb640 1 -- 172.21.15.100:0/2655297008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009560 con 0x7f892819c000 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.298+0000 7f892ebeb640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 0x7f892819c4b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f892000eb10 tx=0x7f892000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.298+0000 7f891f7fe640 1 -- 172.21.15.100:0/2655297008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920004b20 con 0x7f892819c000 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.298+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89281a42b0 con 0x7f892819c000 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.298+0000 7f891f7fe640 1 -- 172.21.15.100:0/2655297008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8920004c80 con 0x7f892819c000 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.298+0000 7f891f7fe640 1 -- 172.21.15.100:0/2655297008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920010650 con 0x7f892819c000 2024-03-12T05:28:06.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.298+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89281a4730 con 0x7f892819c000 2024-03-12T05:28:06.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.300+0000 7f891f7fe640 1 -- 172.21.15.100:0/2655297008 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8920010830 con 0x7f892819c000 2024-03-12T05:28:06.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 Processor -- start 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 -- start start 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cb2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cbdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389c21280 con 0x55e389cb22b0 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389c1e7b0 con 0x55e389cb2bd0 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f8933223740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389c19f90 con 0x55e389cbb7b0 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cb2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f892e3ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cb2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40040/0 (socket says 172.21.15.100:40040) 2024-03-12T05:28:06.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.301+0000 7f892dbe9640 1 -- 172.21.15.100:0/66299107 learned_addr learned my addr 172.21.15.100:0/66299107 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892dbe9640 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cbdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892e3ea640 1 -- 172.21.15.100:0/66299107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 msgr2=0x55e389cbdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f891f7fe640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89140a1ba0 0x7f89140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892e3ea640 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cbdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892e3ea640 1 -- 172.21.15.100:0/66299107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 msgr2=0x55e389cb2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892e3ea640 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cb2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892e3ea640 1 -- 172.21.15.100:0/66299107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e389bd8570 con 0x55e389cb2bd0 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892dbe9640 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cbdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892ebeb640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89140a1ba0 0x7f89140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f892e3ea640 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8924009c00 tx=0x7f8924011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d330c5cb04e87e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f891f7fe640 1 -- 172.21.15.100:0/2655297008 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8920014070 con 0x7f892819c000 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f891cbfe640 1 -- 172.21.15.100:0/66299107 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924004230 con 0x55e389cb2bd0 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f891cbfe640 1 -- 172.21.15.100:0/66299107 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8924009e70 con 0x55e389cb2bd0 2024-03-12T05:28:06.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.302+0000 7f891cbfe640 1 -- 172.21.15.100:0/66299107 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924004d00 con 0x55e389cb2bd0 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 -- 172.21.15.100:0/66299107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 msgr2=0x55e389cbb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f892ebeb640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89140a1ba0 0x7f89140a4060 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89200cb580 tx=0x7f89200c5070 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8924009c00 tx=0x7f8924011810 comp rx=0 tx=0).stop 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 -- 172.21.15.100:0/66299107 shutdown_connections 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cbdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 --2- 172.21.15.100:0/66299107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 -- 172.21.15.100:0/66299107 >> 172.21.15.100:0/66299107 conn(0x55e389cac990 msgr2=0x55e389cacd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 -- 172.21.15.100:0/66299107 shutdown_connections 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.303+0000 7f8933223740 1 -- 172.21.15.100:0/66299107 wait complete. 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 -- start start 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 Processor -- start 2024-03-12T05:28:06.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cb85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cb8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3898ed430 con 0x55e389cb22b0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3898a9e00 con 0x55e389cb2bd0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389b9efd0 con 0x55e389cbb7b0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8933223740 10 client.0 _ll_register_callbacks cb 0x55e389bf4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40042/0 (socket says 172.21.15.100:40042) 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 -- 172.21.15.100:0/2851753865 learned_addr learned my addr 172.21.15.100:0/2851753865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892e3ea640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cb85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cb8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 -- 172.21.15.100:0/2851753865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 msgr2=0x55e389cb8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cb8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 -- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 msgr2=0x55e389cb85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cb85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 -- 172.21.15.100:0/2851753865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8924009830 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f892dbe9640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbb210 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f891802c240 tx=0x7f8918000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6467d76e297439f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.304+0000 7f8911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8911ffb640 1 -- 172.21.15.100:0/2851753865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918004030 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8911ffb640 1 -- 172.21.15.100:0/2851753865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8918004190 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3899197e0 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8911ffb640 1 -- 172.21.15.100:0/2851753865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8918004350 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 10 client.18819 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e389c18610 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8911ffb640 1 -- 172.21.15.100:0/2851753865 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3899197e0 con 0x55e389cb22b0 2024-03-12T05:28:06.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8911ffb640 1 -- 172.21.15.100:0/2851753865 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f8918003780 con 0x55e389cb22b0 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 10 client.18819 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 10 client.18819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 20 client.18819 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389ccb8e0 0x55e389ccdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 4 client.18819 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e389956720 con 0x55e389ccb8e0 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.305+0000 7f892e3ea640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389ccb8e0 0x55e389ccdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.306+0000 7f8911ffb640 10 client.18819 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.306+0000 7f892e3ea640 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389ccb8e0 0x55e389ccdcd0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68b790683430dff server_cookie=90405988206cd8d2 in_seq=0 out_seq=0 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8911ffb640 1 -- 172.21.15.100:0/2851753865 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e389956720 con 0x55e389ccb8e0 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8911ffb640 10 client.18819 handle_command_reply: tid=0 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 client.18819 shutdown 2024-03-12T05:28:06.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 msgr2=0x55e389cbb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbb210 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f891802c240 tx=0x7f8918000bc0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 shutdown_connections 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389ccb8e0 0x55e389ccdcd0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8911ffb640 0 client.18819 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389cb8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cb85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 --2- 172.21.15.100:0/2851753865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbb210 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 >> 172.21.15.100:0/2851753865 conn(0x55e389cac990 msgr2=0x55e389cae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 shutdown_connections 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 1 -- 172.21.15.100:0/2851753865 wait complete. 2024-03-12T05:28:06.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.307+0000 7f8933223740 20 client.18819 trim_cache size 0 max 16384 2024-03-12T05:28:06.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:06 smithi100 ceph-mon[32210]: osdmap e108: 12 total, 12 up, 12 in 2024-03-12T05:28:06.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:06 smithi100 ceph-mon[32210]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-03-12T05:28:06.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 Processor -- start 2024-03-12T05:28:06.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 -- start start 2024-03-12T05:28:06.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cb22b0 0x55e389cbacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cbb7b0 0x55e389cb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389956720 con 0x55e389cbb7b0 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3899197e0 con 0x55e389cb2bd0 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.323+0000 7f8933223740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389cb9a00 con 0x55e389cb22b0 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cbb7b0 0x55e389cb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47286/0 (socket says 172.21.15.100:47286) 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 -- 172.21.15.100:0/3067042735 learned_addr learned my addr 172.21.15.100:0/3067042735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892dbe9640 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cb22b0 0x55e389cbacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 -- 172.21.15.100:0/3067042735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cb22b0 msgr2=0x55e389cbacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cb22b0 0x55e389cbacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 -- 172.21.15.100:0/3067042735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cbb7b0 msgr2=0x55e389cb5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cbb7b0 0x55e389cb5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 -- 172.21.15.100:0/3067042735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e389bb4ac0 con 0x55e389cb2bd0 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892dbe9640 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cbb7b0 0x55e389cb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f892e3ea640 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89240048b0 tx=0x7f8924004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bacda46cba981b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.324+0000 7f8913fff640 1 -- 172.21.15.100:0/3067042735 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924019b00 con 0x55e389cb2bd0 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8913fff640 1 -- 172.21.15.100:0/3067042735 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8924019570 con 0x55e389cb2bd0 2024-03-12T05:28:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8913fff640 1 -- 172.21.15.100:0/3067042735 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924026dd0 con 0x55e389cb2bd0 2024-03-12T05:28:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 -- 172.21.15.100:0/3067042735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 msgr2=0x55e389cbb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89240048b0 tx=0x7f8924004980 comp rx=0 tx=0).stop 2024-03-12T05:28:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 -- 172.21.15.100:0/3067042735 shutdown_connections 2024-03-12T05:28:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cbb7b0 0x55e389cb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbb210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 --2- 172.21.15.100:0/3067042735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cb22b0 0x55e389cbacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 -- 172.21.15.100:0/3067042735 >> 172.21.15.100:0/3067042735 conn(0x55e389cac990 msgr2=0x55e389cb38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 -- 172.21.15.100:0/3067042735 shutdown_connections 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.325+0000 7f8933223740 1 -- 172.21.15.100:0/3067042735 wait complete. 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 -- start start 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 Processor -- start 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389d21db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389cc03e0 con 0x55e389cb22b0 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389d222f0 con 0x55e389cb2bd0 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e389d22480 con 0x55e389cbb7b0 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f8933223740 10 client.0 _ll_register_callbacks cb 0x7f8924009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40058/0 (socket says 172.21.15.100:40058) 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 -- 172.21.15.100:0/3648134376 learned_addr learned my addr 172.21.15.100:0/3648134376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892e3ea640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389d21db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 -- 172.21.15.100:0/3648134376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 msgr2=0x55e389d21db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389d21db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 -- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 msgr2=0x55e389cbfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.326+0000 7f892dbe9640 1 -- 172.21.15.100:0/3648134376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e389c18610 con 0x55e389cb22b0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f892dbe9640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbf760 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f891802c210 tx=0x7f8918052040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c884185b8eb4420 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8911ffb640 1 -- 172.21.15.100:0/3648134376 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918004030 con 0x55e389cb22b0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8911ffb640 1 -- 172.21.15.100:0/3648134376 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8918004190 con 0x55e389cb22b0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8924009e70 con 0x55e389cb22b0 2024-03-12T05:28:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8911ffb640 1 -- 172.21.15.100:0/3648134376 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8918004350 con 0x55e389cb22b0 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 10 client.18825 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f892401d040 con 0x55e389cb22b0 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8911ffb640 1 -- 172.21.15.100:0/3648134376 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f891806a080 con 0x55e389cb22b0 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8911ffb640 1 -- 172.21.15.100:0/3648134376 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f8918066510 con 0x55e389cb22b0 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 10 client.18825 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 10 client.18825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 20 client.18825 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389d27d90 0x55e389d2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 4 client.18825 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.327+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3899a5890 con 0x55e389d27d90 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f892e3ea640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389d27d90 0x55e389d2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8911ffb640 10 client.18825 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f892e3ea640 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389d27d90 0x55e389d2a180 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b68e155f7b7c84 server_cookie=26d6333ad624b87 in_seq=0 out_seq=0 2024-03-12T05:28:06.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8911ffb640 1 -- 172.21.15.100:0/3648134376 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e3899a5890 con 0x55e389d27d90 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8911ffb640 10 client.18825 handle_command_reply: tid=0 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 client.18825 shutdown 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 msgr2=0x55e389cbf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbf760 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f891802c210 tx=0x7f8918052040 comp rx=0 tx=0).stop 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 shutdown_connections 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e389d27d90 0x55e389d2a180 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389cbb7b0 0x55e389d21db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e389cb2bd0 0x55e389cbfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8911ffb640 0 client.18825 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 --2- 172.21.15.100:0/3648134376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e389cb22b0 0x55e389cbf760 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 >> 172.21.15.100:0/3648134376 conn(0x55e389cac990 msgr2=0x55e389cb6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 shutdown_connections 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 1 -- 172.21.15.100:0/3648134376 wait complete. 2024-03-12T05:28:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.328+0000 7f8933223740 20 client.18825 trim_cache size 0 max 16384 2024-03-12T05:28:06.329 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:28:06.329 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:28:06.329 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:28:06.329 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89140a1ba0 msgr2=0x7f89140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89140a1ba0 0x7f89140a4060 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89200cb580 tx=0x7f89200c5070 comp rx=0 tx=0).stop 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 msgr2=0x7f892819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 0x7f892819c4b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f892000eb10 tx=0x7f892000c2d0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 shutdown_connections 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89140a1ba0 0x7f89140a4060 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892819c000 0x7f892819c4b0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8928105340 0x7f8928197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 --2- 172.21.15.100:0/2655297008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8928100270 0x7f8928197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 >> 172.21.15.100:0/2655297008 conn(0x7f8928077230 msgr2=0x7f89280685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 shutdown_connections 2024-03-12T05:28:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.329+0000 7f893064e640 1 -- 172.21.15.100:0/2655297008 wait complete. 2024-03-12T05:28:06.336 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:28:06.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:28:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:06 smithi156 ceph-mon[36270]: osdmap e108: 12 total, 12 up, 12 in 2024-03-12T05:28:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:06 smithi156 ceph-mon[36270]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 -- 172.21.15.100:0/592907183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 msgr2=0x7fcc9010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 --2- 172.21.15.100:0/592907183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9010e050 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009930 tx=0x7fcc8802bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 -- 172.21.15.100:0/592907183 shutdown_connections 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 --2- 172.21.15.100:0/592907183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9010e050 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 --2- 172.21.15.100:0/592907183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc90101d60 0x7fcc90105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 --2- 172.21.15.100:0/592907183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc901013b0 0x7fcc90101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 -- 172.21.15.100:0/592907183 >> 172.21.15.100:0/592907183 conn(0x7fcc900fcc10 msgr2=0x7fcc900ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 -- 172.21.15.100:0/592907183 shutdown_connections 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.467+0000 7fcc972d0640 1 -- 172.21.15.100:0/592907183 wait complete. 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 Processor -- start 2024-03-12T05:28:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 -- start start 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc901013b0 0x7fcc9019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90197610 0x7fcc90197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90198000 con 0x7fcc90106480 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90198170 con 0x7fcc901013b0 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc972d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90198330 con 0x7fcc90197610 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40068/0 (socket says 172.21.15.100:40068) 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 -- 172.21.15.100:0/1521244553 learned_addr learned my addr 172.21.15.100:0/1521244553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9586d640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90197610 0x7fcc90197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9506c640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc901013b0 0x7fcc9019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 -- 172.21.15.100:0/1521244553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90197610 msgr2=0x7fcc90197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90197610 0x7fcc90197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 -- 172.21.15.100:0/1521244553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc901013b0 msgr2=0x7fcc9019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc901013b0 0x7fcc9019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.468+0000 7fcc9486b640 1 -- 172.21.15.100:0/1521244553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88009560 con 0x7fcc90106480 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.469+0000 7fcc9486b640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9019d9a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000e9c0 tx=0x7fcc8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.469+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80009700 con 0x7fcc90106480 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.469+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc901a4320 con 0x7fcc90106480 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.469+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc901a4880 con 0x7fcc90106480 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.469+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc8000cd80 con 0x7fcc90106480 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.469+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004ea0 con 0x7fcc90106480 2024-03-12T05:28:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.470+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74005290 con 0x7fcc90106480 2024-03-12T05:28:06.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.471+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fcc800046c0 con 0x7fcc90106480 2024-03-12T05:28:06.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.472+0000 7fcc867fc640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc7c0a1ba0 0x7fcc7c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.472+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fcc80014070 con 0x7fcc90106480 2024-03-12T05:28:06.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.472+0000 7fcc9506c640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc7c0a1ba0 0x7fcc7c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.472+0000 7fcc9506c640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc7c0a1ba0 0x7fcc7c0a4060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005fd0 tx=0x7fcc8c009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.475+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc80096a30 con 0x7fcc90106480 2024-03-12T05:28:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:06 smithi161 ceph-mon[36043]: osdmap e108: 12 total, 12 up, 12 in 2024-03-12T05:28:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:06 smithi161 ceph-mon[36043]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-03-12T05:28:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.722+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc740058f0 con 0x7fcc90106480 2024-03-12T05:28:06.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.723+0000 7fcc867fc640 1 -- 172.21.15.100:0/1521244553 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fcc740058f0 con 0x7fcc90106480 2024-03-12T05:28:06.717 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:28:06.717 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:27:28.572302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:28:06.718 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 29 2024-03-12T05:28:06.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc7c0a1ba0 msgr2=0x7fcc7c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc7c0a1ba0 0x7fcc7c0a4060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005fd0 tx=0x7fcc8c009500 comp rx=0 tx=0).stop 2024-03-12T05:28:06.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 msgr2=0x7fcc9019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9019d9a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000e9c0 tx=0x7fcc8000ee80 comp rx=0 tx=0).stop 2024-03-12T05:28:06.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 shutdown_connections 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc7c0a1ba0 0x7fcc7c0a4060 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90197610 0x7fcc90197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc90106480 0x7fcc9019d9a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 --2- 172.21.15.100:0/1521244553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc901013b0 0x7fcc9019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.725+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 >> 172.21.15.100:0/1521244553 conn(0x7fcc900fcc10 msgr2=0x7fcc900fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.726+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 shutdown_connections 2024-03-12T05:28:06.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.726+0000 7fcc972d0640 1 -- 172.21.15.100:0/1521244553 wait complete. 2024-03-12T05:28:06.732 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:28:06.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:28:06.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/1558177202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08101380 msgr2=0x7f3b08101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/1558177202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08101380 0x7f3b08101760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009900 tx=0x7f3af802bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/1558177202 shutdown_connections 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/1558177202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08106450 0x7f3b0810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/1558177202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08101d30 0x7f3b08105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/1558177202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08101380 0x7f3b08101760 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/1558177202 >> 172.21.15.100:0/1558177202 conn(0x7f3b080fcc20 msgr2=0x7f3b080ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/1558177202 shutdown_connections 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.857+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/1558177202 wait complete. 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 Processor -- start 2024-03-12T05:28:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 -- start start 2024-03-12T05:28:06.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 0x7f3b0819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08106450 0x7f3b0819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08199830 0x7f3b08199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0819a2e0 con 0x7f3b08101380 2024-03-12T05:28:06.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0819a450 con 0x7f3b08199830 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0e6d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0819a5c0 con 0x7f3b08106450 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 0x7f3b0819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 0x7f3b0819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40084/0 (socket says 172.21.15.100:40084) 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b07fff640 1 -- 172.21.15.100:0/3049610476 learned_addr learned my addr 172.21.15.100:0/3049610476 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08199830 0x7f3b08199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.858+0000 7f3b077fe640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08106450 0x7f3b0819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0cc71640 1 -- 172.21.15.100:0/3049610476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08106450 msgr2=0x7f3b0819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08106450 0x7f3b0819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0cc71640 1 -- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 msgr2=0x7f3b0819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 0x7f3b0819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0cc71640 1 -- 172.21.15.100:0/3049610476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00009840 con 0x7f3b08199830 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b07fff640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 0x7f3b0819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:06.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08199830 0x7f3b08199ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000ec50 tx=0x7f3b0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b057fa640 1 -- 172.21.15.100:0/3049610476 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00003b70 con 0x7f3b08199830 2024-03-12T05:28:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b057fa640 1 -- 172.21.15.100:0/3049610476 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b0000ce70 con 0x7f3b08199830 2024-03-12T05:28:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af8009560 con 0x7f3b08199830 2024-03-12T05:28:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b057fa640 1 -- 172.21.15.100:0/3049610476 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00010830 con 0x7f3b08199830 2024-03-12T05:28:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.859+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b081a6870 con 0x7f3b08199830 2024-03-12T05:28:06.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.863+0000 7f3b057fa640 1 -- 172.21.15.100:0/3049610476 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3b00004020 con 0x7f3b08199830 2024-03-12T05:28:06.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.863+0000 7f3b11380740 1 Processor -- start 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.863+0000 7f3b057fa640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af00a1c20 0x7f3af00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.863+0000 7f3b11380740 1 -- start start 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.863+0000 7f3b057fa640 1 -- 172.21.15.100:0/3049610476 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f3b00014070 con 0x7f3b08199830 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b077fe640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af00a1c20 0x7f3af00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b11380740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137250 0x55c91e137630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b11380740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e140210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b11380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b11380740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e0f2960 con 0x55c91e137b70 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b11380740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e0a6310 con 0x55c91e137250 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b11380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e0a3840 con 0x55c91e140750 2024-03-12T05:28:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40094/0 (socket says 172.21.15.100:40094) 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b077fe640 1 -- 172.21.15.100:0/4026591754 learned_addr learned my addr 172.21.15.100:0/4026591754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b07fff640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137250 0x55c91e137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b0cc71640 1 -- 172.21.15.100:0/4026591754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137250 msgr2=0x55c91e137630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137250 0x55c91e137630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b0cc71640 1 -- 172.21.15.100:0/4026591754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 msgr2=0x55c91e140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b0cc71640 1 -- 172.21.15.100:0/4026591754 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c91e061060 con 0x55c91e140750 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b077fe640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.864+0000 7f3b07fff640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137250 0x55c91e137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.865+0000 7f3b077fe640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af00a1c20 0x7f3af00a40e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3afc013970 tx=0x7f3afc013460 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.865+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e142b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b000cb5d0 tx=0x7f3b00096700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e24e462f788fdc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.865+0000 7f3aee7fc640 1 -- 172.21.15.100:0/4026591754 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0009a410 con 0x55c91e140750 2024-03-12T05:28:06.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.865+0000 7f3aee7fc640 1 -- 172.21.15.100:0/4026591754 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b0009b1b0 con 0x55c91e140750 2024-03-12T05:28:06.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.865+0000 7f3aee7fc640 1 -- 172.21.15.100:0/4026591754 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00004440 con 0x55c91e140750 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 -- 172.21.15.100:0/4026591754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 msgr2=0x55c91e142b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e142b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b000cb5d0 tx=0x7f3b00096700 comp rx=0 tx=0).stop 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 -- 172.21.15.100:0/4026591754 shutdown_connections 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e142b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 --2- 172.21.15.100:0/4026591754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137250 0x55c91e137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 -- 172.21.15.100:0/4026591754 >> 172.21.15.100:0/4026591754 conn(0x55c91e131980 msgr2=0x55c91e131d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 -- 172.21.15.100:0/4026591754 shutdown_connections 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 -- 172.21.15.100:0/4026591754 wait complete. 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.866+0000 7f3b11380740 1 -- start start 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.867+0000 7f3b11380740 1 Processor -- start 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.867+0000 7f3b11380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 0x55c91e13d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.867+0000 7f3b11380740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e13dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 0x55c91e13d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 0x55c91e13d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55298/0 (socket says 172.21.15.100:55298) 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 -- 172.21.15.100:0/1158242766 learned_addr learned my addr 172.21.15.100:0/1158242766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e09f020 con 0x55c91e137b70 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91dd72430 con 0x55c91e140750 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91dd2ee00 con 0x55c91e137250 2024-03-12T05:28:06.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b11380740 10 client.0 _ll_register_callbacks cb 0x55c91e046cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b077fe640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e13dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 -- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 msgr2=0x55c91e13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 -- 172.21.15.100:0/1158242766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 msgr2=0x55c91e13dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e13dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.868+0000 7f3b07fff640 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b081a6870 con 0x55c91e137250 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3b07fff640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 0x55c91e13d7e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006010 tx=0x7f3af802ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf919d85e3b5ec6d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3ae77fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3ae77fe640 1 -- 172.21.15.100:0/1158242766 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004490 con 0x55c91e137250 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3ae77fe640 1 -- 172.21.15.100:0/1158242766 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f3af8004df0 con 0x55c91e137250 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c91dd9e7e0 con 0x55c91e137250 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3ae77fe640 1 -- 172.21.15.100:0/1158242766 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3af8004fb0 con 0x55c91e137250 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3b11380740 10 client.28141 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:06.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c91e09d6a0 con 0x55c91e137250 2024-03-12T05:28:06.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.869+0000 7f3ae77fe640 1 -- 172.21.15.100:0/1158242766 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c91dd9e7e0 con 0x55c91e137250 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3ae77fe640 1 -- 172.21.15.100:0/1158242766 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3af803c040 con 0x55c91e137250 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b11380740 10 client.28141 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b11380740 10 client.28141 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b11380740 20 client.28141 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e150bd0 0x55c91e152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b11380740 4 client.28141 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c91dddb720 con 0x55c91e150bd0 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3b077fe640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e150bd0 0x55c91e152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.870+0000 7f3ae77fe640 10 client.28141 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.871+0000 7f3b077fe640 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e150bd0 0x55c91e152ff0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e866e112d2ed475e server_cookie=70006ba717374e67 in_seq=0 out_seq=0 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3ae77fe640 1 -- 172.21.15.100:0/1158242766 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c91dddb720 con 0x55c91e150bd0 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3ae77fe640 10 client.28141 handle_command_reply: tid=0 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 client.28141 shutdown 2024-03-12T05:28:06.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 msgr2=0x55c91e13d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 0x55c91e13d7e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006010 tx=0x7f3af802ccf0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 shutdown_connections 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e150bd0 0x55c91e152ff0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137b70 0x55c91e13dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 --2- 172.21.15.100:0/1158242766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137250 0x55c91e13d7e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3ae77fe640 0 client.28141 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 >> 172.21.15.100:0/1158242766 conn(0x55c91e131980 msgr2=0x55c91e1336d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 shutdown_connections 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 1 -- 172.21.15.100:0/1158242766 wait complete. 2024-03-12T05:28:06.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.873+0000 7f3b11380740 20 client.28141 trim_cache size 0 max 16384 2024-03-12T05:28:06.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.886+0000 7f3b11380740 1 Processor -- start 2024-03-12T05:28:06.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 -- start start 2024-03-12T05:28:06.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137b70 0x55c91e13e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e024130 con 0x55c91e137250 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e132980 con 0x55c91e140750 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b11380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e132bb0 con 0x55c91e137b70 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40108/0 (socket says 172.21.15.100:40108) 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 -- 172.21.15.100:0/1348385988 learned_addr learned my addr 172.21.15.100:0/1348385988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b077fe640 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137b70 0x55c91e13e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 -- 172.21.15.100:0/1348385988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137b70 msgr2=0x55c91e13e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137b70 0x55c91e13e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 -- 172.21.15.100:0/1348385988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 msgr2=0x55c91e13aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 -- 172.21.15.100:0/1348385988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c91e039c20 con 0x55c91e137250 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.887+0000 7f3b07fff640 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e13ff10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af802cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d18b4860d866879 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3aedffb640 1 -- 172.21.15.100:0/1348385988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004490 con 0x55c91e137250 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3aedffb640 1 -- 172.21.15.100:0/1348385988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3af8003ad0 con 0x55c91e137250 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3aedffb640 1 -- 172.21.15.100:0/1348385988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8034a00 con 0x55c91e137250 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 -- 172.21.15.100:0/1348385988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 msgr2=0x55c91e13ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e13ff10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af802cbe0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 -- 172.21.15.100:0/1348385988 shutdown_connections 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e140750 0x55c91e13aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e137b70 0x55c91e13e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 --2- 172.21.15.100:0/1348385988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e13ff10 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 -- 172.21.15.100:0/1348385988 >> 172.21.15.100:0/1348385988 conn(0x55c91e131980 msgr2=0x55c91e138ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 -- 172.21.15.100:0/1348385988 shutdown_connections 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.888+0000 7f3b11380740 1 -- 172.21.15.100:0/1348385988 wait complete. 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 -- start start 2024-03-12T05:28:06.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 Processor -- start 2024-03-12T05:28:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137b70 0x55c91e146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e1a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e1472c0 con 0x55c91e137250 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e1a75e0 con 0x55c91e137b70 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91e1a7720 con 0x55c91e140750 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b11380740 10 client.0 _ll_register_callbacks cb 0x7f3b00005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40122/0 (socket says 172.21.15.100:40122) 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b07fff640 1 -- 172.21.15.100:0/473222643 learned_addr learned my addr 172.21.15.100:0/473222643 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b077fe640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137b70 0x55c91e146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.889+0000 7f3b0cc71640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e1a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b07fff640 1 -- 172.21.15.100:0/473222643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 msgr2=0x55c91e1a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b07fff640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e1a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b07fff640 1 -- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137b70 msgr2=0x55c91e146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b07fff640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137b70 0x55c91e146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b07fff640 1 -- 172.21.15.100:0/473222643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c91e09d6a0 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b07fff640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e146680 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3af8036040 tx=0x7f3af8003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9536b1a99ebb7c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3ae77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3ae77fe640 1 -- 172.21.15.100:0/473222643 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8062070 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3ae77fe640 1 -- 172.21.15.100:0/473222643 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f3af8004490 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b0009b1b0 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3ae77fe640 1 -- 172.21.15.100:0/473222643 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3af8003d80 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 10 client.18849 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c91e05f8c0 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3ae77fe640 1 -- 172.21.15.100:0/473222643 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b0009b1b0 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3ae77fe640 1 -- 172.21.15.100:0/473222643 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3af8065020 con 0x55c91e137250 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 10 client.18849 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 10 client.18849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 20 client.18849 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e1a8a60 0x55c91e1aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:06.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 4 client.18849 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:28:06.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.890+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c91de2a890 con 0x55c91e1a8a60 2024-03-12T05:28:06.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.891+0000 7f3b077fe640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e1a8a60 0x55c91e1aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:06.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.891+0000 7f3ae77fe640 10 client.18849 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:06.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.891+0000 7f3b077fe640 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e1a8a60 0x55c91e1aae50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19ebf551bcbe3bf2 server_cookie=d37cad3eeeb4baa2 in_seq=0 out_seq=0 2024-03-12T05:28:06.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3ae77fe640 1 -- 172.21.15.100:0/473222643 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c91de2a890 con 0x55c91e1a8a60 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3ae77fe640 10 client.18849 handle_command_reply: tid=0 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 client.18849 shutdown 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 msgr2=0x55c91e146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e146680 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3af8036040 tx=0x7f3af8003960 comp rx=0 tx=0).stop 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 shutdown_connections 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55c91e1a8a60 0x55c91e1aae50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c91e140750 0x55c91e1a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c91e137b70 0x55c91e146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3ae77fe640 0 client.18849 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 --2- 172.21.15.100:0/473222643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91e137250 0x55c91e146680 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 >> 172.21.15.100:0/473222643 conn(0x55c91e131980 msgr2=0x55c91e1302f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 shutdown_connections 2024-03-12T05:28:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 1 -- 172.21.15.100:0/473222643 wait complete. 2024-03-12T05:28:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.901+0000 7f3b11380740 20 client.18849 trim_cache size 0 max 16384 2024-03-12T05:28:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af00a1c20 msgr2=0x7f3af00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af00a1c20 0x7f3af00a40e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3afc013970 tx=0x7f3afc013460 comp rx=0 tx=0).stop 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08199830 msgr2=0x7f3b08199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08199830 0x7f3b08199ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000ec50 tx=0x7f3b0000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 shutdown_connections 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3af00a1c20 0x7f3af00a40e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b08199830 0x7f3b08199ce0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08106450 0x7f3b0819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 --2- 172.21.15.100:0/3049610476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08101380 0x7f3b0819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 >> 172.21.15.100:0/3049610476 conn(0x7f3b080fcc20 msgr2=0x7f3b080fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 shutdown_connections 2024-03-12T05:28:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:06.902+0000 7f3b0e6d4640 1 -- 172.21.15.100:0/3049610476 wait complete. 2024-03-12T05:28:06.908 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:28:06.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:28:07.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.043+0000 7f103c1b5640 1 -- 172.21.15.100:0/3960018319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 msgr2=0x7f1034103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.043+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3960018319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 0x7f1034103970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f103001ace0 tx=0x7f103003d150 comp rx=0 tx=0).stop 2024-03-12T05:28:07.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.044+0000 7f103c1b5640 1 -- 172.21.15.100:0/3960018319 shutdown_connections 2024-03-12T05:28:07.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.044+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3960018319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034108660 0x7f1034110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.044+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3960018319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034103f40 0x7f1034107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.044+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3960018319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 0x7f1034103970 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.044+0000 7f103c1b5640 1 -- 172.21.15.100:0/3960018319 >> 172.21.15.100:0/3960018319 conn(0x7f10340fedf0 msgr2=0x7f1034101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.046+0000 7f103c1b5640 1 -- 172.21.15.100:0/3960018319 shutdown_connections 2024-03-12T05:28:07.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.048+0000 7f103c1b5640 1 -- 172.21.15.100:0/3960018319 wait complete. 2024-03-12T05:28:07.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.049+0000 7f103c1b5640 1 Processor -- start 2024-03-12T05:28:07.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.049+0000 7f103c1b5640 1 -- start start 2024-03-12T05:28:07.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.049+0000 7f103c1b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 0x7f10341101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.049+0000 7f103c1b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034108660 0x7f1034072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.049+0000 7f103c1b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034070da0 0x7f1034071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f1039750640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034108660 0x7f1034072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f1039750640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034108660 0x7f1034072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55346/0 (socket says 172.21.15.100:55346) 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f103c1b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10340718f0 con 0x7f1034103590 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f103c1b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034071a60 con 0x7f1034070da0 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f103c1b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034071c20 con 0x7f1034108660 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f1039750640 1 -- 172.21.15.100:0/3777348597 learned_addr learned my addr 172.21.15.100:0/3777348597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.050+0000 7f103a752640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034070da0 0x7f1034071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f1039f51640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 0x7f10341101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f103a752640 1 -- 172.21.15.100:0/3777348597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034108660 msgr2=0x7f1034072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f103a752640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034108660 0x7f1034072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f103a752640 1 -- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 msgr2=0x7f10341101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f103a752640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 0x7f10341101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f103a752640 1 -- 172.21.15.100:0/3777348597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103001a910 con 0x7f1034070da0 2024-03-12T05:28:07.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.051+0000 7f103a752640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034070da0 0x7f1034071250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f102c00b410 tx=0x7f102c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.052+0000 7f102affd640 1 -- 172.21.15.100:0/3777348597 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011830 con 0x7f1034070da0 2024-03-12T05:28:07.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.052+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10341a22f0 con 0x7f1034070da0 2024-03-12T05:28:07.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.052+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341a27a0 con 0x7f1034070da0 2024-03-12T05:28:07.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.052+0000 7f102affd640 1 -- 172.21.15.100:0/3777348597 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f102c011e70 con 0x7f1034070da0 2024-03-12T05:28:07.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.053+0000 7f102affd640 1 -- 172.21.15.100:0/3777348597 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0193b0 con 0x7f1034070da0 2024-03-12T05:28:07.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.056+0000 7f102affd640 1 -- 172.21.15.100:0/3777348597 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f102c011990 con 0x7f1034070da0 2024-03-12T05:28:07.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.056+0000 7f102affd640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f10200a1a80 0x7f10200a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 Processor -- start 2024-03-12T05:28:07.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f102affd640 1 -- 172.21.15.100:0/3777348597 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f102c0cd2f0 con 0x7f1034070da0 2024-03-12T05:28:07.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f1039f51640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f10200a1a80 0x7f10200a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 -- start start 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a372960 con 0x55d97a3c0750 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a326310 con 0x55d97a3b7b70 2024-03-12T05:28:07.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103ed88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a323840 con 0x55d97a3b7250 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.057+0000 7f103a752640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40160/0 (socket says 172.21.15.100:40160) 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 -- 172.21.15.100:0/2057147796 learned_addr learned my addr 172.21.15.100:0/2057147796 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f1039f51640 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f1039750640 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f1039f51640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f10200a1a80 0x7f10200a3f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f103003d760 tx=0x7f1030005f80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 -- 172.21.15.100:0/2057147796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 msgr2=0x55d97a3b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 -- 172.21.15.100:0/2057147796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 msgr2=0x55d97a3c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 -- 172.21.15.100:0/2057147796 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d97a2e1060 con 0x55d97a3c0750 2024-03-12T05:28:07.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.058+0000 7f103a752640 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3c2b40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f102c096420 tx=0x7f102c09a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff47f4fb9d5e147 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.059+0000 7f101ffff640 1 -- 172.21.15.100:0/2057147796 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0a2760 con 0x55d97a3c0750 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.059+0000 7f101ffff640 1 -- 172.21.15.100:0/2057147796 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f102c0982c0 con 0x55d97a3c0750 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.059+0000 7f101ffff640 1 -- 172.21.15.100:0/2057147796 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c099670 con 0x55d97a3c0750 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 -- 172.21.15.100:0/2057147796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 msgr2=0x55d97a3c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3c2b40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f102c096420 tx=0x7f102c09a6c0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 -- 172.21.15.100:0/2057147796 shutdown_connections 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3c2b40 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 --2- 172.21.15.100:0/2057147796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 -- 172.21.15.100:0/2057147796 >> 172.21.15.100:0/2057147796 conn(0x55d97a3b1980 msgr2=0x55d97a3b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 -- 172.21.15.100:0/2057147796 shutdown_connections 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 -- 172.21.15.100:0/2057147796 wait complete. 2024-03-12T05:28:07.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 -- start start 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.060+0000 7f103ed88740 1 Processor -- start 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.061+0000 7f103ed88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.061+0000 7f103ed88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3b7b70 0x55d97a3bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.061+0000 7f103ed88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 0x55d97a3be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.061+0000 7f103ed88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a31f020 con 0x55d97a3b7b70 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.061+0000 7f103ed88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d979ff2430 con 0x55d97a3c0750 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.061+0000 7f103ed88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d979faee00 con 0x55d97a3b7250 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f103ed88740 10 client.0 _ll_register_callbacks cb 0x55d97a2c6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f103a752640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 0x55d97a3be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f103a752640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 0x55d97a3be260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47408/0 (socket says 172.21.15.100:47408) 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f103a752640 1 -- 172.21.15.100:0/1954522005 learned_addr learned my addr 172.21.15.100:0/1954522005 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f1039750640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3b7b70 0x55d97a3bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f1039750640 1 -- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 msgr2=0x55d97a3bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f1039750640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f1039750640 1 -- 172.21.15.100:0/1954522005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 msgr2=0x55d97a3be260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f1039750640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 0x55d97a3be260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f1039750640 1 -- 172.21.15.100:0/1954522005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341a27a0 con 0x55d97a3b7b70 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.062+0000 7f103a752640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 0x55d97a3be260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.063+0000 7f1039750640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3b7b70 0x55d97a3bdd20 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f10340727e0 tx=0x7f10240119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b827d2999c5be8c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.063+0000 7f101d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.063+0000 7f101d7fa640 1 -- 172.21.15.100:0/1954522005 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024013670 con 0x55d97a3b7b70 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.063+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d97a01e7e0 con 0x55d97a3b7b70 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.063+0000 7f101d7fa640 1 -- 172.21.15.100:0/1954522005 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f10240137d0 con 0x55d97a3b7b70 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.063+0000 7f101d7fa640 1 -- 172.21.15.100:0/1954522005 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1024013990 con 0x55d97a3b7b70 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.064+0000 7f103ed88740 10 client.18861 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:07.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.064+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d97a31d6a0 con 0x55d97a3b7b70 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.064+0000 7f101d7fa640 1 -- 172.21.15.100:0/1954522005 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d97a01e7e0 con 0x55d97a3b7b70 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.064+0000 7f101d7fa640 1 -- 172.21.15.100:0/1954522005 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f102401b020 con 0x55d97a3b7b70 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.064+0000 7f103ed88740 10 client.18861 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.064+0000 7f103ed88740 10 client.18861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.065+0000 7f103ed88740 20 client.18861 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.065+0000 7f103ed88740 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a3e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.065+0000 7f103ed88740 4 client.18861 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.065+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d97a05b720 con 0x55d97a3e1a30 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.066+0000 7f1039f51640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a3e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.067+0000 7f1039f51640 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a3e3e50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f7eb7b8b2a58ae8 server_cookie=6ffd03cbd8163816 in_seq=0 out_seq=0 2024-03-12T05:28:07.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.067+0000 7f101d7fa640 10 client.18861 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:07.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.070+0000 7f101d7fa640 1 -- 172.21.15.100:0/1954522005 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d97a05b720 con 0x55d97a3e1a30 2024-03-12T05:28:07.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.070+0000 7f101d7fa640 10 client.18861 handle_command_reply: tid=0 2024-03-12T05:28:07.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.070+0000 7f103ed88740 1 client.18861 shutdown 2024-03-12T05:28:07.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.071+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3b7b70 msgr2=0x55d97a3bdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.071+0000 7f103ed88740 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3b7b70 0x55d97a3bdd20 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f10340727e0 tx=0x7f10240119d0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.071+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 shutdown_connections 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.071+0000 7f103ed88740 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a3e3e50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.071+0000 7f101d7fa640 0 client.18861 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.071+0000 7f103ed88740 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3c0750 0x55d97a3be260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.072+0000 7f103ed88740 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3b7b70 0x55d97a3bdd20 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.072+0000 7f103ed88740 1 --2- 172.21.15.100:0/1954522005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.072+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 >> 172.21.15.100:0/1954522005 conn(0x55d97a3b1980 msgr2=0x55d97a3b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.072+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 shutdown_connections 2024-03-12T05:28:07.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.072+0000 7f103ed88740 1 -- 172.21.15.100:0/1954522005 wait complete. 2024-03-12T05:28:07.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.072+0000 7f103ed88740 20 client.18861 trim_cache size 0 max 16384 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 Processor -- start 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 -- start start 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7250 0x55d97a3bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 0x55d97a3be080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a2a4130 con 0x55d97a3c0750 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a3ad5c0 con 0x55d97a3b7250 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103ed88740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a3e9080 con 0x55d97a3b7b70 2024-03-12T05:28:07.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039f51640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7250 0x55d97a3bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 0x55d97a3be080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103a752640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 0x55d97a3be080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55374/0 (socket says 172.21.15.100:55374) 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 -- 172.21.15.100:0/2224943588 learned_addr learned my addr 172.21.15.100:0/2224943588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f103a752640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3baa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40176/0 (socket says 172.21.15.100:40176) 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 -- 172.21.15.100:0/2224943588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7250 msgr2=0x55d97a3bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7250 0x55d97a3bff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 -- 172.21.15.100:0/2224943588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 msgr2=0x55d97a3baa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3baa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.089+0000 7f1039750640 1 -- 172.21.15.100:0/2224943588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d97a2b9c20 con 0x55d97a3b7b70 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.090+0000 7f1039750640 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 0x55d97a3be080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1024014560 tx=0x7f1024014590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e166b8aab8397b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.090+0000 7f101f7fe640 1 -- 172.21.15.100:0/2224943588 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024013040 con 0x55d97a3b7b70 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.090+0000 7f101f7fe640 1 -- 172.21.15.100:0/2224943588 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1024014e10 con 0x55d97a3b7b70 2024-03-12T05:28:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.090+0000 7f101f7fe640 1 -- 172.21.15.100:0/2224943588 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024025450 con 0x55d97a3b7b70 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 -- 172.21.15.100:0/2224943588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 msgr2=0x55d97a3be080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 0x55d97a3be080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1024014560 tx=0x7f1024014590 comp rx=0 tx=0).stop 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 -- 172.21.15.100:0/2224943588 shutdown_connections 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3baa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7b70 0x55d97a3be080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 --2- 172.21.15.100:0/2224943588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7250 0x55d97a3bff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 -- 172.21.15.100:0/2224943588 >> 172.21.15.100:0/2224943588 conn(0x55d97a3b1980 msgr2=0x55d97a3b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 -- 172.21.15.100:0/2224943588 shutdown_connections 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.091+0000 7f103ed88740 1 -- 172.21.15.100:0/2224943588 wait complete. 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.092+0000 7f103ed88740 1 -- start start 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.092+0000 7f103ed88740 1 Processor -- start 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.092+0000 7f103ed88740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.092+0000 7f103ed88740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.092+0000 7f103ed88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103ed88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a3d8ef0 con 0x55d97a3c0750 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039f51640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103ed88740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a437f60 con 0x55d97a3b7b70 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039f51640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3d6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55382/0 (socket says 172.21.15.100:55382) 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97a4380a0 con 0x55d97a3b7250 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039f51640 1 -- 172.21.15.100:0/735370208 learned_addr learned my addr 172.21.15.100:0/735370208 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103a752640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103ed88740 10 client.0 _ll_register_callbacks cb 0x7f102c099d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 -- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 msgr2=0x55d97a3d6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3d6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 -- 172.21.15.100:0/735370208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 msgr2=0x55d97a3d7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3d7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 -- 172.21.15.100:0/735370208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d97a31d6a0 con 0x55d97a3b7b70 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103a752640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039f51640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f1039750640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3d7530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55d97a3de7d0 tx=0x7f10240119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26f453fbc99aa474 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f101d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f101d7fa640 1 -- 172.21.15.100:0/735370208 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024013800 con 0x55d97a3b7b70 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f102c0982c0 con 0x55d97a3b7b70 2024-03-12T05:28:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.093+0000 7f101d7fa640 1 -- 172.21.15.100:0/735370208 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f1024013b30 con 0x55d97a3b7b70 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f101d7fa640 1 -- 172.21.15.100:0/735370208 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1024013d10 con 0x55d97a3b7b70 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 10 client.28226 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d97a2df8c0 con 0x55d97a3b7b70 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f101d7fa640 1 -- 172.21.15.100:0/735370208 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f102403d020 con 0x55d97a3b7b70 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f101d7fa640 1 -- 172.21.15.100:0/735370208 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f102401b040 con 0x55d97a3b7b70 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 10 client.28226 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 10 client.28226 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 20 client.28226 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a43b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 4 client.28226 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d97a0aa890 con 0x55d97a3e1a30 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.094+0000 7f1039f51640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a43b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.095+0000 7f101d7fa640 10 client.28226 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.095+0000 7f1039f51640 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a43b330 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=998ca91e5d35e75d server_cookie=a942edd62d0ffcab in_seq=0 out_seq=0 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.097+0000 7f101d7fa640 1 -- 172.21.15.100:0/735370208 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d97a0aa890 con 0x55d97a3e1a30 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.097+0000 7f101d7fa640 10 client.28226 handle_command_reply: tid=0 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.097+0000 7f103ed88740 1 client.28226 shutdown 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 msgr2=0x55d97a3d7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3d7530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55d97a3de7d0 tx=0x7f10240119d0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 shutdown_connections 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d97a3c0750 0x55d97a3d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d97a3e1a30 0x55d97a43b330 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d97a3b7b70 0x55d97a3d7530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f101d7fa640 0 client.28226 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 --2- 172.21.15.100:0/735370208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d97a3b7250 0x55d97a3d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.098+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 >> 172.21.15.100:0/735370208 conn(0x55d97a3b1980 msgr2=0x55d97a3b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 shutdown_connections 2024-03-12T05:28:07.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103ed88740 1 -- 172.21.15.100:0/735370208 wait complete. 2024-03-12T05:28:07.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103ed88740 20 client.28226 trim_cache size 0 max 16384 2024-03-12T05:28:07.101 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f10200a1a80 msgr2=0x7f10200a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f10200a1a80 0x7f10200a3f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f103003d760 tx=0x7f1030005f80 comp rx=0 tx=0).stop 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034070da0 msgr2=0x7f1034071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.099+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034070da0 0x7f1034071250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f102c00b410 tx=0x7f102c00b8d0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 shutdown_connections 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f10200a1a80 0x7f10200a3f40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1034070da0 0x7f1034071250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034108660 0x7f1034072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 --2- 172.21.15.100:0/3777348597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1034103590 0x7f10341101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 >> 172.21.15.100:0/3777348597 conn(0x7f10340fedf0 msgr2=0x7f10341001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 shutdown_connections 2024-03-12T05:28:07.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.100+0000 7f103c1b5640 1 -- 172.21.15.100:0/3777348597 wait complete. 2024-03-12T05:28:07.107 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:28:07.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:28:07.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 -- 172.21.15.100:0/3042840002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107250 msgr2=0x7ff474107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 --2- 172.21.15.100:0/3042840002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107250 0x7ff474107630 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff464009900 tx=0x7ff46402bcd0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 -- 172.21.15.100:0/3042840002 shutdown_connections 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 --2- 172.21.15.100:0/3042840002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff474069f00 0x7ff47410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 --2- 172.21.15.100:0/3042840002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff474069540 0x7ff4740699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 --2- 172.21.15.100:0/3042840002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107250 0x7ff474107630 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 -- 172.21.15.100:0/3042840002 >> 172.21.15.100:0/3042840002 conn(0x7ff474077110 msgr2=0x7ff474077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 -- 172.21.15.100:0/3042840002 shutdown_connections 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 -- 172.21.15.100:0/3042840002 wait complete. 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.236+0000 7ff479320640 1 Processor -- start 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 -- start start 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 0x7ff4741000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff474069f00 0x7ff4741005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff474107250 0x7ff474100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741a5b30 con 0x7ff474069540 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741a5ca0 con 0x7ff474069f00 2024-03-12T05:28:07.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff479320640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741a5e60 con 0x7ff474107250 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 0x7ff4741000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 0x7ff4741000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40196/0 (socket says 172.21.15.100:40196) 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 -- 172.21.15.100:0/3410064257 learned_addr learned my addr 172.21.15.100:0/3410064257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff474069f00 0x7ff4741005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff4737fe640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff474107250 0x7ff474100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 -- 172.21.15.100:0/3410064257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff474107250 msgr2=0x7ff474100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff474107250 0x7ff474100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 -- 172.21.15.100:0/3410064257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff474069f00 msgr2=0x7ff4741005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff474069f00 0x7ff4741005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.237+0000 7ff472ffd640 1 -- 172.21.15.100:0/3410064257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009560 con 0x7ff474069540 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.238+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 0x7ff4741000b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff464005940 tx=0x7ff46402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.238+0000 7ff470ff9640 1 -- 172.21.15.100:0/3410064257 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640043d0 con 0x7ff474069540 2024-03-12T05:28:07.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.238+0000 7ff470ff9640 1 -- 172.21.15.100:0/3410064257 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4640052d0 con 0x7ff474069540 2024-03-12T05:28:07.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.238+0000 7ff470ff9640 1 -- 172.21.15.100:0/3410064257 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640043d0 con 0x7ff474069540 2024-03-12T05:28:07.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.238+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4741a60e0 con 0x7ff474069540 2024-03-12T05:28:07.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.238+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4741a65a0 con 0x7ff474069540 2024-03-12T05:28:07.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.241+0000 7ff470ff9640 1 -- 172.21.15.100:0/3410064257 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff464005440 con 0x7ff474069540 2024-03-12T05:28:07.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.241+0000 7ff470ff9640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4600a1c00 0x7ff4600a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102695 30.75 MB/sec execute 50 sec latency 66.245 ms 2024-03-12T05:28:07.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.241+0000 7ff47bfcb740 1 Processor -- start 2024-03-12T05:28:07.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.241+0000 7ff470ff9640 1 -- 172.21.15.100:0/3410064257 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff4640efb60 con 0x7ff474069540 2024-03-12T05:28:07.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.241+0000 7ff47bfcb740 1 -- start start 2024-03-12T05:28:07.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.241+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4600a1c00 0x7ff4600a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4600a1c00 0x7ff4600a40c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff474101b40 tx=0x7ff45c00b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb40630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb49210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6bafb960 con 0x55fd6bb40250 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6baaf310 con 0x55fd6bb49750 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6baac840 con 0x55fd6bb40b70 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb40630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb49210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb49210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55398/0 (socket says 172.21.15.100:55398) 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb40630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40198/0 (socket says 172.21.15.100:40198) 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 -- 172.21.15.100:0/2521239103 learned_addr learned my addr 172.21.15.100:0/2521239103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff4737fe640 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 -- 172.21.15.100:0/2521239103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 msgr2=0x55fd6bb4bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 -- 172.21.15.100:0/2521239103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 msgr2=0x55fd6bb40630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb40630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.242+0000 7ff46bfff640 1 -- 172.21.15.100:0/2521239103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6ba6a060 con 0x55fd6bb40b70 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.243+0000 7ff46bfff640 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb49210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff45c011010 tx=0x7ff45c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb157c9c9321d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.243+0000 7ff4697fa640 1 -- 172.21.15.100:0/2521239103 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c01e620 con 0x55fd6bb40b70 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.243+0000 7ff4697fa640 1 -- 172.21.15.100:0/2521239103 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff45c015070 con 0x55fd6bb40b70 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.243+0000 7ff4697fa640 1 -- 172.21.15.100:0/2521239103 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c026af0 con 0x55fd6bb40b70 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 -- 172.21.15.100:0/2521239103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 msgr2=0x55fd6bb49210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb49210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff45c011010 tx=0x7ff45c013000 comp rx=0 tx=0).stop 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 -- 172.21.15.100:0/2521239103 shutdown_connections 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb49210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/2521239103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb40630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 -- 172.21.15.100:0/2521239103 >> 172.21.15.100:0/2521239103 conn(0x55fd6bb3a980 msgr2=0x55fd6bb3ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 -- 172.21.15.100:0/2521239103 shutdown_connections 2024-03-12T05:28:07.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 -- 172.21.15.100:0/2521239103 wait complete. 2024-03-12T05:28:07.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 -- start start 2024-03-12T05:28:07.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.244+0000 7ff47bfcb740 1 Processor -- start 2024-03-12T05:28:07.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40250 0x55fd6bb467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb46ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb47220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6baa8020 con 0x55fd6bb49750 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b77b430 con 0x55fd6bb40250 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b737e00 con 0x55fd6bb40b70 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb47220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb47220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40214/0 (socket says 172.21.15.100:40214) 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 -- 172.21.15.100:0/3191623704 learned_addr learned my addr 172.21.15.100:0/3191623704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb46ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40250 0x55fd6bb467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff47bfcb740 10 client.0 _ll_register_callbacks cb 0x55fd6ba4fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 -- 172.21.15.100:0/3191623704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 msgr2=0x55fd6bb46ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb46ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 -- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40250 msgr2=0x55fd6bb467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40250 0x55fd6bb467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 -- 172.21.15.100:0/3191623704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c00bce0 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff4737fe640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb47220 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7ff46c011d40 tx=0x7ff46c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8d3ac459e9618b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.245+0000 7ff45a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3191623704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c016070 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd6b7a77e0 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3191623704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff46c004b20 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3191623704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff46c00fe60 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 10 client.18879 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fd6baa66a0 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3191623704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd6b7a77e0 con 0x55fd6bb49750 2024-03-12T05:28:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3191623704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff46c01b470 con 0x55fd6bb49750 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 10 client.18879 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 20 client.18879 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bb5c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 4 client.18879 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.246+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd6b7e4720 con 0x55fd6bb59bd0 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.247+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bb5c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.247+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bb5c010 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec58f64457c001df server_cookie=6173c77eea8f1245 in_seq=0 out_seq=0 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.247+0000 7ff45a7fc640 10 client.18879 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3191623704 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd6b7e4720 con 0x55fd6bb59bd0 2024-03-12T05:28:07.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff45a7fc640 10 client.18879 handle_command_reply: tid=0 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 client.18879 shutdown 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 msgr2=0x55fd6bb47220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb47220 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7ff46c011d40 tx=0x7ff46c00f790 comp rx=0 tx=0).stop 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 shutdown_connections 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bb5c010 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb47220 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff45a7fc640 0 client.18879 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb46ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3191623704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40250 0x55fd6bb467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 >> 172.21.15.100:0/3191623704 conn(0x55fd6bb3a980 msgr2=0x55fd6bb3c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 shutdown_connections 2024-03-12T05:28:07.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3191623704 wait complete. 2024-03-12T05:28:07.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.248+0000 7ff47bfcb740 20 client.18879 trim_cache size 0 max 16384 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.261+0000 7ff47bfcb740 1 Processor -- start 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.261+0000 7ff47bfcb740 1 -- start start 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40250 0x55fd6bb47040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 0x55fd6bb39a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ba2d130 con 0x55fd6bb49750 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6bb3b760 con 0x55fd6bb40b70 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6bb56680 con 0x55fd6bb40250 2024-03-12T05:28:07.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff46bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 0x55fd6bb39a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40250 0x55fd6bb47040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff46bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 0x55fd6bb39a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47458/0 (socket says 172.21.15.100:47458) 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff46bfff640 1 -- 172.21.15.100:0/3820019713 learned_addr learned my addr 172.21.15.100:0/3820019713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff4737fe640 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff472ffd640 1 -- 172.21.15.100:0/3820019713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 msgr2=0x55fd6bb39a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 0x55fd6bb39a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff472ffd640 1 -- 172.21.15.100:0/3820019713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 msgr2=0x55fd6bb50600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb50600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff472ffd640 1 -- 172.21.15.100:0/3820019713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6ba42c20 con 0x55fd6bb40250 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.262+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 0x55fd6bb39a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.263+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40250 0x55fd6bb47040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4640ed350 tx=0x7ff464041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b78bcd519c6329 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.263+0000 7ff468ff9640 1 -- 172.21.15.100:0/3820019713 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640c0aa0 con 0x55fd6bb40250 2024-03-12T05:28:07.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.263+0000 7ff468ff9640 1 -- 172.21.15.100:0/3820019713 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4640c50f0 con 0x55fd6bb40250 2024-03-12T05:28:07.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.263+0000 7ff468ff9640 1 -- 172.21.15.100:0/3820019713 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464004df0 con 0x55fd6bb40250 2024-03-12T05:28:07.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3820019713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40250 msgr2=0x55fd6bb47040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40250 0x55fd6bb47040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4640ed350 tx=0x7ff464041040 comp rx=0 tx=0).stop 2024-03-12T05:28:07.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3820019713 shutdown_connections 2024-03-12T05:28:07.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb49750 0x55fd6bb50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb40b70 0x55fd6bb39a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3820019713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40250 0x55fd6bb47040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3820019713 >> 172.21.15.100:0/3820019713 conn(0x55fd6bb3a980 msgr2=0x55fd6bb392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3820019713 shutdown_connections 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3820019713 wait complete. 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 -- start start 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.264+0000 7ff47bfcb740 1 Processor -- start 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb4f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb4f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6bbb0180 con 0x55fd6bb40250 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6bbb02f0 con 0x55fd6bb49750 2024-03-12T05:28:07.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6bbb04b0 con 0x55fd6bb40b70 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 10 client.0 _ll_register_callbacks cb 0x7ff45c00b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb4f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb4f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40232/0 (socket says 172.21.15.100:40232) 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 -- 172.21.15.100:0/3647852158 learned_addr learned my addr 172.21.15.100:0/3647852158 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff4737fe640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 -- 172.21.15.100:0/3647852158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 msgr2=0x55fd6bb4f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb4f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 -- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 msgr2=0x55fd6bb4fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 -- 172.21.15.100:0/3647852158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4741a65a0 con 0x55fd6bb40250 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb4f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff472ffd640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb4f1c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ff464005740 tx=0x7ff4640c0aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dde63230f054a0f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff45a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3647852158 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640bad10 con 0x55fd6bb40250 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3647852158 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff464004df0 con 0x55fd6bb40250 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.265+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff45c015070 con 0x55fd6bb40250 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3647852158 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff464004fb0 con 0x55fd6bb40250 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 10 client.18885 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:07.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fd6ba688c0 con 0x55fd6bb40250 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3647852158 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff45c015070 con 0x55fd6bb40250 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3647852158 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff46403c070 con 0x55fd6bb40250 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 10 client.18885 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 10 client.18885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 20 client.18885 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bbb3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 4 client.18885 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd6b833890 con 0x55fd6bb59bd0 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.266+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bbb3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff45a7fc640 10 client.18885 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff46bfff640 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bbb3820 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9914a497d3ecf4 server_cookie=ac0875150e5e54b9 in_seq=0 out_seq=0 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff45a7fc640 1 -- 172.21.15.100:0/3647852158 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd6b833890 con 0x55fd6bb59bd0 2024-03-12T05:28:07.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff45a7fc640 10 client.18885 handle_command_reply: tid=0 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 client.18885 shutdown 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 msgr2=0x55fd6bb4f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb4f1c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ff464005740 tx=0x7ff4640c0aa0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 shutdown_connections 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55fd6bb59bd0 0x55fd6bbb3820 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd6bb49750 0x55fd6bb4fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd6bb40b70 0x55fd6bb4f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff45a7fc640 0 client.18885 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 --2- 172.21.15.100:0/3647852158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd6bb40250 0x55fd6bb4f1c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 >> 172.21.15.100:0/3647852158 conn(0x55fd6bb3a980 msgr2=0x55fd6bb39b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 shutdown_connections 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.267+0000 7ff47bfcb740 1 -- 172.21.15.100:0/3647852158 wait complete. 2024-03-12T05:28:07.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff47bfcb740 20 client.18885 trim_cache size 0 max 16384 2024-03-12T05:28:07.269 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4600a1c00 msgr2=0x7ff4600a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4600a1c00 0x7ff4600a40c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff474101b40 tx=0x7ff45c00b410 comp rx=0 tx=0).stop 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 msgr2=0x7ff4741000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 0x7ff4741000b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff464005940 tx=0x7ff46402c7a0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 shutdown_connections 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4600a1c00 0x7ff4600a40c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff474107250 0x7ff474100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff474069f00 0x7ff4741005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 --2- 172.21.15.100:0/3410064257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474069540 0x7ff4741000b0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:07.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 >> 172.21.15.100:0/3410064257 conn(0x7ff474077110 msgr2=0x7ff4740fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 shutdown_connections 2024-03-12T05:28:07.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:07.268+0000 7ff479320640 1 -- 172.21.15.100:0/3410064257 wait complete. 2024-03-12T05:28:07.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:07 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1521244553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:28:07.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:07 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1521244553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:28:07.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:07 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1521244553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:28:08.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103218 30.80 MB/sec execute 51 sec latency 256.560 ms 2024-03-12T05:28:08.277 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:28:08.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:28:08.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:28:08.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:08 smithi100 ceph-mon[32210]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 887 op/s 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 -- 172.21.15.100:0/476936268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900100cc0 msgr2=0x7f4900104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 --2- 172.21.15.100:0/476936268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900100cc0 0x7f4900104d00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009930 tx=0x7f48ec02bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 -- 172.21.15.100:0/476936268 shutdown_connections 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 --2- 172.21.15.100:0/476936268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 0x7f490010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 --2- 172.21.15.100:0/476936268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900100cc0 0x7f4900104d00 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 --2- 172.21.15.100:0/476936268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900100310 0x7f49001006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 -- 172.21.15.100:0/476936268 >> 172.21.15.100:0/476936268 conn(0x7f49000771e0 msgr2=0x7f49000775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 -- 172.21.15.100:0/476936268 shutdown_connections 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.418+0000 7f4906bd7640 1 -- 172.21.15.100:0/476936268 wait complete. 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 Processor -- start 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 -- start start 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900100310 0x7f4900198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 0x7f4900198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 0x7f49001a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a3ea0 con 0x7f4900199250 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a4010 con 0x7f49001053e0 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4906bd7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a41d0 con 0x7f4900100310 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4905174640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 0x7f49001a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4905174640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 0x7f49001a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40250/0 (socket says 172.21.15.100:40250) 2024-03-12T05:28:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4905174640 1 -- 172.21.15.100:0/1280668189 learned_addr learned my addr 172.21.15.100:0/1280668189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 0x7f4900198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4904973640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900100310 0x7f4900198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 -- 172.21.15.100:0/1280668189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900100310 msgr2=0x7f4900198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900100310 0x7f4900198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 -- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 msgr2=0x7f49001a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 0x7f49001a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 -- 172.21.15.100:0/1280668189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009560 con 0x7f49001053e0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f4905174640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 0x7f49001a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.419+0000 7f48fffff640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 0x7f4900198cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f48ec02c170 tx=0x7f48ec02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.420+0000 7f48fdffb640 1 -- 172.21.15.100:0/1280668189 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0042d0 con 0x7f49001053e0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.420+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49001a4450 con 0x7f49001053e0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.420+0000 7f48fdffb640 1 -- 172.21.15.100:0/1280668189 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48ec004430 con 0x7f49001053e0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.420+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49001a4980 con 0x7f49001053e0 2024-03-12T05:28:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.420+0000 7f48fdffb640 1 -- 172.21.15.100:0/1280668189 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec003c30 con 0x7f49001053e0 2024-03-12T05:28:08.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 Processor -- start 2024-03-12T05:28:08.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 -- start start 2024-03-12T05:28:08.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f48fdffb640 1 -- 172.21.15.100:0/1280668189 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f48ec004df0 con 0x7f49001053e0 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666cd8b0 0x558c666cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c66632f90 con 0x558c666cd8b0 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c666304c0 con 0x558c666c4380 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f49097af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6662bca0 con 0x558c666c4d30 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f4904973640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f4904973640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55464/0 (socket says 172.21.15.100:55464) 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f4904973640 1 -- 172.21.15.100:0/2484751808 learned_addr learned my addr 172.21.15.100:0/2484751808 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f48fffff640 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666cd8b0 0x558c666cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.423+0000 7f4905174640 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4904973640 1 -- 172.21.15.100:0/2484751808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 msgr2=0x558c666c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4904973640 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4904973640 1 -- 172.21.15.100:0/2484751808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666cd8b0 msgr2=0x558c666cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4904973640 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666cd8b0 0x558c666cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4904973640 1 -- 172.21.15.100:0/2484751808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c665ee0b0 con 0x558c666c4d30 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f48fdffb640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f48f00a1ba0 0x7f48f00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4904973640 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cd2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009d20 tx=0x7f48f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ef80cf6625c2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f4905174640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f48f00a1ba0 0x7f48f00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f48eaffd640 1 -- 172.21.15.100:0/2484751808 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4016650 con 0x558c666c4d30 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f48fdffb640 1 -- 172.21.15.100:0/1280668189 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f48ec0ef9c0 con 0x7f49001053e0 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f48eaffd640 1 -- 172.21.15.100:0/2484751808 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48f4016c90 con 0x558c666c4d30 2024-03-12T05:28:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.424+0000 7f48eaffd640 1 -- 172.21.15.100:0/2484751808 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f401ec80 con 0x558c666c4d30 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 -- 172.21.15.100:0/2484751808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 msgr2=0x558c666cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f4905174640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f48f00a1ba0 0x7f48f00a4060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f48f80049d0 tx=0x7f48f8009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cd2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009d20 tx=0x7f48f4011810 comp rx=0 tx=0).stop 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 -- 172.21.15.100:0/2484751808 shutdown_connections 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666cd8b0 0x558c666cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cd2e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 --2- 172.21.15.100:0/2484751808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 -- 172.21.15.100:0/2484751808 >> 172.21.15.100:0/2484751808 conn(0x558c666be840 msgr2=0x558c666bec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 -- 172.21.15.100:0/2484751808 shutdown_connections 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.425+0000 7f49097af740 1 -- 172.21.15.100:0/2484751808 wait complete. 2024-03-12T05:28:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 -- start start 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 Processor -- start 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4d30 0x558c666cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c665d0f40 con 0x558c666c4d30 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c662ff4b0 con 0x558c666c4380 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c665b1670 con 0x558c666cd8b0 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f4904973640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f48fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4d30 0x558c666cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f4904973640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47520/0 (socket says 172.21.15.100:47520) 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f4904973640 1 -- 172.21.15.100:0/267257582 learned_addr learned my addr 172.21.15.100:0/267257582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f49097af740 10 client.0 _ll_register_callbacks cb 0x558c6661ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f4905174640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f48fffff640 1 -- 172.21.15.100:0/267257582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 msgr2=0x558c666cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f48fffff640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f48fffff640 1 -- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 msgr2=0x558c666ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f48fffff640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.426+0000 7f48fffff640 1 -- 172.21.15.100:0/267257582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f4009950 con 0x558c666c4d30 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f4904973640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48fffff640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4d30 0x558c666cadf0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ed150 tx=0x7f48ec009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0772fe16030a06 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48dffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48dffff640 1 -- 172.21.15.100:0/267257582 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0bc170 con 0x558c666c4d30 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c66605f40 con 0x558c666c4d30 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48dffff640 1 -- 172.21.15.100:0/267257582 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f48ec0bcd30 con 0x558c666c4d30 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48dffff640 1 -- 172.21.15.100:0/267257582 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ec0bbf90 con 0x558c666c4d30 2024-03-12T05:28:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 10 client.18897 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558c6662a320 con 0x558c666c4d30 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48dffff640 1 -- 172.21.15.100:0/267257582 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c66605f40 con 0x558c666c4d30 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f48dffff640 1 -- 172.21.15.100:0/267257582 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f48ec03c070 con 0x558c666c4d30 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 10 client.18897 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 20 client.18897 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.427+0000 7f49097af740 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c666ddc70 0x558c666e00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.428+0000 7f49097af740 4 client.18897 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.428+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c66622310 con 0x558c666ddc70 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.428+0000 7f4904973640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c666ddc70 0x558c666e00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.428+0000 7f48dffff640 10 client.18897 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.428+0000 7f4904973640 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c666ddc70 0x558c666e00b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207b53647f532d2b server_cookie=d8b4426462180d53 in_seq=0 out_seq=0 2024-03-12T05:28:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f48dffff640 1 -- 172.21.15.100:0/267257582 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c66622310 con 0x558c666ddc70 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f48dffff640 10 client.18897 handle_command_reply: tid=0 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 client.18897 shutdown 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4d30 msgr2=0x558c666cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4d30 0x558c666cadf0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ed150 tx=0x7f48ec009290 comp rx=0 tx=0).stop 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 shutdown_connections 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c666ddc70 0x558c666e00b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f48dffff640 0 client.18897 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4d30 0x558c666cadf0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 --2- 172.21.15.100:0/267257582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4380 0x558c666ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 >> 172.21.15.100:0/267257582 conn(0x558c666be840 msgr2=0x558c666c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 shutdown_connections 2024-03-12T05:28:08.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 1 -- 172.21.15.100:0/267257582 wait complete. 2024-03-12T05:28:08.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.429+0000 7f49097af740 20 client.18897 trim_cache size 0 max 16384 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 Processor -- start 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 -- start start 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666cd8b0 0x558c666c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c666be370 con 0x558c666c4380 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c666e5300 con 0x558c666cd8b0 2024-03-12T05:28:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f49097af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c666c8e00 con 0x558c666c4d30 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40268/0 (socket says 172.21.15.100:40268) 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 -- 172.21.15.100:0/1766398888 learned_addr learned my addr 172.21.15.100:0/1766398888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f48fffff640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4905174640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666cd8b0 0x558c666c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 -- 172.21.15.100:0/1766398888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 msgr2=0x558c666cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 -- 172.21.15.100:0/1766398888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666cd8b0 msgr2=0x558c666c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666cd8b0 0x558c666c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4904973640 1 -- 172.21.15.100:0/1766398888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c665c6c00 con 0x558c666c4380 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f48fffff640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.446+0000 7f4905174640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666cd8b0 0x558c666c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.447+0000 7f4904973640 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666ccfe0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009cf0 tx=0x7f48f4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badb4dd13c397feb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.447+0000 7f48ea7fc640 1 -- 172.21.15.100:0/1766398888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4009ed0 con 0x558c666c4380 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.447+0000 7f48ea7fc640 1 -- 172.21.15.100:0/1766398888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48f401e4c0 con 0x558c666c4380 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.447+0000 7f48ea7fc640 1 -- 172.21.15.100:0/1766398888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f402c1b0 con 0x558c666c4380 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 -- 172.21.15.100:0/1766398888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 msgr2=0x558c666ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666ccfe0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009cf0 tx=0x7f48f4011f80 comp rx=0 tx=0).stop 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 -- 172.21.15.100:0/1766398888 shutdown_connections 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666cd8b0 0x558c666c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666c4d30 0x558c666cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 --2- 172.21.15.100:0/1766398888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666ccfe0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 -- 172.21.15.100:0/1766398888 >> 172.21.15.100:0/1766398888 conn(0x558c666be840 msgr2=0x558c666bec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 -- 172.21.15.100:0/1766398888 shutdown_connections 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 -- 172.21.15.100:0/1766398888 wait complete. 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 -- start start 2024-03-12T05:28:08.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.448+0000 7f49097af740 1 Processor -- start 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4d30 0x558c666d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c667341e0 con 0x558c666c4380 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c66734320 con 0x558c666c4d30 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c66734490 con 0x558c666cd8b0 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 10 client.0 _ll_register_callbacks cb 0x7f48f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:08.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40284/0 (socket says 172.21.15.100:40284) 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 -- 172.21.15.100:0/2965627183 learned_addr learned my addr 172.21.15.100:0/2965627183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4905174640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f48fffff640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4d30 0x558c666d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 -- 172.21.15.100:0/2965627183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 msgr2=0x558c666d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666d3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 -- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4d30 msgr2=0x558c666d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4d30 0x558c666d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 -- 172.21.15.100:0/2965627183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c665ee0b0 con 0x558c666c4380 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f4904973640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666d33d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009d20 tx=0x7f48f401fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1879875743999e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f48dffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f48dffff640 1 -- 172.21.15.100:0/2965627183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f403b070 con 0x558c666c4380 2024-03-12T05:28:08.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c66651900 con 0x558c666c4380 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.449+0000 7f48dffff640 1 -- 172.21.15.100:0/2965627183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f48f4005470 con 0x558c666c4380 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f48dffff640 1 -- 172.21.15.100:0/2965627183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f4005630 con 0x558c666c4380 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 10 client.18903 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558c665ec910 con 0x558c666c4380 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f48dffff640 1 -- 172.21.15.100:0/2965627183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c66651900 con 0x558c666c4380 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f48dffff640 1 -- 172.21.15.100:0/2965627183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f48f403e020 con 0x558c666c4380 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 10 client.18903 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 20 client.18903 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c667357a0 0x558c66737b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 4 client.18903 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558c6632b860 con 0x558c667357a0 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.450+0000 7f48fffff640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c667357a0 0x558c66737b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.451+0000 7f48fffff640 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c667357a0 0x558c66737b90 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3896da988d77f6ed server_cookie=7a93d5931507bb78 in_seq=0 out_seq=0 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.451+0000 7f48dffff640 10 client.18903 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.452+0000 7f48dffff640 1 -- 172.21.15.100:0/2965627183 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558c6632b860 con 0x558c667357a0 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.452+0000 7f48dffff640 10 client.18903 handle_command_reply: tid=0 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.452+0000 7f49097af740 1 client.18903 shutdown 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.452+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 msgr2=0x558c666d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.452+0000 7f49097af740 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666d33d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009d20 tx=0x7f48f401fde0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 shutdown_connections 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558c667357a0 0x558c66737b90 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c666cd8b0 0x558c666d3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c666c4d30 0x558c666d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f48dffff640 0 client.18903 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 --2- 172.21.15.100:0/2965627183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c666c4380 0x558c666d33d0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 >> 172.21.15.100:0/2965627183 conn(0x558c666be840 msgr2=0x558c666c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 shutdown_connections 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 1 -- 172.21.15.100:0/2965627183 wait complete. 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f49097af740 20 client.18903 trim_cache size 0 max 16384 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "7f7e4e51-82f6-45a3-8d2d-098c4691cd28", 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:28:08.452 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:28:08.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f48f00a1ba0 msgr2=0x7f48f00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f48f00a1ba0 0x7f48f00a4060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f48f80049d0 tx=0x7f48f8009500 comp rx=0 tx=0).stop 2024-03-12T05:28:08.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 msgr2=0x7f4900198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 0x7f4900198cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f48ec02c170 tx=0x7f48ec02c5d0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 shutdown_connections 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f48f00a1ba0 0x7f48f00a4060 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900199250 0x7f49001a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49001053e0 0x7f4900198cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 --2- 172.21.15.100:0/1280668189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900100310 0x7f4900198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.453+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 >> 172.21.15.100:0/1280668189 conn(0x7f49000771e0 msgr2=0x7f49000fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.454+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 shutdown_connections 2024-03-12T05:28:08.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.454+0000 7f4906bd7640 1 -- 172.21.15.100:0/1280668189 wait complete. 2024-03-12T05:28:08.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:08 smithi156 ceph-mon[36270]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 887 op/s 2024-03-12T05:28:08.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:28:08.460 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:28:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:08 smithi161 ceph-mon[36043]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 887 op/s 2024-03-12T05:28:08.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 -- 172.21.15.100:0/2336289313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb4105340 msgr2=0x7f0cb410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 --2- 172.21.15.100:0/2336289313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb4105340 0x7f0cb410cf10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009c10 tx=0x7f0cac02bef0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 -- 172.21.15.100:0/2336289313 shutdown_connections 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 --2- 172.21.15.100:0/2336289313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb4105340 0x7f0cb410cf10 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 --2- 172.21.15.100:0/2336289313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4101730 0x7f0cb4104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 --2- 172.21.15.100:0/2336289313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cb4100d80 0x7f0cb4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.650+0000 7f0cbad25640 1 -- 172.21.15.100:0/2336289313 >> 172.21.15.100:0/2336289313 conn(0x7f0cb40752e0 msgr2=0x7f0cb40756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.651+0000 7f0cbad25640 1 -- 172.21.15.100:0/2336289313 shutdown_connections 2024-03-12T05:28:08.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.651+0000 7f0cbad25640 1 -- 172.21.15.100:0/2336289313 wait complete. 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.651+0000 7f0cbad25640 1 Processor -- start 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 -- start start 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cb4100d80 0x7f0cb410af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 0x7f0cb410b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb419f140 0x7f0cb41a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41a1b40 con 0x7f0cb419f140 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41a1cb0 con 0x7f0cb4100d80 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cbad25640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41a1e70 con 0x7f0cb4105340 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 0x7f0cb410b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb92c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb419f140 0x7f0cb41a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 0x7f0cb410b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55506/0 (socket says 172.21.15.100:55506) 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb92c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb419f140 0x7f0cb41a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40304/0 (socket says 172.21.15.100:40304) 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb3fff640 1 -- 172.21.15.100:0/36615141 learned_addr learned my addr 172.21.15.100:0/36615141 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cb4100d80 0x7f0cb410af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb3fff640 1 -- 172.21.15.100:0/36615141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cb4100d80 msgr2=0x7f0cb410af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cb4100d80 0x7f0cb410af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.652+0000 7f0cb3fff640 1 -- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb419f140 msgr2=0x7f0cb41a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb419f140 0x7f0cb41a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cb3fff640 1 -- 172.21.15.100:0/36615141 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac009840 con 0x7f0cb4105340 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 0x7f0cb410b490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000e9c0 tx=0x7f0ca000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cb1ffb640 1 -- 172.21.15.100:0/36615141 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000ccc0 con 0x7f0cb4105340 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb41a20f0 con 0x7f0cb4105340 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cb1ffb640 1 -- 172.21.15.100:0/36615141 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ca000ce20 con 0x7f0cb4105340 2024-03-12T05:28:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.653+0000 7f0cb1ffb640 1 -- 172.21.15.100:0/36615141 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0010430 con 0x7f0cb4105340 2024-03-12T05:28:08.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.654+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb41a25b0 con 0x7f0cb4105340 2024-03-12T05:28:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.657+0000 7f0cb1ffb640 1 -- 172.21.15.100:0/36615141 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0ca0004f90 con 0x7f0cb4105340 2024-03-12T05:28:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.657+0000 7f0cb1ffb640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ca40a1ca0 0x7f0ca40a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.658+0000 7f0cb1ffb640 1 -- 172.21.15.100:0/36615141 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0ca0014070 con 0x7f0cb4105340 2024-03-12T05:28:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.659+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ca40a1ca0 0x7f0ca40a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.659+0000 7f0cbd9d1740 1 Processor -- start 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ca40a1ca0 0x7f0ca40a4160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005fd0 tx=0x7f0ca8005850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 -- start start 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ecd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0e33280 con 0x5631e0ec42b0 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0e307b0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0e2bf90 con 0x5631e0ec4bd0 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cb8ac1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ecd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ecd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55520/0 (socket says 172.21.15.100:55520) 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cb3fff640 1 -- 172.21.15.100:0/4292510070 learned_addr learned my addr 172.21.15.100:0/4292510070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.660+0000 7f0cb8ac1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40306/0 (socket says 172.21.15.100:40306) 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0cb3fff640 1 -- 172.21.15.100:0/4292510070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 msgr2=0x5631e0ecfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0cb3fff640 1 -- 172.21.15.100:0/4292510070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 msgr2=0x5631e0ec4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0cb3fff640 1 -- 172.21.15.100:0/4292510070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e0dea570 con 0x5631e0ec4bd0 2024-03-12T05:28:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ecd270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009230 tx=0x7f0ca009a260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8330ca3de3c5d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0c9effd640 1 -- 172.21.15.100:0/4292510070 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0083ea0 con 0x5631e0ec4bd0 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0c9effd640 1 -- 172.21.15.100:0/4292510070 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ca00b8e90 con 0x5631e0ec4bd0 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.661+0000 7f0c9effd640 1 -- 172.21.15.100:0/4292510070 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00cb4c0 con 0x5631e0ec4bd0 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.662+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/4292510070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 msgr2=0x5631e0ecd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.662+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ecd270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009230 tx=0x7f0ca009a260 comp rx=0 tx=0).stop 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/4292510070 shutdown_connections 2024-03-12T05:28:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ecd270 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/4292510070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/4292510070 >> 172.21.15.100:0/4292510070 conn(0x5631e0ebe990 msgr2=0x5631e0ebed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/4292510070 shutdown_connections 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.663+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/4292510070 wait complete. 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.664+0000 7f0cbd9d1740 1 -- start start 2024-03-12T05:28:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.664+0000 7f0cbd9d1740 1 Processor -- start 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ec42b0 0x5631e0ecd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec4bd0 0x5631e0eca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cb8ac1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ec42b0 0x5631e0ecd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cb8ac1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ec42b0 0x5631e0ecd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:47580/0 (socket says 172.21.15.100:47580) 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec4bd0 0x5631e0eca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec4bd0 0x5631e0eca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40322/0 (socket says 172.21.15.100:40322) 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cb8ac1640 1 -- 172.21.15.100:0/2959424617 learned_addr learned my addr 172.21.15.100:0/2959424617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0aff430 con 0x5631e0ec4bd0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0abbe00 con 0x5631e0ec42b0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.665+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0db0fd0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cbd9d1740 10 client.0 _ll_register_callbacks cb 0x5631e0e06fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 -- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ec42b0 msgr2=0x5631e0ecd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ec42b0 0x5631e0ecd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 -- 172.21.15.100:0/2959424617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec4bd0 msgr2=0x5631e0eca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec4bd0 0x5631e0eca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb41a25b0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecad00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0cac036040 tx=0x7f0cac033df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68f40bdab89df08f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0c93fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.666+0000 7f0c93fff640 1 -- 172.21.15.100:0/2959424617 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0337f0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.667+0000 7f0c93fff640 1 -- 172.21.15.100:0/2959424617 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0cac003970 con 0x5631e0ecd7b0 2024-03-12T05:28:08.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.667+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631e0b2b7e0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.667+0000 7f0c93fff640 1 -- 172.21.15.100:0/2959424617 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cac003b30 con 0x5631e0ecd7b0 2024-03-12T05:28:08.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.667+0000 7f0cbd9d1740 10 client.28207 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:08.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.667+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5631e0e2a610 con 0x5631e0ecd7b0 2024-03-12T05:28:08.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.667+0000 7f0c93fff640 1 -- 172.21.15.100:0/2959424617 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5631e0b2b7e0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0c93fff640 1 -- 172.21.15.100:0/2959424617 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0cac047580 con 0x5631e0ecd7b0 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0cbd9d1740 10 client.28207 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0cbd9d1740 10 client.28207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0cbd9d1740 20 client.28207 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0edfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0cbd9d1740 4 client.28207 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.668+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631e0b68720 con 0x5631e0edd700 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.669+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0edfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.670+0000 7f0c93fff640 10 client.28207 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.670+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0edfaf0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4640b2c36a57488 server_cookie=554dc2c97328c8d8 in_seq=0 out_seq=0 2024-03-12T05:28:08.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.671+0000 7f0c93fff640 1 -- 172.21.15.100:0/2959424617 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631e0b68720 con 0x5631e0edd700 2024-03-12T05:28:08.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.671+0000 7f0c93fff640 10 client.28207 handle_command_reply: tid=0 2024-03-12T05:28:08.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.672+0000 7f0cbd9d1740 1 client.28207 shutdown 2024-03-12T05:28:08.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.673+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ecd7b0 msgr2=0x5631e0ecad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.673+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecad00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0cac036040 tx=0x7f0cac033df0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 shutdown_connections 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0edfaf0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ecd7b0 0x5631e0ecad00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec4bd0 0x5631e0eca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0c93fff640 0 client.28207 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/2959424617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ec42b0 0x5631e0ecd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 >> 172.21.15.100:0/2959424617 conn(0x5631e0ebe990 msgr2=0x5631e0ec0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 shutdown_connections 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.674+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/2959424617 wait complete. 2024-03-12T05:28:08.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.675+0000 7f0cbd9d1740 20 client.28207 trim_cache size 0 max 16384 2024-03-12T05:28:08.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 Processor -- start 2024-03-12T05:28:08.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 -- start start 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0eba120 con 0x5631e0ec42b0 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0b2b7e0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0ec3c00 con 0x5631e0ec4bd0 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cb8ac1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cb8ac1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40330/0 (socket says 172.21.15.100:40330) 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cb8ac1640 1 -- 172.21.15.100:0/3712971662 learned_addr learned my addr 172.21.15.100:0/3712971662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.701+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.702+0000 7f0cb92c2640 1 -- 172.21.15.100:0/3712971662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 msgr2=0x5631e0ed4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.702+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.702+0000 7f0cb92c2640 1 -- 172.21.15.100:0/3712971662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 msgr2=0x5631e0ec95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.702+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.702+0000 7f0cb92c2640 1 -- 172.21.15.100:0/3712971662 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e0dc6ac0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.702+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed4b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02c430 tx=0x7f0cac005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daec780dc9cd4f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.703+0000 7f0c9e7fc640 1 -- 172.21.15.100:0/3712971662 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac033e60 con 0x5631e0ecd7b0 2024-03-12T05:28:08.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.703+0000 7f0c9e7fc640 1 -- 172.21.15.100:0/3712971662 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cac0335f0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/3712971662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 msgr2=0x5631e0ed4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed4b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02c430 tx=0x7f0cac005cf0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/3712971662 shutdown_connections 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed4b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/3712971662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ec95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/3712971662 >> 172.21.15.100:0/3712971662 conn(0x5631e0ebe990 msgr2=0x5631e0ec8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/3712971662 shutdown_connections 2024-03-12T05:28:08.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.704+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/3712971662 wait complete. 2024-03-12T05:28:08.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 -- start start 2024-03-12T05:28:08.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 Processor -- start 2024-03-12T05:28:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ed4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0ed3bc0 con 0x5631e0ec42b0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0f33bc0 con 0x5631e0ecd7b0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0f33d00 con 0x5631e0ec4bd0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.705+0000 7f0cbd9d1740 10 client.0 _ll_register_callbacks cb 0x7f0cac009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55536/0 (socket says 172.21.15.100:55536) 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 -- 172.21.15.100:0/1502641660 learned_addr learned my addr 172.21.15.100:0/1502641660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ed4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb92c2640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 -- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 msgr2=0x5631e0ed34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 -- 172.21.15.100:0/1502641660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 msgr2=0x5631e0ed4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ed4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 -- 172.21.15.100:0/1502641660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631e0e2a610 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0cb3fff640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed2e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ca009a6b0 tx=0x7f0ca009a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9faa480a4a4ae50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0c93fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.706+0000 7f0c93fff640 1 -- 172.21.15.100:0/1502641660 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0083ea0 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ca00b8e90 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0c93fff640 1 -- 172.21.15.100:0/1502641660 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0ca009ae20 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0c93fff640 1 -- 172.21.15.100:0/1502641660 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ca0005450 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0cbd9d1740 10 client.28213 fetch_fsmap learned FSMap version 29 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5631e0dec6a0 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0c93fff640 1 -- 172.21.15.100:0/1502641660 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ca00b8e90 con 0x5631e0ec4bd0 2024-03-12T05:28:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.707+0000 7f0c93fff640 1 -- 172.21.15.100:0/1502641660 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0ca001f060 con 0x5631e0ec4bd0 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cbd9d1740 10 client.28213 fetch_fsmap finished waiting for FSMap version 29 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cbd9d1740 10 client.28213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cbd9d1740 20 client.28213 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0f37390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cbd9d1740 4 client.28213 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5631e0bb7890 con 0x5631e0edd700 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0f37390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0c93fff640 10 client.28213 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.708+0000 7f0cb8ac1640 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0f37390 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673675f1b41a5e6c server_cookie=1db812f548e97b9a in_seq=0 out_seq=0 2024-03-12T05:28:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0c93fff640 1 -- 172.21.15.100:0/1502641660 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5631e0bb7890 con 0x5631e0edd700 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0c93fff640 10 client.28213 handle_command_reply: tid=0 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 client.28213 shutdown 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 msgr2=0x5631e0ed2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed2e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ca009a6b0 tx=0x7f0ca009a390 comp rx=0 tx=0).stop 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 shutdown_connections 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5631e0edd700 0x5631e0f37390 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631e0ecd7b0 0x5631e0ed34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631e0ec4bd0 0x5631e0ed2e40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 --2- 172.21.15.100:0/1502641660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631e0ec42b0 0x5631e0ed4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 >> 172.21.15.100:0/1502641660 conn(0x5631e0ebe990 msgr2=0x5631e0ec05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.709+0000 7f0c93fff640 0 client.28213 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.710+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 shutdown_connections 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.710+0000 7f0cbd9d1740 1 -- 172.21.15.100:0/1502641660 wait complete. 2024-03-12T05:28:08.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.710+0000 7f0cbd9d1740 20 client.28213 trim_cache size 0 max 16384 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: "7f7e4e51-82f6-45a3-8d2d-098c4691cd28": { 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "7f7e4e51-82f6-45a3-8d2d-098c4691cd28", 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:28:08.707 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.711+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ca40a1ca0 msgr2=0x7f0ca40a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.711+0000 7f0cbad25640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ca40a1ca0 0x7f0ca40a4160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005fd0 tx=0x7f0ca8005850 comp rx=0 tx=0).stop 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.711+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 msgr2=0x7f0cb410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.711+0000 7f0cbad25640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 0x7f0cb410b490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000e9c0 tx=0x7f0ca000ee80 comp rx=0 tx=0).stop 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.711+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 shutdown_connections 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.711+0000 7f0cbad25640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ca40a1ca0 0x7f0ca40a4160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.712+0000 7f0cbad25640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb419f140 0x7f0cb41a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.712+0000 7f0cbad25640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb4105340 0x7f0cb410b490 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.712+0000 7f0cbad25640 1 --2- 172.21.15.100:0/36615141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cb4100d80 0x7f0cb410af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:08.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.712+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 >> 172.21.15.100:0/36615141 conn(0x7f0cb40752e0 msgr2=0x7f0cb40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:08.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.713+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 shutdown_connections 2024-03-12T05:28:08.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:08.713+0000 7f0cbad25640 1 -- 172.21.15.100:0/36615141 wait complete. 2024-03-12T05:28:08.720 INFO:tasks.cephfs.filesystem:scrub status for tag:7f7e4e51-82f6-45a3-8d2d-098c4691cd28 - {'path': '/', 'tag': '7f7e4e51-82f6-45a3-8d2d-098c4691cd28', 'options': 'recursive,force'} 2024-03-12T05:28:09.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103718 30.75 MB/sec execute 52 sec latency 295.274 ms 2024-03-12T05:28:09.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:28:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:09.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:09 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:09.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:09 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:09 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104230 30.84 MB/sec execute 53 sec latency 98.054 ms 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:28:10.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:10 smithi100 ceph-mon[32210]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 887 op/s 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:28:10.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:10 smithi156 ceph-mon[36270]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 887 op/s 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-12T05:28:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:10 smithi161 ceph-mon[36043]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 887 op/s 2024-03-12T05:28:11.235 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104691 30.49 MB/sec execute 54 sec latency 115.389 ms 2024-03-12T05:28:11.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:11 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:11.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:11 smithi100 ceph-mon[32210]: osdmap e109: 12 total, 12 up, 12 in 2024-03-12T05:28:11.370 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:11 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:11.370 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:11 smithi161 ceph-mon[36043]: osdmap e109: 12 total, 12 up, 12 in 2024-03-12T05:28:11.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:11 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:11.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:11 smithi156 ceph-mon[36270]: osdmap e109: 12 total, 12 up, 12 in 2024-03-12T05:28:12.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105354 30.54 MB/sec execute 55 sec latency 91.140 ms 2024-03-12T05:28:12.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:12 smithi156 ceph-mon[36270]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-03-12T05:28:12.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:12 smithi161 ceph-mon[36043]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-03-12T05:28:12.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:12 smithi100 ceph-mon[32210]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-03-12T05:28:13.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105981 30.57 MB/sec execute 56 sec latency 166.864 ms 2024-03-12T05:28:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106384 30.39 MB/sec execute 57 sec latency 219.199 ms 2024-03-12T05:28:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107136 30.67 MB/sec execute 58 sec latency 81.479 ms 2024-03-12T05:28:15.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:15 smithi100 ceph-mon[32210]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 986 op/s 2024-03-12T05:28:15.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:15 smithi156 ceph-mon[36270]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 986 op/s 2024-03-12T05:28:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:15 smithi161 ceph-mon[36043]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 986 op/s 2024-03-12T05:28:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107790 30.90 MB/sec execute 59 sec latency 97.368 ms 2024-03-12T05:28:16.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:16 smithi100 ceph-mon[32210]: pgmap v508: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 880 op/s 2024-03-12T05:28:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:16 smithi156 ceph-mon[36270]: pgmap v508: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 880 op/s 2024-03-12T05:28:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:16 smithi161 ceph-mon[36043]: pgmap v508: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 880 op/s 2024-03-12T05:28:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108271 30.87 MB/sec execute 60 sec latency 75.613 ms 2024-03-12T05:28:18.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108627 30.62 MB/sec execute 61 sec latency 94.494 ms 2024-03-12T05:28:19.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108951 30.25 MB/sec execute 62 sec latency 361.205 ms 2024-03-12T05:28:19.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:19 smithi100 ceph-mon[32210]: pgmap v509: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-12T05:28:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:19 smithi156 ceph-mon[36270]: pgmap v509: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-12T05:28:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:19 smithi161 ceph-mon[36043]: pgmap v509: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-12T05:28:20.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109635 30.34 MB/sec execute 63 sec latency 144.431 ms 2024-03-12T05:28:20.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:20 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:28:20.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:20 smithi100 ceph-mon[32210]: pgmap v510: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-12T05:28:20.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:20 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:28:20.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:20 smithi156 ceph-mon[36270]: pgmap v510: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-12T05:28:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:20 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:28:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:20 smithi161 ceph-mon[36043]: pgmap v510: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-12T05:28:21.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110311 30.43 MB/sec execute 64 sec latency 169.608 ms 2024-03-12T05:28:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:21 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:28:21.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:21 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:28:21.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:21 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:28:21.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:21 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:28:21.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:21 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:28:21.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:21 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:28:22.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110915 30.46 MB/sec execute 65 sec latency 146.306 ms 2024-03-12T05:28:22.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:22 smithi100 ceph-mon[32210]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 827 op/s 2024-03-12T05:28:22.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:22 smithi156 ceph-mon[36270]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 827 op/s 2024-03-12T05:28:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:22 smithi161 ceph-mon[36043]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 827 op/s 2024-03-12T05:28:23.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111709 30.71 MB/sec execute 66 sec latency 92.405 ms 2024-03-12T05:28:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112122 30.65 MB/sec execute 67 sec latency 452.928 ms 2024-03-12T05:28:25.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112621 30.60 MB/sec execute 68 sec latency 115.975 ms 2024-03-12T05:28:25.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:25 smithi100 ceph-mon[32210]: pgmap v512: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 542 op/s 2024-03-12T05:28:25.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:25 smithi156 ceph-mon[36270]: pgmap v512: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 542 op/s 2024-03-12T05:28:25.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:25 smithi161 ceph-mon[36043]: pgmap v512: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 542 op/s 2024-03-12T05:28:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113100 30.55 MB/sec execute 69 sec latency 80.412 ms 2024-03-12T05:28:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:26 smithi100 ceph-mon[32210]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 515 op/s 2024-03-12T05:28:26.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:26 smithi156 ceph-mon[36270]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 515 op/s 2024-03-12T05:28:26.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:26 smithi161 ceph-mon[36043]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 515 op/s 2024-03-12T05:28:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113640 30.36 MB/sec execute 70 sec latency 100.674 ms 2024-03-12T05:28:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114420 30.58 MB/sec execute 71 sec latency 77.089 ms 2024-03-12T05:28:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114887 30.50 MB/sec execute 72 sec latency 250.776 ms 2024-03-12T05:28:29.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:29 smithi100 ceph-mon[32210]: pgmap v514: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 814 op/s 2024-03-12T05:28:29.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:29 smithi156 ceph-mon[36270]: pgmap v514: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 814 op/s 2024-03-12T05:28:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:29 smithi161 ceph-mon[36043]: pgmap v514: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 814 op/s 2024-03-12T05:28:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115466 30.56 MB/sec execute 73 sec latency 97.343 ms 2024-03-12T05:28:30.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:30 smithi100 ceph-mon[32210]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 547 op/s 2024-03-12T05:28:30.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:30 smithi156 ceph-mon[36270]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 547 op/s 2024-03-12T05:28:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:30 smithi161 ceph-mon[36043]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 547 op/s 2024-03-12T05:28:31.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116088 30.59 MB/sec execute 74 sec latency 68.403 ms 2024-03-12T05:28:31.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:31 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:28:31.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:31 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:28:31.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:31 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:28:32.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116628 30.68 MB/sec execute 75 sec latency 83.459 ms 2024-03-12T05:28:32.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:32 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:28:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:32 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:28:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:32 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:28:32.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:32 smithi100 ceph-mon[32210]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-03-12T05:28:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:32 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:28:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:32 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:28:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:32 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:28:32.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:32 smithi156 ceph-mon[36270]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-03-12T05:28:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:32 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:28:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:32 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:28:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:32 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:28:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:32 smithi161 ceph-mon[36043]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-03-12T05:28:33.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117047 30.50 MB/sec execute 76 sec latency 132.859 ms 2024-03-12T05:28:34.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117487 30.34 MB/sec execute 77 sec latency 301.480 ms 2024-03-12T05:28:34.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:34 smithi100 ceph-mon[32210]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 579 op/s 2024-03-12T05:28:34.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:34 smithi156 ceph-mon[36270]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 579 op/s 2024-03-12T05:28:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:34 smithi161 ceph-mon[36043]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 579 op/s 2024-03-12T05:28:35.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118199 30.44 MB/sec execute 78 sec latency 124.365 ms 2024-03-12T05:28:36.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118937 30.55 MB/sec execute 79 sec latency 78.886 ms 2024-03-12T05:28:37.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 119588 30.63 MB/sec execute 80 sec latency 109.105 ms 2024-03-12T05:28:37.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:37 smithi100 ceph-mon[32210]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 538 op/s 2024-03-12T05:28:37.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:37 smithi156 ceph-mon[36270]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 538 op/s 2024-03-12T05:28:37.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:37 smithi161 ceph-mon[36043]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 538 op/s 2024-03-12T05:28:38.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 120316 30.73 MB/sec execute 81 sec latency 90.315 ms 2024-03-12T05:28:38.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:38 smithi100 ceph-mon[32210]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 765 op/s 2024-03-12T05:28:38.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:38 smithi156 ceph-mon[36270]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 765 op/s 2024-03-12T05:28:38.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:38 smithi161 ceph-mon[36043]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 765 op/s 2024-03-12T05:28:38.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:28:38.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 -- 172.21.15.100:0/2939187076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 msgr2=0x7f950810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2939187076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950810cf10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9500009cd0 tx=0x7f950002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 -- 172.21.15.100:0/2939187076 shutdown_connections 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2939187076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950810cf10 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2939187076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9508100c20 0x7f9508104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2939187076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508100270 0x7f9508100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 -- 172.21.15.100:0/2939187076 >> 172.21.15.100:0/2939187076 conn(0x7f9508077230 msgr2=0x7f9508077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 -- 172.21.15.100:0/2939187076 shutdown_connections 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.879+0000 7f950f4f4640 1 -- 172.21.15.100:0/2939187076 wait complete. 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 Processor -- start 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 -- start start 2024-03-12T05:28:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9508100270 0x7f950819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95081975e0 0x7f9508197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9508197fd0 con 0x7f9508105340 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9508198140 con 0x7f9508100270 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950f4f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9508198300 con 0x7f95081975e0 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950ca8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950ca8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52120/0 (socket says 172.21.15.100:52120) 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950ca8f640 1 -- 172.21.15.100:0/2123218803 learned_addr learned my addr 172.21.15.100:0/2123218803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950d290640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9508100270 0x7f950819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.880+0000 7f950da91640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95081975e0 0x7f9508197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950ca8f640 1 -- 172.21.15.100:0/2123218803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95081975e0 msgr2=0x7f9508197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950ca8f640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95081975e0 0x7f9508197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950ca8f640 1 -- 172.21.15.100:0/2123218803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9508100270 msgr2=0x7f950819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950ca8f640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9508100270 0x7f950819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950ca8f640 1 -- 172.21.15.100:0/2123218803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500009900 con 0x7f9508105340 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950ca8f640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950819d970 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e9c0 tx=0x7f94f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f94fe7fc640 1 -- 172.21.15.100:0/2123218803 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8009700 con 0x7f9508105340 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95081a42f0 con 0x7f9508105340 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f94fe7fc640 1 -- 172.21.15.100:0/2123218803 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94f800cd80 con 0x7f9508105340 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f94fe7fc640 1 -- 172.21.15.100:0/2123218803 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8010430 con 0x7f9508105340 2024-03-12T05:28:38.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.881+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95081a4850 con 0x7f9508105340 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.883+0000 7f94fe7fc640 1 -- 172.21.15.100:0/2123218803 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f94f8005000 con 0x7f9508105340 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.884+0000 7f94fe7fc640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94f40a1c20 0x7f94f40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.884+0000 7f94fe7fc640 1 -- 172.21.15.100:0/2123218803 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f94f8014070 con 0x7f9508105340 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.884+0000 7f950d290640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94f40a1c20 0x7f94f40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.884+0000 7f95121a5740 1 Processor -- start 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.884+0000 7f95121a5740 1 -- start start 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.884+0000 7f950d290640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94f40a1c20 0x7f94f40a40e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9504009bd0 tx=0x7f9504005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f95121a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607bdc2b0 0x556607bdc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f95121a5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f95121a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 0x556607be7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f95121a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607b4b280 con 0x556607be57b0 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f95121a5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607b487b0 con 0x556607bdcbd0 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f95121a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607b43f90 con 0x556607bdc2b0 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 0x556607be7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950d290640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607bdc2b0 0x556607bdc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950d290640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607bdc2b0 0x556607bdc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:41918/0 (socket says 172.21.15.100:41918) 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950d290640 1 -- 172.21.15.100:0/3993755785 learned_addr learned my addr 172.21.15.100:0/3993755785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 0x556607be7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52136/0 (socket says 172.21.15.100:52136) 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950ca8f640 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 -- 172.21.15.100:0/3993755785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607bdc2b0 msgr2=0x556607bdc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607bdc2b0 0x556607bdc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 -- 172.21.15.100:0/3993755785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 msgr2=0x556607be5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 -- 172.21.15.100:0/3993755785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556607b02570 con 0x556607be57b0 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950ca8f640 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f950da91640 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 0x556607be7ba0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f950002c630 tx=0x7f9500032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2449ab15afd81d3c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f94f37fe640 1 -- 172.21.15.100:0/3993755785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500034440 con 0x556607be57b0 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f94f37fe640 1 -- 172.21.15.100:0/3993755785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95000345a0 con 0x556607be57b0 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.885+0000 7f94f37fe640 1 -- 172.21.15.100:0/3993755785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500003b70 con 0x556607be57b0 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 -- 172.21.15.100:0/3993755785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 msgr2=0x556607be7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 0x556607be7ba0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f950002c630 tx=0x7f9500032d70 comp rx=0 tx=0).stop 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 -- 172.21.15.100:0/3993755785 shutdown_connections 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607be57b0 0x556607be7ba0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 --2- 172.21.15.100:0/3993755785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607bdc2b0 0x556607bdc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 -- 172.21.15.100:0/3993755785 >> 172.21.15.100:0/3993755785 conn(0x556607bd6990 msgr2=0x556607bd6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:38.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 -- 172.21.15.100:0/3993755785 shutdown_connections 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 -- 172.21.15.100:0/3993755785 wait complete. 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 -- start start 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.886+0000 7f95121a5740 1 Processor -- start 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607817430 con 0x556607bdc2b0 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566077d3e00 con 0x556607bdcbd0 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607ac8fd0 con 0x556607be57b0 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f95121a5740 10 client.0 _ll_register_callbacks cb 0x556607b1efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52146/0 (socket says 172.21.15.100:52146) 2024-03-12T05:28:38.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 -- 172.21.15.100:0/1344193733 learned_addr learned my addr 172.21.15.100:0/1344193733 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950da91640 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 -- 172.21.15.100:0/1344193733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 msgr2=0x556607be2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 -- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 msgr2=0x556607be2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 -- 172.21.15.100:0/1344193733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950810d720 con 0x556607bdc2b0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f950d290640 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be5190 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9504003a10 tx=0x7f9504004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769fafc3811b9aef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.887+0000 7f94f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f94f0ff9640 1 -- 172.21.15.100:0/1344193733 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950401fdc0 con 0x556607bdc2b0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f94f0ff9640 1 -- 172.21.15.100:0/1344193733 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95040273e0 con 0x556607bdc2b0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566078437e0 con 0x556607bdc2b0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f94f0ff9640 1 -- 172.21.15.100:0/1344193733 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95040275a0 con 0x556607bdc2b0 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f95121a5740 10 client.18933 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:38.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556607b42610 con 0x556607bdc2b0 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.888+0000 7f94f0ff9640 1 -- 172.21.15.100:0/1344193733 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5566078437e0 con 0x556607bdc2b0 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f94f0ff9640 1 -- 172.21.15.100:0/1344193733 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f95040288b0 con 0x556607bdc2b0 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f95121a5740 10 client.18933 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f95121a5740 10 client.18933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f95121a5740 20 client.18933 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f95121a5740 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607bf5960 0x556607bf7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f95121a5740 4 client.18933 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556607880720 con 0x556607bf5960 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f950ca8f640 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607bf5960 0x556607bf7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f94f0ff9640 10 client.18933 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.889+0000 7f950ca8f640 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607bf5960 0x556607bf7d70 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd75c322870497a server_cookie=c2140e055cfbf402 in_seq=0 out_seq=0 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f94f0ff9640 1 -- 172.21.15.100:0/1344193733 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556607880720 con 0x556607bf5960 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f94f0ff9640 10 client.18933 handle_command_reply: tid=0 2024-03-12T05:28:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 client.18933 shutdown 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 msgr2=0x556607be5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be5190 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9504003a10 tx=0x7f9504004730 comp rx=0 tx=0).stop 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 shutdown_connections 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607bf5960 0x556607bf7d70 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 --2- 172.21.15.100:0/1344193733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be5190 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f94f0ff9640 0 client.18933 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 >> 172.21.15.100:0/1344193733 conn(0x556607bd6990 msgr2=0x556607bd8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 shutdown_connections 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.890+0000 7f95121a5740 1 -- 172.21.15.100:0/1344193733 wait complete. 2024-03-12T05:28:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.891+0000 7f95121a5740 20 client.18933 trim_cache size 0 max 16384 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 Processor -- start 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 -- start start 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607bd79e0 con 0x556607bdc2b0 2024-03-12T05:28:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607be3630 con 0x556607bdcbd0 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f95121a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607bd22b0 con 0x556607be57b0 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52156/0 (socket says 172.21.15.100:52156) 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 -- 172.21.15.100:0/964635704 learned_addr learned my addr 172.21.15.100:0/964635704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950ca8f640 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 -- 172.21.15.100:0/964635704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 msgr2=0x556607be5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 -- 172.21.15.100:0/964635704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 msgr2=0x556607be4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 -- 172.21.15.100:0/964635704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556607adeac0 con 0x556607bdc2b0 2024-03-12T05:28:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.908+0000 7f950d290640 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be24c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9504028be0 tx=0x7f950400e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2043e97f941e61c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.909+0000 7f94f2ffd640 1 -- 172.21.15.100:0/964635704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950401f800 con 0x556607bdc2b0 2024-03-12T05:28:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.909+0000 7f94f2ffd640 1 -- 172.21.15.100:0/964635704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9504028920 con 0x556607bdc2b0 2024-03-12T05:28:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.909+0000 7f94f2ffd640 1 -- 172.21.15.100:0/964635704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950400e920 con 0x556607bdc2b0 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.909+0000 7f95121a5740 1 -- 172.21.15.100:0/964635704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 msgr2=0x556607be24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be24c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9504028be0 tx=0x7f950400e460 comp rx=0 tx=0).stop 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- 172.21.15.100:0/964635704 shutdown_connections 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607be5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdcbd0 0x556607be4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- 172.21.15.100:0/964635704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdc2b0 0x556607be24c0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- 172.21.15.100:0/964635704 >> 172.21.15.100:0/964635704 conn(0x556607bd6990 msgr2=0x556607be1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- 172.21.15.100:0/964635704 shutdown_connections 2024-03-12T05:28:38.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- 172.21.15.100:0/964635704 wait complete. 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- start start 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 Processor -- start 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdc2b0 0x556607be9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 0x556607be9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607c47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607c48380 con 0x556607bdcbd0 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607c484f0 con 0x556607bdc2b0 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556607c486b0 con 0x556607be57b0 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.910+0000 7f95121a5740 10 client.0 _ll_register_callbacks cb 0x7f94f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 0x556607be9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 0x556607be9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52172/0 (socket says 172.21.15.100:52172) 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 -- 172.21.15.100:0/2980966667 learned_addr learned my addr 172.21.15.100:0/2980966667 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950d290640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdc2b0 0x556607be9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950da91640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607c47e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 -- 172.21.15.100:0/2980966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 msgr2=0x556607c47e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607c47e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 -- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdc2b0 msgr2=0x556607be9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdc2b0 0x556607be9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 -- 172.21.15.100:0/2980966667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556607b42610 con 0x556607bdcbd0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f950ca8f640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 0x556607be9d70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f94f809a4a0 tx=0x7f94f8085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96012029efcb2f8d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f94f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f94f0ff9640 1 -- 172.21.15.100:0/2980966667 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f809e8f0 con 0x556607bdcbd0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f94f0ff9640 1 -- 172.21.15.100:0/2980966667 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f94f80a2f40 con 0x556607bdcbd0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95000345a0 con 0x556607bdcbd0 2024-03-12T05:28:38.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f94f0ff9640 1 -- 172.21.15.100:0/2980966667 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94f8098aa0 con 0x556607bdcbd0 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f95121a5740 10 client.18939 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.911+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556607b046a0 con 0x556607bdcbd0 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f94f0ff9640 1 -- 172.21.15.100:0/2980966667 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95000345a0 con 0x556607bdcbd0 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f94f0ff9640 1 -- 172.21.15.100:0/2980966667 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f94f801f060 con 0x556607bdcbd0 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f95121a5740 10 client.18939 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f95121a5740 10 client.18939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f95121a5740 20 client.18939 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f95121a5740 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607c522c0 0x556607c546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f95121a5740 4 client.18939 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566078cf890 con 0x556607c522c0 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f950d290640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607c522c0 0x556607c546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f94f0ff9640 10 client.18939 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:38.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.912+0000 7f950d290640 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607c522c0 0x556607c546b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f96eca91cd06df server_cookie=638850720693454b in_seq=0 out_seq=0 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f94f0ff9640 1 -- 172.21.15.100:0/2980966667 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5566078cf890 con 0x556607c522c0 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f94f0ff9640 10 client.18939 handle_command_reply: tid=0 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 client.18939 shutdown 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 msgr2=0x556607be9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 0x556607be9d70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f94f809a4a0 tx=0x7f94f8085040 comp rx=0 tx=0).stop 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 shutdown_connections 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556607c522c0 0x556607c546b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607be57b0 0x556607c47e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556607bdcbd0 0x556607be9d70 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 --2- 172.21.15.100:0/2980966667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556607bdc2b0 0x556607be9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f94f0ff9640 0 client.18939 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 >> 172.21.15.100:0/2980966667 conn(0x556607bd6990 msgr2=0x556607bd5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 shutdown_connections 2024-03-12T05:28:38.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 1 -- 172.21.15.100:0/2980966667 wait complete. 2024-03-12T05:28:38.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.913+0000 7f95121a5740 20 client.18939 trim_cache size 0 max 16384 2024-03-12T05:28:38.916 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:28:38.916 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:28:38.916 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:28:38.916 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:28:38.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94f40a1c20 msgr2=0x7f94f40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94f40a1c20 0x7f94f40a40e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9504009bd0 tx=0x7f9504005f90 comp rx=0 tx=0).stop 2024-03-12T05:28:38.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 msgr2=0x7f950819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:38.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950819d970 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e9c0 tx=0x7f94f800ee80 comp rx=0 tx=0).stop 2024-03-12T05:28:38.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 shutdown_connections 2024-03-12T05:28:38.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94f40a1c20 0x7f94f40a40e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95081975e0 0x7f9508197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9508105340 0x7f950819d970 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 --2- 172.21.15.100:0/2123218803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9508100270 0x7f950819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:38.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 >> 172.21.15.100:0/2123218803 conn(0x7f9508077230 msgr2=0x7f9508068520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:38.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 shutdown_connections 2024-03-12T05:28:38.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:38.914+0000 7f950f4f4640 1 -- 172.21.15.100:0/2123218803 wait complete. 2024-03-12T05:28:38.919 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:28:38.919 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:28:39.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 -- 172.21.15.100:0/3136120003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a80102d30 msgr2=0x7f6a8010b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 --2- 172.21.15.100:0/3136120003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a80102d30 0x7f6a8010b210 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009900 tx=0x7f6a6c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 -- 172.21.15.100:0/3136120003 shutdown_connections 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 --2- 172.21.15.100:0/3136120003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a8010b7e0 0x7f6a8010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 --2- 172.21.15.100:0/3136120003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a80102d30 0x7f6a8010b210 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 --2- 172.21.15.100:0/3136120003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a80102410 0x7f6a801027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 -- 172.21.15.100:0/3136120003 >> 172.21.15.100:0/3136120003 conn(0x7f6a80077180 msgr2=0x7f6a80077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 -- 172.21.15.100:0/3136120003 shutdown_connections 2024-03-12T05:28:39.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.032+0000 7f6a86df4640 1 -- 172.21.15.100:0/3136120003 wait complete. 2024-03-12T05:28:39.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 Processor -- start 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 -- start start 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a80102410 0x7f6a8019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a80102d30 0x7f6a8019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 0x7f6a80197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80197c60 con 0x7f6a8010b7e0 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80197dd0 con 0x7f6a80102d30 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a86df4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80197f90 con 0x7f6a80102410 2024-03-12T05:28:39.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 0x7f6a80197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 0x7f6a80197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52190/0 (socket says 172.21.15.100:52190) 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 -- 172.21.15.100:0/2190146238 learned_addr learned my addr 172.21.15.100:0/2190146238 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a84b90640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a80102410 0x7f6a8019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a7ffff640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a80102d30 0x7f6a8019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 -- 172.21.15.100:0/2190146238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a80102410 msgr2=0x7f6a8019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a80102410 0x7f6a8019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 -- 172.21.15.100:0/2190146238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a80102d30 msgr2=0x7f6a8019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a80102d30 0x7f6a8019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 -- 172.21.15.100:0/2190146238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009560 con 0x7f6a8010b7e0 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.033+0000 7f6a85391640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 0x7f6a80197610 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800eb10 tx=0x7f6a7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.034+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004b20 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.034+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a78004c80 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.034+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a80198270 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.034+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78010650 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.034+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801a4500 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.035+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80067f70 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.036+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6a78005020 con 0x7f6a8010b7e0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.036+0000 7f6a7dffb640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a700a1c20 0x7f6a700a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.036+0000 7f6a84b90640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a700a1c20 0x7f6a700a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.037+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6a78014070 con 0x7f6a8010b7e0 2024-03-12T05:28:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.037+0000 7f6a84b90640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a700a1c20 0x7f6a700a40e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009bd0 tx=0x7f6a74005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.041+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a78096bc0 con 0x7f6a8010b7e0 2024-03-12T05:28:39.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 120775 30.76 MB/sec execute 82 sec latency 347.296 ms 2024-03-12T05:28:39.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.310+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a80198b40 con 0x7f6a8010b7e0 2024-03-12T05:28:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.311+0000 7f6a7dffb640 1 -- 172.21.15.100:0/2190146238 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f6a7809a280 con 0x7f6a8010b7e0 2024-03-12T05:28:39.306 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:28:39.306 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:28:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:28:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a700a1c20 msgr2=0x7f6a700a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a700a1c20 0x7f6a700a40e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009bd0 tx=0x7f6a74005f90 comp rx=0 tx=0).stop 2024-03-12T05:28:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 msgr2=0x7f6a80197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 0x7f6a80197610 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800eb10 tx=0x7f6a7800c2d0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 shutdown_connections 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a700a1c20 0x7f6a700a40e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8010b7e0 0x7f6a80197610 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a80102d30 0x7f6a8019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 --2- 172.21.15.100:0/2190146238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a80102410 0x7f6a8019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 >> 172.21.15.100:0/2190146238 conn(0x7f6a80077180 msgr2=0x7f6a800fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 shutdown_connections 2024-03-12T05:28:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.314+0000 7f6a86df4640 1 -- 172.21.15.100:0/2190146238 wait complete. 2024-03-12T05:28:39.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:28:39.319 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:28:39.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.436+0000 7f1367055640 1 -- 172.21.15.100:0/1149230858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360100790 msgr2=0x7f1360100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.436+0000 7f1367055640 1 --2- 172.21.15.100:0/1149230858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360100790 0x7f1360100c10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f134c009900 tx=0x7f134c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- 172.21.15.100:0/1149230858 shutdown_connections 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 --2- 172.21.15.100:0/1149230858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1360101150 0x7f136010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 --2- 172.21.15.100:0/1149230858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360100790 0x7f1360100c10 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 --2- 172.21.15.100:0/1149230858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1360105070 0x7f1360105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- 172.21.15.100:0/1149230858 >> 172.21.15.100:0/1149230858 conn(0x7f1360076640 msgr2=0x7f13600fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- 172.21.15.100:0/1149230858 shutdown_connections 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- 172.21.15.100:0/1149230858 wait complete. 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 Processor -- start 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- start start 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1360100790 0x7f13601965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1360101150 0x7f1360196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 0x7f136019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136019b4b0 con 0x7f1360105070 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136019b620 con 0x7f1360101150 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f1367055640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136019b7e0 con 0x7f1360100790 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.437+0000 7f13655f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 0x7f136019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f1364df1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1360100790 0x7f13601965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 0x7f136019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52202/0 (socket says 172.21.15.100:52202) 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f135ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1360101150 0x7f1360196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 -- 172.21.15.100:0/1006629771 learned_addr learned my addr 172.21.15.100:0/1006629771 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 -- 172.21.15.100:0/1006629771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1360100790 msgr2=0x7f13601965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1360100790 0x7f13601965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 -- 172.21.15.100:0/1006629771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1360101150 msgr2=0x7f1360196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1360101150 0x7f1360196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 -- 172.21.15.100:0/1006629771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c009560 con 0x7f1360105070 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f13655f2640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 0x7f136019aeb0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f135800eb10 tx=0x7f135800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f135dffb640 1 -- 172.21.15.100:0/1006629771 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358004b20 con 0x7f1360105070 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f135dffb640 1 -- 172.21.15.100:0/1006629771 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1358004c80 con 0x7f1360105070 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136019bac0 con 0x7f1360105070 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f135dffb640 1 -- 172.21.15.100:0/1006629771 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358010650 con 0x7f1360105070 2024-03-12T05:28:39.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.438+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13600ff860 con 0x7f1360105070 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.440+0000 7f135dffb640 1 -- 172.21.15.100:0/1006629771 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1358005020 con 0x7f1360105070 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 Processor -- start 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 -- start start 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f135dffb640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13500a1ba0 0x7f13500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426165630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e426170b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e426120960 con 0x55e42616e750 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4260d4310 con 0x55e426165b70 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1369c2d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4260d1840 con 0x55e426165250 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e426170b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f135ffff640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13500a1ba0 0x7f13500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f1364df1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e426170b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52212/0 (socket says 172.21.15.100:52212) 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 -- 172.21.15.100:0/3515283790 learned_addr learned my addr 172.21.15.100:0/3515283790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f135dffb640 1 -- 172.21.15.100:0/1006629771 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1358014070 con 0x7f1360105070 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f135ffff640 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 -- 172.21.15.100:0/3515283790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 msgr2=0x55e426165630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426165630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 -- 172.21.15.100:0/3515283790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 msgr2=0x55e42616e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.441+0000 7f13655f2640 1 -- 172.21.15.100:0/3515283790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e42608f060 con 0x55e42616e750 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f135ffff640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13500a1ba0 0x7f13500a4060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f134c006010 tx=0x7f134c02c8a0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f13655f2640 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e426170b40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f135809a6d0 tx=0x7f135800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37b55fdfeaa9ab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f134affd640 1 -- 172.21.15.100:0/3515283790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358065480 con 0x55e42616e750 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f134affd640 1 -- 172.21.15.100:0/3515283790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13580ba040 con 0x55e42616e750 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f134affd640 1 -- 172.21.15.100:0/3515283790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135809ebb0 con 0x55e42616e750 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f1369c2d740 1 -- 172.21.15.100:0/3515283790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 msgr2=0x55e426170b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.442+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e426170b40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f135809a6d0 tx=0x7f135800cee0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- 172.21.15.100:0/3515283790 shutdown_connections 2024-03-12T05:28:39.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e426170b40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3515283790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426165630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- 172.21.15.100:0/3515283790 >> 172.21.15.100:0/3515283790 conn(0x55e42615f980 msgr2=0x55e42615fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- 172.21.15.100:0/3515283790 shutdown_connections 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- 172.21.15.100:0/3515283790 wait complete. 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- start start 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 Processor -- start 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 0x55e42616b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e42616c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4260cd020 con 0x55e426165250 2024-03-12T05:28:39.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e425da0430 con 0x55e426165b70 2024-03-12T05:28:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e425d5ce00 con 0x55e42616e750 2024-03-12T05:28:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1369c2d740 10 client.0 _ll_register_callbacks cb 0x55e426074cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1364df1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 0x55e42616b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1364df1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 0x55e42616b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52216/0 (socket says 172.21.15.100:52216) 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f1364df1640 1 -- 172.21.15.100:0/2556380934 learned_addr learned my addr 172.21.15.100:0/2556380934 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.443+0000 7f135ffff640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f13655f2640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e42616c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1364df1640 1 -- 172.21.15.100:0/2556380934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 msgr2=0x55e42616c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1364df1640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e42616c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1364df1640 1 -- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 msgr2=0x55e42616bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1364df1640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1364df1640 1 -- 172.21.15.100:0/2556380934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13600ff860 con 0x55e426165250 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1364df1640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 0x55e42616b810 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1354012840 tx=0x7f1354012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18725dd51713794c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f133ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f133ffff640 1 -- 172.21.15.100:0/2556380934 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354017070 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f133ffff640 1 -- 172.21.15.100:0/2556380934 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f135400f8c0 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e425dcc7e0 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f133ffff640 1 -- 172.21.15.100:0/2556380934 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13540122e0 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1369c2d740 10 client.18957 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e4260cb6a0 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.444+0000 7f133ffff640 1 -- 172.21.15.100:0/2556380934 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e425dcc7e0 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f133ffff640 1 -- 172.21.15.100:0/2556380934 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1354010cf0 con 0x55e426165250 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f1369c2d740 10 client.18957 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f1369c2d740 10 client.18957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f1369c2d740 20 client.18957 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e42617eca0 0x55e4261810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f1369c2d740 4 client.18957 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e425e09720 con 0x55e42617eca0 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f135ffff640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e42617eca0 0x55e4261810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f135ffff640 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e42617eca0 0x55e4261810c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34c68a155a0da426 server_cookie=4818a6d9748526d in_seq=0 out_seq=0 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.445+0000 7f133ffff640 10 client.18957 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f133ffff640 1 -- 172.21.15.100:0/2556380934 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e425e09720 con 0x55e42617eca0 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f133ffff640 10 client.18957 handle_command_reply: tid=0 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 client.18957 shutdown 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 msgr2=0x55e42616b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 0x55e42616b810 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1354012840 tx=0x7f1354012d00 comp rx=0 tx=0).stop 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 shutdown_connections 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e42617eca0 0x55e4261810c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e42616c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e42616bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2556380934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165250 0x55e42616b810 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f133ffff640 0 client.18957 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 >> 172.21.15.100:0/2556380934 conn(0x55e42615f980 msgr2=0x55e4261616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 shutdown_connections 2024-03-12T05:28:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 1 -- 172.21.15.100:0/2556380934 wait complete. 2024-03-12T05:28:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.449+0000 7f1369c2d740 20 client.18957 trim_cache size 0 max 16384 2024-03-12T05:28:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 Processor -- start 2024-03-12T05:28:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 -- start start 2024-03-12T05:28:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165250 0x55e42616df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 0x55e42616c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e426168a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e426052130 con 0x55e426165b70 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e426186310 con 0x55e426165250 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.461+0000 7f1369c2d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42615b080 con 0x55e42616e750 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 0x55e42616c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 0x55e42616c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52226/0 (socket says 172.21.15.100:52226) 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 -- 172.21.15.100:0/2193641686 learned_addr learned my addr 172.21.15.100:0/2193641686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f13655f2640 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e426168a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 -- 172.21.15.100:0/2193641686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 msgr2=0x55e426168a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e426168a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 -- 172.21.15.100:0/2193641686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165250 msgr2=0x55e42616df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165250 0x55e42616df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 -- 172.21.15.100:0/2193641686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e426067c20 con 0x55e426165b70 2024-03-12T05:28:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f135ffff640 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 0x55e42616c0b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f134c02c170 tx=0x7f134c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cc20e74fda597d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f134a7fc640 1 -- 172.21.15.100:0/2193641686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c004510 con 0x55e426165b70 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f134a7fc640 1 -- 172.21.15.100:0/2193641686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f134c02cbb0 con 0x55e426165b70 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.462+0000 7f134a7fc640 1 -- 172.21.15.100:0/2193641686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c054560 con 0x55e426165b70 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 -- 172.21.15.100:0/2193641686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 msgr2=0x55e42616c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 0x55e42616c0b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f134c02c170 tx=0x7f134c046000 comp rx=0 tx=0).stop 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 -- 172.21.15.100:0/2193641686 shutdown_connections 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42616e750 0x55e426168a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e426165b70 0x55e42616c0b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 --2- 172.21.15.100:0/2193641686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165250 0x55e42616df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 -- 172.21.15.100:0/2193641686 >> 172.21.15.100:0/2193641686 conn(0x55e42615f980 msgr2=0x55e426166ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 -- 172.21.15.100:0/2193641686 shutdown_connections 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 -- 172.21.15.100:0/2193641686 wait complete. 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 -- start start 2024-03-12T05:28:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.463+0000 7f1369c2d740 1 Processor -- start 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426174c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e426175180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e4261ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4261ce840 con 0x55e42616e750 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4261ce9b0 con 0x55e426165b70 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4261ceb70 con 0x55e426165250 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 10 client.0 _ll_register_callbacks cb 0x7f1358096630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e4261ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1364df1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426174c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e4261ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52238/0 (socket says 172.21.15.100:52238) 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1364df1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426174c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42022/0 (socket says 172.21.15.100:42022) 2024-03-12T05:28:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 -- 172.21.15.100:0/3428255947 learned_addr learned my addr 172.21.15.100:0/3428255947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f135ffff640 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e426175180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 -- 172.21.15.100:0/3428255947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 msgr2=0x55e426174c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426174c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 -- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 msgr2=0x55e426175180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e426175180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 -- 172.21.15.100:0/3428255947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c049070 con 0x55e42616e750 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f13655f2640 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e4261ce300 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f135809a4a0 tx=0x7f13580a3170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e3bcb991682656 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f133ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f133ffff640 1 -- 172.21.15.100:0/3428255947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135809eb20 con 0x55e42616e750 2024-03-12T05:28:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f133ffff640 1 -- 172.21.15.100:0/3428255947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f13580853b0 con 0x55e42616e750 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13580ba040 con 0x55e42616e750 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.464+0000 7f133ffff640 1 -- 172.21.15.100:0/3428255947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135807b080 con 0x55e42616e750 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 10 client.18963 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e42608d8c0 con 0x55e42616e750 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f133ffff640 1 -- 172.21.15.100:0/3428255947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13580ba040 con 0x55e42616e750 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f133ffff640 1 -- 172.21.15.100:0/3428255947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f135801f060 con 0x55e42616e750 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 10 client.18963 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 10 client.18963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 20 client.18963 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e4261d7220 0x55e4261d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 4 client.18963 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e425e58890 con 0x55e4261d7220 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1364df1640 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e4261d7220 0x55e4261d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f133ffff640 10 client.18963 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.465+0000 7f1364df1640 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e4261d7220 0x55e4261d9640 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c059b0de7f72519c server_cookie=d2d9de536fc26768 in_seq=0 out_seq=0 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.468+0000 7f133ffff640 1 -- 172.21.15.100:0/3428255947 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e425e58890 con 0x55e4261d7220 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.468+0000 7f133ffff640 10 client.18963 handle_command_reply: tid=0 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.468+0000 7f1369c2d740 1 client.18963 shutdown 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 msgr2=0x55e4261ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e4261ce300 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f135809a4a0 tx=0x7f13580a3170 comp rx=0 tx=0).stop 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 shutdown_connections 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55e4261d7220 0x55e4261d9640 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42616e750 0x55e4261ce300 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e426165b70 0x55e426175180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f133ffff640 0 client.18963 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 --2- 172.21.15.100:0/3428255947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e426165250 0x55e426174c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 >> 172.21.15.100:0/3428255947 conn(0x55e42615f980 msgr2=0x55e42615e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 shutdown_connections 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 1 -- 172.21.15.100:0/3428255947 wait complete. 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1369c2d740 20 client.18963 trim_cache size 0 max 16384 2024-03-12T05:28:39.468 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:28:39.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13500a1ba0 msgr2=0x7f13500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13500a1ba0 0x7f13500a4060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f134c006010 tx=0x7f134c02c8a0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 msgr2=0x7f136019aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 0x7f136019aeb0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f135800eb10 tx=0x7f135800c2d0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 shutdown_connections 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f13500a1ba0 0x7f13500a4060 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1360105070 0x7f136019aeb0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1360101150 0x7f1360196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 --2- 172.21.15.100:0/1006629771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1360100790 0x7f13601965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.469+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 >> 172.21.15.100:0/1006629771 conn(0x7f1360076640 msgr2=0x7f1360102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.470+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 shutdown_connections 2024-03-12T05:28:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.470+0000 7f1367055640 1 -- 172.21.15.100:0/1006629771 wait complete. 2024-03-12T05:28:39.475 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:28:39.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:28:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 -- 172.21.15.100:0/1625136369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 msgr2=0x7f263c10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 --2- 172.21.15.100:0/1625136369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c10cf00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2634009cd0 tx=0x7f263402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 -- 172.21.15.100:0/1625136369 shutdown_connections 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 --2- 172.21.15.100:0/1625136369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c10cf00 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 --2- 172.21.15.100:0/1625136369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f263c100c10 0x7f263c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 --2- 172.21.15.100:0/1625136369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c100260 0x7f263c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 -- 172.21.15.100:0/1625136369 >> 172.21.15.100:0/1625136369 conn(0x7f263c077180 msgr2=0x7f263c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 -- 172.21.15.100:0/1625136369 shutdown_connections 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 -- 172.21.15.100:0/1625136369 wait complete. 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 Processor -- start 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.595+0000 7f2643815640 1 -- start start 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f263c100260 0x7f263c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c19af90 0x7f263c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19b980 con 0x7f263c105330 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19baf0 con 0x7f263c100260 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19bc60 con 0x7f263c19af90 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52258/0 (socket says 172.21.15.100:52258) 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 -- 172.21.15.100:0/2904271845 learned_addr learned my addr 172.21.15.100:0/2904271845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f26415b1640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f263c100260 0x7f263c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2641db2640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c19af90 0x7f263c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 -- 172.21.15.100:0/2904271845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c19af90 msgr2=0x7f263c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c19af90 0x7f263c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 -- 172.21.15.100:0/2904271845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f263c100260 msgr2=0x7f263c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f263c100260 0x7f263c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 -- 172.21.15.100:0/2904271845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634009900 con 0x7f263c105330 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2640db0640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c196c60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f262c00e9c0 tx=0x7f262c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f26327fc640 1 -- 172.21.15.100:0/2904271845 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c009700 con 0x7f263c105330 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f26327fc640 1 -- 172.21.15.100:0/2904271845 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f262c00cd80 con 0x7f263c105330 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0682e0 con 0x7f263c105330 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f26327fc640 1 -- 172.21.15.100:0/2904271845 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c010430 con 0x7f263c105330 2024-03-12T05:28:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.596+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c068790 con 0x7f263c105330 2024-03-12T05:28:39.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26327fc640 1 -- 172.21.15.100:0/2904271845 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f262c005000 con 0x7f263c105330 2024-03-12T05:28:39.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 Processor -- start 2024-03-12T05:28:39.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 -- start start 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee9c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 0x557c1ee9eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ee4e960 con 0x557c1ee9c750 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ee02310 con 0x557c1ee93b70 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26463e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1edff840 con 0x557c1ee93250 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26327fc640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f26280a1ba0 0x7f26280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f2640db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 0x557c1ee9eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f2641db2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26415b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee9c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f2640db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 0x557c1ee9eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52268/0 (socket says 172.21.15.100:52268) 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f26415b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee9c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59718/0 (socket says 172.21.15.100:59718) 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f2640db0640 1 -- 172.21.15.100:0/1888875678 learned_addr learned my addr 172.21.15.100:0/1888875678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.599+0000 7f2641db2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee93630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42050/0 (socket says 172.21.15.100:42050) 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2641db2640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f26280a1ba0 0x7f26280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f26327fc640 1 -- 172.21.15.100:0/2904271845 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f262c014070 con 0x7f263c105330 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2640db0640 1 -- 172.21.15.100:0/1888875678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 msgr2=0x557c1ee93630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2640db0640 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee93630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2640db0640 1 -- 172.21.15.100:0/1888875678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 msgr2=0x557c1ee9c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2640db0640 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee9c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2640db0640 1 -- 172.21.15.100:0/1888875678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1edbd060 con 0x557c1ee9c750 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f26415b1640 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee9c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2641db2640 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2640db0640 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 0x557c1ee9eb40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f262c000c00 tx=0x7f262c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff8f8cf254b8386 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f2641db2640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f26280a1ba0 0x7f26280a4060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2634005cc0 tx=0x7f2634005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f26277fe640 1 -- 172.21.15.100:0/1888875678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c09aea0 con 0x557c1ee9c750 2024-03-12T05:28:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f26277fe640 1 -- 172.21.15.100:0/1888875678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f262c09a7a0 con 0x557c1ee9c750 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.600+0000 7f26277fe640 1 -- 172.21.15.100:0/1888875678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0052b0 con 0x557c1ee9c750 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 -- 172.21.15.100:0/1888875678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 msgr2=0x557c1ee9eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 0x557c1ee9eb40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f262c000c00 tx=0x7f262c004a50 comp rx=0 tx=0).stop 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 -- 172.21.15.100:0/1888875678 shutdown_connections 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee9c750 0x557c1ee9eb40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee9c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 --2- 172.21.15.100:0/1888875678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 -- 172.21.15.100:0/1888875678 >> 172.21.15.100:0/1888875678 conn(0x557c1ee8d980 msgr2=0x557c1ee8dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 -- 172.21.15.100:0/1888875678 shutdown_connections 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 -- 172.21.15.100:0/1888875678 wait complete. 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 -- start start 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.601+0000 7f26463e2740 1 Processor -- start 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1ee99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee99d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee9c750 0x557c1ee9a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1edfb020 con 0x557c1ee93250 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1eace430 con 0x557c1ee93b70 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ea8ae00 con 0x557c1ee9c750 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 10 client.0 _ll_register_callbacks cb 0x557c1eda2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1ee99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1ee99830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52280/0 (socket says 172.21.15.100:52280) 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 -- 172.21.15.100:0/537581443 learned_addr learned my addr 172.21.15.100:0/537581443 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f2640db0640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee99d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f2641db2640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee9c750 0x557c1ee9a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 -- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee9c750 msgr2=0x557c1ee9a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee9c750 0x557c1ee9a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 -- 172.21.15.100:0/537581443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 msgr2=0x557c1ee99d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee99d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 -- 172.21.15.100:0/537581443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c068790 con 0x557c1ee93250 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26415b1640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1ee99830 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f26380126e0 tx=0x7f2638012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983e0f3643bbb0f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f2624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f2624ff9640 1 -- 172.21.15.100:0/537581443 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638017070 con 0x557c1ee93250 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f2624ff9640 1 -- 172.21.15.100:0/537581443 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f26380121d0 con 0x557c1ee93250 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.602+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c1eafa7e0 con 0x557c1ee93250 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f2624ff9640 1 -- 172.21.15.100:0/537581443 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2638004050 con 0x557c1ee93250 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 10 client.18975 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c1edf96a0 con 0x557c1ee93250 2024-03-12T05:28:39.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f2624ff9640 1 -- 172.21.15.100:0/537581443 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557c1eafa7e0 con 0x557c1ee93250 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f2624ff9640 1 -- 172.21.15.100:0/537581443 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2638022070 con 0x557c1ee93250 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 10 client.18975 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 10 client.18975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 20 client.18975 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1eeacc20 0x557c1eeaf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 4 client.18975 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c1eb37720 con 0x557c1eeacc20 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f2640db0640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1eeacc20 0x557c1eeaf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.603+0000 7f2640db0640 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1eeacc20 0x557c1eeaf060 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d25164c8f21d6553 server_cookie=38452a6da8b54ad1 in_seq=0 out_seq=0 2024-03-12T05:28:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.604+0000 7f2624ff9640 10 client.18975 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:39.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f2624ff9640 1 -- 172.21.15.100:0/537581443 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c1eb37720 con 0x557c1eeacc20 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f2624ff9640 10 client.18975 handle_command_reply: tid=0 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 client.18975 shutdown 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 msgr2=0x557c1ee99830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1ee99830 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f26380126e0 tx=0x7f2638012ba0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 shutdown_connections 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee9c750 0x557c1ee9a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1eeacc20 0x557c1eeaf060 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee93b70 0x557c1ee99d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 --2- 172.21.15.100:0/537581443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1ee99830 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f2624ff9640 0 client.18975 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 >> 172.21.15.100:0/537581443 conn(0x557c1ee8d980 msgr2=0x557c1ee8f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 shutdown_connections 2024-03-12T05:28:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 1 -- 172.21.15.100:0/537581443 wait complete. 2024-03-12T05:28:39.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.606+0000 7f26463e2740 20 client.18975 trim_cache size 0 max 16384 2024-03-12T05:28:39.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.618+0000 7f26463e2740 1 Processor -- start 2024-03-12T05:28:39.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.618+0000 7f26463e2740 1 -- start start 2024-03-12T05:28:39.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26463e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee9bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26463e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 0x557c1ee9a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26463e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1ee96ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26463e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ed80130 con 0x557c1ee93b70 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26463e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ee892b0 con 0x557c1ee9c750 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26463e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ee97d80 con 0x557c1ee93250 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 0x557c1ee9a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 0x557c1ee9a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52282/0 (socket says 172.21.15.100:52282) 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 -- 172.21.15.100:0/3577835062 learned_addr learned my addr 172.21.15.100:0/3577835062 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f26415b1640 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee9bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2641db2640 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1ee96ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 -- 172.21.15.100:0/3577835062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 msgr2=0x557c1ee9bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee9bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 -- 172.21.15.100:0/3577835062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 msgr2=0x557c1ee96ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1ee96ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 -- 172.21.15.100:0/3577835062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1ed95c20 con 0x557c1ee93b70 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2640db0640 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 0x557c1ee9a0d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f262c0cb8c0 tx=0x7f262c096a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24f3e4217c78fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2626ffd640 1 -- 172.21.15.100:0/3577835062 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0cdcb0 con 0x557c1ee93b70 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2626ffd640 1 -- 172.21.15.100:0/3577835062 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f262c09e8f0 con 0x557c1ee93b70 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.619+0000 7f2626ffd640 1 -- 172.21.15.100:0/3577835062 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0047c0 con 0x557c1ee93b70 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 -- 172.21.15.100:0/3577835062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 msgr2=0x557c1ee9a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 0x557c1ee9a0d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f262c0cb8c0 tx=0x7f262c096a40 comp rx=0 tx=0).stop 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 -- 172.21.15.100:0/3577835062 shutdown_connections 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1ee96ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93b70 0x557c1ee9a0d0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 --2- 172.21.15.100:0/3577835062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93250 0x557c1ee9bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 -- 172.21.15.100:0/3577835062 >> 172.21.15.100:0/3577835062 conn(0x557c1ee8d980 msgr2=0x557c1ee98100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 -- 172.21.15.100:0/3577835062 shutdown_connections 2024-03-12T05:28:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 -- 172.21.15.100:0/3577835062 wait complete. 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 -- start start 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.620+0000 7f26463e2740 1 Processor -- start 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1eea3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93b70 0x557c1eea1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1eea2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1eea2c90 con 0x557c1ee93250 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ef03190 con 0x557c1ee9c750 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ef032d0 con 0x557c1ee93b70 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 10 client.0 _ll_register_callbacks cb 0x7f262c0963a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1eea3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1eea3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52288/0 (socket says 172.21.15.100:52288) 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 -- 172.21.15.100:0/535936221 learned_addr learned my addr 172.21.15.100:0/535936221 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f2640db0640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93b70 0x557c1eea1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f2641db2640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1eea2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 -- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93b70 msgr2=0x557c1eea1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93b70 0x557c1eea1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 -- 172.21.15.100:0/535936221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 msgr2=0x557c1eea2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1eea2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 -- 172.21.15.100:0/535936221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c1edbd060 con 0x557c1ee93250 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f2641db2640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1eea2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26415b1640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1eea3ab0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f26380039a0 tx=0x7f2638009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21421c6cf1a86246 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f2624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f2624ff9640 1 -- 172.21.15.100:0/535936221 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638017040 con 0x557c1ee93250 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f262c09a7a0 con 0x557c1ee93250 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.621+0000 7f2624ff9640 1 -- 172.21.15.100:0/535936221 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f263800fd30 con 0x557c1ee93250 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f2624ff9640 1 -- 172.21.15.100:0/535936221 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26380122d0 con 0x557c1ee93250 2024-03-12T05:28:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 10 client.18981 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c1edbb8c0 con 0x557c1ee93250 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f2624ff9640 1 -- 172.21.15.100:0/535936221 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2638039080 con 0x557c1ee93250 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f2624ff9640 1 -- 172.21.15.100:0/535936221 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2638022040 con 0x557c1ee93250 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 10 client.18981 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 10 client.18981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 20 client.18981 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1ef04a80 0x557c1ef06ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 4 client.18981 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c1eb86890 con 0x557c1ef04a80 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.622+0000 7f2640db0640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1ef04a80 0x557c1ef06ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.623+0000 7f2624ff9640 10 client.18981 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.623+0000 7f2640db0640 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1ef04a80 0x557c1ef06ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5344a0f7b0368a0d server_cookie=4b012ac7473b860e in_seq=0 out_seq=0 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f2624ff9640 1 -- 172.21.15.100:0/535936221 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c1eb86890 con 0x557c1ef04a80 2024-03-12T05:28:39.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f2624ff9640 10 client.18981 handle_command_reply: tid=0 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 client.18981 shutdown 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 msgr2=0x557c1eea3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1eea3ab0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f26380039a0 tx=0x7f2638009f50 comp rx=0 tx=0).stop 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 shutdown_connections 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x557c1ef04a80 0x557c1ef06ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c1ee9c750 0x557c1eea2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c1ee93b70 0x557c1eea1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f2624ff9640 0 client.18981 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 --2- 172.21.15.100:0/535936221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c1ee93250 0x557c1eea3ab0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 >> 172.21.15.100:0/535936221 conn(0x557c1ee8d980 msgr2=0x557c1ee8c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 shutdown_connections 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.624+0000 7f26463e2740 1 -- 172.21.15.100:0/535936221 wait complete. 2024-03-12T05:28:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f26463e2740 20 client.18981 trim_cache size 0 max 16384 2024-03-12T05:28:39.626 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f26280a1ba0 msgr2=0x7f26280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f26280a1ba0 0x7f26280a4060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2634005cc0 tx=0x7f2634005c50 comp rx=0 tx=0).stop 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 msgr2=0x7f263c196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c196c60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f262c00e9c0 tx=0x7f262c00ee80 comp rx=0 tx=0).stop 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 shutdown_connections 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f26280a1ba0 0x7f26280a4060 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c19af90 0x7f263c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263c105330 0x7f263c196c60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 --2- 172.21.15.100:0/2904271845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f263c100260 0x7f263c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 >> 172.21.15.100:0/2904271845 conn(0x7f263c077180 msgr2=0x7f263c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 shutdown_connections 2024-03-12T05:28:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.625+0000 7f2643815640 1 -- 172.21.15.100:0/2904271845 wait complete. 2024-03-12T05:28:39.629 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:39 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2190146238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:28:39.630 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:28:39.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:28:39.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:39 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2190146238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:28:39.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:39 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2190146238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:28:39.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 -- 172.21.15.100:0/280463820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 msgr2=0x7f8ffc0ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 --2- 172.21.15.100:0/280463820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc0ffcd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009860 tx=0x7f8fec02bb70 comp rx=0 tx=0).stop 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 -- 172.21.15.100:0/280463820 shutdown_connections 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 --2- 172.21.15.100:0/280463820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc106500 0x7f8ffc10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 --2- 172.21.15.100:0/280463820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc1002a0 0x7f8ffc105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 --2- 172.21.15.100:0/280463820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc0ffcd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 -- 172.21.15.100:0/280463820 >> 172.21.15.100:0/280463820 conn(0x7f8ffc076640 msgr2=0x7f8ffc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 -- 172.21.15.100:0/280463820 shutdown_connections 2024-03-12T05:28:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.746+0000 7f9000f02640 1 -- 172.21.15.100:0/280463820 wait complete. 2024-03-12T05:28:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 Processor -- start 2024-03-12T05:28:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 -- start start 2024-03-12T05:28:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc106500 0x7f8ffc19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc197660 0x7f8ffc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc198130 con 0x7f8ffc106500 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1982a0 con 0x7f8ffc197660 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f9000f02640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc198460 con 0x7f8ffc0ff8f0 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f8ff9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc106500 0x7f8ffc19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f8ff9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc106500 0x7f8ffc19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52308/0 (socket says 172.21.15.100:52308) 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/2785550849 learned_addr learned my addr 172.21.15.100:0/2785550849 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.747+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc197660 0x7f8ffc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffad9d640 1 -- 172.21.15.100:0/2785550849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 msgr2=0x7f8ffc19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffad9d640 1 -- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc106500 msgr2=0x7f8ffc19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc106500 0x7f8ffc19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffad9d640 1 -- 172.21.15.100:0/2785550849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec009490 con 0x7f8ffc197660 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc197660 0x7f8ffc197ae0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009fd0 tx=0x7f8ff400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8feb7fe640 1 -- 172.21.15.100:0/2785550849 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400ce20 con 0x7f8ffc197660 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1a4400 con 0x7f8ffc197660 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8feb7fe640 1 -- 172.21.15.100:0/2785550849 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff4005020 con 0x7f8ffc197660 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f8feb7fe640 1 -- 172.21.15.100:0/2785550849 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4010690 con 0x7f8ffc197660 2024-03-12T05:28:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.748+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1a4930 con 0x7f8ffc197660 2024-03-12T05:28:39.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.750+0000 7f8feb7fe640 1 -- 172.21.15.100:0/2785550849 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8ff4005500 con 0x7f8ffc197660 2024-03-12T05:28:39.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 Processor -- start 2024-03-12T05:28:39.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8feb7fe640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8fe40a1ba0 0x7f8fe40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8fe40a1ba0 0x7f8fe40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8feb7fe640 1 -- 172.21.15.100:0/2785550849 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8ff4014070 con 0x7f8ffc197660 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 -- start start 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a19c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a19c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1974960 con 0x5651a19b9b70 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1928310 con 0x5651a19c2750 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f9003bb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1925840 con 0x5651a19b9250 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ff9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a19c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ffa59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ffa59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19b9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42110/0 (socket says 172.21.15.100:42110) 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ffa59c640 1 -- 172.21.15.100:0/1881109213 learned_addr learned my addr 172.21.15.100:0/1881109213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ff9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a19c2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52322/0 (socket says 172.21.15.100:52322) 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a19c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.751+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8fe40a1ba0 0x7f8fe40a4060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc198eb0 tx=0x7f8ff000b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/1881109213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 msgr2=0x5651a19b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/1881109213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 msgr2=0x5651a19c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a19c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/1881109213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651a18e3060 con 0x5651a19b9b70 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a19c2210 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0003c10 tx=0x7f8ff0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1215c9ece0a0f9b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8fe8ff9640 1 -- 172.21.15.100:0/1881109213 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0003e40 con 0x5651a19b9b70 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8fe8ff9640 1 -- 172.21.15.100:0/1881109213 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff0015070 con 0x5651a19b9b70 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.752+0000 7f8fe8ff9640 1 -- 172.21.15.100:0/1881109213 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff001d730 con 0x5651a19b9b70 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- 172.21.15.100:0/1881109213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 msgr2=0x5651a19c2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a19c2210 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0003c10 tx=0x7f8ff0013000 comp rx=0 tx=0).stop 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- 172.21.15.100:0/1881109213 shutdown_connections 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a19c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a19c2210 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1881109213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- 172.21.15.100:0/1881109213 >> 172.21.15.100:0/1881109213 conn(0x5651a19b3980 msgr2=0x5651a19b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- 172.21.15.100:0/1881109213 shutdown_connections 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- 172.21.15.100:0/1881109213 wait complete. 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- start start 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 Processor -- start 2024-03-12T05:28:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9b70 0x5651a19bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1921020 con 0x5651a19c2750 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a15f4430 con 0x5651a19b9b70 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.753+0000 7f9003bb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a15b0e00 con 0x5651a19b9250 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f9003bb1740 10 client.0 _ll_register_callbacks cb 0x5651a18c8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19c01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52332/0 (socket says 172.21.15.100:52332) 2024-03-12T05:28:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 -- 172.21.15.100:0/1371878181 learned_addr learned my addr 172.21.15.100:0/1371878181 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9b70 0x5651a19bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 -- 172.21.15.100:0/1371878181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 msgr2=0x5651a19bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 -- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9b70 msgr2=0x5651a19bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9b70 0x5651a19bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 -- 172.21.15.100:0/1371878181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff00038f0 con 0x5651a19c2750 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19c01b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4000f80 tx=0x7f8ff409b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad2459b9a69bfeb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/1371878181 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40a2f90 con 0x5651a19c2750 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/1371878181 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8ff40a3220 con 0x5651a19c2750 2024-03-12T05:28:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651a16207e0 con 0x5651a19c2750 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/1371878181 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff4099ed0 con 0x5651a19c2750 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f9003bb1740 10 client.18993 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5651a191f6a0 con 0x5651a19c2750 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.754+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/1371878181 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5651a16207e0 con 0x5651a19c2750 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/1371878181 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ff401f070 con 0x5651a19c2750 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f9003bb1740 10 client.18993 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f9003bb1740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f9003bb1740 20 client.18993 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a19d2ae0 0x5651a19d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f9003bb1740 4 client.18993 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651a165d720 con 0x5651a19d2ae0 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a19d2ae0 0x5651a19d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f8fe1ffb640 10 client.18993 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.755+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a19d2ae0 0x5651a19d4f00 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9de23f706a3373 server_cookie=f317085c0df34a14 in_seq=0 out_seq=0 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/1371878181 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651a165d720 con 0x5651a19d2ae0 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f8fe1ffb640 10 client.18993 handle_command_reply: tid=0 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 client.18993 shutdown 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 msgr2=0x5651a19c01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19c01b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4000f80 tx=0x7f8ff409b240 comp rx=0 tx=0).stop 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 shutdown_connections 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a19d2ae0 0x5651a19d4f00 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19c01b0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f8fe1ffb640 0 client.18993 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9b70 0x5651a19bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 --2- 172.21.15.100:0/1371878181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a19bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.756+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 >> 172.21.15.100:0/1371878181 conn(0x5651a19b3980 msgr2=0x5651a19b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.757+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 shutdown_connections 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.757+0000 7f9003bb1740 1 -- 172.21.15.100:0/1371878181 wait complete. 2024-03-12T05:28:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.757+0000 7f9003bb1740 20 client.18993 trim_cache size 0 max 16384 2024-03-12T05:28:39.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.770+0000 7f9003bb1740 1 Processor -- start 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.770+0000 7f9003bb1740 1 -- start start 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.770+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9250 0x5651a19c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.770+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9b70 0x5651a19c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f9003bb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a18a6130 con 0x5651a19c2750 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f9003bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a19b8340 con 0x5651a19b9250 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f9003bb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a19cf640 con 0x5651a19b9b70 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19bffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52340/0 (socket says 172.21.15.100:52340) 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 -- 172.21.15.100:0/211871889 learned_addr learned my addr 172.21.15.100:0/211871889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9b70 0x5651a19c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9250 0x5651a19c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 -- 172.21.15.100:0/211871889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9b70 msgr2=0x5651a19c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9b70 0x5651a19c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 -- 172.21.15.100:0/211871889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9250 msgr2=0x5651a19c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9250 0x5651a19c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 -- 172.21.15.100:0/211871889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651a18bbc20 con 0x5651a19c2750 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19bffd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4098d30 tx=0x7f8ff409a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e38a56419045855 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8fe3fff640 1 -- 172.21.15.100:0/211871889 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4079ea0 con 0x5651a19c2750 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8fe3fff640 1 -- 172.21.15.100:0/211871889 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff409a850 con 0x5651a19c2750 2024-03-12T05:28:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.771+0000 7f8fe3fff640 1 -- 172.21.15.100:0/211871889 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40a2ff0 con 0x5651a19c2750 2024-03-12T05:28:39.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 -- 172.21.15.100:0/211871889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 msgr2=0x5651a19bffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19bffd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4098d30 tx=0x7f8ff409a3e0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 -- 172.21.15.100:0/211871889 shutdown_connections 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19c2750 0x5651a19bffd0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9b70 0x5651a19c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 --2- 172.21.15.100:0/211871889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19b9250 0x5651a19c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 -- 172.21.15.100:0/211871889 >> 172.21.15.100:0/211871889 conn(0x5651a19b3980 msgr2=0x5651a19baa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 -- 172.21.15.100:0/211871889 shutdown_connections 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 -- 172.21.15.100:0/211871889 wait complete. 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 -- start start 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.772+0000 7f9003bb1740 1 Processor -- start 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a1a26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a1a27440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a1a21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1a21920 con 0x5651a19b9b70 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1a21a90 con 0x5651a19c2750 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a1a21c50 con 0x5651a19b9250 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f9003bb1740 10 client.0 _ll_register_callbacks cb 0x7f8ff0004240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a1a27440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ffa59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a1a26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a1a27440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52356/0 (socket says 172.21.15.100:52356) 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/898256731 learned_addr learned my addr 172.21.15.100:0/898256731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ffad9d640 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a1a21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/898256731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 msgr2=0x5651a1a26f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a1a26f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 msgr2=0x5651a1a21360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a1a21360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 -- 172.21.15.100:0/898256731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651a191f6a0 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8ff9d9b640 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a1a27440 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8ff001ef90 tx=0x7f8ff0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffe7f9a9adda6b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.773+0000 7f8fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/898256731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff002b040 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/898256731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8ff0017070 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ff0015070 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/898256731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff0004640 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 10 client.18999 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5651a18e18c0 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/898256731 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff0015070 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/898256731 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ff0024020 con 0x5651a19b9b70 2024-03-12T05:28:39.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 10 client.18999 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 10 client.18999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 20 client.18999 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a1a2ed90 0x5651a1a311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 4 client.18999 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651a16ac890 con 0x5651a1a2ed90 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.774+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a1a2ed90 0x5651a1a311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f8fe1ffb640 10 client.18999 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f8ffa59c640 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a1a2ed90 0x5651a1a311b0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4678705002cfea server_cookie=d8836209d12966d in_seq=0 out_seq=0 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f8fe1ffb640 1 -- 172.21.15.100:0/898256731 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651a16ac890 con 0x5651a1a2ed90 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f8fe1ffb640 10 client.18999 handle_command_reply: tid=0 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 client.18999 shutdown 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 msgr2=0x5651a1a27440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a1a27440 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8ff001ef90 tx=0x7f8ff0005270 comp rx=0 tx=0).stop 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 shutdown_connections 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5651a1a2ed90 0x5651a1a311b0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651a19c2750 0x5651a1a21360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a19b9b70 0x5651a1a27440 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 --2- 172.21.15.100:0/898256731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651a19b9250 0x5651a1a26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 >> 172.21.15.100:0/898256731 conn(0x5651a19b3980 msgr2=0x5651a19b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.775+0000 7f8fe1ffb640 0 client.18999 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 shutdown_connections 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9003bb1740 1 -- 172.21.15.100:0/898256731 wait complete. 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9003bb1740 20 client.18999 trim_cache size 0 max 16384 2024-03-12T05:28:39.776 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:28:39.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8fe40a1ba0 msgr2=0x7f8fe40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8fe40a1ba0 0x7f8fe40a4060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc198eb0 tx=0x7f8ff000b410 comp rx=0 tx=0).stop 2024-03-12T05:28:39.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc197660 msgr2=0x7f8ffc197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc197660 0x7f8ffc197ae0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009fd0 tx=0x7f8ff400ef10 comp rx=0 tx=0).stop 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 shutdown_connections 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8fe40a1ba0 0x7f8fe40a4060 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc197660 0x7f8ffc197ae0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc106500 0x7f8ffc19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 --2- 172.21.15.100:0/2785550849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc0ff8f0 0x7f8ffc19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 >> 172.21.15.100:0/2785550849 conn(0x7f8ffc076640 msgr2=0x7f8ffc104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 shutdown_connections 2024-03-12T05:28:39.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:39.776+0000 7f9000f02640 1 -- 172.21.15.100:0/2785550849 wait complete. 2024-03-12T05:28:40.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 121215 30.75 MB/sec execute 83 sec latency 165.728 ms 2024-03-12T05:28:40.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:40 smithi156 ceph-mon[36270]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.1 MiB/s wr, 466 op/s 2024-03-12T05:28:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:40 smithi161 ceph-mon[36043]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.1 MiB/s wr, 466 op/s 2024-03-12T05:28:40.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:28:40.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:28:40.783 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:28:40.809 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:40 smithi100 ceph-mon[32210]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.1 MiB/s wr, 466 op/s 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 -- 172.21.15.100:0/178251033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3500105330 msgr2=0x7f350010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 --2- 172.21.15.100:0/178251033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3500105330 0x7f350010cf00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009cd0 tx=0x7f34f802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 -- 172.21.15.100:0/178251033 shutdown_connections 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 --2- 172.21.15.100:0/178251033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3500105330 0x7f350010cf00 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 --2- 172.21.15.100:0/178251033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3500100c10 0x7f3500104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 --2- 172.21.15.100:0/178251033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500100260 0x7f3500100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.908+0000 7f3507e84640 1 -- 172.21.15.100:0/178251033 >> 172.21.15.100:0/178251033 conn(0x7f3500077180 msgr2=0x7f3500077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 -- 172.21.15.100:0/178251033 shutdown_connections 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 -- 172.21.15.100:0/178251033 wait complete. 2024-03-12T05:28:40.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 Processor -- start 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 -- start start 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3500100260 0x7f35001966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500105330 0x7f3500196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 0x7f350019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350019b950 con 0x7f350019af60 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350019bac0 con 0x7f3500105330 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3507e84640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350019bc30 con 0x7f3500100260 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3506421640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 0x7f350019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3506421640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 0x7f350019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58094/0 (socket says 172.21.15.100:58094) 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f3506421640 1 -- 172.21.15.100:0/2621373142 learned_addr learned my addr 172.21.15.100:0/2621373142 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.909+0000 7f350541f640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500105330 0x7f3500196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3505c20640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3500100260 0x7f35001966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3506421640 1 -- 172.21.15.100:0/2621373142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3500100260 msgr2=0x7f35001966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3506421640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3500100260 0x7f35001966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3506421640 1 -- 172.21.15.100:0/2621373142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500105330 msgr2=0x7f3500196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3506421640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500105330 0x7f3500196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3506421640 1 -- 172.21.15.100:0/2621373142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8009900 con 0x7f350019af60 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3506421640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 0x7f350019b410 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f34f8004af0 tx=0x7f34f8032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f34f6ffd640 1 -- 172.21.15.100:0/2621373142 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80328b0 con 0x7f350019af60 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f34f6ffd640 1 -- 172.21.15.100:0/2621373142 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34f8003e50 con 0x7f350019af60 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35000682e0 con 0x7f350019af60 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f34f6ffd640 1 -- 172.21.15.100:0/2621373142 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004740 con 0x7f350019af60 2024-03-12T05:28:40.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.910+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3500068790 con 0x7f350019af60 2024-03-12T05:28:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.912+0000 7f34f6ffd640 1 -- 172.21.15.100:0/2621373142 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f34f800a040 con 0x7f350019af60 2024-03-12T05:28:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.912+0000 7f350aa58740 1 Processor -- start 2024-03-12T05:28:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 -- start start 2024-03-12T05:28:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b2f8b0 0x562e86b31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86a94f90 con 0x562e86b26d30 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86a924c0 con 0x562e86b26380 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350aa58740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86a8dca0 con 0x562e86b2f8b0 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f34f6ffd640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f34ec0a1ba0 0x7f34ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f3505c20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b26760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:54082/0 (socket says 172.21.15.100:54082) 2024-03-12T05:28:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f3506421640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f34ec0a1ba0 0x7f34ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 -- 172.21.15.100:0/2096861607 learned_addr learned my addr 172.21.15.100:0/2096861607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b2f8b0 0x562e86b31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f34f6ffd640 1 -- 172.21.15.100:0/2621373142 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f34f80f02e0 con 0x7f350019af60 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 -- 172.21.15.100:0/2096861607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b2f8b0 msgr2=0x562e86b31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b2f8b0 0x562e86b31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 -- 172.21.15.100:0/2096861607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 msgr2=0x562e86b2f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.913+0000 7f350541f640 1 -- 172.21.15.100:0/2096861607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e86a500b0 con 0x562e86b26380 2024-03-12T05:28:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.914+0000 7f3506421640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f34ec0a1ba0 0x7f34ec0a4060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f34f80edbb0 tx=0x7f34f808f1e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.914+0000 7f350541f640 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b26760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e970 tx=0x7f34f0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=545036bc6b33a512 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.914+0000 7f34ebfff640 1 -- 172.21.15.100:0/2096861607 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0013070 con 0x562e86b26380 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.914+0000 7f34ebfff640 1 -- 172.21.15.100:0/2096861607 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34f0014040 con 0x562e86b26380 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.914+0000 7f34ebfff640 1 -- 172.21.15.100:0/2096861607 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004a90 con 0x562e86b26380 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- 172.21.15.100:0/2096861607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 msgr2=0x562e86b26760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b26760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e970 tx=0x7f34f0004730 comp rx=0 tx=0).stop 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- 172.21.15.100:0/2096861607 shutdown_connections 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b2f8b0 0x562e86b31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- 172.21.15.100:0/2096861607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b26760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- 172.21.15.100:0/2096861607 >> 172.21.15.100:0/2096861607 conn(0x562e86b20840 msgr2=0x562e86b20c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- 172.21.15.100:0/2096861607 shutdown_connections 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- 172.21.15.100:0/2096861607 wait complete. 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- start start 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 Processor -- start 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b2c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b2d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86a32f40 con 0x562e86b26d30 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e867614b0 con 0x562e86b2f8b0 2024-03-12T05:28:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86a13670 con 0x562e86b26380 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.915+0000 7f350aa58740 10 client.0 _ll_register_callbacks cb 0x562e86a80f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58114/0 (socket says 172.21.15.100:58114) 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 -- 172.21.15.100:0/1764265852 learned_addr learned my addr 172.21.15.100:0/1764265852 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f350541f640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b2c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f350541f640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b2d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 -- 172.21.15.100:0/1764265852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 msgr2=0x562e86b2c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b2c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 -- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 msgr2=0x562e86b2d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b2d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 -- 172.21.15.100:0/1764265852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f000e650 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f3505c20640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2ce20 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0126e0 tx=0x7f34fc012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a83a946c9d5388 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f34e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f34e97fa640 1 -- 172.21.15.100:0/1764265852 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc017070 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f34e97fa640 1 -- 172.21.15.100:0/1764265852 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f34fc012190 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e86a67f40 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f34e97fa640 1 -- 172.21.15.100:0/1764265852 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34fc00f920 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f350aa58740 10 client.19011 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562e86a8c320 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.916+0000 7f34e97fa640 1 -- 172.21.15.100:0/1764265852 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34fc027020 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f34e97fa640 1 -- 172.21.15.100:0/1764265852 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f34fc010b30 con 0x562e86b26d30 2024-03-12T05:28:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350aa58740 10 client.19011 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350aa58740 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350aa58740 20 client.19011 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350aa58740 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b3ff70 0x562e86b423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350aa58740 4 client.19011 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e86a67f40 con 0x562e86b3ff70 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350541f640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b3ff70 0x562e86b423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f350541f640 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b3ff70 0x562e86b423b0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea90b117d87dab8 server_cookie=d8a4a18e5e460d67 in_seq=0 out_seq=0 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.917+0000 7f34e97fa640 10 client.19011 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f34e97fa640 1 -- 172.21.15.100:0/1764265852 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e86a67f40 con 0x562e86b3ff70 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f34e97fa640 10 client.19011 handle_command_reply: tid=0 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 client.19011 shutdown 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 msgr2=0x562e86b2ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2ce20 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0126e0 tx=0x7f34fc012ba0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 shutdown_connections 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b3ff70 0x562e86b423b0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b2d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b2ce20 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 --2- 172.21.15.100:0/1764265852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f34e97fa640 0 client.19011 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 >> 172.21.15.100:0/1764265852 conn(0x562e86b20840 msgr2=0x562e86b227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 shutdown_connections 2024-03-12T05:28:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 1 -- 172.21.15.100:0/1764265852 wait complete. 2024-03-12T05:28:40.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.918+0000 7f350aa58740 20 client.19011 trim_cache size 0 max 16384 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 Processor -- start 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 -- start start 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b2d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26d30 0x562e86b22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 0x562e86b368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86b21bf0 con 0x562e86b2f8b0 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86b1c110 con 0x562e86b26380 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350aa58740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86b20370 con 0x562e86b26d30 2024-03-12T05:28:40.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350541f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 0x562e86b368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350541f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 0x562e86b368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58122/0 (socket says 172.21.15.100:58122) 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350541f640 1 -- 172.21.15.100:0/2205345798 learned_addr learned my addr 172.21.15.100:0/2205345798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f3505c20640 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26d30 0x562e86b22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.934+0000 7f350541f640 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b2d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350541f640 1 -- 172.21.15.100:0/2205345798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26d30 msgr2=0x562e86b22530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350541f640 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26d30 0x562e86b22530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350541f640 1 -- 172.21.15.100:0/2205345798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 msgr2=0x562e86b2d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350541f640 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b2d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350541f640 1 -- 172.21.15.100:0/2205345798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e86a28c00 con 0x562e86b2f8b0 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350541f640 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 0x562e86b368e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f34f00097a0 tx=0x7f34f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa24e185ae149575 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f34eb7fe640 1 -- 172.21.15.100:0/2205345798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0033070 con 0x562e86b2f8b0 2024-03-12T05:28:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f34eb7fe640 1 -- 172.21.15.100:0/2205345798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34f0004050 con 0x562e86b2f8b0 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f34eb7fe640 1 -- 172.21.15.100:0/2205345798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0013040 con 0x562e86b2f8b0 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350aa58740 1 -- 172.21.15.100:0/2205345798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 msgr2=0x562e86b368e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.935+0000 7f350aa58740 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 0x562e86b368e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f34f00097a0 tx=0x7f34f0004730 comp rx=0 tx=0).stop 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- 172.21.15.100:0/2205345798 shutdown_connections 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b2f8b0 0x562e86b368e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26d30 0x562e86b22530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 --2- 172.21.15.100:0/2205345798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b26380 0x562e86b2d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- 172.21.15.100:0/2205345798 >> 172.21.15.100:0/2205345798 conn(0x562e86b20840 msgr2=0x562e86b21e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- 172.21.15.100:0/2205345798 shutdown_connections 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- 172.21.15.100:0/2205345798 wait complete. 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- start start 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 Processor -- start 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b36540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b8f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86b8fc10 con 0x562e86b26d30 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86b8fd80 con 0x562e86b2f8b0 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e86b8ff40 con 0x562e86b26380 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350aa58740 10 client.0 _ll_register_callbacks cb 0x7f34f0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f3505c20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b36540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f3505c20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b36540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58128/0 (socket says 172.21.15.100:58128) 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f3505c20640 1 -- 172.21.15.100:0/2109766752 learned_addr learned my addr 172.21.15.100:0/2109766752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.936+0000 7f350541f640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350541f640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b8f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f3505c20640 1 -- 172.21.15.100:0/2109766752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 msgr2=0x562e86b36000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f3505c20640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b36000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f3505c20640 1 -- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 msgr2=0x562e86b8f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f3505c20640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b8f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f3505c20640 1 -- 172.21.15.100:0/2109766752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f0022040 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350541f640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b36000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f3505c20640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b36540 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x562e86b2deb0 tx=0x7f34fc00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1822a48cb7e700 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f34e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f34e97fa640 1 -- 172.21.15.100:0/2109766752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc017030 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f34e97fa640 1 -- 172.21.15.100:0/2109766752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f34fc028080 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e86ab3900 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f34e97fa640 1 -- 172.21.15.100:0/2109766752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34fc00f920 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 10 client.19017 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562e86a4e910 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f34e97fa640 1 -- 172.21.15.100:0/2109766752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34fc038020 con 0x562e86b26d30 2024-03-12T05:28:40.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f34e97fa640 1 -- 172.21.15.100:0/2109766752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f34fc0035f0 con 0x562e86b26d30 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 10 client.19017 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 20 client.19017 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b98a80 0x562e86b9aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 4 client.19017 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.937+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562e86a84310 con 0x562e86b98a80 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.938+0000 7f350541f640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b98a80 0x562e86b9aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.938+0000 7f34e97fa640 10 client.19017 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.938+0000 7f350541f640 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b98a80 0x562e86b9aea0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ed81e62220e59c server_cookie=b43ee91d4b7c411b in_seq=0 out_seq=0 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.939+0000 7f34e97fa640 1 -- 172.21.15.100:0/2109766752 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562e86a84310 con 0x562e86b98a80 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.939+0000 7f34e97fa640 10 client.19017 handle_command_reply: tid=0 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.939+0000 7f350aa58740 1 client.19017 shutdown 2024-03-12T05:28:40.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.939+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 msgr2=0x562e86b36540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.939+0000 7f350aa58740 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b36540 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x562e86b2deb0 tx=0x7f34fc00f810 comp rx=0 tx=0).stop 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 shutdown_connections 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x562e86b98a80 0x562e86b9aea0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e86b2f8b0 0x562e86b8f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e86b26d30 0x562e86b36540 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f34e97fa640 0 client.19017 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 --2- 172.21.15.100:0/2109766752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e86b26380 0x562e86b36000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 >> 172.21.15.100:0/2109766752 conn(0x562e86b20840 msgr2=0x562e86b2ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 shutdown_connections 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 1 -- 172.21.15.100:0/2109766752 wait complete. 2024-03-12T05:28:40.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f350aa58740 20 client.19017 trim_cache size 0 max 16384 2024-03-12T05:28:40.942 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:28:40.942 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:28:40.942 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "1f5a5d76-d3c5-40c7-b6f5-17c0336c5400", 2024-03-12T05:28:40.942 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:28:40.942 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f34ec0a1ba0 msgr2=0x7f34ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f34ec0a1ba0 0x7f34ec0a4060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f34f80edbb0 tx=0x7f34f808f1e0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 msgr2=0x7f350019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 0x7f350019b410 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f34f8004af0 tx=0x7f34f8032570 comp rx=0 tx=0).stop 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 shutdown_connections 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f34ec0a1ba0 0x7f34ec0a4060 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350019af60 0x7f350019b410 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500105330 0x7f3500196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 --2- 172.21.15.100:0/2621373142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3500100260 0x7f35001966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:40.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 >> 172.21.15.100:0/2621373142 conn(0x7f3500077180 msgr2=0x7f35000fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:40.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 shutdown_connections 2024-03-12T05:28:40.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:40.940+0000 7f3507e84640 1 -- 172.21.15.100:0/2621373142 wait complete. 2024-03-12T05:28:40.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:28:40.945 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:28:41.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 -- 172.21.15.100:0/684518605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd741002b0 msgr2=0x7efd74105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 --2- 172.21.15.100:0/684518605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd741002b0 0x7efd74105de0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009900 tx=0x7efd5c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:28:41.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 -- 172.21.15.100:0/684518605 shutdown_connections 2024-03-12T05:28:41.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 --2- 172.21.15.100:0/684518605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 0x7efd7410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 --2- 172.21.15.100:0/684518605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd741002b0 0x7efd74105de0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 --2- 172.21.15.100:0/684518605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd740ff900 0x7efd740ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.106+0000 7efd78dc7640 1 -- 172.21.15.100:0/684518605 >> 172.21.15.100:0/684518605 conn(0x7efd74076590 msgr2=0x7efd740fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.107+0000 7efd78dc7640 1 -- 172.21.15.100:0/684518605 shutdown_connections 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.107+0000 7efd78dc7640 1 -- 172.21.15.100:0/684518605 wait complete. 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.107+0000 7efd78dc7640 1 Processor -- start 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.107+0000 7efd78dc7640 1 -- start start 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd78dc7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd740ff900 0x7efd74197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd78dc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 0x7efd74197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd78dc7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 0x7efd7419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd78dc7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7419c9f0 con 0x7efd7419c000 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd78dc7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7419cb60 con 0x7efd74106510 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd78dc7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7419ccd0 con 0x7efd740ff900 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd72d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 0x7efd7419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 0x7efd74197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd72d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 0x7efd7419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58148/0 (socket says 172.21.15.100:58148) 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd72d9d640 1 -- 172.21.15.100:0/2794935095 learned_addr learned my addr 172.21.15.100:0/2794935095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd7259c640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd740ff900 0x7efd74197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd69d9b640 1 -- 172.21.15.100:0/2794935095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd740ff900 msgr2=0x7efd74197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd69d9b640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd740ff900 0x7efd74197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd69d9b640 1 -- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 msgr2=0x7efd7419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd69d9b640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 0x7efd7419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd69d9b640 1 -- 172.21.15.100:0/2794935095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c009560 con 0x7efd74106510 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.108+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 0x7efd7419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.109+0000 7efd69d9b640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 0x7efd74197cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd5c036040 tx=0x7efd5c02c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.109+0000 7efd6bfff640 1 -- 172.21.15.100:0/2794935095 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0042d0 con 0x7efd74106510 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.109+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd741a42b0 con 0x7efd74106510 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.109+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd741a4730 con 0x7efd74106510 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.109+0000 7efd6bfff640 1 -- 172.21.15.100:0/2794935095 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd5c004430 con 0x7efd74106510 2024-03-12T05:28:41.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.109+0000 7efd6bfff640 1 -- 172.21.15.100:0/2794935095 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c003b50 con 0x7efd74106510 2024-03-12T05:28:41.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.111+0000 7efd7b99a740 1 Processor -- start 2024-03-12T05:28:41.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 -- start start 2024-03-12T05:28:41.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd6bfff640 1 -- 172.21.15.100:0/2794935095 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7efd5c003cb0 con 0x7efd74106510 2024-03-12T05:28:41.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8d2b0 0x55615ff8d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8dbd0 0x55615ff96270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fefc280 con 0x55615ff967b0 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fef97b0 con 0x55615ff8dbd0 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7b99a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fef4f90 con 0x55615ff8d2b0 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd72d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8d2b0 0x55615ff8d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8dbd0 0x55615ff96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd72d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8d2b0 0x55615ff8d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:48296/0 (socket says 172.21.15.100:48296) 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd7259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8dbd0 0x55615ff96270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:54124/0 (socket says 172.21.15.100:54124) 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd72d9d640 1 -- 172.21.15.100:0/187411499 learned_addr learned my addr 172.21.15.100:0/187411499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd69d9b640 1 -- 172.21.15.100:0/187411499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8d2b0 msgr2=0x55615ff8d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd69d9b640 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8d2b0 0x55615ff8d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd69d9b640 1 -- 172.21.15.100:0/187411499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8dbd0 msgr2=0x55615ff96270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd69d9b640 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8dbd0 0x55615ff96270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd69d9b640 1 -- 172.21.15.100:0/187411499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55615feb3570 con 0x55615ff967b0 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd6bfff640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efd600a1ba0 0x7efd600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.112+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efd600a1ba0 0x7efd600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.113+0000 7efd6bfff640 1 -- 172.21.15.100:0/2794935095 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efd5c0ef520 con 0x7efd74106510 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.113+0000 7efd69d9b640 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff98ba0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ecd40 tx=0x7efd5c0f5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5a3591db7f9e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.113+0000 7efd68d99640 1 -- 172.21.15.100:0/187411499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b84c0 con 0x55615ff967b0 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.113+0000 7efd68d99640 1 -- 172.21.15.100:0/187411499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd5c0bbff0 con 0x55615ff967b0 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.113+0000 7efd68d99640 1 -- 172.21.15.100:0/187411499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0bbb80 con 0x55615ff967b0 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 -- 172.21.15.100:0/187411499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 msgr2=0x55615ff98ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efd600a1ba0 0x7efd600a4060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efd6c013ea0 tx=0x7efd6c015040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff98ba0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ecd40 tx=0x7efd5c0f5540 comp rx=0 tx=0).stop 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 -- 172.21.15.100:0/187411499 shutdown_connections 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff98ba0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8dbd0 0x55615ff96270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 --2- 172.21.15.100:0/187411499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8d2b0 0x55615ff8d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 -- 172.21.15.100:0/187411499 >> 172.21.15.100:0/187411499 conn(0x55615ff87990 msgr2=0x55615ff87d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 -- 172.21.15.100:0/187411499 shutdown_connections 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 -- 172.21.15.100:0/187411499 wait complete. 2024-03-12T05:28:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 -- start start 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.114+0000 7efd7b99a740 1 Processor -- start 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff93490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615ff939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff96100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58172/0 (socket says 172.21.15.100:58172) 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7259c640 1 -- 172.21.15.100:0/707260485 learned_addr learned my addr 172.21.15.100:0/707260485 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fbc8430 con 0x55615ff8d2b0 2024-03-12T05:28:41.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fb84e00 con 0x55615ff967b0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fe79fd0 con 0x55615ff8dbd0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd7b99a740 10 client.0 _ll_register_callbacks cb 0x55615fecffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd69d9b640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff93490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.115+0000 7efd72d9d640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615ff939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd72d9d640 1 -- 172.21.15.100:0/707260485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 msgr2=0x55615ff93490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd72d9d640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff93490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd72d9d640 1 -- 172.21.15.100:0/707260485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 msgr2=0x55615ff96100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd72d9d640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff96100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd72d9d640 1 -- 172.21.15.100:0/707260485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd741a4730 con 0x55615ff967b0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd72d9d640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615ff939f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00ec10 tx=0x7efd6c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beeff903fdcdd84 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd59ffb640 1 -- 172.21.15.100:0/707260485 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c01dac0 con 0x55615ff967b0 2024-03-12T05:28:41.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55615fbf47e0 con 0x55615ff967b0 2024-03-12T05:28:41.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.116+0000 7efd59ffb640 1 -- 172.21.15.100:0/707260485 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efd6c00ee80 con 0x55615ff967b0 2024-03-12T05:28:41.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.117+0000 7efd59ffb640 1 -- 172.21.15.100:0/707260485 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd6c01dc20 con 0x55615ff967b0 2024-03-12T05:28:41.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.117+0000 7efd7b99a740 10 client.28376 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:41.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.117+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55615fef3610 con 0x55615ff967b0 2024-03-12T05:28:41.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.118+0000 7efd59ffb640 1 -- 172.21.15.100:0/707260485 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55615fbf47e0 con 0x55615ff967b0 2024-03-12T05:28:41.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.119+0000 7efd59ffb640 1 -- 172.21.15.100:0/707260485 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efd6c0323d0 con 0x55615ff967b0 2024-03-12T05:28:41.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.119+0000 7efd7b99a740 10 client.28376 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.119+0000 7efd7b99a740 10 client.28376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.119+0000 7efd7b99a740 20 client.28376 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.119+0000 7efd7b99a740 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615ffa6790 0x55615ffa8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.120+0000 7efd7b99a740 4 client.28376 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.120+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55615fc31720 con 0x55615ffa6790 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.120+0000 7efd7259c640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615ffa6790 0x55615ffa8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.120+0000 7efd59ffb640 10 client.28376 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.120+0000 7efd7259c640 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615ffa6790 0x55615ffa8bb0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5485706eb7d7de server_cookie=d1a739652cbbc19 in_seq=0 out_seq=0 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd59ffb640 1 -- 172.21.15.100:0/707260485 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55615fc31720 con 0x55615ffa6790 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd59ffb640 10 client.28376 handle_command_reply: tid=0 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 client.28376 shutdown 2024-03-12T05:28:41.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 msgr2=0x55615ff939f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615ff939f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00ec10 tx=0x7efd6c004ab0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 shutdown_connections 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615ffa6790 0x55615ffa8bb0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615ff939f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd59ffb640 0 client.28376 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff93490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 --2- 172.21.15.100:0/707260485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff96100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.123+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 >> 172.21.15.100:0/707260485 conn(0x55615ff87990 msgr2=0x55615ff89730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.124+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 shutdown_connections 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.124+0000 7efd7b99a740 1 -- 172.21.15.100:0/707260485 wait complete. 2024-03-12T05:28:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.124+0000 7efd7b99a740 20 client.28376 trim_cache size 0 max 16384 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 Processor -- start 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 -- start start 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8d2b0 0x55615ff95bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff9d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615ff83120 con 0x55615ff967b0 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fc31750 con 0x55615ff8d2b0 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd7b99a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615ff8cf50 con 0x55615ff8dbd0 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd72d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff9d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd72d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff9d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58188/0 (socket says 172.21.15.100:58188) 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd72d9d640 1 -- 172.21.15.100:0/1714182928 learned_addr learned my addr 172.21.15.100:0/1714182928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.148+0000 7efd69d9b640 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd72d9d640 1 -- 172.21.15.100:0/1714182928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 msgr2=0x55615ff96100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd72d9d640 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff96100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd72d9d640 1 -- 172.21.15.100:0/1714182928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8d2b0 msgr2=0x55615ff95bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd72d9d640 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8d2b0 0x55615ff95bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd72d9d640 1 -- 172.21.15.100:0/1714182928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55615fe8fac0 con 0x55615ff967b0 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd72d9d640 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff9d440 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7efd6c032830 tx=0x7efd6c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7a14169d9203f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd5bfff640 1 -- 172.21.15.100:0/1714182928 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00ee80 con 0x55615ff967b0 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd5bfff640 1 -- 172.21.15.100:0/1714182928 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd6c019040 con 0x55615ff967b0 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.149+0000 7efd5bfff640 1 -- 172.21.15.100:0/1714182928 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c03d680 con 0x55615ff967b0 2024-03-12T05:28:41.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 -- 172.21.15.100:0/1714182928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 msgr2=0x55615ff9d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff9d440 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7efd6c032830 tx=0x7efd6c00ec90 comp rx=0 tx=0).stop 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 -- 172.21.15.100:0/1714182928 shutdown_connections 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff967b0 0x55615ff9d440 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff96100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 --2- 172.21.15.100:0/1714182928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff8d2b0 0x55615ff95bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 -- 172.21.15.100:0/1714182928 >> 172.21.15.100:0/1714182928 conn(0x55615ff87990 msgr2=0x55615ffad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 -- 172.21.15.100:0/1714182928 shutdown_connections 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 -- 172.21.15.100:0/1714182928 wait complete. 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.150+0000 7efd7b99a740 1 -- start start 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 Processor -- start 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff9a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff9a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615fffcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615ff9b060 con 0x55615ff8d2b0 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fffd1d0 con 0x55615ff967b0 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615fffd310 con 0x55615ff8dbd0 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff9a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff9a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58190/0 (socket says 172.21.15.100:58190) 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7259c640 1 -- 172.21.15.100:0/2793891058 learned_addr learned my addr 172.21.15.100:0/2793891058 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd7b99a740 10 client.0 _ll_register_callbacks cb 0x7efd5c0f2f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.151+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615fffcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd72d9d640 1 -- 172.21.15.100:0/2793891058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 msgr2=0x55615ff9a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff9a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd72d9d640 1 -- 172.21.15.100:0/2793891058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 msgr2=0x55615ff9a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:28:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff9a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd72d9d640 1 -- 172.21.15.100:0/2793891058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55615fef3610 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd7259c640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff9a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd72d9d640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615fffcc90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000f80 tx=0x7efd6c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10fdd9344f86f9c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd59ffb640 1 -- 172.21.15.100:0/2793891058 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c032510 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd59ffb640 1 -- 172.21.15.100:0/2793891058 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efd6c01d790 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.152+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd5c0bbff0 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.153+0000 7efd59ffb640 1 -- 172.21.15.100:0/2793891058 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd6c01d950 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.153+0000 7efd7b99a740 10 client.28379 fetch_fsmap learned FSMap version 30 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.153+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55615feb56a0 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.153+0000 7efd59ffb640 1 -- 172.21.15.100:0/2793891058 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd6c051080 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.153+0000 7efd59ffb640 1 -- 172.21.15.100:0/2793891058 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efd6c0055f0 con 0x55615ff967b0 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7b99a740 10 client.28379 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7b99a740 10 client.28379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:28:41.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7b99a740 20 client.28379 populate_metadata read hostname 'smithi100' 2024-03-12T05:28:41.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7b99a740 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615fffe6e0 0x556160000ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:28:41.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7b99a740 4 client.28379 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:28:41.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55615fc80890 con 0x55615fffe6e0 2024-03-12T05:28:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.154+0000 7efd7259c640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615fffe6e0 0x556160000ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:28:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.155+0000 7efd7259c640 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615fffe6e0 0x556160000ad0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e365eab6e2a76e6 server_cookie=abed132e035ad2d7 in_seq=0 out_seq=0 2024-03-12T05:28:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.155+0000 7efd59ffb640 10 client.28379 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd59ffb640 1 -- 172.21.15.100:0/2793891058 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55615fc80890 con 0x55615fffe6e0 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd59ffb640 10 client.28379 handle_command_reply: tid=0 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 client.28379 shutdown 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 msgr2=0x55615fffcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615fffcc90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000f80 tx=0x7efd6c005c80 comp rx=0 tx=0).stop 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 shutdown_connections 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55615fffe6e0 0x556160000ad0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55615ff967b0 0x55615fffcc90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd59ffb640 0 client.28379 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55615ff8dbd0 0x55615ff9a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 --2- 172.21.15.100:0/2793891058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55615ff8d2b0 0x55615ff9a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 >> 172.21.15.100:0/2793891058 conn(0x55615ff87990 msgr2=0x55615ff861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 shutdown_connections 2024-03-12T05:28:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 1 -- 172.21.15.100:0/2793891058 wait complete. 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.156+0000 7efd7b99a740 20 client.28379 trim_cache size 0 max 16384 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: "1f5a5d76-d3c5-40c7-b6f5-17c0336c5400": { 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "1f5a5d76-d3c5-40c7-b6f5-17c0336c5400", 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:28:41.156 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:28:41.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efd600a1ba0 msgr2=0x7efd600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efd600a1ba0 0x7efd600a4060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efd6c013ea0 tx=0x7efd6c015040 comp rx=0 tx=0).stop 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 msgr2=0x7efd74197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 0x7efd74197cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd5c036040 tx=0x7efd5c02c640 comp rx=0 tx=0).stop 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 shutdown_connections 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7efd600a1ba0 0x7efd600a4060 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7419c000 0x7efd7419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd74106510 0x7efd74197cd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 --2- 172.21.15.100:0/2794935095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd740ff900 0x7efd74197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 >> 172.21.15.100:0/2794935095 conn(0x7efd74076590 msgr2=0x7efd74104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 shutdown_connections 2024-03-12T05:28:41.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:28:41.157+0000 7efd78dc7640 1 -- 172.21.15.100:0/2794935095 wait complete. 2024-03-12T05:28:41.164 INFO:tasks.cephfs.filesystem:scrub status for tag:1f5a5d76-d3c5-40c7-b6f5-17c0336c5400 - {'path': '/', 'tag': '1f5a5d76-d3c5-40c7-b6f5-17c0336c5400', 'options': 'recursive,force'} 2024-03-12T05:28:41.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 121575 30.53 MB/sec execute 84 sec latency 190.245 ms 2024-03-12T05:28:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:41 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:28:41.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:41 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:41.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:41 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:41.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:41 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:41.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:41 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:41.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:41 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:28:41.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:41 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:41.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:41 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:41.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:41 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:41.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:41 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:41.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:41 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:28:41.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:41 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:41.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:41 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:41.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:41 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:41.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:41 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 122038 30.42 MB/sec execute 85 sec latency 119.510 ms 2024-03-12T05:28:42.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:42 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:42.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:42 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:42 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:28:42.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:42 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:42 smithi156 ceph-mon[36270]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 688 op/s 2024-03-12T05:28:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:42 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:42 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:42 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:28:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:42 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:42 smithi161 ceph-mon[36043]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 688 op/s 2024-03-12T05:28:42.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:42 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:42.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:42 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:42 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:28:42.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:42 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:42.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:42 smithi100 ceph-mon[32210]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 688 op/s 2024-03-12T05:28:43.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 122705 30.45 MB/sec execute 86 sec latency 50.609 ms 2024-03-12T05:28:43.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:43 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:43.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:43 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:43.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:43 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:28:44.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 123202 30.34 MB/sec execute 87 sec latency 280.982 ms 2024-03-12T05:28:44.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:44 smithi156 ceph-mon[36270]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-03-12T05:28:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:44 smithi161 ceph-mon[36043]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-03-12T05:28:44.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:44 smithi100 ceph-mon[32210]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-03-12T05:28:45.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 123836 30.48 MB/sec execute 88 sec latency 146.142 ms 2024-03-12T05:28:46.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 124548 30.58 MB/sec execute 89 sec latency 143.844 ms 2024-03-12T05:28:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:46 smithi100 ceph-mon[32210]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.6 MiB/s wr, 514 op/s 2024-03-12T05:28:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:46 smithi156 ceph-mon[36270]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.6 MiB/s wr, 514 op/s 2024-03-12T05:28:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:46 smithi161 ceph-mon[36043]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.6 MiB/s wr, 514 op/s 2024-03-12T05:28:47.236 INFO:tasks.workunit.client.0.smithi100.stdout: 10 125134 30.63 MB/sec execute 90 sec latency 115.768 ms 2024-03-12T05:28:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 125616 30.56 MB/sec execute 91 sec latency 99.588 ms 2024-03-12T05:28:48.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:48 smithi100 ceph-mon[32210]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 748 op/s 2024-03-12T05:28:48.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:48 smithi156 ceph-mon[36270]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 748 op/s 2024-03-12T05:28:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:48 smithi161 ceph-mon[36043]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 748 op/s 2024-03-12T05:28:49.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 126004 30.41 MB/sec execute 92 sec latency 406.379 ms 2024-03-12T05:28:50.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 126603 30.42 MB/sec execute 93 sec latency 102.984 ms 2024-03-12T05:28:50.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:50 smithi100 ceph-mon[32210]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 521 op/s 2024-03-12T05:28:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:50 smithi156 ceph-mon[36270]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 521 op/s 2024-03-12T05:28:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:50 smithi161 ceph-mon[36043]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 521 op/s 2024-03-12T05:28:51.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 127279 30.48 MB/sec execute 94 sec latency 67.562 ms 2024-03-12T05:28:52.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 127859 30.50 MB/sec execute 95 sec latency 105.524 ms 2024-03-12T05:28:52.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:52 smithi100 ceph-mon[32210]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-03-12T05:28:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:52 smithi156 ceph-mon[36270]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-03-12T05:28:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:52 smithi161 ceph-mon[36043]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-03-12T05:28:53.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 128625 30.63 MB/sec execute 96 sec latency 81.429 ms 2024-03-12T05:28:54.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 129136 30.61 MB/sec execute 97 sec latency 228.270 ms 2024-03-12T05:28:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:54 smithi156 ceph-mon[36270]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 543 op/s 2024-03-12T05:28:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:54 smithi161 ceph-mon[36043]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 543 op/s 2024-03-12T05:28:54.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:54 smithi100 ceph-mon[32210]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 543 op/s 2024-03-12T05:28:55.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 129580 30.57 MB/sec execute 98 sec latency 306.220 ms 2024-03-12T05:28:55.352 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:55 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:28:55.465 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:55 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:28:55.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:55 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:28:56.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 130034 30.51 MB/sec execute 99 sec latency 155.007 ms 2024-03-12T05:28:56.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:56 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:28:56.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:56 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:28:56.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:56 smithi156 ceph-mon[36270]: pgmap v528: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.3 MiB/s wr, 478 op/s 2024-03-12T05:28:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:56 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:28:56.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:56 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:28:56.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:56 smithi161 ceph-mon[36043]: pgmap v528: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.3 MiB/s wr, 478 op/s 2024-03-12T05:28:56.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:56 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:28:56.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:56 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:28:56.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:56 smithi100 ceph-mon[32210]: pgmap v528: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.3 MiB/s wr, 478 op/s 2024-03-12T05:28:57.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 130510 30.43 MB/sec execute 100 sec latency 161.229 ms 2024-03-12T05:28:58.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 131099 30.36 MB/sec execute 101 sec latency 139.844 ms 2024-03-12T05:28:58.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:28:58 smithi100 ceph-mon[32210]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 728 op/s 2024-03-12T05:28:58.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:28:58 smithi156 ceph-mon[36270]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 728 op/s 2024-03-12T05:28:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:28:58 smithi161 ceph-mon[36043]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 728 op/s 2024-03-12T05:28:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 131581 30.29 MB/sec execute 102 sec latency 147.561 ms 2024-03-12T05:29:00.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 132123 30.33 MB/sec execute 103 sec latency 285.246 ms 2024-03-12T05:29:00.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:00 smithi156 ceph-mon[36270]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 494 op/s 2024-03-12T05:29:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:00 smithi161 ceph-mon[36043]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 494 op/s 2024-03-12T05:29:00.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:00 smithi100 ceph-mon[32210]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 494 op/s 2024-03-12T05:29:01.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 132597 30.26 MB/sec execute 104 sec latency 128.408 ms 2024-03-12T05:29:02.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 132885 30.15 MB/sec execute 105 sec latency 126.081 ms 2024-03-12T05:29:02.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:02 smithi156 ceph-mon[36270]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-03-12T05:29:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:02 smithi161 ceph-mon[36043]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-03-12T05:29:02.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:02 smithi100 ceph-mon[32210]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-03-12T05:29:03.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 133469 30.23 MB/sec execute 106 sec latency 78.048 ms 2024-03-12T05:29:04.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 134007 30.21 MB/sec execute 107 sec latency 70.667 ms 2024-03-12T05:29:04.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:04 smithi156 ceph-mon[36270]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-12T05:29:04.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:04 smithi161 ceph-mon[36043]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-12T05:29:04.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:04 smithi100 ceph-mon[32210]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-12T05:29:05.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 134375 30.13 MB/sec execute 108 sec latency 233.344 ms 2024-03-12T05:29:06.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 134942 30.09 MB/sec execute 109 sec latency 132.610 ms 2024-03-12T05:29:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:06 smithi156 ceph-mon[36270]: osdmap e110: 12 total, 12 up, 12 in 2024-03-12T05:29:06.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:06 smithi156 ceph-mon[36270]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 765 op/s 2024-03-12T05:29:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:06 smithi161 ceph-mon[36043]: osdmap e110: 12 total, 12 up, 12 in 2024-03-12T05:29:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:06 smithi161 ceph-mon[36043]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 765 op/s 2024-03-12T05:29:06.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:06 smithi100 ceph-mon[32210]: osdmap e110: 12 total, 12 up, 12 in 2024-03-12T05:29:06.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:06 smithi100 ceph-mon[32210]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 765 op/s 2024-03-12T05:29:07.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 135577 30.15 MB/sec execute 110 sec latency 83.742 ms 2024-03-12T05:29:08.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 136283 30.14 MB/sec execute 111 sec latency 302.216 ms 2024-03-12T05:29:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:08 smithi156 ceph-mon[36270]: pgmap v535: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 781 op/s 2024-03-12T05:29:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:08 smithi161 ceph-mon[36043]: pgmap v535: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 781 op/s 2024-03-12T05:29:08.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:08 smithi100 ceph-mon[32210]: pgmap v535: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 781 op/s 2024-03-12T05:29:09.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 136921 30.24 MB/sec execute 112 sec latency 1302.249 ms 2024-03-12T05:29:10.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 137302 30.20 MB/sec execute 113 sec latency 2302.290 ms 2024-03-12T05:29:10.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:10 smithi156 ceph-mon[36270]: pgmap v536: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 781 op/s 2024-03-12T05:29:10.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:10 smithi161 ceph-mon[36043]: pgmap v536: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 781 op/s 2024-03-12T05:29:10.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:10 smithi100 ceph-mon[32210]: pgmap v536: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 781 op/s 2024-03-12T05:29:11.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:29:11.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:29:11.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 137664 30.09 MB/sec execute 114 sec latency 3302.335 ms 2024-03-12T05:29:11.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.290+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3238596325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 msgr2=0x7f1efc100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.290+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3238596325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc100650 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009930 tx=0x7f1eec02bc30 comp rx=0 tx=0).stop 2024-03-12T05:29:11.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3238596325 shutdown_connections 2024-03-12T05:29:11.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3238596325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc105340 0x7f1efc10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3238596325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc100c20 0x7f1efc104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3238596325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc100650 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3238596325 >> 172.21.15.100:0/3238596325 conn(0x7f1efc077230 msgr2=0x7f1efc077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3238596325 shutdown_connections 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3238596325 wait complete. 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 Processor -- start 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- start start 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc105340 0x7f1efc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc1990b0 0x7f1efc1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a3d00 con 0x7f1efc100270 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a3e70 con 0x7f1efc1990b0 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.291+0000 7f1f01f3a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a4030 con 0x7f1efc105340 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42750/0 (socket says 172.21.15.100:42750) 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 -- 172.21.15.100:0/3412038540 learned_addr learned my addr 172.21.15.100:0/3412038540 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efaffd640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc105340 0x7f1efc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efbfff640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc1990b0 0x7f1efc1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 -- 172.21.15.100:0/3412038540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc105340 msgr2=0x7f1efc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc105340 0x7f1efc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 -- 172.21.15.100:0/3412038540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc1990b0 msgr2=0x7f1efc1a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc1990b0 0x7f1efc1a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 -- 172.21.15.100:0/3412038540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec009560 con 0x7f1efc100270 2024-03-12T05:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc1985e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009230 tx=0x7f1eec02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1ef8ff9640 1 -- 172.21.15.100:0/3412038540 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec004350 con 0x7f1efc100270 2024-03-12T05:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc1a42b0 con 0x7f1efc100270 2024-03-12T05:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1ef8ff9640 1 -- 172.21.15.100:0/3412038540 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1eec0044b0 con 0x7f1efc100270 2024-03-12T05:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1ef8ff9640 1 -- 172.21.15.100:0/3412038540 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec003d70 con 0x7f1efc100270 2024-03-12T05:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.292+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc1a4810 con 0x7f1efc100270 2024-03-12T05:29:11.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1ef8ff9640 1 -- 172.21.15.100:0/3412038540 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1eec004df0 con 0x7f1efc100270 2024-03-12T05:29:11.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 Processor -- start 2024-03-12T05:29:11.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 -- start start 2024-03-12T05:29:11.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x55925984e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259857270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259859ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592597bd280 con 0x5592598577b0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592597ba7b0 con 0x55925984e2b0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1f04be3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592597b5f90 con 0x55925984ebd0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1ef8ff9640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1ee40a1ba0 0x7f1ee40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1efaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259859ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259857270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1efaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259859ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42760/0 (socket says 172.21.15.100:42760) 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1efaffd640 1 -- 172.21.15.100:0/1181365684 learned_addr learned my addr 172.21.15.100:0/1181365684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1efbfff640 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x55925984e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.295+0000 7f1efbfff640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1ee40a1ba0 0x7f1ee40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efaffd640 1 -- 172.21.15.100:0/1181365684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 msgr2=0x559259857270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efaffd640 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259857270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efaffd640 1 -- 172.21.15.100:0/1181365684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 msgr2=0x55925984e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efaffd640 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x55925984e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efaffd640 1 -- 172.21.15.100:0/1181365684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559259774570 con 0x5592598577b0 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1ef8ff9640 1 -- 172.21.15.100:0/3412038540 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1eec0ef7d0 con 0x7f1efc100270 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efaffd640 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259859ba0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009c60 tx=0x7f1ef0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a26e81b3805e966 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1efbfff640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1ee40a1ba0 0x7f1ee40a4060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009fd0 tx=0x7f1ef4009600 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1ee1ffb640 1 -- 172.21.15.100:0/1181365684 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0004230 con 0x5592598577b0 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1ee1ffb640 1 -- 172.21.15.100:0/1181365684 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0004390 con 0x5592598577b0 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.296+0000 7f1ee1ffb640 1 -- 172.21.15.100:0/1181365684 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0019740 con 0x5592598577b0 2024-03-12T05:29:11.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 -- 172.21.15.100:0/1181365684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 msgr2=0x559259859ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259859ba0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009c60 tx=0x7f1ef0011810 comp rx=0 tx=0).stop 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 -- 172.21.15.100:0/1181365684 shutdown_connections 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259859ba0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259857270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 --2- 172.21.15.100:0/1181365684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x55925984e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 -- 172.21.15.100:0/1181365684 >> 172.21.15.100:0/1181365684 conn(0x559259848990 msgr2=0x559259848d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 -- 172.21.15.100:0/1181365684 shutdown_connections 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 -- 172.21.15.100:0/1181365684 wait complete. 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 -- start start 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.297+0000 7f1f04be3740 1 Processor -- start 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x5592598571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259854530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259854b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559259489430 con 0x5592598577b0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559259445e00 con 0x55925984e2b0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925973afd0 con 0x55925984ebd0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 10 client.0 _ll_register_callbacks cb 0x559259790fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259854b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259854b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42762/0 (socket says 172.21.15.100:42762) 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 -- 172.21.15.100:0/4247246074 learned_addr learned my addr 172.21.15.100:0/4247246074 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efaffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259854530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x5592598571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 -- 172.21.15.100:0/4247246074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 msgr2=0x559259854530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259854530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 -- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 msgr2=0x5592598571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x5592598571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 -- 172.21.15.100:0/4247246074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef0009890 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1efbfff640 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259854b40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400ab90 tx=0x7f1ef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175a48d7f35ff00b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1edaffd640 1 -- 172.21.15.100:0/4247246074 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef401ebb0 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1edaffd640 1 -- 172.21.15.100:0/4247246074 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1ef401ed10 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.298+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592594b57e0 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1edaffd640 1 -- 172.21.15.100:0/4247246074 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ef4004b20 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 10 client.19047 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5592597b4610 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1edaffd640 1 -- 172.21.15.100:0/4247246074 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ef4034020 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1edaffd640 1 -- 172.21.15.100:0/4247246074 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ef4005700 con 0x5592598577b0 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 10 client.19047 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:11.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 20 client.19047 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559259867960 0x559259869d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 4 client.19047 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592594f2720 con 0x559259867960 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.299+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559259867960 0x559259869d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.300+0000 7f1edaffd640 10 client.19047 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.300+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559259867960 0x559259869d80 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292063af76604cb7 server_cookie=7f8160e00ffaad6e in_seq=0 out_seq=0 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1edaffd640 1 -- 172.21.15.100:0/4247246074 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592594f2720 con 0x559259867960 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1edaffd640 10 client.19047 handle_command_reply: tid=0 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 client.19047 shutdown 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 msgr2=0x559259854b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259854b40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400ab90 tx=0x7f1ef4004a40 comp rx=0 tx=0).stop 2024-03-12T05:29:11.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 shutdown_connections 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559259867960 0x559259869d80 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1edaffd640 0 client.19047 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x559259854b40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984ebd0 0x559259854530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 --2- 172.21.15.100:0/4247246074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984e2b0 0x5592598571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 >> 172.21.15.100:0/4247246074 conn(0x559259848990 msgr2=0x55925984a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 shutdown_connections 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 1 -- 172.21.15.100:0/4247246074 wait complete. 2024-03-12T05:29:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.301+0000 7f1f04be3740 20 client.19047 trim_cache size 0 max 16384 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 Processor -- start 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 -- start start 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x5592598478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984ebd0 0x559259856c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x5592598571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925984df10 con 0x5592598577b0 2024-03-12T05:29:11.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925984f320 con 0x55925984ebd0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1f04be3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925984d4b0 con 0x55925984e2b0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x5592598571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x5592598571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42766/0 (socket says 172.21.15.100:42766) 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 -- 172.21.15.100:0/131973168 learned_addr learned my addr 172.21.15.100:0/131973168 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x5592598478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efaffd640 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984ebd0 0x559259856c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 -- 172.21.15.100:0/131973168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 msgr2=0x5592598478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x5592598478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 -- 172.21.15.100:0/131973168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984ebd0 msgr2=0x559259856c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984ebd0 0x559259856c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efbfff640 1 -- 172.21.15.100:0/131973168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559259750ac0 con 0x5592598577b0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.316+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x5592598478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1efbfff640 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x5592598571a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40098b0 tx=0x7f1ef400afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c146346ecd807f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1ee17fa640 1 -- 172.21.15.100:0/131973168 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4005bd0 con 0x5592598577b0 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1ee17fa640 1 -- 172.21.15.100:0/131973168 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ef4034040 con 0x5592598577b0 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1ee17fa640 1 -- 172.21.15.100:0/131973168 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4028d30 con 0x5592598577b0 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 -- 172.21.15.100:0/131973168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 msgr2=0x5592598571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x5592598571a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40098b0 tx=0x7f1ef400afb0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 -- 172.21.15.100:0/131973168 shutdown_connections 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592598577b0 0x5592598571a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925984ebd0 0x559259856c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 --2- 172.21.15.100:0/131973168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x5592598478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 -- 172.21.15.100:0/131973168 >> 172.21.15.100:0/131973168 conn(0x559259848990 msgr2=0x5592598471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 -- 172.21.15.100:0/131973168 shutdown_connections 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.317+0000 7f1f04be3740 1 -- 172.21.15.100:0/131973168 wait complete. 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 -- start start 2024-03-12T05:29:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 Processor -- start 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x55925985bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 0x55925985c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592598577b0 0x55925985e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925985e5a0 con 0x55925984ebd0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925985e710 con 0x5592598577b0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925985e8d0 con 0x55925984e2b0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1f04be3740 10 client.0 _ll_register_callbacks cb 0x7f1ef0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 0x55925985c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x55925985bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 0x55925985c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42778/0 (socket says 172.21.15.100:42778) 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 -- 172.21.15.100:0/2898485822 learned_addr learned my addr 172.21.15.100:0/2898485822 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efbfff640 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592598577b0 0x55925985e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 -- 172.21.15.100:0/2898485822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 msgr2=0x55925985bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x55925985bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 -- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592598577b0 msgr2=0x55925985e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592598577b0 0x55925985e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.318+0000 7f1efaffd640 1 -- 172.21.15.100:0/2898485822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592597b4610 con 0x55925984ebd0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1efaffd640 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 0x55925985c040 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009c60 tx=0x7f1ef0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2ae5c92bfce958 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1edaffd640 1 -- 172.21.15.100:0/2898485822 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00052c0 con 0x55925984ebd0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1edaffd640 1 -- 172.21.15.100:0/2898485822 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1ef0005420 con 0x55925984ebd0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ef0004390 con 0x55925984ebd0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1edaffd640 1 -- 172.21.15.100:0/2898485822 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ef00055e0 con 0x55925984ebd0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 10 client.19053 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5592597766a0 con 0x55925984ebd0 2024-03-12T05:29:11.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1edaffd640 1 -- 172.21.15.100:0/2898485822 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0004390 con 0x55925984ebd0 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1edaffd640 1 -- 172.21.15.100:0/2898485822 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ef0004950 con 0x55925984ebd0 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 10 client.19053 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 20 client.19053 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5592598c0140 0x5592598c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 4 client.19053 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.319+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559259541890 con 0x5592598c0140 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5592598c0140 0x5592598c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1efb7fe640 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5592598c0140 0x5592598c2560 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6356787f76a9c17f server_cookie=4460a43a37fa7380 in_seq=0 out_seq=0 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1edaffd640 10 client.19053 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1edaffd640 1 -- 172.21.15.100:0/2898485822 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559259541890 con 0x5592598c0140 2024-03-12T05:29:11.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1edaffd640 10 client.19053 handle_command_reply: tid=0 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1f04be3740 1 client.19053 shutdown 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 msgr2=0x55925985c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.320+0000 7f1f04be3740 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 0x55925985c040 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009c60 tx=0x7f1ef0009640 comp rx=0 tx=0).stop 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 shutdown_connections 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5592598c0140 0x5592598c2560 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592598577b0 0x55925985e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55925984ebd0 0x55925985c040 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1edaffd640 0 client.19053 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 --2- 172.21.15.100:0/2898485822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55925984e2b0 0x55925985bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 >> 172.21.15.100:0/2898485822 conn(0x559259848990 msgr2=0x559259847a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 shutdown_connections 2024-03-12T05:29:11.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 1 -- 172.21.15.100:0/2898485822 wait complete. 2024-03-12T05:29:11.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f04be3740 20 client.19053 trim_cache size 0 max 16384 2024-03-12T05:29:11.323 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:29:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:29:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:29:11.323 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:29:11.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1ee40a1ba0 msgr2=0x7f1ee40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1ee40a1ba0 0x7f1ee40a4060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009fd0 tx=0x7f1ef4009600 comp rx=0 tx=0).stop 2024-03-12T05:29:11.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 msgr2=0x7f1efc1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc1985e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009230 tx=0x7f1eec02c6c0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 shutdown_connections 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1ee40a1ba0 0x7f1ee40a4060 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc1990b0 0x7f1efc1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc105340 0x7f1efc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 --2- 172.21.15.100:0/3412038540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1efc100270 0x7f1efc1985e0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.321+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 >> 172.21.15.100:0/3412038540 conn(0x7f1efc077230 msgr2=0x7f1efc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.322+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 shutdown_connections 2024-03-12T05:29:11.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.322+0000 7f1f01f3a640 1 -- 172.21.15.100:0/3412038540 wait complete. 2024-03-12T05:29:11.327 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:29:11.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:29:11.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 -- 172.21.15.100:0/124088337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 msgr2=0x7f667810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 --2- 172.21.15.100:0/124088337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667810e020 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6670009cd0 tx=0x7f667002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 -- 172.21.15.100:0/124088337 shutdown_connections 2024-03-12T05:29:11.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 --2- 172.21.15.100:0/124088337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667810e020 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 --2- 172.21.15.100:0/124088337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678101d30 0x7f6678105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 --2- 172.21.15.100:0/124088337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6678101380 0x7f6678101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.444+0000 7f667f7dd640 1 -- 172.21.15.100:0/124088337 >> 172.21.15.100:0/124088337 conn(0x7f66780fcc20 msgr2=0x7f66780ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 -- 172.21.15.100:0/124088337 shutdown_connections 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 -- 172.21.15.100:0/124088337 wait complete. 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 Processor -- start 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 -- start start 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6678101380 0x7f667819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678197600 0x7f6678197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6678197ff0 con 0x7f6678106450 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6678198160 con 0x7f6678101380 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.445+0000 7f667f7dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6678198320 con 0x7f6678197600 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667d579640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6678101380 0x7f667819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42800/0 (socket says 172.21.15.100:42800) 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 -- 172.21.15.100:0/2074024737 learned_addr learned my addr 172.21.15.100:0/2074024737 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667dd7a640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678197600 0x7f6678197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 -- 172.21.15.100:0/2074024737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678197600 msgr2=0x7f6678197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678197600 0x7f6678197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 -- 172.21.15.100:0/2074024737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6678101380 msgr2=0x7f667819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6678101380 0x7f667819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 -- 172.21.15.100:0/2074024737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009900 con 0x7f6678106450 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667cd78640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667819d990 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f666800e9f0 tx=0x7f666800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680096c0 con 0x7f6678106450 2024-03-12T05:29:11.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66780733b0 con 0x7f6678106450 2024-03-12T05:29:11.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6668005500 con 0x7f6678106450 2024-03-12T05:29:11.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668010430 con 0x7f6678106450 2024-03-12T05:29:11.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.446+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66780738c0 con 0x7f6678106450 2024-03-12T05:29:11.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.447+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678067f70 con 0x7f6678106450 2024-03-12T05:29:11.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.448+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6668005000 con 0x7f6678106450 2024-03-12T05:29:11.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.449+0000 7f666e7fc640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f66640a1ba0 0x7f66640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.449+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6668014070 con 0x7f6678106450 2024-03-12T05:29:11.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.449+0000 7f667d579640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f66640a1ba0 0x7f66640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.450+0000 7f667d579640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f66640a1ba0 0x7f66640a4060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6674009bd0 tx=0x7f6674005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.453+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6668096bd0 con 0x7f6678106450 2024-03-12T05:29:11.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.686+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6678109740 con 0x7f6678106450 2024-03-12T05:29:11.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.687+0000 7f666e7fc640 1 -- 172.21.15.100:0/2074024737 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f666809a290 con 0x7f6678106450 2024-03-12T05:29:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:29:11.681 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:29:11.681 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f66640a1ba0 msgr2=0x7f66640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f66640a1ba0 0x7f66640a4060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6674009bd0 tx=0x7f6674005f90 comp rx=0 tx=0).stop 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 msgr2=0x7f667819d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667819d990 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f666800e9f0 tx=0x7f666800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 shutdown_connections 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f66640a1ba0 0x7f66640a4060 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678197600 0x7f6678197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678106450 0x7f667819d990 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 --2- 172.21.15.100:0/2074024737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6678101380 0x7f667819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 >> 172.21.15.100:0/2074024737 conn(0x7f66780fcc20 msgr2=0x7f66780fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 shutdown_connections 2024-03-12T05:29:11.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.689+0000 7f667f7dd640 1 -- 172.21.15.100:0/2074024737 wait complete. 2024-03-12T05:29:11.694 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:29:11.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:29:11.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 -- 172.21.15.100:0/171948696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc106450 msgr2=0x7fb8dc10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 --2- 172.21.15.100:0/171948696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc106450 0x7fb8dc10e020 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009cd0 tx=0x7fb8d402c0f0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 -- 172.21.15.100:0/171948696 shutdown_connections 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 --2- 172.21.15.100:0/171948696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc106450 0x7fb8dc10e020 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 --2- 172.21.15.100:0/171948696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc101d30 0x7fb8dc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 --2- 172.21.15.100:0/171948696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc101380 0x7fb8dc101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 -- 172.21.15.100:0/171948696 >> 172.21.15.100:0/171948696 conn(0x7fb8dc078540 msgr2=0x7fb8dc078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 -- 172.21.15.100:0/171948696 shutdown_connections 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.816+0000 7fb8e253a640 1 -- 172.21.15.100:0/171948696 wait complete. 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 Processor -- start 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 -- start start 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc101380 0x7fb8dc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc106450 0x7fb8dc197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 0x7fb8dc19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19ca10 con 0x7fb8dc19c020 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19cb80 con 0x7fb8dc101380 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e253a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19ccf0 con 0x7fb8dc106450 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e0ad7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 0x7fb8dc19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e0ad7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 0x7fb8dc19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42808/0 (socket says 172.21.15.100:42808) 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8e0ad7640 1 -- 172.21.15.100:0/89438231 learned_addr learned my addr 172.21.15.100:0/89438231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc106450 0x7fb8dc197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.817+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc101380 0x7fb8dc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e0ad7640 1 -- 172.21.15.100:0/89438231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc106450 msgr2=0x7fb8dc197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e0ad7640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc106450 0x7fb8dc197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e0ad7640 1 -- 172.21.15.100:0/89438231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc101380 msgr2=0x7fb8dc197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e0ad7640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc101380 0x7fb8dc197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e0ad7640 1 -- 172.21.15.100:0/89438231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4009900 con 0x7fb8dc19c020 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e0ad7640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 0x7fb8dc19c4d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005e30 tx=0x7fb8d40325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8d97fa640 1 -- 172.21.15.100:0/89438231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4034e20 con 0x7fb8dc19c020 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc1a42d0 con 0x7fb8dc19c020 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8d97fa640 1 -- 172.21.15.100:0/89438231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8d4003e50 con 0x7fb8dc19c020 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8d97fa640 1 -- 172.21.15.100:0/89438231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4004750 con 0x7fb8dc19c020 2024-03-12T05:29:11.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.818+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc1a46f0 con 0x7fb8dc19c020 2024-03-12T05:29:11.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.820+0000 7fb8d97fa640 1 -- 172.21.15.100:0/89438231 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb8d400a040 con 0x7fb8dc19c020 2024-03-12T05:29:11.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 Processor -- start 2024-03-12T05:29:11.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 -- start start 2024-03-12T05:29:11.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474be210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f474c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f47470960 con 0x555f474b5250 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f47424310 con 0x555f474b5b70 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f47421840 con 0x555f474be750 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8d97fa640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8c40a1ba0 0x7fb8c40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42810/0 (socket says 172.21.15.100:42810) 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 -- 172.21.15.100:0/586919095 learned_addr learned my addr 172.21.15.100:0/586919095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f474c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8e0ad7640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8c40a1ba0 0x7fb8c40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 -- 172.21.15.100:0/586919095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 msgr2=0x555f474c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f474c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 -- 172.21.15.100:0/586919095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 msgr2=0x555f474be210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474be210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8db7fe640 1 -- 172.21.15.100:0/586919095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f473df060 con 0x555f474b5250 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.821+0000 7fb8d97fa640 1 -- 172.21.15.100:0/89438231 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb8d40f0380 con 0x7fb8dc19c020 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b5630 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f70 tx=0x7fb8d0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90b8dd166ebfe0f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8c27fc640 1 -- 172.21.15.100:0/586919095 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004050 con 0x555f474b5250 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8c27fc640 1 -- 172.21.15.100:0/586919095 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8d00041b0 con 0x555f474b5250 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8c27fc640 1 -- 172.21.15.100:0/586919095 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000f720 con 0x555f474b5250 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 -- 172.21.15.100:0/586919095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 msgr2=0x555f474b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e0ad7640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8c40a1ba0 0x7fb8c40a4060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40edc50 tx=0x7fb8d40095c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b5630 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f70 tx=0x7fb8d0005850 comp rx=0 tx=0).stop 2024-03-12T05:29:11.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 -- 172.21.15.100:0/586919095 shutdown_connections 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f474c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474be210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/586919095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b5630 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 -- 172.21.15.100:0/586919095 >> 172.21.15.100:0/586919095 conn(0x555f474af980 msgr2=0x555f474afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.822+0000 7fb8e51e7740 1 -- 172.21.15.100:0/586919095 shutdown_connections 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 -- 172.21.15.100:0/586919095 wait complete. 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 -- start start 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 Processor -- start 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5250 0x555f474bb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 0x555f474bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4741d020 con 0x555f474be750 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f470f0430 con 0x555f474b5250 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f470ace00 con 0x555f474b5b70 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8e51e7740 10 client.0 _ll_register_callbacks cb 0x555f473c4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 0x555f474bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8dbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 0x555f474bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42814/0 (socket says 172.21.15.100:42814) 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8db7fe640 1 -- 172.21.15.100:0/647289821 learned_addr learned my addr 172.21.15.100:0/647289821 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.823+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5250 0x555f474bb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8dbfff640 1 -- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5250 msgr2=0x555f474bb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5250 0x555f474bb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8dbfff640 1 -- 172.21.15.100:0/647289821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 msgr2=0x555f474bc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 0x555f474bc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8dbfff640 1 -- 172.21.15.100:0/647289821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0009cc0 con 0x555f474b5b70 2024-03-12T05:29:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 0x555f474bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474bbc80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0126d0 tx=0x7fb8cc012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7e2bf80afafc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/647289821 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc017070 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f4711c7e0 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/647289821 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb8cc00f8c0 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/647289821 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8cc00fa80 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8e51e7740 10 client.28357 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555f4741b6a0 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.824+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/647289821 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc027020 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/647289821 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8cc01c420 con 0x555f474b5b70 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8e51e7740 10 client.28357 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8e51e7740 10 client.28357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8e51e7740 20 client.28357 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f474cec30 0x555f474d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8e51e7740 4 client.28357 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f47159720 con 0x555f474cec30 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f474cec30 0x555f474d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f474cec30 0x555f474d1050 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fe311a149ab1c52 server_cookie=9ca665f96e4a92d9 in_seq=0 out_seq=0 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.825+0000 7fb8bb7fe640 10 client.28357 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/647289821 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f47159720 con 0x555f474cec30 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8bb7fe640 10 client.28357 handle_command_reply: tid=0 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 client.28357 shutdown 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 msgr2=0x555f474bbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474bbc80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0126d0 tx=0x7fb8cc012b90 comp rx=0 tx=0).stop 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 shutdown_connections 2024-03-12T05:29:11.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f474cec30 0x555f474d1050 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474be750 0x555f474bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474bbc80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/647289821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5250 0x555f474bb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8bb7fe640 0 client.28357 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 >> 172.21.15.100:0/647289821 conn(0x555f474af980 msgr2=0x555f474b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 shutdown_connections 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 1 -- 172.21.15.100:0/647289821 wait complete. 2024-03-12T05:29:11.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.828+0000 7fb8e51e7740 20 client.28357 trim_cache size 0 max 16384 2024-03-12T05:29:11.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 Processor -- start 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 -- start start 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474be750 0x555f474ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f473a2130 con 0x555f474b5250 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f474af7f0 con 0x555f474be750 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f474cb610 con 0x555f474b5b70 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42830/0 (socket says 172.21.15.100:42830) 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 -- 172.21.15.100:0/3041246057 learned_addr learned my addr 172.21.15.100:0/3041246057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474be750 0x555f474ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 -- 172.21.15.100:0/3041246057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 msgr2=0x555f474b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 -- 172.21.15.100:0/3041246057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474be750 msgr2=0x555f474ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474be750 0x555f474ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 -- 172.21.15.100:0/3041246057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f473b7c20 con 0x555f474b5250 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b89a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0013ea0 tx=0x7fb8d0003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcbab53c18deda2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.841+0000 7fb8c1ffb640 1 -- 172.21.15.100:0/3041246057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00037f0 con 0x555f474b5250 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8c1ffb640 1 -- 172.21.15.100:0/3041246057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8d0005500 con 0x555f474b5250 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8c1ffb640 1 -- 172.21.15.100:0/3041246057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004c40 con 0x555f474b5250 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 -- 172.21.15.100:0/3041246057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 msgr2=0x555f474b89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b89a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0013ea0 tx=0x7fb8d0003d90 comp rx=0 tx=0).stop 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 -- 172.21.15.100:0/3041246057 shutdown_connections 2024-03-12T05:29:11.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474be750 0x555f474ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474b5b70 0x555f474b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/3041246057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474b89a0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 -- 172.21.15.100:0/3041246057 >> 172.21.15.100:0/3041246057 conn(0x555f474af980 msgr2=0x555f474ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 -- 172.21.15.100:0/3041246057 shutdown_connections 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.842+0000 7fb8e51e7740 1 -- 172.21.15.100:0/3041246057 wait complete. 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 -- start start 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 Processor -- start 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f47525100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f474c52f0 con 0x555f474b5250 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f47525640 con 0x555f474b5b70 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f475257b0 con 0x555f474be750 2024-03-12T05:29:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8e51e7740 10 client.0 _ll_register_callbacks cb 0x7fb8d00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42840/0 (socket says 172.21.15.100:42840) 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8db7fe640 1 -- 172.21.15.100:0/2663368182 learned_addr learned my addr 172.21.15.100:0/2663368182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.843+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f47525100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8db7fe640 1 -- 172.21.15.100:0/2663368182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 msgr2=0x555f47525100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f47525100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8db7fe640 1 -- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 msgr2=0x555f474c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8db7fe640 1 -- 172.21.15.100:0/2663368182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f473df060 con 0x555f474b5250 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8db7fe640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474c46b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00059f0 tx=0x7fb8d0004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3044aab718c2ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:11.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/2663368182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00036b0 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/2663368182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb8d0003810 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8d00041b0 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/2663368182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8d00039d0 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 10 client.19077 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555f473dd8c0 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/2663368182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8d0003b30 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/2663368182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8d0043070 con 0x555f474b5250 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 10 client.19077 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 10 client.19077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 20 client.19077 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f47526af0 0x555f47528ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 4 client.19077 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:29:11.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.844+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fb8d00041b0 con 0x555f47526af0 2024-03-12T05:29:11.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.845+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f47526af0 0x555f47528ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.845+0000 7fb8bb7fe640 10 client.19077 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:11.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.845+0000 7fb8dbfff640 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f47526af0 0x555f47528ee0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb20f66895dac0e server_cookie=53cb05601c710a4c in_seq=0 out_seq=0 2024-03-12T05:29:11.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8bb7fe640 1 -- 172.21.15.100:0/2663368182 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fb8d00041b0 con 0x555f47526af0 2024-03-12T05:29:11.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8bb7fe640 10 client.19077 handle_command_reply: tid=0 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 client.19077 shutdown 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 msgr2=0x555f474c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474c46b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00059f0 tx=0x7fb8d0004820 comp rx=0 tx=0).stop 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 shutdown_connections 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555f47526af0 0x555f47528ee0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f474be750 0x555f47525100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555f474b5b70 0x555f474c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8bb7fe640 0 client.19077 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 --2- 172.21.15.100:0/2663368182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f474b5250 0x555f474c46b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 >> 172.21.15.100:0/2663368182 conn(0x555f474af980 msgr2=0x555f474aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 shutdown_connections 2024-03-12T05:29:11.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 1 -- 172.21.15.100:0/2663368182 wait complete. 2024-03-12T05:29:11.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e51e7740 20 client.19077 trim_cache size 0 max 16384 2024-03-12T05:29:11.848 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.850+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8c40a1ba0 msgr2=0x7fb8c40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8c40a1ba0 0x7fb8c40a4060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40edc50 tx=0x7fb8d40095c0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 msgr2=0x7fb8dc19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 0x7fb8dc19c4d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005e30 tx=0x7fb8d40325e0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 shutdown_connections 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8c40a1ba0 0x7fb8c40a4060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8dc19c020 0x7fb8dc19c4d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc106450 0x7fb8dc197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 --2- 172.21.15.100:0/89438231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc101380 0x7fb8dc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 >> 172.21.15.100:0/89438231 conn(0x7fb8dc078540 msgr2=0x7fb8dc0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 shutdown_connections 2024-03-12T05:29:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.851+0000 7fb8e253a640 1 -- 172.21.15.100:0/89438231 wait complete. 2024-03-12T05:29:11.856 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:29:11.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:29:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 -- 172.21.15.100:0/2918007447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b80685f0 msgr2=0x7f40b80689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 --2- 172.21.15.100:0/2918007447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b80685f0 0x7f40b80689d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009930 tx=0x7f40b402bc30 comp rx=0 tx=0).stop 2024-03-12T05:29:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 -- 172.21.15.100:0/2918007447 shutdown_connections 2024-03-12T05:29:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 --2- 172.21.15.100:0/2918007447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40b81064c0 0x7f40b810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 --2- 172.21.15.100:0/2918007447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40b8068fa0 0x7f40b8105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 --2- 172.21.15.100:0/2918007447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b80685f0 0x7f40b80689d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 -- 172.21.15.100:0/2918007447 >> 172.21.15.100:0/2918007447 conn(0x7f40b8077180 msgr2=0x7f40b8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 -- 172.21.15.100:0/2918007447 shutdown_connections 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.974+0000 7f40c0239640 1 -- 172.21.15.100:0/2918007447 wait complete. 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 Processor -- start 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 -- start start 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40b80685f0 0x7f40b819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 0x7f40b819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40b8197610 0x7f40b8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8198000 con 0x7f40b81064c0 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8198170 con 0x7f40b80685f0 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40c0239640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8198330 con 0x7f40b8197610 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 0x7f40b819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 0x7f40b819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42872/0 (socket says 172.21.15.100:42872) 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 -- 172.21.15.100:0/1769229131 learned_addr learned my addr 172.21.15.100:0/1769229131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40be7d6640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40b8197610 0x7f40b8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40b80685f0 0x7f40b819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 -- 172.21.15.100:0/1769229131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40b8197610 msgr2=0x7f40b8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40b8197610 0x7f40b8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 -- 172.21.15.100:0/1769229131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40b80685f0 msgr2=0x7f40b819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40b80685f0 0x7f40b819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 -- 172.21.15.100:0/1769229131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4009560 con 0x7f40b81064c0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 0x7f40b819d9a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e9f0 tx=0x7f40a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.975+0000 7f40aeffd640 1 -- 172.21.15.100:0/1769229131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80096a0 con 0x7f40b81064c0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.976+0000 7f40aeffd640 1 -- 172.21.15.100:0/1769229131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40a800ccc0 con 0x7f40b81064c0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.976+0000 7f40aeffd640 1 -- 172.21.15.100:0/1769229131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8018860 con 0x7f40b81064c0 2024-03-12T05:29:11.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.976+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b81a4320 con 0x7f40b81064c0 2024-03-12T05:29:11.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.976+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b81a4830 con 0x7f40b81064c0 2024-03-12T05:29:11.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40aeffd640 1 -- 172.21.15.100:0/1769229131 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f40a8010aa0 con 0x7f40b81064c0 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 Processor -- start 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 -- start start 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfcda630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfc95960 con 0x563cbfce3750 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfc49310 con 0x563cbfcdab70 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.978+0000 7f40c2e08740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfc46840 con 0x563cbfcda250 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40aeffd640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40a40a1ba0 0x7f40a40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40bd7d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfcda630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40bdfd5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfcda630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:53150/0 (socket says 172.21.15.100:53150) 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 -- 172.21.15.100:0/2116455649 learned_addr learned my addr 172.21.15.100:0/2116455649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40a40a1ba0 0x7f40a40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40aeffd640 1 -- 172.21.15.100:0/1769229131 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40a8014070 con 0x7f40b81064c0 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 -- 172.21.15.100:0/2116455649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 msgr2=0x563cbfce3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 -- 172.21.15.100:0/2116455649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 msgr2=0x563cbfce5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 -- 172.21.15.100:0/2116455649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbfc04060 con 0x563cbfcda250 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40a40a1ba0 0x7f40a40a4060 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40b000fc90 tx=0x7f40b001e000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfcda630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40b0011e20 tx=0x7f40b000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b4d08814fb93fc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40a3fff640 1 -- 172.21.15.100:0/2116455649 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b001dcb0 con 0x563cbfcda250 2024-03-12T05:29:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.979+0000 7f40a3fff640 1 -- 172.21.15.100:0/2116455649 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40b0009e50 con 0x563cbfcda250 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40a3fff640 1 -- 172.21.15.100:0/2116455649 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0004230 con 0x563cbfcda250 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 -- 172.21.15.100:0/2116455649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 msgr2=0x563cbfcda630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfcda630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40b0011e20 tx=0x7f40b000f790 comp rx=0 tx=0).stop 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 -- 172.21.15.100:0/2116455649 shutdown_connections 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2116455649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfcda630 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 -- 172.21.15.100:0/2116455649 >> 172.21.15.100:0/2116455649 conn(0x563cbfcd4980 msgr2=0x563cbfcd4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 -- 172.21.15.100:0/2116455649 shutdown_connections 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.980+0000 7f40c2e08740 1 -- 172.21.15.100:0/2116455649 wait complete. 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 -- start start 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 Processor -- start 2024-03-12T05:29:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcda250 0x563cbfce0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcdab70 0x563cbfce0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfc42020 con 0x563cbfce3750 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbf915430 con 0x563cbfcda250 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbf8d1e00 con 0x563cbfcdab70 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40c2e08740 10 client.0 _ll_register_callbacks cb 0x563cbfbe9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40be7d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40bdfd5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcda250 0x563cbfce0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40be7d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42884/0 (socket says 172.21.15.100:42884) 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.981+0000 7f40be7d6640 1 -- 172.21.15.100:0/2711165056 learned_addr learned my addr 172.21.15.100:0/2711165056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcdab70 0x563cbfce0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcdab70 msgr2=0x563cbfce0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcdab70 0x563cbfce0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2711165056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 msgr2=0x563cbfce11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2711165056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b0019040 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcda250 0x563cbfce0740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009230 tx=0x7f40b402ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95074a11cf05039d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40a17fa640 1 -- 172.21.15.100:0/2711165056 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004450 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40a17fa640 1 -- 172.21.15.100:0/2711165056 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40b402cc00 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.982+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cbf97e720 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40a17fa640 1 -- 172.21.15.100:0/2711165056 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b402c5d0 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 10 client.28433 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563cbfc406a0 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40a17fa640 1 -- 172.21.15.100:0/2711165056 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40b4055080 con 0x563cbfcda250 2024-03-12T05:29:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40a17fa640 1 -- 172.21.15.100:0/2711165056 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40b403c040 con 0x563cbfcda250 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 10 client.28433 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 10 client.28433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 20 client.28433 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfcf3a50 0x563cbfcf5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 4 client.28433 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cbf9cd890 con 0x563cbfcf3a50 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.983+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfcf3a50 0x563cbfcf5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.984+0000 7f40a17fa640 10 client.28433 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.984+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfcf3a50 0x563cbfcf5e40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2cd93a139f837570 server_cookie=289c29c626851230 in_seq=0 out_seq=0 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40a17fa640 1 -- 172.21.15.100:0/2711165056 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cbf9cd890 con 0x563cbfcf3a50 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40a17fa640 10 client.28433 handle_command_reply: tid=0 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 client.28433 shutdown 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcda250 msgr2=0x563cbfce0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcda250 0x563cbfce0740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009230 tx=0x7f40b402ca10 comp rx=0 tx=0).stop 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 shutdown_connections 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfce11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcdab70 0x563cbfce0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfcf3a50 0x563cbfcf5e40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2711165056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcda250 0x563cbfce0740 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40a17fa640 0 client.28433 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 >> 172.21.15.100:0/2711165056 conn(0x563cbfcd4980 msgr2=0x563cbfcd66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 shutdown_connections 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 1 -- 172.21.15.100:0/2711165056 wait complete. 2024-03-12T05:29:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.985+0000 7f40c2e08740 20 client.28433 trim_cache size 0 max 16384 2024-03-12T05:29:11.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.997+0000 7f40c2e08740 1 Processor -- start 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.997+0000 7f40c2e08740 1 -- start start 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfce2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfcdd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40c2e08740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfbc7130 con 0x563cbfce3750 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40c2e08740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfcd0080 con 0x563cbfcdab70 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40c2e08740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfcf3800 con 0x563cbfcda250 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfcdd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfcdd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42886/0 (socket says 172.21.15.100:42886) 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40bdfd5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfce2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 -- 172.21.15.100:0/2997842801 learned_addr learned my addr 172.21.15.100:0/2997842801 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 -- 172.21.15.100:0/2997842801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 msgr2=0x563cbfce2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfce2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 -- 172.21.15.100:0/2997842801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 msgr2=0x563cbfce0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 -- 172.21.15.100:0/2997842801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbfbdcc20 con 0x563cbfce3750 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfcdd9a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f40b0003ef0 tx=0x7f40b0011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee1d225fbd198e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.998+0000 7f40a37fe640 1 -- 172.21.15.100:0/2997842801 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00119f0 con 0x563cbfce3750 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40a37fe640 1 -- 172.21.15.100:0/2997842801 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40b0005070 con 0x563cbfce3750 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40a37fe640 1 -- 172.21.15.100:0/2997842801 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b002f680 con 0x563cbfce3750 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 -- 172.21.15.100:0/2997842801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 msgr2=0x563cbfcdd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfcdd9a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f40b0003ef0 tx=0x7f40b0011cd0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 -- 172.21.15.100:0/2997842801 shutdown_connections 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfce3750 0x563cbfcdd9a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2997842801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfcda250 0x563cbfce2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:11.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 -- 172.21.15.100:0/2997842801 >> 172.21.15.100:0/2997842801 conn(0x563cbfcd4980 msgr2=0x563cbfcfa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:11.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 -- 172.21.15.100:0/2997842801 shutdown_connections 2024-03-12T05:29:11.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:11.999+0000 7f40c2e08740 1 -- 172.21.15.100:0/2997842801 wait complete. 2024-03-12T05:29:11.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 -- start start 2024-03-12T05:29:11.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 Processor -- start 2024-03-12T05:29:11.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 0x563cbfce91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:11.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfce3750 0x563cbfce9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfd4a010 con 0x563cbfcda250 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfd4a180 con 0x563cbfcdab70 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbfd4a340 con 0x563cbfce3750 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40c2e08740 10 client.0 _ll_register_callbacks cb 0x7f40b0011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 0x563cbfce91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 0x563cbfce91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42890/0 (socket says 172.21.15.100:42890) 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2160441451 learned_addr learned my addr 172.21.15.100:0/2160441451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40be7d6640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfce3750 0x563cbfce9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfce3750 msgr2=0x563cbfce9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfce3750 0x563cbfce9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2160441451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 msgr2=0x563cbfce9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.000+0000 7f40bdfd5640 1 -- 172.21.15.100:0/2160441451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cbfc04060 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40bdfd5640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 0x563cbfce91f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f40b4003800 tx=0x7f40b4004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826dec4bf3b3c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40a17fa640 1 -- 172.21.15.100:0/2160441451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004450 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40a17fa640 1 -- 172.21.15.100:0/2160441451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40b4000e40 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cbf9417e0 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40a17fa640 1 -- 172.21.15.100:0/2160441451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b4004af0 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 10 client.19095 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563cbfc028c0 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40a17fa640 1 -- 172.21.15.100:0/2160441451 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563cbf9417e0 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40a17fa640 1 -- 172.21.15.100:0/2160441451 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40b4034440 con 0x563cbfcda250 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 10 client.19095 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 10 client.19095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 20 client.19095 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfd4b8d0 0x563cbfd4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 4 client.19095 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.001+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563cbf8ed700 con 0x563cbfd4b8d0 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfd4b8d0 0x563cbfd4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40a17fa640 10 client.19095 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40bd7d4640 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfd4b8d0 0x563cbfd4dcc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=489834a05eeea8f5 server_cookie=f1e4343932b40dbb in_seq=0 out_seq=0 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40a17fa640 1 -- 172.21.15.100:0/2160441451 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cbf8ed700 con 0x563cbfd4b8d0 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40a17fa640 10 client.19095 handle_command_reply: tid=0 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 client.19095 shutdown 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 msgr2=0x563cbfce91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 0x563cbfce91f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f40b4003800 tx=0x7f40b4004a80 comp rx=0 tx=0).stop 2024-03-12T05:29:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 shutdown_connections 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563cbfce3750 0x563cbfce9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x563cbfd4b8d0 0x563cbfd4dcc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cbfcdab70 0x563cbfce9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 --2- 172.21.15.100:0/2160441451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563cbfcda250 0x563cbfce91f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40a17fa640 0 client.19095 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 >> 172.21.15.100:0/2160441451 conn(0x563cbfcd4980 msgr2=0x563cbfcd32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 shutdown_connections 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 1 -- 172.21.15.100:0/2160441451 wait complete. 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.002+0000 7f40c2e08740 20 client.19095 trim_cache size 0 max 16384 2024-03-12T05:29:12.003 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:29:12.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40a40a1ba0 msgr2=0x7f40a40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40a40a1ba0 0x7f40a40a4060 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40b000fc90 tx=0x7f40b001e000 comp rx=0 tx=0).stop 2024-03-12T05:29:12.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 msgr2=0x7f40b819d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 0x7f40b819d9a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e9f0 tx=0x7f40a800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 shutdown_connections 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f40a40a1ba0 0x7f40a40a4060 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40b8197610 0x7f40b8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81064c0 0x7f40b819d9a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 --2- 172.21.15.100:0/1769229131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40b80685f0 0x7f40b819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 >> 172.21.15.100:0/1769229131 conn(0x7f40b8077180 msgr2=0x7f40b80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 shutdown_connections 2024-03-12T05:29:12.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.003+0000 7f40c0239640 1 -- 172.21.15.100:0/1769229131 wait complete. 2024-03-12T05:29:12.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:11 smithi100 ceph-mon[32210]: osdmap e111: 12 total, 12 up, 12 in 2024-03-12T05:29:12.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:11 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2074024737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:29:12.008 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:29:12.008 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:29:12.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 -- 172.21.15.100:0/2411731603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102b70 msgr2=0x7f193010b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 --2- 172.21.15.100:0/2411731603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102b70 0x7f193010b050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402bcd0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 -- 172.21.15.100:0/2411731603 shutdown_connections 2024-03-12T05:29:12.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 --2- 172.21.15.100:0/2411731603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193010b620 0x7f193010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 --2- 172.21.15.100:0/2411731603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102b70 0x7f193010b050 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 --2- 172.21.15.100:0/2411731603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1930102250 0x7f1930102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 -- 172.21.15.100:0/2411731603 >> 172.21.15.100:0/2411731603 conn(0x7f19300fb9d0 msgr2=0x7f19300fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 -- 172.21.15.100:0/2411731603 shutdown_connections 2024-03-12T05:29:12.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 -- 172.21.15.100:0/2411731603 wait complete. 2024-03-12T05:29:12.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 Processor -- start 2024-03-12T05:29:12.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.124+0000 7f19357a0640 1 -- start start 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 0x7f1930197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102b70 0x7f1930197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f193010b620 0x7f193019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193019c550 con 0x7f1930102250 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193019c6c0 con 0x7f193010b620 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193019c880 con 0x7f1930102b70 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 0x7f1930197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 0x7f1930197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42908/0 (socket says 172.21.15.100:42908) 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 -- 172.21.15.100:0/4230141585 learned_addr learned my addr 172.21.15.100:0/4230141585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:12.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192f7fe640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102b70 0x7f1930197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f1934f9f640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f193010b620 0x7f193019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 -- 172.21.15.100:0/4230141585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102b70 msgr2=0x7f1930197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102b70 0x7f1930197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 -- 172.21.15.100:0/4230141585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f193010b620 msgr2=0x7f193019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f193010b620 0x7f193019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 -- 172.21.15.100:0/4230141585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1924009560 con 0x7f1930102250 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192ffff640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 0x7f1930197690 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f192000e9f0 tx=0x7f192000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192d7fa640 1 -- 172.21.15.100:0/4230141585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920009700 con 0x7f1930102250 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192d7fa640 1 -- 172.21.15.100:0/4230141585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f192000cd80 con 0x7f1930102250 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193019cb60 con 0x7f1930102250 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f192d7fa640 1 -- 172.21.15.100:0/4230141585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920018920 con 0x7f1930102250 2024-03-12T05:29:12.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.125+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19301a43e0 con 0x7f1930102250 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.127+0000 7f192d7fa640 1 -- 172.21.15.100:0/4230141585 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1920018a80 con 0x7f1930102250 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 Processor -- start 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 -- start start 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192d7fa640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f19180a1ba0 0x7f19180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af74550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5af2f960 con 0x559b5af74a90 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5aee3310 con 0x559b5af81d50 2024-03-12T05:29:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f19395d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5aee0840 con 0x559b5af74170 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192d7fa640 1 -- 172.21.15.100:0/4230141585 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1920014070 con 0x7f1930102250 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f1934f9f640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f19180a1ba0 0x7f19180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af81810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42920/0 (socket says 172.21.15.100:42920) 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af74550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 -- 172.21.15.100:0/3939294618 learned_addr learned my addr 172.21.15.100:0/3939294618 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f1934f9f640 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 -- 172.21.15.100:0/3939294618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 msgr2=0x559b5af74550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af74550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 -- 172.21.15.100:0/3939294618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 msgr2=0x559b5af84140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af84140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192f7fe640 1 -- 172.21.15.100:0/3939294618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5ae9e060 con 0x559b5af74a90 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.128+0000 7f192ffff640 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af74550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f192f7fe640 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af81810 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f19240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdab86639988f03 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f1934f9f640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f19180a1ba0 0x7f19180a4060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1928005fd0 tx=0x7f1928009dd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f19167fc640 1 -- 172.21.15.100:0/3939294618 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19240048b0 con 0x559b5af74a90 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f19167fc640 1 -- 172.21.15.100:0/3939294618 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1924004a10 con 0x559b5af74a90 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f19167fc640 1 -- 172.21.15.100:0/3939294618 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19240051f0 con 0x559b5af74a90 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f19395d0740 1 -- 172.21.15.100:0/3939294618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 msgr2=0x559b5af81810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.129+0000 7f19395d0740 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af81810 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f19240046c0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- 172.21.15.100:0/3939294618 shutdown_connections 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af84140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af81810 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 --2- 172.21.15.100:0/3939294618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af74550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- 172.21.15.100:0/3939294618 >> 172.21.15.100:0/3939294618 conn(0x559b5af6eb70 msgr2=0x559b5af6ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- 172.21.15.100:0/3939294618 shutdown_connections 2024-03-12T05:29:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- 172.21.15.100:0/3939294618 wait complete. 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- start start 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 Processor -- start 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 0x559b5af7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af81d50 0x559b5af7f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5aedc020 con 0x559b5af74170 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5abaf430 con 0x559b5af74a90 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5ab6be00 con 0x559b5af81d50 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.130+0000 7f19395d0740 10 client.0 _ll_register_callbacks cb 0x559b5ae83cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 0x559b5af7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 0x559b5af7ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42926/0 (socket says 172.21.15.100:42926) 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f1934f9f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af81d50 0x559b5af7f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 -- 172.21.15.100:0/2413872232 learned_addr learned my addr 172.21.15.100:0/2413872232 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 -- 172.21.15.100:0/2413872232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af81d50 msgr2=0x559b5af7f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af81d50 0x559b5af7f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 -- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 msgr2=0x559b5af7f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af7f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 -- 172.21.15.100:0/2413872232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193010e740 con 0x559b5af74170 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f192ffff640 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 0x559b5af7ed20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f192009a540 tx=0x7f192009a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ef058dd2f20ace server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f190f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f190f7fe640 1 -- 172.21.15.100:0/2413872232 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192009e8e0 con 0x559b5af74170 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f190f7fe640 1 -- 172.21.15.100:0/2413872232 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f19200a2f30 con 0x559b5af74170 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b5abdb7e0 con 0x559b5af74170 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f190f7fe640 1 -- 172.21.15.100:0/2413872232 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1920096eb0 con 0x559b5af74170 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f19395d0740 10 client.19107 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559b5aeda6a0 con 0x559b5af74170 2024-03-12T05:29:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.131+0000 7f190f7fe640 1 -- 172.21.15.100:0/2413872232 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559b5abdb7e0 con 0x559b5af74170 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f190f7fe640 1 -- 172.21.15.100:0/2413872232 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f19200ba0b0 con 0x559b5af74170 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f19395d0740 10 client.19107 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f19395d0740 10 client.19107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f19395d0740 20 client.19107 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f19395d0740 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5af91fc0 0x559b5af94400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f19395d0740 4 client.19107 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b5ac18720 con 0x559b5af91fc0 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f192f7fe640 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5af91fc0 0x559b5af94400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f192f7fe640 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5af91fc0 0x559b5af94400 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fb3004d2307ea5 server_cookie=7ed220862f6404b6 in_seq=0 out_seq=0 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.132+0000 7f190f7fe640 10 client.19107 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f190f7fe640 1 -- 172.21.15.100:0/2413872232 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b5ac18720 con 0x559b5af91fc0 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f190f7fe640 10 client.19107 handle_command_reply: tid=0 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 client.19107 shutdown 2024-03-12T05:29:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 msgr2=0x559b5af7ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 0x559b5af7ed20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f192009a540 tx=0x7f192009a1a0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 shutdown_connections 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5af91fc0 0x559b5af94400 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af81d50 0x559b5af7f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af7f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f190f7fe640 0 client.19107 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 --2- 172.21.15.100:0/2413872232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74170 0x559b5af7ed20 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 >> 172.21.15.100:0/2413872232 conn(0x559b5af6eb70 msgr2=0x559b5af708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 shutdown_connections 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 1 -- 172.21.15.100:0/2413872232 wait complete. 2024-03-12T05:29:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.133+0000 7f19395d0740 20 client.19107 trim_cache size 0 max 16384 2024-03-12T05:29:12.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 Processor -- start 2024-03-12T05:29:12.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 -- start start 2024-03-12T05:29:12.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af70650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5ae61130 con 0x559b5af74a90 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5ac18720 con 0x559b5af81d50 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.145+0000 7f19395d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5af6f740 con 0x559b5af74170 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af70650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af70650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42938/0 (socket says 172.21.15.100:42938) 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 -- 172.21.15.100:0/90088601 learned_addr learned my addr 172.21.15.100:0/90088601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f1934f9f640 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192ffff640 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 -- 172.21.15.100:0/90088601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 msgr2=0x559b5af7f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af7f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 -- 172.21.15.100:0/90088601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 msgr2=0x559b5af88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 -- 172.21.15.100:0/90088601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5ae76c20 con 0x559b5af74a90 2024-03-12T05:29:12.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f192f7fe640 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af70650 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1924047000 tx=0x7f1924004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879230f8ff53e995 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f1915ffb640 1 -- 172.21.15.100:0/90088601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924051b70 con 0x559b5af74a90 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f1915ffb640 1 -- 172.21.15.100:0/90088601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1924051870 con 0x559b5af74a90 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.146+0000 7f1915ffb640 1 -- 172.21.15.100:0/90088601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192402ce60 con 0x559b5af74a90 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 -- 172.21.15.100:0/90088601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 msgr2=0x559b5af70650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af70650 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1924047000 tx=0x7f1924004e90 comp rx=0 tx=0).stop 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 -- 172.21.15.100:0/90088601 shutdown_connections 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af81d50 0x559b5af88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af74a90 0x559b5af70650 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 --2- 172.21.15.100:0/90088601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af7f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 -- 172.21.15.100:0/90088601 >> 172.21.15.100:0/90088601 conn(0x559b5af6eb70 msgr2=0x559b5af6ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 -- 172.21.15.100:0/90088601 shutdown_connections 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 -- 172.21.15.100:0/90088601 wait complete. 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 -- start start 2024-03-12T05:29:12.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.147+0000 7f19395d0740 1 Processor -- start 2024-03-12T05:29:12.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af87570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af87ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 0x559b5af87ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5afe8530 con 0x559b5af81d50 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5afe86a0 con 0x559b5af74a90 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5afe8860 con 0x559b5af74170 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 10 client.0 _ll_register_callbacks cb 0x7f1924009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 0x559b5af87ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 0x559b5af87ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42944/0 (socket says 172.21.15.100:42944) 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 -- 172.21.15.100:0/1350980832 learned_addr learned my addr 172.21.15.100:0/1350980832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f192ffff640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af87570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f192f7fe640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af87ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 -- 172.21.15.100:0/1350980832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 msgr2=0x559b5af87570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af87570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 -- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 msgr2=0x559b5af87ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:12.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af87ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 -- 172.21.15.100:0/1350980832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b5ae9e060 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f192f7fe640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af87ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f1934f9f640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 0x559b5af87ff0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f192800ed50 tx=0x7f192801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed26ace457aaacf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f190f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f190f7fe640 1 -- 172.21.15.100:0/1350980832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928025070 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f190f7fe640 1 -- 172.21.15.100:0/1350980832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f192800ed80 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.148+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1924004a10 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f190f7fe640 1 -- 172.21.15.100:0/1350980832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f192802a460 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 10 client.19113 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559b5ae9c8c0 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f190f7fe640 1 -- 172.21.15.100:0/1350980832 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1924004a10 con 0x559b5af81d50 2024-03-12T05:29:12.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f190f7fe640 1 -- 172.21.15.100:0/1350980832 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1928030070 con 0x559b5af81d50 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 10 client.19113 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 10 client.19113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 20 client.19113 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5afea140 0x559b5afec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 4 client.19113 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b5ac67890 con 0x559b5afea140 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f192ffff640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5afea140 0x559b5afec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f190f7fe640 10 client.19113 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.149+0000 7f192ffff640 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5afea140 0x559b5afec560 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ca2d4bb99f6048 server_cookie=6fbc34311b1de405 in_seq=0 out_seq=0 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f190f7fe640 1 -- 172.21.15.100:0/1350980832 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b5ac67890 con 0x559b5afea140 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f190f7fe640 10 client.19113 handle_command_reply: tid=0 2024-03-12T05:29:12.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 client.19113 shutdown 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 msgr2=0x559b5af87ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 0x559b5af87ff0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f192800ed50 tx=0x7f192801c000 comp rx=0 tx=0).stop 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 shutdown_connections 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559b5afea140 0x559b5afec560 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b5af81d50 0x559b5af87ff0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f190f7fe640 0 client.19113 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559b5af74a90 0x559b5af87ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 --2- 172.21.15.100:0/1350980832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b5af74170 0x559b5af87570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 >> 172.21.15.100:0/1350980832 conn(0x559b5af6eb70 msgr2=0x559b5af707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 shutdown_connections 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 1 -- 172.21.15.100:0/1350980832 wait complete. 2024-03-12T05:29:12.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.150+0000 7f19395d0740 20 client.19113 trim_cache size 0 max 16384 2024-03-12T05:29:12.152 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:29:12.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f19180a1ba0 msgr2=0x7f19180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f19180a1ba0 0x7f19180a4060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1928005fd0 tx=0x7f1928009dd0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 msgr2=0x7f1930197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 0x7f1930197690 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f192000e9f0 tx=0x7f192000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 shutdown_connections 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f19180a1ba0 0x7f19180a4060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f193010b620 0x7f193019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102b70 0x7f1930197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 --2- 172.21.15.100:0/4230141585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1930102250 0x7f1930197690 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 >> 172.21.15.100:0/4230141585 conn(0x7f19300fb9d0 msgr2=0x7f19300fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 shutdown_connections 2024-03-12T05:29:12.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:12.151+0000 7f19357a0640 1 -- 172.21.15.100:0/4230141585 wait complete. 2024-03-12T05:29:12.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:11 smithi156 ceph-mon[36270]: osdmap e111: 12 total, 12 up, 12 in 2024-03-12T05:29:12.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:11 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2074024737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:29:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:11 smithi161 ceph-mon[36043]: osdmap e111: 12 total, 12 up, 12 in 2024-03-12T05:29:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:11 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2074024737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:29:12.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 138084 30.00 MB/sec execute 115 sec latency 4302.363 ms 2024-03-12T05:29:13.156 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:29:13.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:29:13.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:29:13.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:12 smithi100 ceph-mon[32210]: pgmap v538: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 696 op/s 2024-03-12T05:29:13.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:12 smithi156 ceph-mon[36270]: pgmap v538: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 696 op/s 2024-03-12T05:29:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:12 smithi161 ceph-mon[36043]: pgmap v538: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 696 op/s 2024-03-12T05:29:13.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 138568 29.97 MB/sec execute 116 sec latency 5302.397 ms 2024-03-12T05:29:13.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 -- 172.21.15.100:0/3394751473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308106450 msgr2=0x7fe30810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 --2- 172.21.15.100:0/3394751473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308106450 0x7fe30810e020 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe300009cd0 tx=0x7fe30002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 -- 172.21.15.100:0/3394751473 shutdown_connections 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 --2- 172.21.15.100:0/3394751473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308106450 0x7fe30810e020 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 --2- 172.21.15.100:0/3394751473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe308100240 0x7fe308105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 --2- 172.21.15.100:0/3394751473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3080ff890 0x7fe3080ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 -- 172.21.15.100:0/3394751473 >> 172.21.15.100:0/3394751473 conn(0x7fe3080765a0 msgr2=0x7fe3080fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 -- 172.21.15.100:0/3394751473 shutdown_connections 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.282+0000 7fe30e165640 1 -- 172.21.15.100:0/3394751473 wait complete. 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 Processor -- start 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 -- start start 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3080ff890 0x7fe30819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe308106450 0x7fe30819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 0x7fe308197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308197ff0 con 0x7fe308197600 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308198160 con 0x7fe308106450 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe30e165640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308198320 con 0x7fe3080ff890 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 0x7fe308197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe3077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3080ff890 0x7fe30819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 0x7fe308197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42962/0 (socket says 172.21.15.100:42962) 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 -- 172.21.15.100:0/3896586605 learned_addr learned my addr 172.21.15.100:0/3896586605 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe306ffd640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe308106450 0x7fe30819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 -- 172.21.15.100:0/3896586605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3080ff890 msgr2=0x7fe30819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3080ff890 0x7fe30819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 -- 172.21.15.100:0/3896586605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe308106450 msgr2=0x7fe30819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe308106450 0x7fe30819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.283+0000 7fe307fff640 1 -- 172.21.15.100:0/3896586605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300009900 con 0x7fe308197600 2024-03-12T05:29:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.284+0000 7fe307fff640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 0x7fe308197ab0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fe300004af0 tx=0x7fe300034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.284+0000 7fe304ff9640 1 -- 172.21.15.100:0/3896586605 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000328c0 con 0x7fe308197600 2024-03-12T05:29:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.284+0000 7fe304ff9640 1 -- 172.21.15.100:0/3896586605 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe300003e50 con 0x7fe308197600 2024-03-12T05:29:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.284+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3081a4310 con 0x7fe308197600 2024-03-12T05:29:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.284+0000 7fe304ff9640 1 -- 172.21.15.100:0/3896586605 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300004860 con 0x7fe308197600 2024-03-12T05:29:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.284+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3081a4790 con 0x7fe308197600 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe304ff9640 1 -- 172.21.15.100:0/3896586605 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe30000a040 con 0x7fe308197600 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 Processor -- start 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 -- start start 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe304ff9640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2f00a1ba0 0x7fe2f00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 0x55da8d464640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464c10 0x55da8d46d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d46e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d3d2f90 con 0x55da8d46dd10 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d3d04c0 con 0x55da8d464c10 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.286+0000 7fe310e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d3cbca0 con 0x55da8d464260 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe306ffd640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2f00a1ba0 0x7fe2f00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d46e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 0x55da8d464640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 0x55da8d464640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:53268/0 (socket says 172.21.15.100:53268) 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d46e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42978/0 (socket says 172.21.15.100:42978) 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 -- 172.21.15.100:0/174582932 learned_addr learned my addr 172.21.15.100:0/174582932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe306ffd640 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464c10 0x55da8d46d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe304ff9640 1 -- 172.21.15.100:0/3896586605 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe3000f0400 con 0x7fe308197600 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 -- 172.21.15.100:0/174582932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464c10 msgr2=0x55da8d46d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464c10 0x55da8d46d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 -- 172.21.15.100:0/174582932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 msgr2=0x55da8d46e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d46e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 -- 172.21.15.100:0/174582932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da8d38e0b0 con 0x55da8d464260 2024-03-12T05:29:13.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe307fff640 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d46e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe306ffd640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2f00a1ba0 0x7fe2f00a4060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe3080febe0 tx=0x7fe2fc009a30 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe3077fe640 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 0x55da8d464640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800bcd0 tx=0x7fe2f80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1b0f86e2c8575f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe2edffb640 1 -- 172.21.15.100:0/174582932 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f801c680 con 0x55da8d464260 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe2edffb640 1 -- 172.21.15.100:0/174582932 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2f8015070 con 0x55da8d464260 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.287+0000 7fe2edffb640 1 -- 172.21.15.100:0/174582932 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8011930 con 0x55da8d464260 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 -- 172.21.15.100:0/174582932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 msgr2=0x55da8d464640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 0x55da8d464640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800bcd0 tx=0x7fe2f80107a0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 -- 172.21.15.100:0/174582932 shutdown_connections 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d46e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464c10 0x55da8d46d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 --2- 172.21.15.100:0/174582932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464260 0x55da8d464640 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 -- 172.21.15.100:0/174582932 >> 172.21.15.100:0/174582932 conn(0x55da8d45e840 msgr2=0x55da8d45ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 -- 172.21.15.100:0/174582932 shutdown_connections 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 -- 172.21.15.100:0/174582932 wait complete. 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.288+0000 7fe310e13740 1 -- start start 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 Processor -- start 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d471730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d471c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d4721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d370f40 con 0x55da8d46dd10 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d09f4b0 con 0x55da8d464260 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d351670 con 0x55da8d464c10 2024-03-12T05:29:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 10 client.0 _ll_register_callbacks cb 0x55da8d3bef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d4721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d4721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42980/0 (socket says 172.21.15.100:42980) 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe3077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d471730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 -- 172.21.15.100:0/1821472790 learned_addr learned my addr 172.21.15.100:0/1821472790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe306ffd640 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d471c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 -- 172.21.15.100:0/1821472790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 msgr2=0x55da8d471c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d471c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 -- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 msgr2=0x55da8d471730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d471730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 -- 172.21.15.100:0/1821472790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f800b900 con 0x55da8d46dd10 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe307fff640 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d4721b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe3000bce00 tx=0x7fe3000b93a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d927d57ca7c61c5f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe2e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1821472790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000bca60 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1821472790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe3000bd800 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.289+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da8d3a5f40 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1821472790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30009f4b0 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 10 client.19125 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55da8d3ca320 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1821472790 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da8d3a5f40 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1821472790 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe3000edd90 con 0x55da8d46dd10 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 10 client.19125 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 10 client.19125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 20 client.19125 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:13.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d484420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 4 client.19125 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da8d3c2310 con 0x55da8d482000 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.290+0000 7fe3077fe640 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d484420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe2e6ffd640 10 client.19125 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe3077fe640 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d484420 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a082da575b9053fb server_cookie=ee463497cd38c27c in_seq=0 out_seq=0 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1821472790 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da8d3c2310 con 0x55da8d482000 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe2e6ffd640 10 client.19125 handle_command_reply: tid=0 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 client.19125 shutdown 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 msgr2=0x55da8d4721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d4721b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe3000bce00 tx=0x7fe3000b93a0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 shutdown_connections 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d484420 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d4721b0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.291+0000 7fe310e13740 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d471c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.292+0000 7fe2e6ffd640 0 client.19125 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.292+0000 7fe310e13740 1 --2- 172.21.15.100:0/1821472790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d471730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.292+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 >> 172.21.15.100:0/1821472790 conn(0x55da8d45e840 msgr2=0x55da8d4607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.292+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 shutdown_connections 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.292+0000 7fe310e13740 1 -- 172.21.15.100:0/1821472790 wait complete. 2024-03-12T05:29:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.292+0000 7fe310e13740 20 client.19125 trim_cache size 0 max 16384 2024-03-12T05:29:13.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 Processor -- start 2024-03-12T05:29:13.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 -- start start 2024-03-12T05:29:13.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d473e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 0x55da8d46f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d46dd10 0x55da8d470730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d3c2310 con 0x55da8d464c10 2024-03-12T05:29:13.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d3a5f40 con 0x55da8d464260 2024-03-12T05:29:13.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe310e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d474b80 con 0x55da8d46dd10 2024-03-12T05:29:13.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe306ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 0x55da8d46f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe306ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 0x55da8d46f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42988/0 (socket says 172.21.15.100:42988) 2024-03-12T05:29:13.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe306ffd640 1 -- 172.21.15.100:0/2197640851 learned_addr learned my addr 172.21.15.100:0/2197640851 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe307fff640 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d46dd10 0x55da8d470730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.307+0000 7fe3077fe640 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d473e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe306ffd640 1 -- 172.21.15.100:0/2197640851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d46dd10 msgr2=0x55da8d470730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe306ffd640 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d46dd10 0x55da8d470730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe306ffd640 1 -- 172.21.15.100:0/2197640851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 msgr2=0x55da8d473e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe306ffd640 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d473e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe306ffd640 1 -- 172.21.15.100:0/2197640851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da8d366c00 con 0x55da8d464c10 2024-03-12T05:29:13.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe3077fe640 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d473e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe306ffd640 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 0x55da8d46f100 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe308198740 tx=0x7fe2fc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9b171f41ad50cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe2ed7fa640 1 -- 172.21.15.100:0/2197640851 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc026070 con 0x55da8d464c10 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe2ed7fa640 1 -- 172.21.15.100:0/2197640851 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2fc011e00 con 0x55da8d464c10 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe2ed7fa640 1 -- 172.21.15.100:0/2197640851 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc021770 con 0x55da8d464c10 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe310e13740 1 -- 172.21.15.100:0/2197640851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 msgr2=0x55da8d46f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.308+0000 7fe310e13740 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 0x55da8d46f100 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe308198740 tx=0x7fe2fc021000 comp rx=0 tx=0).stop 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- 172.21.15.100:0/2197640851 shutdown_connections 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d46dd10 0x55da8d470730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d464c10 0x55da8d46f100 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 --2- 172.21.15.100:0/2197640851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d473e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- 172.21.15.100:0/2197640851 >> 172.21.15.100:0/2197640851 conn(0x55da8d45e840 msgr2=0x55da8d470020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- 172.21.15.100:0/2197640851 shutdown_connections 2024-03-12T05:29:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- 172.21.15.100:0/2197640851 wait complete. 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- start start 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 Processor -- start 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d4d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d4d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d478390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d4789a0 con 0x55da8d46dd10 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d478b10 con 0x55da8d464260 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8d478cd0 con 0x55da8d464c10 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe310e13740 10 client.0 _ll_register_callbacks cb 0x7fe2f800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:13.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe3077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d4d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe307fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d478390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe3077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d4d2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:60408/0 (socket says 172.21.15.100:60408) 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe3077fe640 1 -- 172.21.15.100:0/1573608990 learned_addr learned my addr 172.21.15.100:0/1573608990 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.309+0000 7fe306ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d4d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe307fff640 1 -- 172.21.15.100:0/1573608990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 msgr2=0x55da8d4d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe307fff640 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d4d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe307fff640 1 -- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 msgr2=0x55da8d4d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe307fff640 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d4d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe307fff640 1 -- 172.21.15.100:0/1573608990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc01b040 con 0x55da8d46dd10 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe3077fe640 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d4d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:13.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe307fff640 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d478390 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe300009430 tx=0x7fe3000b94d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f891445a300c4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe2e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1573608990 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000bca60 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1573608990 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe3000bd800 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da8d3f1900 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1573608990 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30009f4b0 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe310e13740 10 client.19131 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55da8d38c910 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1573608990 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da8d3f1900 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1573608990 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe3000c5410 con 0x55da8d46dd10 2024-03-12T05:29:13.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe310e13740 10 client.19131 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.310+0000 7fe310e13740 10 client.19131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe310e13740 20 client.19131 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe310e13740 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d4dc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe310e13740 4 client.19131 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55da8d0cb860 con 0x55da8d482000 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe3077fe640 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d4dc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe3077fe640 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d4dc630 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49734d55bbe86d27 server_cookie=c57d7d144d4abed8 in_seq=0 out_seq=0 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.311+0000 7fe2e6ffd640 10 client.19131 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe2e6ffd640 1 -- 172.21.15.100:0/1573608990 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55da8d0cb860 con 0x55da8d482000 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe2e6ffd640 10 client.19131 handle_command_reply: tid=0 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 client.19131 shutdown 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 msgr2=0x55da8d478390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d478390 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe300009430 tx=0x7fe3000b94d0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 shutdown_connections 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55da8d482000 0x55da8d4dc630 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da8d46dd10 0x55da8d478390 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe2e6ffd640 0 client.19131 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da8d464c10 0x55da8d4d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 --2- 172.21.15.100:0/1573608990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da8d464260 0x55da8d4d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 >> 172.21.15.100:0/1573608990 conn(0x55da8d45e840 msgr2=0x55da8d4708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.312+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 shutdown_connections 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe310e13740 1 -- 172.21.15.100:0/1573608990 wait complete. 2024-03-12T05:29:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe310e13740 20 client.19131 trim_cache size 0 max 16384 2024-03-12T05:29:13.322 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:29:13.322 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:29:13.322 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "185f8e47-e66f-4591-bd2f-ad5de7392b4f", 2024-03-12T05:29:13.322 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:29:13.322 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:29:13.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2f00a1ba0 msgr2=0x7fe2f00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2f00a1ba0 0x7fe2f00a4060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe3080febe0 tx=0x7fe2fc009a30 comp rx=0 tx=0).stop 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 msgr2=0x7fe308197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 0x7fe308197ab0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fe300004af0 tx=0x7fe300034cc0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 shutdown_connections 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe2f00a1ba0 0x7fe2f00a4060 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe308197600 0x7fe308197ab0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe308106450 0x7fe30819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 --2- 172.21.15.100:0/3896586605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3080ff890 0x7fe30819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 >> 172.21.15.100:0/3896586605 conn(0x7fe3080765a0 msgr2=0x7fe308104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 shutdown_connections 2024-03-12T05:29:13.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.313+0000 7fe30e165640 1 -- 172.21.15.100:0/3896586605 wait complete. 2024-03-12T05:29:13.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:29:13.325 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:29:13.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 -- 172.21.15.100:0/2529361219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c90106460 msgr2=0x7f0c9010e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 --2- 172.21.15.100:0/2529361219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c90106460 0x7f0c9010e030 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009cd0 tx=0x7f0c8802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 -- 172.21.15.100:0/2529361219 shutdown_connections 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 --2- 172.21.15.100:0/2529361219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c90106460 0x7f0c9010e030 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 --2- 172.21.15.100:0/2529361219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90101d40 0x7f0c90105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 --2- 172.21.15.100:0/2529361219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c90101390 0x7f0c90101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 -- 172.21.15.100:0/2529361219 >> 172.21.15.100:0/2529361219 conn(0x7f0c900fcc30 msgr2=0x7f0c900ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 -- 172.21.15.100:0/2529361219 shutdown_connections 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.482+0000 7f0c95afb640 1 -- 172.21.15.100:0/2529361219 wait complete. 2024-03-12T05:29:13.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.483+0000 7f0c95afb640 1 Processor -- start 2024-03-12T05:29:13.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.483+0000 7f0c95afb640 1 -- start start 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c95afb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90101390 0x7f0c9019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c95afb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c90106460 0x7f0c9019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c95afb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 0x7f0c90197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c95afb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c90197fa0 con 0x7f0c901975b0 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c95afb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c90198110 con 0x7f0c90101390 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c95afb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c901982d0 con 0x7f0c90106460 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 0x7f0c90197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 0x7f0c90197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43008/0 (socket says 172.21.15.100:43008) 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 -- 172.21.15.100:0/1065231718 learned_addr learned my addr 172.21.15.100:0/1065231718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8effd640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c90106460 0x7f0c9019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90101390 0x7f0c9019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 -- 172.21.15.100:0/1065231718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c90106460 msgr2=0x7f0c9019d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c90106460 0x7f0c9019d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 -- 172.21.15.100:0/1065231718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90101390 msgr2=0x7f0c9019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90101390 0x7f0c9019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 -- 172.21.15.100:0/1065231718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88009900 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.484+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 0x7f0c90197a60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0c88004af0 tx=0x7f0c88032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.485+0000 7f0c8cff9640 1 -- 172.21.15.100:0/1065231718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880328b0 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.485+0000 7f0c8cff9640 1 -- 172.21.15.100:0/1065231718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c88003e50 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.485+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c901a42c0 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.485+0000 7f0c8cff9640 1 -- 172.21.15.100:0/1065231718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88004740 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.485+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c901a4740 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.487+0000 7f0c8cff9640 1 -- 172.21.15.100:0/1065231718 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0c8800a040 con 0x7f0c901975b0 2024-03-12T05:29:13.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.487+0000 7f0c987ae740 1 Processor -- start 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.487+0000 7f0c987ae740 1 -- start start 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.487+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d42b0 0x55cce04d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.487+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d4bd0 0x55cce04dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.487+0000 7f0c8cff9640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c780a1ba0 0x7f0c780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 0x55cce04dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c987ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce0443280 con 0x55cce04dd7b0 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c987ae740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce04407b0 con 0x55cce04d42b0 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c987ae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce043bf90 con 0x55cce04d4bd0 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 0x55cce04dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8effd640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c780a1ba0 0x7f0c780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 0x55cce04dfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43022/0 (socket says 172.21.15.100:43022) 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 -- 172.21.15.100:0/836491969 learned_addr learned my addr 172.21.15.100:0/836491969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d4bd0 0x55cce04dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8effd640 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d42b0 0x55cce04d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8cff9640 1 -- 172.21.15.100:0/1065231718 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c880f02b0 con 0x7f0c901975b0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 -- 172.21.15.100:0/836491969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d4bd0 msgr2=0x55cce04dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d4bd0 0x55cce04dd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 -- 172.21.15.100:0/836491969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d42b0 msgr2=0x55cce04d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d42b0 0x55cce04d4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 -- 172.21.15.100:0/836491969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce03fa570 con 0x55cce04dd7b0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8effd640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c780a1ba0 0x7f0c780a4060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0c90105dc0 tx=0x7f0c84009ad0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 0x55cce04dfba0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0c880b9410 tx=0x7f0c8803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cf7a37db86c0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c75ffb640 1 -- 172.21.15.100:0/836491969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880d5ec0 con 0x55cce04dd7b0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c75ffb640 1 -- 172.21.15.100:0/836491969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c880d5ce0 con 0x55cce04dd7b0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.488+0000 7f0c75ffb640 1 -- 172.21.15.100:0/836491969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880c5750 con 0x55cce04dd7b0 2024-03-12T05:29:13.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 -- 172.21.15.100:0/836491969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 msgr2=0x55cce04dfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 0x55cce04dfba0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0c880b9410 tx=0x7f0c8803d000 comp rx=0 tx=0).stop 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 -- 172.21.15.100:0/836491969 shutdown_connections 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04dd7b0 0x55cce04dfba0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d4bd0 0x55cce04dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 --2- 172.21.15.100:0/836491969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d42b0 0x55cce04d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 -- 172.21.15.100:0/836491969 >> 172.21.15.100:0/836491969 conn(0x55cce04ce990 msgr2=0x55cce04ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 -- 172.21.15.100:0/836491969 shutdown_connections 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 -- 172.21.15.100:0/836491969 wait complete. 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.489+0000 7f0c987ae740 1 -- start start 2024-03-12T05:29:13.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 Processor -- start 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce04e7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce04e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce04e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce010f430 con 0x55cce04d42b0 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce00cbe00 con 0x55cce04d4bd0 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce03c0fd0 con 0x55cce04dd7b0 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c987ae740 10 client.0 _ll_register_callbacks cb 0x55cce0416fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce04e7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c8ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce04e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce04e7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43038/0 (socket says 172.21.15.100:43038) 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/512372646 learned_addr learned my addr 172.21.15.100:0/512372646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c8effd640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce04e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.490+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/512372646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 msgr2=0x55cce04e16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce04e16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 msgr2=0x55cce04e77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce04e77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/512372646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c901a4740 con 0x55cce04d42b0 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce04e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce04e7290 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0c80012740 tx=0x7f0c80012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d23aeb30ed04e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c6effd640 1 -- 172.21.15.100:0/512372646 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80017070 con 0x55cce04d42b0 2024-03-12T05:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c6effd640 1 -- 172.21.15.100:0/512372646 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c8000f920 con 0x55cce04d42b0 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cce013b7e0 con 0x55cce04d42b0 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c6effd640 1 -- 172.21.15.100:0/512372646 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8000fb20 con 0x55cce04d42b0 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 10 client.19143 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cce043a610 con 0x55cce04d42b0 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c6effd640 1 -- 172.21.15.100:0/512372646 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cce013b7e0 con 0x55cce04d42b0 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c6effd640 1 -- 172.21.15.100:0/512372646 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0c8001c420 con 0x55cce04d42b0 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 10 client.19143 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 10 client.19143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 20 client.19143 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce04f1c80 0x55cce04f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 4 client.19143 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.491+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cce0178720 con 0x55cce04f1c80 2024-03-12T05:29:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.492+0000 7f0c8effd640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce04f1c80 0x55cce04f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.492+0000 7f0c6effd640 10 client.19143 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.492+0000 7f0c8effd640 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce04f1c80 0x55cce04f40a0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725fdae19cb8673f server_cookie=5984581a99586994 in_seq=0 out_seq=0 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c6effd640 1 -- 172.21.15.100:0/512372646 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cce0178720 con 0x55cce04f1c80 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c6effd640 10 client.19143 handle_command_reply: tid=0 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 client.19143 shutdown 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 msgr2=0x55cce04e7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce04e7290 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0c80012740 tx=0x7f0c80012c00 comp rx=0 tx=0).stop 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 shutdown_connections 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce04f1c80 0x55cce04f40a0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce04e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce04e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 --2- 172.21.15.100:0/512372646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce04e7290 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c6effd640 0 client.19143 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 >> 172.21.15.100:0/512372646 conn(0x55cce04ce990 msgr2=0x55cce04d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 shutdown_connections 2024-03-12T05:29:13.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 1 -- 172.21.15.100:0/512372646 wait complete. 2024-03-12T05:29:13.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.493+0000 7f0c987ae740 20 client.19143 trim_cache size 0 max 16384 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 Processor -- start 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 -- start start 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d42b0 0x55cce04e1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 0x55cce04e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04dd7b0 0x55cce04e7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce04d3d20 con 0x55cce04d4bd0 2024-03-12T05:29:13.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce04cfa10 con 0x55cce04dd7b0 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c987ae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce04cf790 con 0x55cce04d42b0 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 0x55cce04e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 0x55cce04e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43042/0 (socket says 172.21.15.100:43042) 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 -- 172.21.15.100:0/730312382 learned_addr learned my addr 172.21.15.100:0/730312382 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d42b0 0x55cce04e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04dd7b0 0x55cce04e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 -- 172.21.15.100:0/730312382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d42b0 msgr2=0x55cce04e1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d42b0 0x55cce04e1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 -- 172.21.15.100:0/730312382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04dd7b0 msgr2=0x55cce04e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04dd7b0 0x55cce04e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8effd640 1 -- 172.21.15.100:0/730312382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce03d6ac0 con 0x55cce04d4bd0 2024-03-12T05:29:13.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.510+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04dd7b0 0x55cce04e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:13.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.511+0000 7f0c8effd640 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 0x55cce04e6df0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0c840098d0 tx=0x7f0c84021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ac41fd09903413 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.511+0000 7f0c757fa640 1 -- 172.21.15.100:0/730312382 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84016070 con 0x55cce04d4bd0 2024-03-12T05:29:13.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.511+0000 7f0c757fa640 1 -- 172.21.15.100:0/730312382 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c8400fde0 con 0x55cce04d4bd0 2024-03-12T05:29:13.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.511+0000 7f0c757fa640 1 -- 172.21.15.100:0/730312382 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84004990 con 0x55cce04d4bd0 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- 172.21.15.100:0/730312382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 msgr2=0x55cce04e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 0x55cce04e6df0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0c840098d0 tx=0x7f0c84021000 comp rx=0 tx=0).stop 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- 172.21.15.100:0/730312382 shutdown_connections 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04dd7b0 0x55cce04e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d4bd0 0x55cce04e6df0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- 172.21.15.100:0/730312382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04d42b0 0x55cce04e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- 172.21.15.100:0/730312382 >> 172.21.15.100:0/730312382 conn(0x55cce04ce990 msgr2=0x55cce04e0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- 172.21.15.100:0/730312382 shutdown_connections 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- 172.21.15.100:0/730312382 wait complete. 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- start start 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 Processor -- start 2024-03-12T05:29:13.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce0542320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce0542860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce053c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce053ccf0 con 0x55cce04d42b0 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce053ce60 con 0x55cce04d4bd0 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce053d020 con 0x55cce04dd7b0 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.512+0000 7f0c987ae740 10 client.0 _ll_register_callbacks cb 0x7f0c880b8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce0542320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce0542320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43048/0 (socket says 172.21.15.100:43048) 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/2465480813 learned_addr learned my addr 172.21.15.100:0/2465480813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:13.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8effd640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce0542860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce053c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/2465480813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 msgr2=0x55cce053c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce053c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 msgr2=0x55cce0542860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce0542860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 -- 172.21.15.100:0/2465480813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c84019040 con 0x55cce04d42b0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8ffff640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce053c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c8f7fe640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce0542320 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0c80012740 tx=0x7f0c8000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eb1eff7cd66ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c6effd640 1 -- 172.21.15.100:0/2465480813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80017070 con 0x55cce04d42b0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c6effd640 1 -- 172.21.15.100:0/2465480813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c8000fb20 con 0x55cce04d42b0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c880d5ce0 con 0x55cce04d42b0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c6effd640 1 -- 172.21.15.100:0/2465480813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8000fce0 con 0x55cce04d42b0 2024-03-12T05:29:13.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c987ae740 10 client.19149 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cce03fc6a0 con 0x55cce04d42b0 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c6effd640 1 -- 172.21.15.100:0/2465480813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c880d5ce0 con 0x55cce04d42b0 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.513+0000 7f0c6effd640 1 -- 172.21.15.100:0/2465480813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0c80005380 con 0x55cce04d42b0 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 10 client.19149 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 10 client.19149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 20 client.19149 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce054a230 0x55cce054c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 4 client.19149 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cce01c7890 con 0x55cce054a230 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c8effd640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce054a230 0x55cce054c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c8effd640 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce054a230 0x55cce054c650 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf98d4f3f765b0e server_cookie=6f56b1c50c08ec65 in_seq=0 out_seq=0 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c6effd640 10 client.19149 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c6effd640 1 -- 172.21.15.100:0/2465480813 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cce01c7890 con 0x55cce054a230 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c6effd640 10 client.19149 handle_command_reply: tid=0 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 1 client.19149 shutdown 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 msgr2=0x55cce0542320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.514+0000 7f0c987ae740 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce0542320 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0c80012740 tx=0x7f0c8000f790 comp rx=0 tx=0).stop 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 shutdown_connections 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55cce054a230 0x55cce054c650 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cce04dd7b0 0x55cce053c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cce04d4bd0 0x55cce0542860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c6effd640 0 client.19149 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 --2- 172.21.15.100:0/2465480813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cce04d42b0 0x55cce0542320 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 >> 172.21.15.100:0/2465480813 conn(0x55cce04ce990 msgr2=0x55cce04cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 shutdown_connections 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 1 -- 172.21.15.100:0/2465480813 wait complete. 2024-03-12T05:29:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c987ae740 20 client.19149 trim_cache size 0 max 16384 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: "185f8e47-e66f-4591-bd2f-ad5de7392b4f": { 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "185f8e47-e66f-4591-bd2f-ad5de7392b4f", 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:29:13.519 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:29:13.520 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:29:13.520 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:29:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c780a1ba0 msgr2=0x7f0c780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c780a1ba0 0x7f0c780a4060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0c90105dc0 tx=0x7f0c84009ad0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 msgr2=0x7f0c90197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 0x7f0c90197a60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0c88004af0 tx=0x7f0c88032570 comp rx=0 tx=0).stop 2024-03-12T05:29:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 shutdown_connections 2024-03-12T05:29:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c780a1ba0 0x7f0c780a4060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c901975b0 0x7f0c90197a60 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c90106460 0x7f0c9019d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 --2- 172.21.15.100:0/1065231718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90101390 0x7f0c9019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 >> 172.21.15.100:0/1065231718 conn(0x7f0c900fcc30 msgr2=0x7f0c900fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.515+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 shutdown_connections 2024-03-12T05:29:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:13.516+0000 7f0c95afb640 1 -- 172.21.15.100:0/1065231718 wait complete. 2024-03-12T05:29:13.522 INFO:tasks.cephfs.filesystem:scrub status for tag:185f8e47-e66f-4591-bd2f-ad5de7392b4f - {'path': '/', 'tag': '185f8e47-e66f-4591-bd2f-ad5de7392b4f', 'options': 'recursive,force'} 2024-03-12T05:29:14.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:13 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:14.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:13 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:14.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:13 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:14.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:13 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 138963 29.88 MB/sec execute 117 sec latency 6302.431 ms 2024-03-12T05:29:14.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:13 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:14.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:13 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:15.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:29:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:14 smithi156 ceph-mon[36270]: pgmap v539: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 681 op/s 2024-03-12T05:29:15.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:29:15.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:15.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:15.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:14 smithi161 ceph-mon[36043]: pgmap v539: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 681 op/s 2024-03-12T05:29:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 139479 29.81 MB/sec execute 118 sec latency 7302.465 ms 2024-03-12T05:29:15.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:29:15.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:15.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:15.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:15.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:14 smithi100 ceph-mon[32210]: pgmap v539: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 681 op/s 2024-03-12T05:29:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: 10 140226 30.01 MB/sec execute 119 sec latency 8302.494 ms 2024-03-12T05:29:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:16 smithi156 ceph-mon[36270]: pgmap v540: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 605 op/s 2024-03-12T05:29:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:16 smithi161 ceph-mon[36043]: pgmap v540: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 605 op/s 2024-03-12T05:29:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:16 smithi100 ceph-mon[32210]: pgmap v540: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 605 op/s 2024-03-12T05:29:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 140658 29.96 MB/sec execute 120 sec latency 9302.529 ms 2024-03-12T05:29:17.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:17 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:17.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:17 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:17.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:17 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:18.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 141125 29.89 MB/sec execute 121 sec latency 10302.576 ms 2024-03-12T05:29:18.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:18 smithi156 ceph-mon[36270]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 625 op/s 2024-03-12T05:29:18.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:18 smithi161 ceph-mon[36043]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 625 op/s 2024-03-12T05:29:18.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:18 smithi100 ceph-mon[32210]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 625 op/s 2024-03-12T05:29:19.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 141667 29.84 MB/sec execute 122 sec latency 10917.060 ms 2024-03-12T05:29:19.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:19 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:19.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:19 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:20.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:19 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:20.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 142023 29.77 MB/sec execute 123 sec latency 2792.477 ms 2024-03-12T05:29:20.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:20 smithi156 ceph-mon[36270]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 625 op/s 2024-03-12T05:29:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:20 smithi161 ceph-mon[36043]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 625 op/s 2024-03-12T05:29:21.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:20 smithi100 ceph-mon[32210]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 625 op/s 2024-03-12T05:29:21.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 142533 29.74 MB/sec execute 124 sec latency 3792.506 ms 2024-03-12T05:29:22.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 143109 29.77 MB/sec execute 125 sec latency 4792.533 ms 2024-03-12T05:29:22.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:22 smithi156 ceph-mon[36270]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 738 op/s 2024-03-12T05:29:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:22 smithi161 ceph-mon[36043]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 738 op/s 2024-03-12T05:29:22.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:22 smithi100 ceph-mon[32210]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 738 op/s 2024-03-12T05:29:23.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 143626 29.74 MB/sec execute 126 sec latency 5792.590 ms 2024-03-12T05:29:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 144092 29.66 MB/sec execute 127 sec latency 6792.647 ms 2024-03-12T05:29:24.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:24 smithi156 ceph-mon[36270]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 563 op/s 2024-03-12T05:29:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:24 smithi161 ceph-mon[36043]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 563 op/s 2024-03-12T05:29:24.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:24 smithi100 ceph-mon[32210]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 563 op/s 2024-03-12T05:29:25.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 144536 29.62 MB/sec execute 128 sec latency 7792.696 ms 2024-03-12T05:29:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 144956 29.62 MB/sec execute 129 sec latency 8792.724 ms 2024-03-12T05:29:26.365 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:26 smithi100 ceph-mon[32210]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 523 op/s 2024-03-12T05:29:26.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:26 smithi156 ceph-mon[36270]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 523 op/s 2024-03-12T05:29:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:26 smithi161 ceph-mon[36043]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 523 op/s 2024-03-12T05:29:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 145527 29.62 MB/sec execute 130 sec latency 8950.547 ms 2024-03-12T05:29:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 146048 29.62 MB/sec execute 131 sec latency 1642.079 ms 2024-03-12T05:29:28.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:28 smithi156 ceph-mon[36270]: pgmap v546: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 748 op/s 2024-03-12T05:29:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:28 smithi161 ceph-mon[36043]: pgmap v546: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 748 op/s 2024-03-12T05:29:28.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:28 smithi100 ceph-mon[32210]: pgmap v546: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 748 op/s 2024-03-12T05:29:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 146571 29.63 MB/sec execute 132 sec latency 2642.111 ms 2024-03-12T05:29:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 146951 29.54 MB/sec execute 133 sec latency 3642.138 ms 2024-03-12T05:29:30.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:30 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:29:30.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:30 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:29:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:30 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:29:31.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 147360 29.51 MB/sec execute 134 sec latency 4642.193 ms 2024-03-12T05:29:31.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:31 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:29:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:31 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:29:31.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:31 smithi100 ceph-mon[32210]: pgmap v547: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 468 op/s 2024-03-12T05:29:31.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:31 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:29:31.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:31 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:29:31.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:31 smithi156 ceph-mon[36270]: pgmap v547: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 468 op/s 2024-03-12T05:29:31.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:31 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:29:31.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:31 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:29:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:31 smithi161 ceph-mon[36043]: pgmap v547: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 468 op/s 2024-03-12T05:29:32.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 147775 29.43 MB/sec execute 135 sec latency 4731.769 ms 2024-03-12T05:29:32.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:32 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:29:32.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:32 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:29:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:32 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:29:33.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 148233 29.36 MB/sec execute 136 sec latency 198.808 ms 2024-03-12T05:29:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:33 smithi100 ceph-mon[32210]: pgmap v548: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 638 op/s 2024-03-12T05:29:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:33 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:29:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:33 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:29:33.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:33 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:29:33.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:33 smithi156 ceph-mon[36270]: pgmap v548: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 638 op/s 2024-03-12T05:29:33.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:33 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:29:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:33 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:29:33.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:33 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:29:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:33 smithi161 ceph-mon[36043]: pgmap v548: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 638 op/s 2024-03-12T05:29:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:33 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:29:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:33 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:29:33.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:33 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:29:34.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 148811 29.35 MB/sec execute 137 sec latency 171.062 ms 2024-03-12T05:29:35.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 149308 29.30 MB/sec execute 138 sec latency 307.664 ms 2024-03-12T05:29:35.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:35 smithi100 ceph-mon[32210]: pgmap v549: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 508 op/s 2024-03-12T05:29:35.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:35 smithi156 ceph-mon[36270]: pgmap v549: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 508 op/s 2024-03-12T05:29:35.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:35 smithi161 ceph-mon[36043]: pgmap v549: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 508 op/s 2024-03-12T05:29:36.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 149879 29.42 MB/sec execute 139 sec latency 1195.559 ms 2024-03-12T05:29:36.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:36 smithi100 ceph-mon[32210]: pgmap v550: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.9 MiB/s wr, 424 op/s 2024-03-12T05:29:36.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:36 smithi156 ceph-mon[36270]: pgmap v550: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.9 MiB/s wr, 424 op/s 2024-03-12T05:29:36.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:36 smithi161 ceph-mon[36043]: pgmap v550: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.9 MiB/s wr, 424 op/s 2024-03-12T05:29:37.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 150438 29.42 MB/sec execute 140 sec latency 2195.599 ms 2024-03-12T05:29:38.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 150885 29.38 MB/sec execute 141 sec latency 3195.642 ms 2024-03-12T05:29:38.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:38 smithi156 ceph-mon[36270]: pgmap v551: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-03-12T05:29:38.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:38 smithi161 ceph-mon[36043]: pgmap v551: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-03-12T05:29:38.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:38 smithi100 ceph-mon[32210]: pgmap v551: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-03-12T05:29:39.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 151299 29.30 MB/sec execute 142 sec latency 4195.670 ms 2024-03-12T05:29:40.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 151842 29.30 MB/sec execute 143 sec latency 5195.700 ms 2024-03-12T05:29:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:40 smithi156 ceph-mon[36270]: pgmap v552: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 415 op/s 2024-03-12T05:29:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:40 smithi161 ceph-mon[36043]: pgmap v552: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 415 op/s 2024-03-12T05:29:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:40 smithi100 ceph-mon[32210]: pgmap v552: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 415 op/s 2024-03-12T05:29:41.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 152268 29.27 MB/sec execute 144 sec latency 6195.737 ms 2024-03-12T05:29:42.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 152722 29.23 MB/sec execute 145 sec latency 7195.767 ms 2024-03-12T05:29:42.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:42 smithi156 ceph-mon[36270]: pgmap v553: 97 pgs: 97 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 597 op/s 2024-03-12T05:29:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:42 smithi161 ceph-mon[36043]: pgmap v553: 97 pgs: 97 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 597 op/s 2024-03-12T05:29:42.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:42 smithi100 ceph-mon[32210]: pgmap v553: 97 pgs: 97 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 597 op/s 2024-03-12T05:29:43.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 153296 29.24 MB/sec execute 146 sec latency 8195.799 ms 2024-03-12T05:29:43.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:29:43.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:29:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 -- 172.21.15.100:0/1942986442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 msgr2=0x7fe054069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 --2- 172.21.15.100:0/1942986442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe054069e00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe050009930 tx=0x7fe05002bc30 comp rx=0 tx=0).stop 2024-03-12T05:29:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 -- 172.21.15.100:0/1942986442 shutdown_connections 2024-03-12T05:29:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 --2- 172.21.15.100:0/1942986442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe054106450 0x7fe05410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 --2- 172.21.15.100:0/1942986442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05406a3d0 0x7fe054105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 --2- 172.21.15.100:0/1942986442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe054069e00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.652+0000 7fe05b281640 1 -- 172.21.15.100:0/1942986442 >> 172.21.15.100:0/1942986442 conn(0x7fe054076e20 msgr2=0x7fe054077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 -- 172.21.15.100:0/1942986442 shutdown_connections 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 -- 172.21.15.100:0/1942986442 wait complete. 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 Processor -- start 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 -- start start 2024-03-12T05:29:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe0541975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe054106450 0x7fe054197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05419be60 0x7fe05419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419c850 con 0x7fe054069a20 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419c9c0 con 0x7fe05419be60 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.653+0000 7fe05b281640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419cb30 con 0x7fe054106450 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05901d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe0541975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05419be60 0x7fe05419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05901d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe0541975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42230/0 (socket says 172.21.15.100:42230) 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05901d640 1 -- 172.21.15.100:0/2612054582 learned_addr learned my addr 172.21.15.100:0/2612054582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05881c640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe054106450 0x7fe054197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 -- 172.21.15.100:0/2612054582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe054106450 msgr2=0x7fe054197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe054106450 0x7fe054197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 -- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 msgr2=0x7fe0541975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe0541975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 -- 172.21.15.100:0/2612054582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050009560 con 0x7fe05419be60 2024-03-12T05:29:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.654+0000 7fe05981e640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05419be60 0x7fe05419c310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009fd0 tx=0x7fe04c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.655+0000 7fe04a7fc640 1 -- 172.21.15.100:0/2612054582 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00ce20 con 0x7fe05419be60 2024-03-12T05:29:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.655+0000 7fe04a7fc640 1 -- 172.21.15.100:0/2612054582 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe04c005020 con 0x7fe05419be60 2024-03-12T05:29:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.655+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0541a4110 con 0x7fe05419be60 2024-03-12T05:29:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.655+0000 7fe04a7fc640 1 -- 172.21.15.100:0/2612054582 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c010690 con 0x7fe05419be60 2024-03-12T05:29:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.655+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0541a45c0 con 0x7fe05419be60 2024-03-12T05:29:43.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.657+0000 7fe04a7fc640 1 -- 172.21.15.100:0/2612054582 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe04c004b20 con 0x7fe05419be60 2024-03-12T05:29:43.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 Processor -- start 2024-03-12T05:29:43.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 -- start start 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 0x56503f877690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f880270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8807b0 0x56503f882ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f7e6280 con 0x56503f877bd0 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f7e37b0 con 0x56503f8807b0 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05de4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f7def90 con 0x56503f8772b0 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05901d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f880270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05881c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 0x56503f877690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05901d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f880270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42238/0 (socket says 172.21.15.100:42238) 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05881c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 0x56503f877690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:60684/0 (socket says 172.21.15.100:60684) 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe04a7fc640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0400a1cf0 0x7fe0400a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05901d640 1 -- 172.21.15.100:0/4107213009 learned_addr learned my addr 172.21.15.100:0/4107213009 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.658+0000 7fe05881c640 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8807b0 0x56503f882ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05981e640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0400a1cf0 0x7fe0400a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05901d640 1 -- 172.21.15.100:0/4107213009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 msgr2=0x56503f877690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05901d640 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 0x56503f877690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05901d640 1 -- 172.21.15.100:0/4107213009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8807b0 msgr2=0x56503f882ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05901d640 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8807b0 0x56503f882ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe04a7fc640 1 -- 172.21.15.100:0/2612054582 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe04c014070 con 0x7fe05419be60 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05901d640 1 -- 172.21.15.100:0/4107213009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56503f79d570 con 0x56503f877bd0 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05881c640 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 0x56503f877690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05901d640 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f880270 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe050005ec0 tx=0x7fe05002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547bfeb7df312672 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe05981e640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0400a1cf0 0x7fe0400a41b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe04c01a040 tx=0x7fe04c079040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe03f7fe640 1 -- 172.21.15.100:0/4107213009 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050004a10 con 0x56503f877bd0 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe03f7fe640 1 -- 172.21.15.100:0/4107213009 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe050004030 con 0x56503f877bd0 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.659+0000 7fe03f7fe640 1 -- 172.21.15.100:0/4107213009 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500053a0 con 0x56503f877bd0 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 -- 172.21.15.100:0/4107213009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 msgr2=0x56503f880270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f880270 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe050005ec0 tx=0x7fe05002c5d0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 -- 172.21.15.100:0/4107213009 shutdown_connections 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8807b0 0x56503f882ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f880270 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4107213009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8772b0 0x56503f877690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 -- 172.21.15.100:0/4107213009 >> 172.21.15.100:0/4107213009 conn(0x56503f871990 msgr2=0x56503f871d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 -- 172.21.15.100:0/4107213009 shutdown_connections 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.660+0000 7fe05de4f740 1 -- 172.21.15.100:0/4107213009 wait complete. 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 -- start start 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 Processor -- start 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f87db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f87e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f4b2430 con 0x56503f877bd0 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f46ee00 con 0x56503f8772b0 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f763fd0 con 0x56503f8807b0 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05de4f740 10 client.0 _ll_register_callbacks cb 0x56503f7b9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f87db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f87db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42252/0 (socket says 172.21.15.100:42252) 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 -- 172.21.15.100:0/4127062087 learned_addr learned my addr 172.21.15.100:0/4127062087 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05881c640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f87e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05881c640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 -- 172.21.15.100:0/4127062087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 msgr2=0x56503f87e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f87e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 -- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 msgr2=0x56503f87d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.661+0000 7fe05901d640 1 -- 172.21.15.100:0/4127062087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05410e810 con 0x56503f877bd0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe05901d640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f87db70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe050009850 tx=0x7fe050004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b7c2193d743c65 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe03cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe03cff9640 1 -- 172.21.15.100:0/4127062087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500050a0 con 0x56503f877bd0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe03cff9640 1 -- 172.21.15.100:0/4127062087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe05004f070 con 0x56503f877bd0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56503f4de7e0 con 0x56503f877bd0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe03cff9640 1 -- 172.21.15.100:0/4127062087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe050005200 con 0x56503f877bd0 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe05de4f740 10 client.19161 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:43.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56503f7dd610 con 0x56503f877bd0 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.662+0000 7fe03cff9640 1 -- 172.21.15.100:0/4127062087 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56503f4de7e0 con 0x56503f877bd0 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe03cff9640 1 -- 172.21.15.100:0/4127062087 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe05003c050 con 0x56503f877bd0 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05de4f740 10 client.19161 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05de4f740 10 client.19161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05de4f740 20 client.19161 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f890500 0x56503f892920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05de4f740 4 client.19161 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56503f51b720 con 0x56503f890500 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05881c640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f890500 0x56503f892920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe03cff9640 10 client.19161 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.663+0000 7fe05881c640 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f890500 0x56503f892920 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc58f5d077b1b6e server_cookie=59ac05ba72a979fd in_seq=0 out_seq=0 2024-03-12T05:29:43.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe03cff9640 1 -- 172.21.15.100:0/4127062087 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56503f51b720 con 0x56503f890500 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe03cff9640 10 client.19161 handle_command_reply: tid=0 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 client.19161 shutdown 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 msgr2=0x56503f87db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f87db70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe050009850 tx=0x7fe050004e50 comp rx=0 tx=0).stop 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 shutdown_connections 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f890500 0x56503f892920 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f87e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f877bd0 0x56503f87db70 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4127062087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe03cff9640 0 client.19161 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.664+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 >> 172.21.15.100:0/4127062087 conn(0x56503f871990 msgr2=0x56503f873730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.665+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 shutdown_connections 2024-03-12T05:29:43.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.665+0000 7fe05de4f740 1 -- 172.21.15.100:0/4127062087 wait complete. 2024-03-12T05:29:43.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.665+0000 7fe05de4f740 20 client.19161 trim_cache size 0 max 16384 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 Processor -- start 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 -- start start 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f877bd0 0x56503f87ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 0x56503f87c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f876cf0 con 0x56503f8807b0 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f8729e0 con 0x56503f8772b0 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05de4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f8711d0 con 0x56503f877bd0 2024-03-12T05:29:43.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 0x56503f87c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 0x56503f87c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42268/0 (socket says 172.21.15.100:42268) 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 -- 172.21.15.100:0/4078667138 learned_addr learned my addr 172.21.15.100:0/4078667138 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05901d640 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f877bd0 0x56503f87ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 -- 172.21.15.100:0/4078667138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f877bd0 msgr2=0x56503f87ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f877bd0 0x56503f87ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 -- 172.21.15.100:0/4078667138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 msgr2=0x56503f87fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 -- 172.21.15.100:0/4078667138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56503f779ac0 con 0x56503f8807b0 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.690+0000 7fe05881c640 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 0x56503f87c610 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe0541025a0 tx=0x7fe044013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdb8155a8a4e4ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.691+0000 7fe03effd640 1 -- 172.21.15.100:0/4078667138 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044016040 con 0x56503f8807b0 2024-03-12T05:29:43.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.691+0000 7fe03effd640 1 -- 172.21.15.100:0/4078667138 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe04402e070 con 0x56503f8807b0 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.691+0000 7fe03effd640 1 -- 172.21.15.100:0/4078667138 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044004b00 con 0x56503f8807b0 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.691+0000 7fe05de4f740 1 -- 172.21.15.100:0/4078667138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 msgr2=0x56503f87c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.691+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 0x56503f87c610 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe0541025a0 tx=0x7fe044013040 comp rx=0 tx=0).stop 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 -- 172.21.15.100:0/4078667138 shutdown_connections 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8807b0 0x56503f87c610 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f877bd0 0x56503f87ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4078667138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f8772b0 0x56503f87fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 -- 172.21.15.100:0/4078667138 >> 172.21.15.100:0/4078667138 conn(0x56503f871990 msgr2=0x56503f87bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 -- 172.21.15.100:0/4078667138 shutdown_connections 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 -- 172.21.15.100:0/4078667138 wait complete. 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 -- start start 2024-03-12T05:29:43.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 Processor -- start 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.692+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8772b0 0x56503f8863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f877bd0 0x56503f8868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05de4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f8e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05de4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f8871c0 con 0x56503f8772b0 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05de4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f8e6f50 con 0x56503f877bd0 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05de4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503f8e7090 con 0x56503f8807b0 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05de4f740 10 client.0 _ll_register_callbacks cb 0x7fe04c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05881c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8772b0 0x56503f8863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05881c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8772b0 0x56503f8863b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42276/0 (socket says 172.21.15.100:42276) 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05881c640 1 -- 172.21.15.100:0/4010469064 learned_addr learned my addr 172.21.15.100:0/4010469064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f877bd0 0x56503f8868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05881c640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f8e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 -- 172.21.15.100:0/4010469064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 msgr2=0x56503f8e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f8e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 -- 172.21.15.100:0/4010469064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8772b0 msgr2=0x56503f8863b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8772b0 0x56503f8863b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 -- 172.21.15.100:0/4010469064 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe044004450 con 0x56503f877bd0 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe05901d640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f877bd0 0x56503f8868f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56503f87be10 tx=0x7fe050003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed96f70eee2d406 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.693+0000 7fe03cff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe03cff9640 1 -- 172.21.15.100:0/4010469064 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500050d0 con 0x56503f877bd0 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe03cff9640 1 -- 172.21.15.100:0/4010469064 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe05004f040 con 0x56503f877bd0 2024-03-12T05:29:43.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe050004030 con 0x56503f877bd0 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe03cff9640 1 -- 172.21.15.100:0/4010469064 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0500044b0 con 0x56503f877bd0 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 10 client.28511 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56503f79f6a0 con 0x56503f877bd0 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe03cff9640 1 -- 172.21.15.100:0/4010469064 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe05006a020 con 0x56503f877bd0 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe03cff9640 1 -- 172.21.15.100:0/4010469064 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe05003c040 con 0x56503f877bd0 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 10 client.28511 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 10 client.28511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 20 client.28511 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f8e83c0 0x56503f8ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 4 client.28511 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:29:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.694+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56503f56a890 con 0x56503f8e83c0 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05881c640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f8e83c0 0x56503f8ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe03cff9640 10 client.28511 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05881c640 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f8e83c0 0x56503f8ea7e0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39c5f74d00ff0c8 server_cookie=242cb41cd5174e24 in_seq=0 out_seq=0 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe03cff9640 1 -- 172.21.15.100:0/4010469064 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56503f56a890 con 0x56503f8e83c0 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe03cff9640 10 client.28511 handle_command_reply: tid=0 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 client.28511 shutdown 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f877bd0 msgr2=0x56503f8868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f877bd0 0x56503f8868f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56503f87be10 tx=0x7fe050003820 comp rx=0 tx=0).stop 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 shutdown_connections 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56503f8e83c0 0x56503f8ea7e0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56503f8807b0 0x56503f8e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe03cff9640 0 client.28511 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56503f877bd0 0x56503f8868f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 --2- 172.21.15.100:0/4010469064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56503f8772b0 0x56503f8863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.695+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 >> 172.21.15.100:0/4010469064 conn(0x56503f871990 msgr2=0x56503f8701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 shutdown_connections 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05de4f740 1 -- 172.21.15.100:0/4010469064 wait complete. 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05de4f740 20 client.28511 trim_cache size 0 max 16384 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:29:43.697 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:29:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0400a1cf0 msgr2=0x7fe0400a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0400a1cf0 0x7fe0400a41b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe04c01a040 tx=0x7fe04c079040 comp rx=0 tx=0).stop 2024-03-12T05:29:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05419be60 msgr2=0x7fe05419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05419be60 0x7fe05419c310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009fd0 tx=0x7fe04c00ef10 comp rx=0 tx=0).stop 2024-03-12T05:29:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 shutdown_connections 2024-03-12T05:29:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe0400a1cf0 0x7fe0400a41b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe05419be60 0x7fe05419c310 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe054106450 0x7fe054197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 --2- 172.21.15.100:0/2612054582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe054069a20 0x7fe0541975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 >> 172.21.15.100:0/2612054582 conn(0x7fe054076e20 msgr2=0x7fe0540fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 shutdown_connections 2024-03-12T05:29:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.696+0000 7fe05b281640 1 -- 172.21.15.100:0/2612054582 wait complete. 2024-03-12T05:29:43.702 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:29:43.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 -- 172.21.15.100:0/2879161598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce60105370 msgr2=0x7fce6010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 --2- 172.21.15.100:0/2879161598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce60105370 0x7fce6010cf40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fce58009cd0 tx=0x7fce5802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 -- 172.21.15.100:0/2879161598 shutdown_connections 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 --2- 172.21.15.100:0/2879161598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce60105370 0x7fce6010cf40 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 --2- 172.21.15.100:0/2879161598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60100c50 0x7fce60104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 --2- 172.21.15.100:0/2879161598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce60100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 -- 172.21.15.100:0/2879161598 >> 172.21.15.100:0/2879161598 conn(0x7fce60076e30 msgr2=0x7fce600fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 -- 172.21.15.100:0/2879161598 shutdown_connections 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 -- 172.21.15.100:0/2879161598 wait complete. 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 Processor -- start 2024-03-12T05:29:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.825+0000 7fce685bc640 1 -- start start 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce685bc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce6019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce685bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 0x7fce6019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce685bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce601975d0 0x7fce60197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce685bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60197fc0 con 0x7fce601975d0 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce685bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60198130 con 0x7fce60105370 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce685bc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601982f0 con 0x7fce601002a0 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce66358640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce6019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 0x7fce6019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce66b59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce601975d0 0x7fce60197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce66358640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce6019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:60740/0 (socket says 172.21.15.100:60740) 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 0x7fce6019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41788/0 (socket says 172.21.15.100:41788) 2024-03-12T05:29:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce66358640 1 -- 172.21.15.100:0/1902531884 learned_addr learned my addr 172.21.15.100:0/1902531884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 -- 172.21.15.100:0/1902531884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 msgr2=0x7fce6019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce6019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 -- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce601975d0 msgr2=0x7fce60197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce601975d0 0x7fce60197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 -- 172.21.15.100:0/1902531884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58009900 con 0x7fce60105370 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce66b59640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce601975d0 0x7fce60197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce66358640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce6019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.826+0000 7fce65b57640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 0x7fce6019d960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e9c0 tx=0x7fce5000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.827+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000ccc0 con 0x7fce60105370 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.827+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce601a42e0 con 0x7fce60105370 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.827+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce5000ce20 con 0x7fce60105370 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.827+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50018880 con 0x7fce60105370 2024-03-12T05:29:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.827+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce601a4840 con 0x7fce60105370 2024-03-12T05:29:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.828+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce600676e0 con 0x7fce60105370 2024-03-12T05:29:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.829+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fce500189e0 con 0x7fce60105370 2024-03-12T05:29:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.830+0000 7fce577fe640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fce4c0a1ba0 0x7fce4c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.830+0000 7fce66358640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fce4c0a1ba0 0x7fce4c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.830+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fce50014070 con 0x7fce60105370 2024-03-12T05:29:43.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.830+0000 7fce66358640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fce4c0a1ba0 0x7fce4c0a4060 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fce5c011500 tx=0x7fce5c005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:43.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:43.836+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce50096e80 con 0x7fce60105370 2024-03-12T05:29:44.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.100+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce60198b00 con 0x7fce60105370 2024-03-12T05:29:44.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.101+0000 7fce577fe640 1 -- 172.21.15.100:0/1902531884 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fce5009a540 con 0x7fce60105370 2024-03-12T05:29:44.095 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:29:44.095 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:29:44.095 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:29:44.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fce4c0a1ba0 msgr2=0x7fce4c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fce4c0a1ba0 0x7fce4c0a4060 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fce5c011500 tx=0x7fce5c005f90 comp rx=0 tx=0).stop 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 msgr2=0x7fce6019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 0x7fce6019d960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e9c0 tx=0x7fce5000ee80 comp rx=0 tx=0).stop 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 shutdown_connections 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fce4c0a1ba0 0x7fce4c0a4060 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce601975d0 0x7fce60197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce60105370 0x7fce6019d960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 --2- 172.21.15.100:0/1902531884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce601002a0 0x7fce6019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 >> 172.21.15.100:0/1902531884 conn(0x7fce60076e30 msgr2=0x7fce600697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 shutdown_connections 2024-03-12T05:29:44.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.104+0000 7fce685bc640 1 -- 172.21.15.100:0/1902531884 wait complete. 2024-03-12T05:29:44.109 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:29:44.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:29:44.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 -- 172.21.15.100:0/1964222431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 msgr2=0x7f9a2010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 --2- 172.21.15.100:0/1964222431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a2010dc30 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009ca0 tx=0x7f9a1802c180 comp rx=0 tx=0).stop 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 -- 172.21.15.100:0/1964222431 shutdown_connections 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 --2- 172.21.15.100:0/1964222431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a2010dc30 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 --2- 172.21.15.100:0/1964222431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a20102d10 0x7f9a2010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 --2- 172.21.15.100:0/1964222431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 0x7f9a201027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 -- 172.21.15.100:0/1964222431 >> 172.21.15.100:0/1964222431 conn(0x7f9a200771e0 msgr2=0x7f9a200775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 -- 172.21.15.100:0/1964222431 shutdown_connections 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 -- 172.21.15.100:0/1964222431 wait complete. 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.228+0000 7f9a267d0640 1 Processor -- start 2024-03-12T05:29:44.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 -- start start 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 0x7f9a2019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a20102d10 0x7f9a2019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a20197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20197c60 con 0x7f9a2010b7c0 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20197dd0 con 0x7f9a201023f0 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a267d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20197f90 con 0x7f9a20102d10 2024-03-12T05:29:44.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a20197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 0x7f9a2019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a20102d10 0x7f9a2019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a20197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42304/0 (socket says 172.21.15.100:42304) 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 -- 172.21.15.100:0/65029571 learned_addr learned my addr 172.21.15.100:0/65029571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 0x7f9a2019d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41818/0 (socket says 172.21.15.100:41818) 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 -- 172.21.15.100:0/65029571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a20102d10 msgr2=0x7f9a2019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a20102d10 0x7f9a2019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 -- 172.21.15.100:0/65029571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 msgr2=0x7f9a2019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 0x7f9a2019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 -- 172.21.15.100:0/65029571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18009900 con 0x7f9a2010b7c0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.229+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a20197610 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9a1802c790 tx=0x7f9a18034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.230+0000 7f9a1d7fa640 1 -- 172.21.15.100:0/65029571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18003b70 con 0x7f9a2010b7c0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.230+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a20198210 con 0x7f9a2010b7c0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.230+0000 7f9a1d7fa640 1 -- 172.21.15.100:0/65029571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a18003cd0 con 0x7f9a2010b7c0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.230+0000 7f9a1d7fa640 1 -- 172.21.15.100:0/65029571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a180046c0 con 0x7f9a2010b7c0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.230+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a20100150 con 0x7f9a2010b7c0 2024-03-12T05:29:44.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.232+0000 7f9a1d7fa640 1 -- 172.21.15.100:0/65029571 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9a1800a040 con 0x7f9a2010b7c0 2024-03-12T05:29:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.232+0000 7f9a293a3740 1 Processor -- start 2024-03-12T05:29:44.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.232+0000 7f9a1d7fa640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1c20 0x7f9a080a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 -- start start 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1c20 0x7f9a080a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1d7fa640 1 -- 172.21.15.100:0/65029571 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a180f3900 con 0x7f9a2010b7c0 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x564297276630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297281b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564297231960 con 0x564297276b70 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642971e5310 con 0x564297276250 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a293a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642971e2840 con 0x56429727f750 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42314/0 (socket says 172.21.15.100:42314) 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1976592084 learned_addr learned my addr 172.21.15.100:0/1976592084 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x564297276630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297281b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1c20 0x7f9a080a40e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009c20 tx=0x7f9a14011810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1976592084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 msgr2=0x564297281b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297281b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1976592084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 msgr2=0x564297276630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x564297276630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/1976592084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642971a0060 con 0x564297276b70 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.233+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727f210 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009c00 tx=0x7f9a10011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f964389662acc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a067fc640 1 -- 172.21.15.100:0/1976592084 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10004310 con 0x564297276b70 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a067fc640 1 -- 172.21.15.100:0/1976592084 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a10004470 con 0x564297276b70 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a067fc640 1 -- 172.21.15.100:0/1976592084 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10019910 con 0x564297276b70 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 -- 172.21.15.100:0/1976592084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 msgr2=0x56429727f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727f210 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009c00 tx=0x7f9a10011440 comp rx=0 tx=0).stop 2024-03-12T05:29:44.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 -- 172.21.15.100:0/1976592084 shutdown_connections 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297281b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727f210 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 --2- 172.21.15.100:0/1976592084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x564297276630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 -- 172.21.15.100:0/1976592084 >> 172.21.15.100:0/1976592084 conn(0x564297270980 msgr2=0x564297270d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.234+0000 7f9a293a3740 1 -- 172.21.15.100:0/1976592084 shutdown_connections 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 -- 172.21.15.100:0/1976592084 wait complete. 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 -- start start 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 Processor -- start 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x56429727d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642971de020 con 0x564297276b70 2024-03-12T05:29:44.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564296eb1430 con 0x564297276250 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564296e6de00 con 0x56429727f750 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a293a3740 10 client.0 _ll_register_callbacks cb 0x564297185cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.235+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42318/0 (socket says 172.21.15.100:42318) 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/2603085658 learned_addr learned my addr 172.21.15.100:0/2603085658 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x56429727d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/2603085658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 msgr2=0x56429727d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x56429727d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 msgr2=0x56429727c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/2603085658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a10009830 con 0x564297276b70 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x56429727d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727cd00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9a100096e0 tx=0x7f9a10005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421d4a09995b8da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f99ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f99ff7fe640 1 -- 172.21.15.100:0/2603085658 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10019890 con 0x564297276b70 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f99ff7fe640 1 -- 172.21.15.100:0/2603085658 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a10005620 con 0x564297276b70 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564296edd7e0 con 0x564297276b70 2024-03-12T05:29:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f99ff7fe640 1 -- 172.21.15.100:0/2603085658 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a100199f0 con 0x564297276b70 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a293a3740 10 client.19185 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5642971dc6a0 con 0x564297276b70 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.236+0000 7f99ff7fe640 1 -- 172.21.15.100:0/2603085658 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564296edd7e0 con 0x564297276b70 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f99ff7fe640 1 -- 172.21.15.100:0/2603085658 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a10003680 con 0x564297276b70 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a293a3740 10 client.19185 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a293a3740 10 client.19185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a293a3740 20 client.19185 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a293a3740 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56429728fab0 0x564297291ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a293a3740 4 client.19185 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564296f1a720 con 0x56429728fab0 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56429728fab0 0x564297291ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56429728fab0 0x564297291ed0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9942611044dde905 server_cookie=713e9843c405511 in_seq=0 out_seq=0 2024-03-12T05:29:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.237+0000 7f99ff7fe640 10 client.19185 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.240+0000 7f99ff7fe640 1 -- 172.21.15.100:0/2603085658 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564296f1a720 con 0x56429728fab0 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.240+0000 7f99ff7fe640 10 client.19185 handle_command_reply: tid=0 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 client.19185 shutdown 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 msgr2=0x56429727cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727cd00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9a100096e0 tx=0x7f9a10005430 comp rx=0 tx=0).stop 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 shutdown_connections 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x56429728fab0 0x564297291ed0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x56429727d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727cd00 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 --2- 172.21.15.100:0/2603085658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 >> 172.21.15.100:0/2603085658 conn(0x564297270980 msgr2=0x5642972726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f99ff7fe640 0 client.19185 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:44.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 shutdown_connections 2024-03-12T05:29:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 1 -- 172.21.15.100:0/2603085658 wait complete. 2024-03-12T05:29:44.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.241+0000 7f9a293a3740 20 client.19185 trim_cache size 0 max 16384 2024-03-12T05:29:44.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 153788 29.25 MB/sec execute 147 sec latency 8652.361 ms 2024-03-12T05:29:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.253+0000 7f9a293a3740 1 Processor -- start 2024-03-12T05:29:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.253+0000 7f9a293a3740 1 -- start start 2024-03-12T05:29:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297296870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a293a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564297163130 con 0x564297276b70 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a293a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564297282110 con 0x564297276250 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a293a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429728c6e0 con 0x56429727f750 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42322/0 (socket says 172.21.15.100:42322) 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/3775935219 learned_addr learned my addr 172.21.15.100:0/3775935219 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297296870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/3775935219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 msgr2=0x564297296870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297296870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/3775935219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 msgr2=0x56429727eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 -- 172.21.15.100:0/3775935219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564297178c20 con 0x564297276b70 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a1f7fe640 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727d060 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9a10011940 tx=0x7f9a10009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ae9689f182d801 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a05ffb640 1 -- 172.21.15.100:0/3775935219 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10004fd0 con 0x564297276b70 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a05ffb640 1 -- 172.21.15.100:0/3775935219 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a100035f0 con 0x564297276b70 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.254+0000 7f9a05ffb640 1 -- 172.21.15.100:0/3775935219 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1001a440 con 0x564297276b70 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 -- 172.21.15.100:0/3775935219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 msgr2=0x56429727d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727d060 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9a10011940 tx=0x7f9a10009ad0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 -- 172.21.15.100:0/3775935219 shutdown_connections 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56429727f750 0x564297296870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564297276b70 0x56429727d060 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 --2- 172.21.15.100:0/3775935219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276250 0x56429727eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 -- 172.21.15.100:0/3775935219 >> 172.21.15.100:0/3775935219 conn(0x564297270980 msgr2=0x56429726ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 -- 172.21.15.100:0/3775935219 shutdown_connections 2024-03-12T05:29:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 -- 172.21.15.100:0/3775935219 wait complete. 2024-03-12T05:29:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.255+0000 7f9a293a3740 1 -- start start 2024-03-12T05:29:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 Processor -- start 2024-03-12T05:29:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564297276250 0x564297286280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276b70 0x5642972867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 0x564297284740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564297284e10 con 0x56429727f750 2024-03-12T05:29:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564297284f80 con 0x564297276b70 2024-03-12T05:29:44.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564297285140 con 0x564297276250 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a293a3740 10 client.0 _ll_register_callbacks cb 0x7f9a10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 0x564297284740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 0x564297284740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42338/0 (socket says 172.21.15.100:42338) 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276b70 0x5642972867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 -- 172.21.15.100:0/920003061 learned_addr learned my addr 172.21.15.100:0/920003061 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564297276250 0x564297286280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 -- 172.21.15.100:0/920003061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564297276250 msgr2=0x564297286280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564297276250 0x564297286280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 -- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276b70 msgr2=0x5642972867c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276b70 0x5642972867c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.256+0000 7f9a24d6d640 1 -- 172.21.15.100:0/920003061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642971dc6a0 con 0x56429727f750 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a24d6d640 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 0x564297284740 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9a180bcfd0 tx=0x7f9a180e0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb0018bb9ad85d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f99ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f99ff7fe640 1 -- 172.21.15.100:0/920003061 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18003e40 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a10004470 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f99ff7fe640 1 -- 172.21.15.100:0/920003061 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9a180c0d00 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f99ff7fe640 1 -- 172.21.15.100:0/920003061 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a180c0600 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 10 client.19191 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56429719e8c0 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f99ff7fe640 1 -- 172.21.15.100:0/920003061 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a10004470 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f99ff7fe640 1 -- 172.21.15.100:0/920003061 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a180bcc70 con 0x56429727f750 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 10 client.19191 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 10 client.19191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 20 client.19191 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5642972e6d60 0x5642972e9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 4 client.19191 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.257+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564296f69890 con 0x5642972e6d60 2024-03-12T05:29:44.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.258+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5642972e6d60 0x5642972e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.258+0000 7f99ff7fe640 10 client.19191 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.258+0000 7f9a1ffff640 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5642972e6d60 0x5642972e9150 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf8bfb211ffbdfeb server_cookie=c3d68e7466ede2e6 in_seq=0 out_seq=0 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.259+0000 7f99ff7fe640 1 -- 172.21.15.100:0/920003061 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564296f69890 con 0x5642972e6d60 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.259+0000 7f99ff7fe640 10 client.19191 handle_command_reply: tid=0 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.259+0000 7f9a293a3740 1 client.19191 shutdown 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 msgr2=0x564297284740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 0x564297284740 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9a180bcfd0 tx=0x7f9a180e0040 comp rx=0 tx=0).stop 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 shutdown_connections 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5642972e6d60 0x5642972e9150 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56429727f750 0x564297284740 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564297276b70 0x5642972867c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f99ff7fe640 0 client.19191 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 --2- 172.21.15.100:0/920003061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564297276250 0x564297286280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 >> 172.21.15.100:0/920003061 conn(0x564297270980 msgr2=0x56429727aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 shutdown_connections 2024-03-12T05:29:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 1 -- 172.21.15.100:0/920003061 wait complete. 2024-03-12T05:29:44.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a293a3740 20 client.19191 trim_cache size 0 max 16384 2024-03-12T05:29:44.259 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1c20 msgr2=0x7f9a080a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1c20 0x7f9a080a40e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009c20 tx=0x7f9a14011810 comp rx=0 tx=0).stop 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 msgr2=0x7f9a20197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a20197610 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9a1802c790 tx=0x7f9a18034670 comp rx=0 tx=0).stop 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 shutdown_connections 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f9a080a1c20 0x7f9a080a40e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a2010b7c0 0x7f9a20197610 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a20102d10 0x7f9a2019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.260+0000 7f9a267d0640 1 --2- 172.21.15.100:0/65029571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a201023f0 0x7f9a2019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.261+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 >> 172.21.15.100:0/65029571 conn(0x7f9a200771e0 msgr2=0x7f9a200fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.261+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 shutdown_connections 2024-03-12T05:29:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.261+0000 7f9a267d0640 1 -- 172.21.15.100:0/65029571 wait complete. 2024-03-12T05:29:44.265 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:29:44.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 -- 172.21.15.100:0/1027581108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4105330 msgr2=0x7f4bd410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/1027581108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4105330 0x7f4bd410cf00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009cd0 tx=0x7f4bcc02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 -- 172.21.15.100:0/1027581108 shutdown_connections 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/1027581108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4105330 0x7f4bd410cf00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/1027581108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bd4100c10 0x7f4bd4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/1027581108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd4100260 0x7f4bd4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.390+0000 7f4bdadd0640 1 -- 172.21.15.100:0/1027581108 >> 172.21.15.100:0/1027581108 conn(0x7f4bd4077180 msgr2=0x7f4bd4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 -- 172.21.15.100:0/1027581108 shutdown_connections 2024-03-12T05:29:44.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 -- 172.21.15.100:0/1027581108 wait complete. 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 Processor -- start 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 -- start start 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bd4100260 0x7f4bd419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd4105330 0x7f4bd419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 0x7f4bd4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4198150 con 0x7f4bd4197680 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd41982c0 con 0x7f4bd4105330 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bdadd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4198480 con 0x7f4bd4100260 2024-03-12T05:29:44.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bd936d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 0x7f4bd4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd4105330 0x7f4bd419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bd8b6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bd4100260 0x7f4bd419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bd936d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 0x7f4bd4197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42356/0 (socket says 172.21.15.100:42356) 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.391+0000 7f4bd936d640 1 -- 172.21.15.100:0/4134020422 learned_addr learned my addr 172.21.15.100:0/4134020422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bd936d640 1 -- 172.21.15.100:0/4134020422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bd4100260 msgr2=0x7f4bd419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bd936d640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bd4100260 0x7f4bd419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bd936d640 1 -- 172.21.15.100:0/4134020422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd4105330 msgr2=0x7f4bd419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bd936d640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd4105330 0x7f4bd419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bd936d640 1 -- 172.21.15.100:0/4134020422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc009900 con 0x7f4bd4197680 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bd936d640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 0x7f4bd4197b00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc004af0 tx=0x7f4bcc032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bc9ffb640 1 -- 172.21.15.100:0/4134020422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0328b0 con 0x7f4bd4197680 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bc9ffb640 1 -- 172.21.15.100:0/4134020422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bcc003e50 con 0x7f4bd4197680 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd41a4370 con 0x7f4bd4197680 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bc9ffb640 1 -- 172.21.15.100:0/4134020422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc004740 con 0x7f4bd4197680 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.392+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd41a47f0 con 0x7f4bd4197680 2024-03-12T05:29:44.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bc9ffb640 1 -- 172.21.15.100:0/4134020422 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4bcc00a040 con 0x7f4bd4197680 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bc9ffb640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bbc0a1ba0 0x7f4bbc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bdd9a4740 1 Processor -- start 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bbc0a1ba0 0x7f4bbc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bdd9a4740 1 -- start start 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bc9ffb640 1 -- 172.21.15.100:0/4134020422 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4bcc0f0270 con 0x7f4bd4197680 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662250 0x55ff50662630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.395+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 0x55ff5066b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff5066db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5061d960 con 0x55ff5066b750 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff505d1310 con 0x55ff50662b70 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff505ce840 con 0x55ff50662250 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bbc0a1ba0 0x7f4bbc0a4060 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009bd0 tx=0x7f4bd0005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 0x55ff5066b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 0x55ff5066b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41872/0 (socket says 172.21.15.100:41872) 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bd8b6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662250 0x55ff50662630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 -- 172.21.15.100:0/3073755070 learned_addr learned my addr 172.21.15.100:0/3073755070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bd936d640 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff5066db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 -- 172.21.15.100:0/3073755070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662250 msgr2=0x55ff50662630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662250 0x55ff50662630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 -- 172.21.15.100:0/3073755070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 msgr2=0x55ff5066db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff5066db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 -- 172.21.15.100:0/3073755070 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff5058c060 con 0x55ff50662b70 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662250 0x55ff50662630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bcbfff640 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 0x55ff5066b210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009c00 tx=0x7f4bc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30cff40585fc0f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bbaffd640 1 -- 172.21.15.100:0/3073755070 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4004230 con 0x55ff50662b70 2024-03-12T05:29:44.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bbaffd640 1 -- 172.21.15.100:0/3073755070 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bc4009e70 con 0x55ff50662b70 2024-03-12T05:29:44.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.396+0000 7f4bbaffd640 1 -- 172.21.15.100:0/3073755070 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4019680 con 0x55ff50662b70 2024-03-12T05:29:44.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/3073755070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 msgr2=0x55ff5066b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 0x55ff5066b210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009c00 tx=0x7f4bc4011810 comp rx=0 tx=0).stop 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/3073755070 shutdown_connections 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff5066db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662b70 0x55ff5066b210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/3073755070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662250 0x55ff50662630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/3073755070 >> 172.21.15.100:0/3073755070 conn(0x55ff5065c980 msgr2=0x55ff5065cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/3073755070 shutdown_connections 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.397+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/3073755070 wait complete. 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 -- start start 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 Processor -- start 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50668770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50668cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff506691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff505ca020 con 0x55ff50662250 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5029d430 con 0x55ff5066b750 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff50259e00 con 0x55ff50662b70 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bdd9a4740 10 client.0 _ll_register_callbacks cb 0x55ff50571cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bd8b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50668770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50668cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bd936d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff506691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bd8b6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50668770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42380/0 (socket says 172.21.15.100:42380) 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.398+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/1608789404 learned_addr learned my addr 172.21.15.100:0/1608789404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 msgr2=0x55ff50668cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50668cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/1608789404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 msgr2=0x55ff506691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff506691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/1608789404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc4009830 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50668770 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000edc0 tx=0x7f4bd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c28cbc444123be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bb3fff640 1 -- 172.21.15.100:0/1608789404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0022070 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bb3fff640 1 -- 172.21.15.100:0/1608789404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4bd001db20 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff502c97e0 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bb3fff640 1 -- 172.21.15.100:0/1608789404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bd001dd20 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bdd9a4740 10 client.19203 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ff505c86a0 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bb3fff640 1 -- 172.21.15.100:0/1608789404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ff502c97e0 con 0x55ff50662250 2024-03-12T05:29:44.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.399+0000 7f4bb3fff640 1 -- 172.21.15.100:0/1608789404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bd0005550 con 0x55ff50662250 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bdd9a4740 10 client.19203 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bdd9a4740 10 client.19203 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bdd9a4740 20 client.19203 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff5067bbd0 0x55ff5067dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bdd9a4740 4 client.19203 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff50306720 con 0x55ff5067bbd0 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bcbfff640 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff5067bbd0 0x55ff5067dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bcbfff640 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff5067bbd0 0x55ff5067dff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5a660e025ae6edc server_cookie=9cfe090d6012f2fb in_seq=0 out_seq=0 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.400+0000 7f4bb3fff640 10 client.19203 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bb3fff640 1 -- 172.21.15.100:0/1608789404 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff50306720 con 0x55ff5067bbd0 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bb3fff640 10 client.19203 handle_command_reply: tid=0 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 client.19203 shutdown 2024-03-12T05:29:44.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 msgr2=0x55ff50668770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50668770 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000edc0 tx=0x7f4bd0004730 comp rx=0 tx=0).stop 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 shutdown_connections 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff5067bbd0 0x55ff5067dff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff506691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50668cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/1608789404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50668770 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bb3fff640 0 client.19203 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 >> 172.21.15.100:0/1608789404 conn(0x55ff5065c980 msgr2=0x55ff5065e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 shutdown_connections 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/1608789404 wait complete. 2024-03-12T05:29:44.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.402+0000 7f4bdd9a4740 20 client.19203 trim_cache size 0 max 16384 2024-03-12T05:29:44.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 Processor -- start 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 -- start start 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 0x55ff5066aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50669010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff50663a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5054f130 con 0x55ff5066b750 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff50661ef0 con 0x55ff50662250 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.420+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff50658080 con 0x55ff50662b70 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 0x55ff5066aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50669010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 0x55ff5066aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41892/0 (socket says 172.21.15.100:41892) 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/949541755 learned_addr learned my addr 172.21.15.100:0/949541755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd936d640 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff50663a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/949541755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 msgr2=0x55ff50669010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50669010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/949541755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 msgr2=0x55ff50663a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff50663a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 -- 172.21.15.100:0/949541755 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff50564c20 con 0x55ff50662250 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bcbfff640 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50669010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 0x55ff5066aea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0026460 tx=0x7f4bd00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f0c20754e34c72 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bba7fc640 1 -- 172.21.15.100:0/949541755 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0022030 con 0x55ff50662250 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.421+0000 7f4bba7fc640 1 -- 172.21.15.100:0/949541755 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bd000e970 con 0x55ff50662250 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.422+0000 7f4bba7fc640 1 -- 172.21.15.100:0/949541755 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0004b60 con 0x55ff50662250 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/949541755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 msgr2=0x55ff5066aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 0x55ff5066aea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0026460 tx=0x7f4bd00057f0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/949541755 shutdown_connections 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff5066b750 0x55ff50663a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50669010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/949541755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff50662250 0x55ff5066aea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/949541755 >> 172.21.15.100:0/949541755 conn(0x55ff5065c980 msgr2=0x55ff50682950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/949541755 shutdown_connections 2024-03-12T05:29:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/949541755 wait complete. 2024-03-12T05:29:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.423+0000 7f4bdd9a4740 1 -- start start 2024-03-12T05:29:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 Processor -- start 2024-03-12T05:29:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50672a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50670de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff50671380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff50671a80 con 0x55ff50662250 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff50671bf0 con 0x55ff5066b750 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff506d20e0 con 0x55ff50662b70 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bdd9a4740 10 client.0 _ll_register_callbacks cb 0x7f4bc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bd936d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff50671380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bd936d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff50671380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41896/0 (socket says 172.21.15.100:41896) 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bd936d640 1 -- 172.21.15.100:0/934531305 learned_addr learned my addr 172.21.15.100:0/934531305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.424+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50672a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd936d640 1 -- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 msgr2=0x55ff50670de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd936d640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50670de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd936d640 1 -- 172.21.15.100:0/934531305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 msgr2=0x55ff50672a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd936d640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50672a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd936d640 1 -- 172.21.15.100:0/934531305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff5058c060 con 0x55ff5066b750 2024-03-12T05:29:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50672a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bd936d640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff50671380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0b9940 tx=0x7f4bcc034da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d49e22ca65163a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bb3fff640 1 -- 172.21.15.100:0/934531305 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0bcf70 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bc4009e70 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bb3fff640 1 -- 172.21.15.100:0/934531305 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4bcc0c10c0 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bb3fff640 1 -- 172.21.15.100:0/934531305 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bcc0c5710 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bdd9a4740 10 client.28553 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.425+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ff505c86a0 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bb3fff640 1 -- 172.21.15.100:0/934531305 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bcc0d9070 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bb3fff640 1 -- 172.21.15.100:0/934531305 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bcc0bd1d0 con 0x55ff5066b750 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bdd9a4740 10 client.28553 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bdd9a4740 10 client.28553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:29:44.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bdd9a4740 20 client.28553 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:44.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff506d3710 0x55ff506d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bdd9a4740 4 client.28553 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:29:44.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff50355890 con 0x55ff506d3710 2024-03-12T05:29:44.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff506d3710 0x55ff506d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.426+0000 7f4bb3fff640 10 client.28553 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:44.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.427+0000 7f4bd8b6c640 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff506d3710 0x55ff506d5b30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f837aec8be8d31d9 server_cookie=5333e9649d71407e in_seq=0 out_seq=0 2024-03-12T05:29:44.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bb3fff640 1 -- 172.21.15.100:0/934531305 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff50355890 con 0x55ff506d3710 2024-03-12T05:29:44.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bb3fff640 10 client.28553 handle_command_reply: tid=0 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 client.28553 shutdown 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 msgr2=0x55ff50671380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff50671380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0b9940 tx=0x7f4bcc034da0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 shutdown_connections 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55ff506d3710 0x55ff506d5b30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff5066b750 0x55ff50671380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bb3fff640 0 client.28553 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff50662b70 0x55ff50670de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 --2- 172.21.15.100:0/934531305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff50662250 0x55ff50672a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 >> 172.21.15.100:0/934531305 conn(0x55ff5065c980 msgr2=0x55ff50663a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 shutdown_connections 2024-03-12T05:29:44.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 1 -- 172.21.15.100:0/934531305 wait complete. 2024-03-12T05:29:44.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.428+0000 7f4bdd9a4740 20 client.28553 trim_cache size 0 max 16384 2024-03-12T05:29:44.427 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:29:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bbc0a1ba0 msgr2=0x7f4bbc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bbc0a1ba0 0x7f4bbc0a4060 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009bd0 tx=0x7f4bd0005f90 comp rx=0 tx=0).stop 2024-03-12T05:29:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 msgr2=0x7f4bd4197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 0x7f4bd4197b00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc004af0 tx=0x7f4bcc032570 comp rx=0 tx=0).stop 2024-03-12T05:29:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 shutdown_connections 2024-03-12T05:29:44.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4bbc0a1ba0 0x7f4bbc0a4060 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd4197680 0x7f4bd4197b00 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd4105330 0x7f4bd419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 --2- 172.21.15.100:0/4134020422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bd4100260 0x7f4bd419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.429+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 >> 172.21.15.100:0/4134020422 conn(0x7f4bd4077180 msgr2=0x7f4bd40686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.430+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 shutdown_connections 2024-03-12T05:29:44.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.430+0000 7f4bdadd0640 1 -- 172.21.15.100:0/4134020422 wait complete. 2024-03-12T05:29:44.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:44 smithi100 ceph-mon[32210]: pgmap v554: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.7 MiB/s wr, 471 op/s 2024-03-12T05:29:44.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:29:44.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:29:44.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:44 smithi156 ceph-mon[36270]: pgmap v554: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.7 MiB/s wr, 471 op/s 2024-03-12T05:29:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:44 smithi161 ceph-mon[36043]: pgmap v554: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.7 MiB/s wr, 471 op/s 2024-03-12T05:29:44.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1936555879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100c10 msgr2=0x7f1b44104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1936555879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100c10 0x7f1b44104c50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009930 tx=0x7f1b3002bc30 comp rx=0 tx=0).stop 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1936555879 shutdown_connections 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1936555879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44105330 0x7f1b4410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1936555879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100c10 0x7f1b44104c50 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1936555879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44100260 0x7f1b44100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1936555879 >> 172.21.15.100:0/1936555879 conn(0x7f1b44077180 msgr2=0x7f1b44077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1936555879 shutdown_connections 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.556+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1936555879 wait complete. 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 Processor -- start 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 -- start start 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 0x7f1b4419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44105330 0x7f1b4419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44197640 0x7f1b44197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44198110 con 0x7f1b44100260 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44198280 con 0x7f1b44105330 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b4ab2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44198440 con 0x7f1b44197640 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 0x7f1b4419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 0x7f1b4419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42406/0 (socket says 172.21.15.100:42406) 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b488c8640 1 -- 172.21.15.100:0/1312635484 learned_addr learned my addr 172.21.15.100:0/1312635484 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b43fff640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44105330 0x7f1b4419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b490c9640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44197640 0x7f1b44197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b488c8640 1 -- 172.21.15.100:0/1312635484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44197640 msgr2=0x7f1b44197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.557+0000 7f1b488c8640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44197640 0x7f1b44197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b488c8640 1 -- 172.21.15.100:0/1312635484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44105330 msgr2=0x7f1b4419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b488c8640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44105330 0x7f1b4419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b488c8640 1 -- 172.21.15.100:0/1312635484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b30009560 con 0x7f1b44100260 2024-03-12T05:29:44.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b43fff640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44105330 0x7f1b4419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b488c8640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 0x7f1b4419d580 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e9f0 tx=0x7f1b3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b41ffb640 1 -- 172.21.15.100:0/1312635484 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38009700 con 0x7f1b44100260 2024-03-12T05:29:44.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b44101ce0 con 0x7f1b44100260 2024-03-12T05:29:44.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b41ffb640 1 -- 172.21.15.100:0/1312635484 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b3800cd80 con 0x7f1b44100260 2024-03-12T05:29:44.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b41ffb640 1 -- 172.21.15.100:0/1312635484 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38018920 con 0x7f1b44100260 2024-03-12T05:29:44.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.558+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b44102210 con 0x7f1b44100260 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.561+0000 7f1b41ffb640 1 -- 172.21.15.100:0/1312635484 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1b38018a80 con 0x7f1b44100260 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.561+0000 7f1b4d7dc740 1 Processor -- start 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 -- start start 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b41ffb640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1b340a1ba0 0x7f1b340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac6fb70 0x55b4cac78210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b41ffb640 1 -- 172.21.15.100:0/1312635484 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1b38014070 con 0x7f1b44100260 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b490c9640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1b340a1ba0 0x7f1b340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac78750 0x55b4cac7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cac2a960 con 0x55b4cac6f250 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cabde310 con 0x55b4cac6fb70 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cabdb840 con 0x55b4cac78750 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac6f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42420/0 (socket says 172.21.15.100:42420) 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 -- 172.21.15.100:0/3509477858 learned_addr learned my addr 172.21.15.100:0/3509477858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b43fff640 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac6fb70 0x55b4cac78210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b490c9640 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac78750 0x55b4cac7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 -- 172.21.15.100:0/3509477858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac78750 msgr2=0x55b4cac7ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac78750 0x55b4cac7ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 -- 172.21.15.100:0/3509477858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac6fb70 msgr2=0x55b4cac78210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac6fb70 0x55b4cac78210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 -- 172.21.15.100:0/3509477858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4cab99060 con 0x55b4cac6f250 2024-03-12T05:29:44.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b490c9640 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac78750 0x55b4cac7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b490c9640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1b340a1ba0 0x7f1b340a4060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c005fd0 tx=0x7f1b3c011810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.562+0000 7f1b488c8640 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac6f630 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1b38097230 tx=0x7f1b3809a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5c7337ffb1b741 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.563+0000 7f1b2effd640 1 -- 172.21.15.100:0/3509477858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38010b80 con 0x55b4cac6f250 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.563+0000 7f1b2effd640 1 -- 172.21.15.100:0/3509477858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b380cb550 con 0x55b4cac6f250 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.563+0000 7f1b2effd640 1 -- 172.21.15.100:0/3509477858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b380ba3b0 con 0x55b4cac6f250 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3509477858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 msgr2=0x55b4cac6f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac6f630 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1b38097230 tx=0x7f1b3809a280 comp rx=0 tx=0).stop 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3509477858 shutdown_connections 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac78750 0x55b4cac7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac6fb70 0x55b4cac78210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3509477858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac6f630 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3509477858 >> 172.21.15.100:0/3509477858 conn(0x55b4cac69980 msgr2=0x55b4cac69d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3509477858 shutdown_connections 2024-03-12T05:29:44.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3509477858 wait complete. 2024-03-12T05:29:44.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 -- start start 2024-03-12T05:29:44.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.564+0000 7f1b4d7dc740 1 Processor -- start 2024-03-12T05:29:44.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6fb70 0x55b4cac75d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac76240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cabd7020 con 0x55b4cac6f250 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4ca8aa430 con 0x55b4cac78750 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4ca866e00 con 0x55b4cac6fb70 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b4d7dc740 10 client.0 _ll_register_callbacks cb 0x55b4cab7ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.565+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac76240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42434/0 (socket says 172.21.15.100:42434) 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac76240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41920/0 (socket says 172.21.15.100:41920) 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b488c8640 1 -- 172.21.15.100:0/3453161832 learned_addr learned my addr 172.21.15.100:0/3453161832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b43fff640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6fb70 0x55b4cac75d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 -- 172.21.15.100:0/3453161832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6fb70 msgr2=0x55b4cac75d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6fb70 0x55b4cac75d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 -- 172.21.15.100:0/3453161832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 msgr2=0x55b4cac757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 -- 172.21.15.100:0/3453161832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b44102210 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b43fff640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6fb70 0x55b4cac75d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b488c8640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b490c9640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac76240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009fd0 tx=0x7f1b3c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74eefdec1906ea68 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b23fff640 1 -- 172.21.15.100:0/3453161832 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00f710 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.566+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4ca8d67e0 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b23fff640 1 -- 172.21.15.100:0/3453161832 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1b3c00f870 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b23fff640 1 -- 172.21.15.100:0/3453161832 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b3c00fa70 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b4d7dc740 10 client.28565 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b4cabd56a0 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b23fff640 1 -- 172.21.15.100:0/3453161832 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b4ca8d67e0 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b23fff640 1 -- 172.21.15.100:0/3453161832 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1b3c01d050 con 0x55b4cac78750 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b4d7dc740 10 client.28565 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:44.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b4d7dc740 10 client.28565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.567+0000 7f1b4d7dc740 20 client.28565 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.568+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cac88a60 0x55b4cac8ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.568+0000 7f1b4d7dc740 4 client.28565 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.568+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4ca913720 con 0x55b4cac88a60 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.568+0000 7f1b488c8640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cac88a60 0x55b4cac8ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.568+0000 7f1b23fff640 10 client.28565 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.568+0000 7f1b488c8640 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cac88a60 0x55b4cac8ae50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af44082a79fc74c server_cookie=9f546ed5c321fad6 in_seq=0 out_seq=0 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b23fff640 1 -- 172.21.15.100:0/3453161832 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4ca913720 con 0x55b4cac88a60 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b23fff640 10 client.28565 handle_command_reply: tid=0 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 client.28565 shutdown 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 msgr2=0x55b4cac76240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac76240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009fd0 tx=0x7f1b3c004ab0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 shutdown_connections 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cac88a60 0x55b4cac8ae50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac76240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b23fff640 0 client.28565 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6fb70 0x55b4cac75d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/3453161832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6f250 0x55b4cac757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 >> 172.21.15.100:0/3453161832 conn(0x55b4cac69980 msgr2=0x55b4cac6b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 shutdown_connections 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/3453161832 wait complete. 2024-03-12T05:29:44.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.569+0000 7f1b4d7dc740 20 client.28565 trim_cache size 0 max 16384 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.582+0000 7f1b4d7dc740 1 Processor -- start 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.582+0000 7f1b4d7dc740 1 -- start start 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac77ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cab5c130 con 0x55b4cac6fb70 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cac6e450 con 0x55b4cac78750 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cac855f0 con 0x55b4cac6f250 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac74090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42450/0 (socket says 172.21.15.100:42450) 2024-03-12T05:29:44.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 -- 172.21.15.100:0/2895436440 learned_addr learned my addr 172.21.15.100:0/2895436440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b490c9640 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b488c8640 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac77ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 -- 172.21.15.100:0/2895436440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 msgr2=0x55b4cac77ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac77ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 -- 172.21.15.100:0/2895436440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 msgr2=0x55b4cac745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 -- 172.21.15.100:0/2895436440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4cab71c20 con 0x55b4cac6fb70 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.583+0000 7f1b43fff640 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac74090 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005ab0 tx=0x7f1b30004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c20b3113d40975 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b2e7fc640 1 -- 172.21.15.100:0/2895436440 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3002c920 con 0x55b4cac6fb70 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b2e7fc640 1 -- 172.21.15.100:0/2895436440 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b30004510 con 0x55b4cac6fb70 2024-03-12T05:29:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b2e7fc640 1 -- 172.21.15.100:0/2895436440 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b300053f0 con 0x55b4cac6fb70 2024-03-12T05:29:44.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/2895436440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 msgr2=0x55b4cac74090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac74090 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005ab0 tx=0x7f1b30004b40 comp rx=0 tx=0).stop 2024-03-12T05:29:44.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/2895436440 shutdown_connections 2024-03-12T05:29:44.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac74090 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/2895436440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac77ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/2895436440 >> 172.21.15.100:0/2895436440 conn(0x55b4cac69980 msgr2=0x55b4cac70ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/2895436440 shutdown_connections 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.584+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/2895436440 wait complete. 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 -- start start 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 Processor -- start 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac7f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac7dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac7e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cac7e940 con 0x55b4cac6fb70 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cac7eab0 con 0x55b4cac78750 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4cacdef60 con 0x55b4cac6f250 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b4d7dc740 10 client.0 _ll_register_callbacks cb 0x7f1b380963e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac7dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac7dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42454/0 (socket says 172.21.15.100:42454) 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b488c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac7f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b43fff640 1 -- 172.21.15.100:0/1257108531 learned_addr learned my addr 172.21.15.100:0/1257108531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.585+0000 7f1b490c9640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac7e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b43fff640 1 -- 172.21.15.100:0/1257108531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 msgr2=0x55b4cac7f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b43fff640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac7f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b43fff640 1 -- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 msgr2=0x55b4cac7e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b43fff640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac7e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b43fff640 1 -- 172.21.15.100:0/1257108531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4410d710 con 0x55b4cac6fb70 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b488c8640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac7f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b43fff640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac7dce0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1b3004a040 tx=0x7f1b300050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e55372b5fe0266 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b23fff640 1 -- 172.21.15.100:0/1257108531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b300054a0 con 0x55b4cac6fb70 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b23fff640 1 -- 172.21.15.100:0/1257108531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1b30005600 con 0x55b4cac6fb70 2024-03-12T05:29:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b380cb550 con 0x55b4cac6fb70 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b23fff640 1 -- 172.21.15.100:0/1257108531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b30051460 con 0x55b4cac6fb70 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 10 client.19227 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b4cab978c0 con 0x55b4cac6fb70 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b23fff640 1 -- 172.21.15.100:0/1257108531 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b380cb550 con 0x55b4cac6fb70 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b23fff640 1 -- 172.21.15.100:0/1257108531 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1b3003c040 con 0x55b4cac6fb70 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 10 client.19227 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 20 client.19227 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cace03d0 0x55b4cace27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 4 client.19227 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.586+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4ca962890 con 0x55b4cace03d0 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b488c8640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cace03d0 0x55b4cace27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b23fff640 10 client.19227 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b488c8640 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cace03d0 0x55b4cace27f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=575b976b1ef6e622 server_cookie=16a2029572279895 in_seq=0 out_seq=0 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b23fff640 1 -- 172.21.15.100:0/1257108531 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4ca962890 con 0x55b4cace03d0 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b23fff640 10 client.19227 handle_command_reply: tid=0 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 client.19227 shutdown 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 msgr2=0x55b4cac7dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac7dce0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1b3004a040 tx=0x7f1b300050a0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 shutdown_connections 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55b4cace03d0 0x55b4cace27f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4cac78750 0x55b4cac7e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4cac6fb70 0x55b4cac7dce0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b23fff640 0 client.19227 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 --2- 172.21.15.100:0/1257108531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cac6f250 0x55b4cac7f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.587+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 >> 172.21.15.100:0/1257108531 conn(0x55b4cac69980 msgr2=0x55b4cac74010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 shutdown_connections 2024-03-12T05:29:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4d7dc740 1 -- 172.21.15.100:0/1257108531 wait complete. 2024-03-12T05:29:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4d7dc740 20 client.19227 trim_cache size 0 max 16384 2024-03-12T05:29:44.589 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1b340a1ba0 msgr2=0x7f1b340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1b340a1ba0 0x7f1b340a4060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c005fd0 tx=0x7f1b3c011810 comp rx=0 tx=0).stop 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 msgr2=0x7f1b4419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 0x7f1b4419d580 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e9f0 tx=0x7f1b3800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 shutdown_connections 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1b340a1ba0 0x7f1b340a4060 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44197640 0x7f1b44197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b44105330 0x7f1b4419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 --2- 172.21.15.100:0/1312635484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b44100260 0x7f1b4419d580 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 >> 172.21.15.100:0/1312635484 conn(0x7f1b44077180 msgr2=0x7f1b440ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 shutdown_connections 2024-03-12T05:29:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:44.588+0000 7f1b4ab2c640 1 -- 172.21.15.100:0/1312635484 wait complete. 2024-03-12T05:29:45.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 154269 29.16 MB/sec execute 148 sec latency 198.596 ms 2024-03-12T05:29:45.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:45 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1902531884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:29:45.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:45 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1902531884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:29:45.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:45 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1902531884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:29:45.594 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:29:45.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:29:45.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:29:45.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2141221929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a341023f0 msgr2=0x7f1a341027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2141221929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a341023f0 0x7f1a341027d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009860 tx=0x7f1a3002bc00 comp rx=0 tx=0).stop 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2141221929 shutdown_connections 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2141221929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a3410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2141221929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a34102d10 0x7f1a3410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2141221929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a341023f0 0x7f1a341027d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2141221929 >> 172.21.15.100:0/2141221929 conn(0x7f1a34077210 msgr2=0x7f1a34077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2141221929 shutdown_connections 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.725+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2141221929 wait complete. 2024-03-12T05:29:45.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 Processor -- start 2024-03-12T05:29:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 -- start start 2024-03-12T05:29:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 0x7f1a34100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34102d10 0x7f1a34100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a34100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a341a5c00 con 0x7f1a3410b7c0 2024-03-12T05:29:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a341a5d70 con 0x7f1a34102d10 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3c4c3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a341a5f30 con 0x7f1a341023f0 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3aa60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a34100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3a25f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 0x7f1a34100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a39a5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34102d10 0x7f1a34100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3aa60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a34100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42466/0 (socket says 172.21.15.100:42466) 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3aa60640 1 -- 172.21.15.100:0/2298484581 learned_addr learned my addr 172.21.15.100:0/2298484581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.726+0000 7f1a3a25f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 0x7f1a34100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:60924/0 (socket says 172.21.15.100:60924) 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3aa60640 1 -- 172.21.15.100:0/2298484581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 msgr2=0x7f1a34100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 0x7f1a34100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3aa60640 1 -- 172.21.15.100:0/2298484581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34102d10 msgr2=0x7f1a34100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34102d10 0x7f1a34100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3aa60640 1 -- 172.21.15.100:0/2298484581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c009900 con 0x7f1a3410b7c0 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 0x7f1a34100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:45.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a34100ac0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00efc0 tx=0x7f1a2c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a2b7fe640 1 -- 172.21.15.100:0/2298484581 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c003970 con 0x7f1a3410b7c0 2024-03-12T05:29:45.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a30009490 con 0x7f1a3410b7c0 2024-03-12T05:29:45.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a2b7fe640 1 -- 172.21.15.100:0/2298484581 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a2c004020 con 0x7f1a3410b7c0 2024-03-12T05:29:45.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a2b7fe640 1 -- 172.21.15.100:0/2298484581 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00b9b0 con 0x7f1a3410b7c0 2024-03-12T05:29:45.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.727+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a34197960 con 0x7f1a3410b7c0 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.730+0000 7f1a2b7fe640 1 -- 172.21.15.100:0/2298484581 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1a2c00a040 con 0x7f1a3410b7c0 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.730+0000 7f1a2b7fe640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a200a1c20 0x7f1a200a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a2b7fe640 1 -- 172.21.15.100:0/2298484581 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1a2c0cd840 con 0x7f1a3410b7c0 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a200a1c20 0x7f1a200a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 Processor -- start 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 -- start start 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a200a1c20 0x7f1a200a40e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a3002c210 tx=0x7f1a3002e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6ed8b0 0x5620eb6efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb652f90 con 0x5620eb6e4d30 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6504c0 con 0x5620eb6e4380 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.731+0000 7f1a3f171740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb64bca0 con 0x5620eb6ed8b0 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:41962/0 (socket says 172.21.15.100:41962) 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 -- 172.21.15.100:0/3787840311 learned_addr learned my addr 172.21.15.100:0/3787840311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6ed8b0 0x5620eb6efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 -- 172.21.15.100:0/3787840311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6ed8b0 msgr2=0x5620eb6efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6ed8b0 0x5620eb6efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 -- 172.21.15.100:0/3787840311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 msgr2=0x5620eb6ed2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6ed2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 -- 172.21.15.100:0/3787840311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620eb60e0b0 con 0x5620eb6e4380 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6e4760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009860 tx=0x7f1a30046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f933044124c463 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a28bfe640 1 -- 172.21.15.100:0/3787840311 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30046070 con 0x5620eb6e4380 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a28bfe640 1 -- 172.21.15.100:0/3787840311 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a30034920 con 0x5620eb6e4380 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.732+0000 7f1a28bfe640 1 -- 172.21.15.100:0/3787840311 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30003850 con 0x5620eb6e4380 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 -- 172.21.15.100:0/3787840311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 msgr2=0x5620eb6e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6e4760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009860 tx=0x7f1a30046000 comp rx=0 tx=0).stop 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 -- 172.21.15.100:0/3787840311 shutdown_connections 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6ed8b0 0x5620eb6efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 --2- 172.21.15.100:0/3787840311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6e4760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 -- 172.21.15.100:0/3787840311 >> 172.21.15.100:0/3787840311 conn(0x5620eb6de840 msgr2=0x5620eb6dec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 -- 172.21.15.100:0/3787840311 shutdown_connections 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.733+0000 7f1a3f171740 1 -- 172.21.15.100:0/3787840311 wait complete. 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 -- start start 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 Processor -- start 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb6ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6ed8b0 0x5620eb6eb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb5f0f40 con 0x5620eb6e4d30 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb31f4b0 con 0x5620eb6ed8b0 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb5d1670 con 0x5620eb6e4380 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3f171740 10 client.0 _ll_register_callbacks cb 0x5620eb63ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a39a5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3a25f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb6ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a39a5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6eadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42490/0 (socket says 172.21.15.100:42490) 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a39a5e640 1 -- 172.21.15.100:0/1980515769 learned_addr learned my addr 172.21.15.100:0/1980515769 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6ed8b0 0x5620eb6eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3a25f640 1 -- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6ed8b0 msgr2=0x5620eb6eb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6ed8b0 0x5620eb6eb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3a25f640 1 -- 172.21.15.100:0/1980515769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 msgr2=0x5620eb6eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.734+0000 7f1a3a25f640 1 -- 172.21.15.100:0/1980515769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3410e8b0 con 0x5620eb6e4380 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb6ea880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009830 tx=0x7f1a3002c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276b6d15dde03472 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1980515769 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30003b10 con 0x5620eb6e4380 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620eb625f40 con 0x5620eb6e4380 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1980515769 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1a30003c70 con 0x5620eb6e4380 2024-03-12T05:29:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1980515769 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a30003e30 con 0x5620eb6e4380 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 10 client.28519 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5620eb64a320 con 0x5620eb6e4380 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1980515769 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a3006d020 con 0x5620eb6e4380 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1980515769 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1a30061940 con 0x5620eb6e4380 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 10 client.28519 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 10 client.28519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 20 client.28519 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb6fdac0 0x5620eb6ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 4 client.28519 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.735+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620eb642310 con 0x5620eb6fdac0 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.736+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb6fdac0 0x5620eb6ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.736+0000 7f1a1dffb640 10 client.28519 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.736+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb6fdac0 0x5620eb6ffeb0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd876e07d6775a09 server_cookie=7364f54304b1ecfa in_seq=0 out_seq=0 2024-03-12T05:29:45.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1980515769 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620eb642310 con 0x5620eb6fdac0 2024-03-12T05:29:45.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a1dffb640 10 client.28519 handle_command_reply: tid=0 2024-03-12T05:29:45.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 client.28519 shutdown 2024-03-12T05:29:45.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 msgr2=0x5620eb6ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb6ea880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009830 tx=0x7f1a3002c910 comp rx=0 tx=0).stop 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 shutdown_connections 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb6fdac0 0x5620eb6ffeb0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6ed8b0 0x5620eb6eb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6e4d30 0x5620eb6eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1980515769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb6ea880 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a1dffb640 0 client.28519 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 >> 172.21.15.100:0/1980515769 conn(0x5620eb6de840 msgr2=0x5620eb6e07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 shutdown_connections 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 1 -- 172.21.15.100:0/1980515769 wait complete. 2024-03-12T05:29:45.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.737+0000 7f1a3f171740 20 client.28519 trim_cache size 0 max 16384 2024-03-12T05:29:45.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 Processor -- start 2024-03-12T05:29:45.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 -- start start 2024-03-12T05:29:45.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4d30 0x5620eb6eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6e36d0 con 0x5620eb6ed8b0 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6da110 con 0x5620eb6e4380 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.755+0000 7f1a3f171740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6e8dd0 con 0x5620eb6e4d30 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a3aa60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a3aa60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6e7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42500/0 (socket says 172.21.15.100:42500) 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a3aa60640 1 -- 172.21.15.100:0/1032452234 learned_addr learned my addr 172.21.15.100:0/1032452234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4d30 0x5620eb6eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 -- 172.21.15.100:0/1032452234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 msgr2=0x5620eb6ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6ecfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 -- 172.21.15.100:0/1032452234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 msgr2=0x5620eb6e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 -- 172.21.15.100:0/1032452234 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620eb5e6c00 con 0x5620eb6e4d30 2024-03-12T05:29:45.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:45.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4d30 0x5620eb6eb120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5620eb6ed6a0 tx=0x7f1a24005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1654893836a7ec64 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a1ffff640 1 -- 172.21.15.100:0/1032452234 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017070 con 0x5620eb6e4d30 2024-03-12T05:29:45.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a1ffff640 1 -- 172.21.15.100:0/1032452234 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a24018040 con 0x5620eb6e4d30 2024-03-12T05:29:45.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.756+0000 7f1a1ffff640 1 -- 172.21.15.100:0/1032452234 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2401d380 con 0x5620eb6e4d30 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 -- 172.21.15.100:0/1032452234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4d30 msgr2=0x5620eb6eb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4d30 0x5620eb6eb120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5620eb6ed6a0 tx=0x7f1a24005f90 comp rx=0 tx=0).stop 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 -- 172.21.15.100:0/1032452234 shutdown_connections 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4d30 0x5620eb6eb120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1032452234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4380 0x5620eb6ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 -- 172.21.15.100:0/1032452234 >> 172.21.15.100:0/1032452234 conn(0x5620eb6de840 msgr2=0x5620eb704820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.757+0000 7f1a3f171740 1 -- 172.21.15.100:0/1032452234 shutdown_connections 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 -- 172.21.15.100:0/1032452234 wait complete. 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 -- start start 2024-03-12T05:29:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 Processor -- start 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb74e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4d30 0x5620eb74ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6f4770 con 0x5620eb6ed8b0 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6f48e0 con 0x5620eb6e4d30 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620eb6f4aa0 con 0x5620eb6e4380 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3f171740 10 client.0 _ll_register_callbacks cb 0x7f1a30037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3aa60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3aa60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42510/0 (socket says 172.21.15.100:42510) 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3a25f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb74e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a3aa60640 1 -- 172.21.15.100:0/1360032726 learned_addr learned my addr 172.21.15.100:0/1360032726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.758+0000 7f1a39a5e640 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4d30 0x5620eb74ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3aa60640 1 -- 172.21.15.100:0/1360032726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 msgr2=0x5620eb74e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb74e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3aa60640 1 -- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4d30 msgr2=0x5620eb74ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4d30 0x5620eb74ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3aa60640 1 -- 172.21.15.100:0/1360032726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a34197960 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3aa60640 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6f41b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c096830 tx=0x7f1a2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f928d527ed8f4a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1360032726 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0b2070 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1360032726 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1a2c09e520 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620eb671900 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1360032726 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a2c0a2b70 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 10 client.19245 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1a24010430 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1360032726 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a2c07d020 con 0x5620eb6ed8b0 2024-03-12T05:29:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1360032726 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1a2c0a2770 con 0x5620eb6ed8b0 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 10 client.19245 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 10 client.19245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 20 client.19245 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.759+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb756a40 0x5620eb758e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.760+0000 7f1a3f171740 4 client.19245 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.760+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5620eb34b860 con 0x5620eb756a40 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.760+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb756a40 0x5620eb758e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.760+0000 7f1a1dffb640 10 client.19245 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.760+0000 7f1a3a25f640 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb756a40 0x5620eb758e60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1b3692c0cb7070 server_cookie=e13e68010b464c78 in_seq=0 out_seq=0 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a1dffb640 1 -- 172.21.15.100:0/1360032726 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5620eb34b860 con 0x5620eb756a40 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a1dffb640 10 client.19245 handle_command_reply: tid=0 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 client.19245 shutdown 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 msgr2=0x5620eb6f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6f41b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c096830 tx=0x7f1a2c009780 comp rx=0 tx=0).stop 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 shutdown_connections 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620eb756a40 0x5620eb758e60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620eb6ed8b0 0x5620eb6f41b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a1dffb640 0 client.19245 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620eb6e4d30 0x5620eb74ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 --2- 172.21.15.100:0/1360032726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620eb6e4380 0x5620eb74e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.761+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 >> 172.21.15.100:0/1360032726 conn(0x5620eb6de840 msgr2=0x5620eb6dd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 shutdown_connections 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3f171740 1 -- 172.21.15.100:0/1360032726 wait complete. 2024-03-12T05:29:45.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3f171740 20 client.19245 trim_cache size 0 max 16384 2024-03-12T05:29:45.761 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:29:45.761 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:29:45.761 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "189a46a5-d405-4665-8582-f9c7738c8777", 2024-03-12T05:29:45.761 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:29:45.761 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a200a1c20 msgr2=0x7f1a200a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a200a1c20 0x7f1a200a40e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a3002c210 tx=0x7f1a3002e040 comp rx=0 tx=0).stop 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 msgr2=0x7f1a34100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a34100ac0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00efc0 tx=0x7f1a2c00c800 comp rx=0 tx=0).stop 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 shutdown_connections 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1a200a1c20 0x7f1a200a40e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3410b7c0 0x7f1a34100ac0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34102d10 0x7f1a34100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 --2- 172.21.15.100:0/2298484581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a341023f0 0x7f1a34100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 >> 172.21.15.100:0/2298484581 conn(0x7f1a34077210 msgr2=0x7f1a340fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 shutdown_connections 2024-03-12T05:29:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.762+0000 7f1a3c4c3640 1 -- 172.21.15.100:0/2298484581 wait complete. 2024-03-12T05:29:45.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:29:45.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:29:45.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 -- 172.21.15.100:0/1507709175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81002d0 msgr2=0x7f35a8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 --2- 172.21.15.100:0/1507709175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81002d0 0x7f35a8100750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3598009930 tx=0x7f359802bc30 comp rx=0 tx=0).stop 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 -- 172.21.15.100:0/1507709175 shutdown_connections 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 --2- 172.21.15.100:0/1507709175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 --2- 172.21.15.100:0/1507709175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81002d0 0x7f35a8100750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 --2- 172.21.15.100:0/1507709175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a8106210 0x7f35a81065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 -- 172.21.15.100:0/1507709175 >> 172.21.15.100:0/1507709175 conn(0x7f35a8076e40 msgr2=0x7f35a8077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 -- 172.21.15.100:0/1507709175 shutdown_connections 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.898+0000 7f35b0615640 1 -- 172.21.15.100:0/1507709175 wait complete. 2024-03-12T05:29:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 Processor -- start 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 -- start start 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a81002d0 0x7f35a810f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8106210 0x7f35a81100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81a5bb0 con 0x7f35a8100c90 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81a5d20 con 0x7f35a81002d0 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35b0615640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81a5ee0 con 0x7f35a8106210 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42520/0 (socket says 172.21.15.100:42520) 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35adbb0640 1 -- 172.21.15.100:0/2998380055 learned_addr learned my addr 172.21.15.100:0/2998380055 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a81002d0 0x7f35a810f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.899+0000 7f35aebb2640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8106210 0x7f35a81100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35adbb0640 1 -- 172.21.15.100:0/2998380055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8106210 msgr2=0x7f35a81100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35adbb0640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8106210 0x7f35a81100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35adbb0640 1 -- 172.21.15.100:0/2998380055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a81002d0 msgr2=0x7f35a810f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35adbb0640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a81002d0 0x7f35a810f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35adbb0640 1 -- 172.21.15.100:0/2998380055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009560 con 0x7f35a8100c90 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35adbb0640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810fb80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3598009230 tx=0x7f359802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f359f7fe640 1 -- 172.21.15.100:0/2998380055 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598004350 con 0x7f35a8100c90 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f359f7fe640 1 -- 172.21.15.100:0/2998380055 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35980044b0 con 0x7f35a8100c90 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a806a480 con 0x7f35a8100c90 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f359f7fe640 1 -- 172.21.15.100:0/2998380055 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598003d00 con 0x7f35a8100c90 2024-03-12T05:29:45.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.900+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a81a64e0 con 0x7f35a8100c90 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f359f7fe640 1 -- 172.21.15.100:0/2998380055 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3598004df0 con 0x7f35a8100c90 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f35b31e1740 1 Processor -- start 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f35b31e1740 1 -- start start 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd558f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd558fbd0 0x55abd5598270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 0x55abd559aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f35b31e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd54fe280 con 0x55abd55987b0 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.903+0000 7f359f7fe640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f35940a1ba0 0x7f35940a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35b31e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd54fb7b0 con 0x55abd558f2b0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35b31e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd54f6f90 con 0x55abd558fbd0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 0x55abd559aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35aebb2640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f35940a1ba0 0x7f35940a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 0x55abd559aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42524/0 (socket says 172.21.15.100:42524) 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35aebb2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd558f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 -- 172.21.15.100:0/3459084956 learned_addr learned my addr 172.21.15.100:0/3459084956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd558fbd0 0x55abd5598270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f359f7fe640 1 -- 172.21.15.100:0/2998380055 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35980ef700 con 0x7f35a8100c90 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 -- 172.21.15.100:0/3459084956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd558fbd0 msgr2=0x55abd5598270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd558fbd0 0x55abd5598270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 -- 172.21.15.100:0/3459084956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 msgr2=0x55abd558f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd558f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 -- 172.21.15.100:0/3459084956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abd54b5570 con 0x55abd55987b0 2024-03-12T05:29:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35aebb2640 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd558f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35adbb0640 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 0x55abd559aba0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f35980ed410 tx=0x7f359802c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163f0c33317839dc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f35aebb2640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f35940a1ba0 0x7f35940a4060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005fd0 tx=0x7f35a0004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f359cbfe640 1 -- 172.21.15.100:0/3459084956 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980bbd60 con 0x55abd55987b0 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f359cbfe640 1 -- 172.21.15.100:0/3459084956 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3598085d00 con 0x55abd55987b0 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.904+0000 7f359cbfe640 1 -- 172.21.15.100:0/3459084956 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980c4ba0 con 0x55abd55987b0 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 -- 172.21.15.100:0/3459084956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 msgr2=0x55abd559aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 0x55abd559aba0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f35980ed410 tx=0x7f359802c740 comp rx=0 tx=0).stop 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 -- 172.21.15.100:0/3459084956 shutdown_connections 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd55987b0 0x55abd559aba0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd558fbd0 0x55abd5598270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3459084956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd558f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 -- 172.21.15.100:0/3459084956 >> 172.21.15.100:0/3459084956 conn(0x55abd5589990 msgr2=0x55abd5589d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 -- 172.21.15.100:0/3459084956 shutdown_connections 2024-03-12T05:29:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.905+0000 7f35b31e1740 1 -- 172.21.15.100:0/3459084956 wait complete. 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 -- start start 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 Processor -- start 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd5595560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd5595560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd5595560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42532/0 (socket says 172.21.15.100:42532) 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35ae3b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5595b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35adbb0640 1 -- 172.21.15.100:0/1827607315 learned_addr learned my addr 172.21.15.100:0/1827607315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd51ca430 con 0x55abd558fbd0 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd5186e00 con 0x55abd558f2b0 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd547bfd0 con 0x55abd55987b0 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35b31e1740 10 client.0 _ll_register_callbacks cb 0x55abd54d1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35aebb2640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5595b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35ae3b1640 1 -- 172.21.15.100:0/1827607315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 msgr2=0x55abd5595b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5595b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.906+0000 7f35ae3b1640 1 -- 172.21.15.100:0/1827607315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 msgr2=0x55abd5595560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd5595560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35ae3b1640 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a81a64e0 con 0x55abd558f2b0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35aebb2640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5595b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35adbb0640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd5595560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55981d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35a40126d0 tx=0x7f35a4012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5073165ea5b82bef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f3591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f3591ffb640 1 -- 172.21.15.100:0/1827607315 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4017070 con 0x55abd558f2b0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abd51f67e0 con 0x55abd558f2b0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f3591ffb640 1 -- 172.21.15.100:0/1827607315 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35a400f920 con 0x55abd558f2b0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f3591ffb640 1 -- 172.21.15.100:0/1827607315 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a400fae0 con 0x55abd558f2b0 2024-03-12T05:29:45.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35b31e1740 10 client.28601 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55abd54f5610 con 0x55abd558f2b0 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f3591ffb640 1 -- 172.21.15.100:0/1827607315 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55abd51f67e0 con 0x55abd558f2b0 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.907+0000 7f3591ffb640 1 -- 172.21.15.100:0/1827607315 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35a4022070 con 0x55abd558f2b0 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35b31e1740 10 client.28601 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35b31e1740 10 client.28601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35b31e1740 20 client.28601 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35b31e1740 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd55b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35b31e1740 4 client.28601 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abd5233720 con 0x55abd55b0e80 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35adbb0640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd55b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f3591ffb640 10 client.28601 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.908+0000 7f35adbb0640 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd55b32a0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbccc43a91aee66 server_cookie=d2a210e451813819 in_seq=0 out_seq=0 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.909+0000 7f3591ffb640 1 -- 172.21.15.100:0/1827607315 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abd5233720 con 0x55abd55b0e80 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.909+0000 7f3591ffb640 10 client.28601 handle_command_reply: tid=0 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.909+0000 7f35b31e1740 1 client.28601 shutdown 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 msgr2=0x55abd55981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55981d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35a40126d0 tx=0x7f35a4012b90 comp rx=0 tx=0).stop 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 shutdown_connections 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd55b32a0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5595b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd5595560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 --2- 172.21.15.100:0/1827607315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55981d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f3591ffb640 0 client.28601 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 >> 172.21.15.100:0/1827607315 conn(0x55abd5589990 msgr2=0x55abd558b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 shutdown_connections 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 1 -- 172.21.15.100:0/1827607315 wait complete. 2024-03-12T05:29:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.910+0000 7f35b31e1740 20 client.28601 trim_cache size 0 max 16384 2024-03-12T05:29:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.927+0000 7f35b31e1740 1 Processor -- start 2024-03-12T05:29:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 -- start start 2024-03-12T05:29:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 0x55abd5597c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558fbd0 0x55abd55981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5594dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd558e3a0 con 0x55abd558f2b0 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd55897f0 con 0x55abd558fbd0 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35b31e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd55b0c80 con 0x55abd55987b0 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 0x55abd5597c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 0x55abd5597c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42536/0 (socket says 172.21.15.100:42536) 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 -- 172.21.15.100:0/3356393793 learned_addr learned my addr 172.21.15.100:0/3356393793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35adbb0640 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558fbd0 0x55abd55981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35aebb2640 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5594dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 -- 172.21.15.100:0/3356393793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 msgr2=0x55abd5594dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5594dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 -- 172.21.15.100:0/3356393793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558fbd0 msgr2=0x55abd55981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558fbd0 0x55abd55981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 -- 172.21.15.100:0/3356393793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abd5491ac0 con 0x55abd558f2b0 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.928+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 0x55abd5597c90 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f35a40098d0 tx=0x7f35a4009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e95f6f8d4efe2d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f3593fff640 1 -- 172.21.15.100:0/3356393793 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4017040 con 0x55abd558f2b0 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f3593fff640 1 -- 172.21.15.100:0/3356393793 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a400f920 con 0x55abd558f2b0 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f3593fff640 1 -- 172.21.15.100:0/3356393793 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4005440 con 0x55abd558f2b0 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 -- 172.21.15.100:0/3356393793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 msgr2=0x55abd5597c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 0x55abd5597c90 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f35a40098d0 tx=0x7f35a4009f20 comp rx=0 tx=0).stop 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 -- 172.21.15.100:0/3356393793 shutdown_connections 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5594dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558fbd0 0x55abd55981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 --2- 172.21.15.100:0/3356393793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558f2b0 0x55abd5597c90 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 -- 172.21.15.100:0/3356393793 >> 172.21.15.100:0/3356393793 conn(0x55abd5589990 msgr2=0x55abd5593e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 -- 172.21.15.100:0/3356393793 shutdown_connections 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.929+0000 7f35b31e1740 1 -- 172.21.15.100:0/3356393793 wait complete. 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 -- start start 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 Processor -- start 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd55a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5607380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd55a5820 con 0x55abd558fbd0 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd56078c0 con 0x55abd558f2b0 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abd5607a00 con 0x55abd55987b0 2024-03-12T05:29:45.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35b31e1740 10 client.0 _ll_register_callbacks cb 0x7f3598005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd55a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd55a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42546/0 (socket says 172.21.15.100:42546) 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 -- 172.21.15.100:0/656938432 learned_addr learned my addr 172.21.15.100:0/656938432 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35aebb2640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5607380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 -- 172.21.15.100:0/656938432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 msgr2=0x55abd5607380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5607380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 -- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 msgr2=0x55abd55a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 -- 172.21.15.100:0/656938432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abd54f5610 con 0x55abd558fbd0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f35adbb0640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd55a4f50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f35980ba6b0 tx=0x7f359802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdce2fc4800ebfe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.930+0000 7f3591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:29:45.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f3591ffb640 1 -- 172.21.15.100:0/656938432 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598085e70 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f3591ffb640 1 -- 172.21.15.100:0/656938432 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35980035f0 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3598085d00 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f3591ffb640 1 -- 172.21.15.100:0/656938432 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35980bc1d0 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 10 client.19263 fetch_fsmap learned FSMap version 30 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55abd54b76a0 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f3591ffb640 1 -- 172.21.15.100:0/656938432 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3598085d00 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f3591ffb640 1 -- 172.21.15.100:0/656938432 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f359803c070 con 0x55abd558fbd0 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 10 client.19263 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 10 client.19263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 20 client.19263 populate_metadata read hostname 'smithi100' 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd560acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 4 client.19263 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:29:45.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55abd5282890 con 0x55abd55b0e80 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.931+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd560acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35ae3b1640 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd560acd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11da90540131ce6 server_cookie=74cf84bd74eb1757 in_seq=0 out_seq=0 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f3591ffb640 10 client.19263 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f3591ffb640 1 -- 172.21.15.100:0/656938432 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55abd5282890 con 0x55abd55b0e80 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f3591ffb640 10 client.19263 handle_command_reply: tid=0 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 client.19263 shutdown 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 msgr2=0x55abd55a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd55a4f50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f35980ba6b0 tx=0x7f359802bc30 comp rx=0 tx=0).stop 2024-03-12T05:29:45.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 shutdown_connections 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55abd55b0e80 0x55abd560acd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f3591ffb640 0 client.19263 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abd55987b0 0x55abd5607380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd558fbd0 0x55abd55a4f50 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 --2- 172.21.15.100:0/656938432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abd558f2b0 0x55abd55a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 >> 172.21.15.100:0/656938432 conn(0x55abd5589990 msgr2=0x55abd55881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 shutdown_connections 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 1 -- 172.21.15.100:0/656938432 wait complete. 2024-03-12T05:29:45.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.932+0000 7f35b31e1740 20 client.19263 trim_cache size 0 max 16384 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: "189a46a5-d405-4665-8582-f9c7738c8777": { 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "189a46a5-d405-4665-8582-f9c7738c8777", 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:29:45.938 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f35940a1ba0 msgr2=0x7f35940a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f35940a1ba0 0x7f35940a4060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005fd0 tx=0x7f35a0004ab0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 msgr2=0x7f35a810fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810fb80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3598009230 tx=0x7f359802c650 comp rx=0 tx=0).stop 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 shutdown_connections 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f35940a1ba0 0x7f35940a4060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8106210 0x7f35a81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8100c90 0x7f35a810fb80 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 --2- 172.21.15.100:0/2998380055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a81002d0 0x7f35a810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 >> 172.21.15.100:0/2998380055 conn(0x7f35a8076e40 msgr2=0x7f35a80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 shutdown_connections 2024-03-12T05:29:45.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:29:45.933+0000 7f35b0615640 1 -- 172.21.15.100:0/2998380055 wait complete. 2024-03-12T05:29:45.941 INFO:tasks.cephfs.filesystem:scrub status for tag:189a46a5-d405-4665-8582-f9c7738c8777 - {'path': '/', 'tag': '189a46a5-d405-4665-8582-f9c7738c8777', 'options': 'recursive,force'} 2024-03-12T05:29:46.238 INFO:tasks.workunit.client.0.smithi100.stdout: 10 154645 29.10 MB/sec execute 149 sec latency 364.249 ms 2024-03-12T05:29:46.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:29:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:46 smithi100 ceph-mon[32210]: pgmap v555: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 442 op/s 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:46 smithi156 ceph-mon[36270]: pgmap v555: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 442 op/s 2024-03-12T05:29:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:29:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:46 smithi161 ceph-mon[36043]: pgmap v555: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 442 op/s 2024-03-12T05:29:47.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 155223 29.18 MB/sec execute 150 sec latency 190.589 ms 2024-03-12T05:29:47.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:47 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:47 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:29:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:47 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:47.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:47 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:47.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:47 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:47 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:29:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:47 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:47 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:47 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:47 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:29:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:47 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:47 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:29:48.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 155674 29.14 MB/sec execute 151 sec latency 181.270 ms 2024-03-12T05:29:48.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:48 smithi156 ceph-mon[36270]: pgmap v556: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-03-12T05:29:48.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:48 smithi161 ceph-mon[36043]: pgmap v556: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-03-12T05:29:48.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:48 smithi100 ceph-mon[32210]: pgmap v556: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-03-12T05:29:49.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 156230 29.14 MB/sec execute 152 sec latency 115.850 ms 2024-03-12T05:29:50.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 156659 29.10 MB/sec execute 153 sec latency 180.761 ms 2024-03-12T05:29:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:50 smithi156 ceph-mon[36270]: pgmap v557: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 494 op/s 2024-03-12T05:29:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:50 smithi161 ceph-mon[36043]: pgmap v557: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 494 op/s 2024-03-12T05:29:50.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:50 smithi100 ceph-mon[32210]: pgmap v557: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 494 op/s 2024-03-12T05:29:51.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 157007 29.01 MB/sec execute 154 sec latency 341.167 ms 2024-03-12T05:29:52.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 157555 29.03 MB/sec execute 155 sec latency 222.629 ms 2024-03-12T05:29:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:52 smithi156 ceph-mon[36270]: pgmap v558: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 651 op/s 2024-03-12T05:29:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:52 smithi161 ceph-mon[36043]: pgmap v558: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 651 op/s 2024-03-12T05:29:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:52 smithi100 ceph-mon[32210]: pgmap v558: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 651 op/s 2024-03-12T05:29:53.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 158047 29.01 MB/sec execute 156 sec latency 163.708 ms 2024-03-12T05:29:54.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 158427 28.97 MB/sec execute 157 sec latency 181.583 ms 2024-03-12T05:29:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:54 smithi156 ceph-mon[36270]: pgmap v559: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 525 op/s 2024-03-12T05:29:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:54 smithi161 ceph-mon[36043]: pgmap v559: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 525 op/s 2024-03-12T05:29:54.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:54 smithi100 ceph-mon[32210]: pgmap v559: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 525 op/s 2024-03-12T05:29:55.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 158891 28.90 MB/sec execute 158 sec latency 168.263 ms 2024-03-12T05:29:56.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 159331 28.84 MB/sec execute 159 sec latency 394.655 ms 2024-03-12T05:29:56.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:56 smithi100 ceph-mon[32210]: pgmap v560: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.6 MiB/s wr, 481 op/s 2024-03-12T05:29:56.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:56 smithi156 ceph-mon[36270]: pgmap v560: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.6 MiB/s wr, 481 op/s 2024-03-12T05:29:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:56 smithi161 ceph-mon[36043]: pgmap v560: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.6 MiB/s wr, 481 op/s 2024-03-12T05:29:57.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 159924 28.91 MB/sec execute 160 sec latency 134.422 ms 2024-03-12T05:29:58.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 160411 28.88 MB/sec execute 161 sec latency 132.198 ms 2024-03-12T05:29:58.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:29:58 smithi156 ceph-mon[36270]: pgmap v561: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 699 op/s 2024-03-12T05:29:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:29:58 smithi161 ceph-mon[36043]: pgmap v561: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 699 op/s 2024-03-12T05:29:58.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:29:58 smithi100 ceph-mon[32210]: pgmap v561: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 699 op/s 2024-03-12T05:29:59.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 160988 28.93 MB/sec execute 162 sec latency 158.334 ms 2024-03-12T05:30:00.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 161529 28.94 MB/sec execute 163 sec latency 171.350 ms 2024-03-12T05:30:00.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:00 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:30:00.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:00 smithi100 ceph-mon[32210]: pgmap v562: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 431 op/s 2024-03-12T05:30:00.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:00 smithi100 ceph-mon[32210]: overall HEALTH_OK 2024-03-12T05:30:00.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:00 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:30:00.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:00 smithi156 ceph-mon[36270]: pgmap v562: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 431 op/s 2024-03-12T05:30:00.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:00 smithi156 ceph-mon[36270]: overall HEALTH_OK 2024-03-12T05:30:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:00 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:30:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:00 smithi161 ceph-mon[36043]: pgmap v562: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 431 op/s 2024-03-12T05:30:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:00 smithi161 ceph-mon[36043]: overall HEALTH_OK 2024-03-12T05:30:01.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 161598 28.79 MB/sec execute 164 sec latency 868.190 ms 2024-03-12T05:30:01.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:01 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:30:01.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:01 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:30:01.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:01 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:30:01.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:01 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:30:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:01 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:30:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:01 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:30:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 161631 28.62 MB/sec execute 165 sec latency 960.870 ms 2024-03-12T05:30:02.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:02 smithi100 ceph-mon[32210]: pgmap v563: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 586 op/s 2024-03-12T05:30:02.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:02 smithi156 ceph-mon[36270]: pgmap v563: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 586 op/s 2024-03-12T05:30:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:02 smithi161 ceph-mon[36043]: pgmap v563: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 586 op/s 2024-03-12T05:30:03.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 161631 28.45 MB/sec execute 166 sec latency 1893.076 ms 2024-03-12T05:30:04.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 161920 28.38 MB/sec execute 167 sec latency 2353.575 ms 2024-03-12T05:30:04.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:04 smithi156 ceph-mon[36270]: pgmap v564: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.6 MiB/s wr, 455 op/s 2024-03-12T05:30:04.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:04 smithi161 ceph-mon[36043]: pgmap v564: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.6 MiB/s wr, 455 op/s 2024-03-12T05:30:04.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:04 smithi100 ceph-mon[32210]: pgmap v564: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.6 MiB/s wr, 455 op/s 2024-03-12T05:30:05.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 162205 28.27 MB/sec execute 168 sec latency 149.971 ms 2024-03-12T05:30:06.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 162470 28.17 MB/sec execute 169 sec latency 411.745 ms 2024-03-12T05:30:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:06 smithi156 ceph-mon[36270]: osdmap e112: 12 total, 12 up, 12 in 2024-03-12T05:30:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:06 smithi156 ceph-mon[36270]: pgmap v566: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.6 MiB/s wr, 479 op/s 2024-03-12T05:30:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:06 smithi161 ceph-mon[36043]: osdmap e112: 12 total, 12 up, 12 in 2024-03-12T05:30:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:06 smithi161 ceph-mon[36043]: pgmap v566: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.6 MiB/s wr, 479 op/s 2024-03-12T05:30:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:06 smithi100 ceph-mon[32210]: osdmap e112: 12 total, 12 up, 12 in 2024-03-12T05:30:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:06 smithi100 ceph-mon[32210]: pgmap v566: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.6 MiB/s wr, 479 op/s 2024-03-12T05:30:07.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 162858 28.14 MB/sec execute 170 sec latency 124.387 ms 2024-03-12T05:30:08.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 163303 28.07 MB/sec execute 171 sec latency 105.736 ms 2024-03-12T05:30:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:08 smithi156 ceph-mon[36270]: pgmap v567: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 560 op/s 2024-03-12T05:30:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:08 smithi161 ceph-mon[36043]: pgmap v567: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 560 op/s 2024-03-12T05:30:08.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:08 smithi100 ceph-mon[32210]: pgmap v567: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 560 op/s 2024-03-12T05:30:09.239 INFO:tasks.workunit.client.0.smithi100.stdout: 10 163947 28.14 MB/sec execute 172 sec latency 142.295 ms 2024-03-12T05:30:10.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 164568 28.22 MB/sec execute 173 sec latency 146.076 ms 2024-03-12T05:30:10.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:10 smithi100 ceph-mon[32210]: pgmap v568: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 560 op/s 2024-03-12T05:30:10.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:10 smithi156 ceph-mon[36270]: pgmap v568: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 560 op/s 2024-03-12T05:30:10.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:10 smithi161 ceph-mon[36043]: pgmap v568: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 560 op/s 2024-03-12T05:30:11.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 164880 28.15 MB/sec execute 174 sec latency 284.548 ms 2024-03-12T05:30:12.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:11 smithi156 ceph-mon[36270]: osdmap e113: 12 total, 12 up, 12 in 2024-03-12T05:30:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:11 smithi161 ceph-mon[36043]: osdmap e113: 12 total, 12 up, 12 in 2024-03-12T05:30:12.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 165347 28.09 MB/sec execute 175 sec latency 194.138 ms 2024-03-12T05:30:12.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:11 smithi100 ceph-mon[32210]: osdmap e113: 12 total, 12 up, 12 in 2024-03-12T05:30:13.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:12 smithi156 ceph-mon[36270]: pgmap v570: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-03-12T05:30:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:12 smithi161 ceph-mon[36043]: pgmap v570: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-03-12T05:30:13.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 165852 28.12 MB/sec execute 176 sec latency 204.379 ms 2024-03-12T05:30:13.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:12 smithi100 ceph-mon[32210]: pgmap v570: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-03-12T05:30:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 166286 28.06 MB/sec execute 177 sec latency 205.398 ms 2024-03-12T05:30:14.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:14 smithi156 ceph-mon[36270]: pgmap v571: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-12T05:30:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:14 smithi161 ceph-mon[36043]: pgmap v571: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-12T05:30:14.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:14 smithi100 ceph-mon[32210]: pgmap v571: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-12T05:30:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 166887 28.13 MB/sec execute 178 sec latency 138.068 ms 2024-03-12T05:30:15.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:30:15.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 -- 172.21.15.100:0/2877963647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 msgr2=0x7f621c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 --2- 172.21.15.100:0/2877963647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c10b1a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f6210009900 tx=0x7f621002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 -- 172.21.15.100:0/2877963647 shutdown_connections 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 --2- 172.21.15.100:0/2877963647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 0x7f621c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 --2- 172.21.15.100:0/2877963647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c10b1a0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 --2- 172.21.15.100:0/2877963647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 -- 172.21.15.100:0/2877963647 >> 172.21.15.100:0/2877963647 conn(0x7f621c077230 msgr2=0x7f621c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 -- 172.21.15.100:0/2877963647 shutdown_connections 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.135+0000 7f62212b5640 1 -- 172.21.15.100:0/2877963647 wait complete. 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.136+0000 7f62212b5640 1 Processor -- start 2024-03-12T05:30:16.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.136+0000 7f62212b5640 1 -- start start 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.136+0000 7f62212b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.136+0000 7f62212b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f62212b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 0x7f621c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f62212b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c19c5d0 con 0x7f621c102cc0 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f62212b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c19c740 con 0x7f621c1023a0 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f62212b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c19c900 con 0x7f621c10b6e0 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37802/0 (socket says 172.21.15.100:37802) 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621affd640 1 -- 172.21.15.100:0/2579073436 learned_addr learned my addr 172.21.15.100:0/2579073436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621b7fe640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 0x7f621c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 -- 172.21.15.100:0/2579073436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 msgr2=0x7f621c19bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 0x7f621c19bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 -- 172.21.15.100:0/2579073436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 msgr2=0x7f621c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 -- 172.21.15.100:0/2579073436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210009560 con 0x7f621c102cc0 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621b7fe640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 0x7f621c19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621affd640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f621a7fc640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c197c50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f621002c210 tx=0x7f621002c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.137+0000 7f6203fff640 1 -- 172.21.15.100:0/2579073436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621002db20 con 0x7f621c102cc0 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.138+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c19cb80 con 0x7f621c102cc0 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.138+0000 7f6203fff640 1 -- 172.21.15.100:0/2579073436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f621002dc80 con 0x7f621c102cc0 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.138+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c1a4460 con 0x7f621c102cc0 2024-03-12T05:30:16.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.138+0000 7f6203fff640 1 -- 172.21.15.100:0/2579073436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210003a80 con 0x7f621c102cc0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6203fff640 1 -- 172.21.15.100:0/2579073436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6210003be0 con 0x7f621c102cc0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 Processor -- start 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 -- start start 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4217690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4220270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4222ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf4186280 con 0x55dbf42207b0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf41837b0 con 0x55dbf42172b0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6223f66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf417ef90 con 0x55dbf4217bd0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f621b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4217690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f621affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4220270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f621affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4220270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42880/0 (socket says 172.21.15.100:42880) 2024-03-12T05:30:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f621b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4217690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37808/0 (socket says 172.21.15.100:37808) 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f621affd640 1 -- 172.21.15.100:0/4152912238 learned_addr learned my addr 172.21.15.100:0/4152912238 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.140+0000 7f6203fff640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62040a1ba0 0x7f62040a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621a7fc640 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4222ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621b7fe640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62040a1ba0 0x7f62040a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621affd640 1 -- 172.21.15.100:0/4152912238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 msgr2=0x55dbf4217690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621affd640 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4217690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621affd640 1 -- 172.21.15.100:0/4152912238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 msgr2=0x55dbf4222ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621affd640 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4222ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621affd640 1 -- 172.21.15.100:0/4152912238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbf413d570 con 0x55dbf4217bd0 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621b7fe640 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4217690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f6203fff640 1 -- 172.21.15.100:0/2579073436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f62100efb10 con 0x7f621c102cc0 2024-03-12T05:30:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621affd640 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4220270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f620c011650 tx=0x7f620c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd3c54e132f47f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f621b7fe640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62040a1ba0 0x7f62040a4060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6214005fd0 tx=0x7f6214004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f62017fa640 1 -- 172.21.15.100:0/4152912238 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f6c0 con 0x55dbf4217bd0 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f62017fa640 1 -- 172.21.15.100:0/4152912238 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f620c00fd00 con 0x55dbf4217bd0 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.141+0000 7f62017fa640 1 -- 172.21.15.100:0/4152912238 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f6c0 con 0x55dbf4217bd0 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 -- 172.21.15.100:0/4152912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 msgr2=0x55dbf4220270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4220270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f620c011650 tx=0x7f620c009f90 comp rx=0 tx=0).stop 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 -- 172.21.15.100:0/4152912238 shutdown_connections 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4222ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4220270 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 --2- 172.21.15.100:0/4152912238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4217690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 -- 172.21.15.100:0/4152912238 >> 172.21.15.100:0/4152912238 conn(0x55dbf4211990 msgr2=0x55dbf4211d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 -- 172.21.15.100:0/4152912238 shutdown_connections 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 -- 172.21.15.100:0/4152912238 wait complete. 2024-03-12T05:30:16.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 -- start start 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 Processor -- start 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 0x55dbf42200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.142+0000 7f6223f66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf4217bd0 0x55dbf421d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf421d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3e52430 con 0x55dbf42172b0 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3e0ee00 con 0x55dbf4217bd0 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf4103fd0 con 0x55dbf42207b0 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 10 client.0 _ll_register_callbacks cb 0x55dbf4159fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 0x55dbf42200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 0x55dbf42200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59984/0 (socket says 172.21.15.100:59984) 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 -- 172.21.15.100:0/660580320 learned_addr learned my addr 172.21.15.100:0/660580320 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621a7fc640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf4217bd0 0x55dbf421d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621b7fe640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf421d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 -- 172.21.15.100:0/660580320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 msgr2=0x55dbf421d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf421d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 -- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf4217bd0 msgr2=0x55dbf421d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf4217bd0 0x55dbf421d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 -- 172.21.15.100:0/660580320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c009e00 con 0x55dbf42172b0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f621affd640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 0x55dbf42200c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f620c019620 tx=0x7f620c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2ab6e4dff161ec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f61feff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f61feff5640 1 -- 172.21.15.100:0/660580320 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f6c0 con 0x55dbf42172b0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f61feff5640 1 -- 172.21.15.100:0/660580320 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f620c00fa30 con 0x55dbf42172b0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbf3e7e7e0 con 0x55dbf42172b0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f61feff5640 1 -- 172.21.15.100:0/660580320 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c0197a0 con 0x55dbf42172b0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 10 client.19275 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.143+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dbf417d610 con 0x55dbf42172b0 2024-03-12T05:30:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f61feff5640 1 -- 172.21.15.100:0/660580320 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbf3e7e7e0 con 0x55dbf42172b0 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f61feff5640 1 -- 172.21.15.100:0/660580320 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f620c030070 con 0x55dbf42172b0 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f6223f66740 10 client.19275 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f6223f66740 10 client.19275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f6223f66740 20 client.19275 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f6223f66740 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf4234c30 0x55dbf4237070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f6223f66740 4 client.19275 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbf3ebb720 con 0x55dbf4234c30 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f621a7fc640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf4234c30 0x55dbf4237070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f61feff5640 10 client.19275 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.144+0000 7f621a7fc640 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf4234c30 0x55dbf4237070 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a2bc374f5b3ae7 server_cookie=2e9bc8bc30c0ed73 in_seq=0 out_seq=0 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f61feff5640 1 -- 172.21.15.100:0/660580320 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbf3ebb720 con 0x55dbf4234c30 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f61feff5640 10 client.19275 handle_command_reply: tid=0 2024-03-12T05:30:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 client.19275 shutdown 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 msgr2=0x55dbf42200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 0x55dbf42200c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f620c019620 tx=0x7f620c0056d0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 shutdown_connections 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf4234c30 0x55dbf4237070 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf421d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf4217bd0 0x55dbf421d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 --2- 172.21.15.100:0/660580320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42172b0 0x55dbf42200c0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 >> 172.21.15.100:0/660580320 conn(0x55dbf4211990 msgr2=0x55dbf4213730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f61feff5640 0 client.19275 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 shutdown_connections 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 1 -- 172.21.15.100:0/660580320 wait complete. 2024-03-12T05:30:16.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.145+0000 7f6223f66740 20 client.19275 trim_cache size 0 max 16384 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 Processor -- start 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 -- start start 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf421fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 0x55dbf42200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf422bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf4212790 con 0x55dbf4217bd0 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.160+0000 7f6223f66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf421e560 con 0x55dbf42172b0 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f6223f66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf4216c80 con 0x55dbf42207b0 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 0x55dbf42200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 0x55dbf42200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59990/0 (socket says 172.21.15.100:59990) 2024-03-12T05:30:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 -- 172.21.15.100:0/515526855 learned_addr learned my addr 172.21.15.100:0/515526855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621affd640 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf421fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621b7fe640 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf422bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 -- 172.21.15.100:0/515526855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 msgr2=0x55dbf422bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf422bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 -- 172.21.15.100:0/515526855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 msgr2=0x55dbf421fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf421fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 -- 172.21.15.100:0/515526855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbf4119ac0 con 0x55dbf4217bd0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f621a7fc640 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 0x55dbf42200c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f621008c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92708e65ed69bd3d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f6200ff9640 1 -- 172.21.15.100:0/515526855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100bc150 con 0x55dbf4217bd0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f6200ff9640 1 -- 172.21.15.100:0/515526855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6210087f90 con 0x55dbf4217bd0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.161+0000 7f6200ff9640 1 -- 172.21.15.100:0/515526855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100baaf0 con 0x55dbf4217bd0 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- 172.21.15.100:0/515526855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 msgr2=0x55dbf42200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 0x55dbf42200c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f621008c890 comp rx=0 tx=0).stop 2024-03-12T05:30:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- 172.21.15.100:0/515526855 shutdown_connections 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf42207b0 0x55dbf422bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf4217bd0 0x55dbf42200c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- 172.21.15.100:0/515526855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf421fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- 172.21.15.100:0/515526855 >> 172.21.15.100:0/515526855 conn(0x55dbf4211990 msgr2=0x55dbf423ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- 172.21.15.100:0/515526855 shutdown_connections 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- 172.21.15.100:0/515526855 wait complete. 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- start start 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 Processor -- start 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4283450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4283990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4227ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf4228490 con 0x55dbf42207b0 2024-03-12T05:30:16.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf4228600 con 0x55dbf42172b0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf42287c0 con 0x55dbf4217bd0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.162+0000 7f6223f66740 10 client.0 _ll_register_callbacks cb 0x7f620c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4227ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4283990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4283450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4227ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60006/0 (socket says 172.21.15.100:60006) 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 -- 172.21.15.100:0/1801384969 learned_addr learned my addr 172.21.15.100:0/1801384969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 -- 172.21.15.100:0/1801384969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 msgr2=0x55dbf4283990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4283990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 -- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 msgr2=0x55dbf4283450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4283450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 -- 172.21.15.100:0/1801384969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c1a4460 con 0x55dbf42207b0 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621affd640 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4283450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:16.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f621b7fe640 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4227ed0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6214009f90 tx=0x7f6214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24abec4a139cd44e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f61feff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f61feff5640 1 -- 172.21.15.100:0/1801384969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214026070 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f620c00fd00 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f61feff5640 1 -- 172.21.15.100:0/1801384969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6214021b60 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f61feff5640 1 -- 172.21.15.100:0/1801384969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6214021d20 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f6223f66740 10 client.19281 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dbf413f6a0 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.163+0000 7f61feff5640 1 -- 172.21.15.100:0/1801384969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6214036020 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f61feff5640 1 -- 172.21.15.100:0/1801384969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f62140056e0 con 0x55dbf42207b0 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 10 client.19281 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 10 client.19281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 20 client.19281 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf428c510 0x55dbf428e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 4 client.19281 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:30:16.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dbf3f0a890 con 0x55dbf428c510 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f621affd640 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf428c510 0x55dbf428e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f61feff5640 10 client.19281 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f621affd640 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf428c510 0x55dbf428e930 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6e05d822ce4ba2 server_cookie=6f3edc68d2f72bb5 in_seq=0 out_seq=0 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f61feff5640 1 -- 172.21.15.100:0/1801384969 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dbf3f0a890 con 0x55dbf428c510 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f61feff5640 10 client.19281 handle_command_reply: tid=0 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.164+0000 7f6223f66740 1 client.19281 shutdown 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 msgr2=0x55dbf4227ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4227ed0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6214009f90 tx=0x7f6214004a40 comp rx=0 tx=0).stop 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 shutdown_connections 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55dbf428c510 0x55dbf428e930 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbf42207b0 0x55dbf4227ed0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f61feff5640 0 client.19281 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4217bd0 0x55dbf4283990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 --2- 172.21.15.100:0/1801384969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf42172b0 0x55dbf4283450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 >> 172.21.15.100:0/1801384969 conn(0x55dbf4211990 msgr2=0x55dbf421bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 shutdown_connections 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 1 -- 172.21.15.100:0/1801384969 wait complete. 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f6223f66740 20 client.19281 trim_cache size 0 max 16384 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:30:16.166 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:30:16.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62040a1ba0 msgr2=0x7f62040a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62040a1ba0 0x7f62040a4060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6214005fd0 tx=0x7f6214004ab0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 msgr2=0x7f621c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c197c50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f621002c210 tx=0x7f621002c690 comp rx=0 tx=0).stop 2024-03-12T05:30:16.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 shutdown_connections 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f62040a1ba0 0x7f62040a4060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f621c10b6e0 0x7f621c19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c102cc0 0x7f621c197c50 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 --2- 172.21.15.100:0/2579073436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c1023a0 0x7f621c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 >> 172.21.15.100:0/2579073436 conn(0x7f621c077230 msgr2=0x7f621c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 shutdown_connections 2024-03-12T05:30:16.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.165+0000 7f62212b5640 1 -- 172.21.15.100:0/2579073436 wait complete. 2024-03-12T05:30:16.170 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:30:16.170 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:30:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 167278 28.05 MB/sec execute 179 sec latency 337.225 ms 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.301+0000 7fae3f8ae640 1 -- 172.21.15.100:0/109043148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3810f9b0 msgr2=0x7fae3810d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.301+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/109043148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3810f9b0 0x7fae3810d4a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae30009cd0 tx=0x7fae3002c0e0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 -- 172.21.15.100:0/109043148 shutdown_connections 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/109043148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3811a960 0x7fae3811cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/109043148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3810d9e0 0x7fae3810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/109043148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3810f9b0 0x7fae3810d4a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 -- 172.21.15.100:0/109043148 >> 172.21.15.100:0/109043148 conn(0x7fae3806c270 msgr2=0x7fae3806c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 -- 172.21.15.100:0/109043148 shutdown_connections 2024-03-12T05:30:16.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.302+0000 7fae3f8ae640 1 -- 172.21.15.100:0/109043148 wait complete. 2024-03-12T05:30:16.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.303+0000 7fae3f8ae640 1 Processor -- start 2024-03-12T05:30:16.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.303+0000 7fae3f8ae640 1 -- start start 2024-03-12T05:30:16.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3f8ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 0x7fae381a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3f8ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3810f9b0 0x7fae381a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3f8ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 0x7fae381ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3f8ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381ab600 con 0x7fae3810d9e0 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3f8ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381ab770 con 0x7fae3811a960 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3f8ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381ab930 con 0x7fae3810f9b0 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3d64a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 0x7fae381a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 0x7fae381ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 0x7fae381ab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:37878/0 (socket says 172.21.15.100:37878) 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 -- 172.21.15.100:0/3052964706 learned_addr learned my addr 172.21.15.100:0/3052964706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3d64a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 0x7fae381a6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60028/0 (socket says 172.21.15.100:60028) 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3ce49640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3810f9b0 0x7fae381a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 -- 172.21.15.100:0/3052964706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3810f9b0 msgr2=0x7fae381a6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3810f9b0 0x7fae381a6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 -- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 msgr2=0x7fae381a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 0x7fae381a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3de4b640 1 -- 172.21.15.100:0/3052964706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009900 con 0x7fae3811a960 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.304+0000 7fae3d64a640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 0x7fae381a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.305+0000 7fae3de4b640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 0x7fae381ab000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae28009fd0 tx=0x7fae2800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.305+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011830 con 0x7fae3811a960 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.305+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae28011e70 con 0x7fae3811a960 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.305+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae381abc10 con 0x7fae3811a960 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.305+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae381b3490 con 0x7fae3811a960 2024-03-12T05:30:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.305+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800f2e0 con 0x7fae3811a960 2024-03-12T05:30:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.306+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae38067f70 con 0x7fae3811a960 2024-03-12T05:30:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.308+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fae280105f0 con 0x7fae3811a960 2024-03-12T05:30:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.308+0000 7fae2e7fc640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae240a1ba0 0x7fae240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.309+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fae280ccf20 con 0x7fae3811a960 2024-03-12T05:30:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.309+0000 7fae3d64a640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae240a1ba0 0x7fae240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.309+0000 7fae3d64a640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae240a1ba0 0x7fae240a4060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae30009ca0 tx=0x7fae30005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.314+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae28095ec0 con 0x7fae3811a960 2024-03-12T05:30:16.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:16 smithi100 ceph-mon[32210]: pgmap v572: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.8 MiB/s wr, 610 op/s 2024-03-12T05:30:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:16 smithi156 ceph-mon[36270]: pgmap v572: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.8 MiB/s wr, 610 op/s 2024-03-12T05:30:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:16 smithi161 ceph-mon[36043]: pgmap v572: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.8 MiB/s wr, 610 op/s 2024-03-12T05:30:16.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.608+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae381a7a90 con 0x7fae3811a960 2024-03-12T05:30:16.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.609+0000 7fae2e7fc640 1 -- 172.21.15.100:0/3052964706 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fae28099580 con 0x7fae3811a960 2024-03-12T05:30:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:30:16.604 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:30:16.604 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.612+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae240a1ba0 msgr2=0x7fae240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae240a1ba0 0x7fae240a4060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae30009ca0 tx=0x7fae30005c50 comp rx=0 tx=0).stop 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 msgr2=0x7fae381ab000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 0x7fae381ab000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae28009fd0 tx=0x7fae2800b8e0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 shutdown_connections 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae240a1ba0 0x7fae240a4060 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3811a960 0x7fae381ab000 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3810f9b0 0x7fae381a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 --2- 172.21.15.100:0/3052964706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3810d9e0 0x7fae381a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 >> 172.21.15.100:0/3052964706 conn(0x7fae3806c270 msgr2=0x7fae38071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 shutdown_connections 2024-03-12T05:30:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.613+0000 7fae3f8ae640 1 -- 172.21.15.100:0/3052964706 wait complete. 2024-03-12T05:30:16.621 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:30:16.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- 172.21.15.100:0/2695886882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 msgr2=0x7f6c58100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- 172.21.15.100:0/2695886882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58100640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009890 tx=0x7f6c5402bb70 comp rx=0 tx=0).stop 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- 172.21.15.100:0/2695886882 shutdown_connections 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- 172.21.15.100:0/2695886882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58105330 0x7f6c5810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- 172.21.15.100:0/2695886882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58100c10 0x7f6c58104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- 172.21.15.100:0/2695886882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58100640 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- 172.21.15.100:0/2695886882 >> 172.21.15.100:0/2695886882 conn(0x7f6c58077180 msgr2=0x7f6c58077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- 172.21.15.100:0/2695886882 shutdown_connections 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- 172.21.15.100:0/2695886882 wait complete. 2024-03-12T05:30:16.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 Processor -- start 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- start start 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58105330 0x7f6c58102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 0x7f6c581a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581a5fd0 con 0x7f6c58102770 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581a6140 con 0x7f6c58100260 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.756+0000 7f6c6016b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581a6300 con 0x7f6c58105330 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 0x7f6c581a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 0x7f6c581a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60036/0 (socket says 172.21.15.100:60036) 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5df07640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 -- 172.21.15.100:0/1687355795 learned_addr learned my addr 172.21.15.100:0/1687355795 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58105330 0x7f6c58102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 -- 172.21.15.100:0/1687355795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58105330 msgr2=0x7f6c58102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58105330 0x7f6c58102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 -- 172.21.15.100:0/1687355795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 msgr2=0x7f6c58101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 -- 172.21.15.100:0/1687355795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50009900 con 0x7f6c58102770 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c5e708640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 0x7f6c581a5a00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000efc0 tx=0x7f6c5000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c4effd640 1 -- 172.21.15.100:0/1687355795 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50003970 con 0x7f6c58102770 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c54009490 con 0x7f6c58102770 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c4effd640 1 -- 172.21.15.100:0/1687355795 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c50004020 con 0x7f6c58102770 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c4effd640 1 -- 172.21.15.100:0/1687355795 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000b9b0 con 0x7f6c58102770 2024-03-12T05:30:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.757+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c581a6960 con 0x7f6c58102770 2024-03-12T05:30:16.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.760+0000 7f6c4effd640 1 -- 172.21.15.100:0/1687355795 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6c5000a040 con 0x7f6c58102770 2024-03-12T05:30:16.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 Processor -- start 2024-03-12T05:30:16.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 -- start start 2024-03-12T05:30:16.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e719630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e722210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 0x55986e724b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e6d4960 con 0x55986e722750 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e688310 con 0x55986e719b70 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c62e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e685840 con 0x55986e719250 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c4effd640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6c440a1c50 0x7f6c440a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c5d706640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 0x55986e724b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c5d706640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 0x55986e724b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60048/0 (socket says 172.21.15.100:60048) 2024-03-12T05:30:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c5d706640 1 -- 172.21.15.100:0/3223211884 learned_addr learned my addr 172.21.15.100:0/3223211884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e719630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5e708640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6c440a1c50 0x7f6c440a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c4effd640 1 -- 172.21.15.100:0/1687355795 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6c500cd750 con 0x7f6c58102770 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.761+0000 7f6c5df07640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e722210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 -- 172.21.15.100:0/3223211884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 msgr2=0x55986e719630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e719630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 -- 172.21.15.100:0/3223211884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 msgr2=0x55986e722210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e722210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 -- 172.21.15.100:0/3223211884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55986e643060 con 0x55986e722750 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 0x55986e724b40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009fc0 tx=0x7f6c48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df23f8ffe8d5f95 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c5e708640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6c440a1c50 0x7f6c440a4110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6c5009a270 tx=0x7f6c5006a380 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c43fff640 1 -- 172.21.15.100:0/3223211884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48018070 con 0x55986e722750 2024-03-12T05:30:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c43fff640 1 -- 172.21.15.100:0/3223211884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c4800f960 con 0x55986e722750 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.762+0000 7f6c43fff640 1 -- 172.21.15.100:0/3223211884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48010880 con 0x55986e722750 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 -- 172.21.15.100:0/3223211884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 msgr2=0x55986e724b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 0x55986e724b40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009fc0 tx=0x7f6c48005f90 comp rx=0 tx=0).stop 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 -- 172.21.15.100:0/3223211884 shutdown_connections 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e722750 0x55986e724b40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e722210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3223211884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e719630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.763+0000 7f6c62e10740 1 -- 172.21.15.100:0/3223211884 >> 172.21.15.100:0/3223211884 conn(0x55986e713980 msgr2=0x55986e713d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 -- 172.21.15.100:0/3223211884 shutdown_connections 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 -- 172.21.15.100:0/3223211884 wait complete. 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 -- start start 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 Processor -- start 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 0x55986e71f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e71fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e722750 0x55986e720230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e681020 con 0x55986e719250 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e354430 con 0x55986e719b70 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e310e00 con 0x55986e722750 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c62e10740 10 client.0 _ll_register_callbacks cb 0x55986e628cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c5d706640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 0x55986e71f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c5d706640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 0x55986e71f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60050/0 (socket says 172.21.15.100:60050) 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.764+0000 7f6c5d706640 1 -- 172.21.15.100:0/1720180621 learned_addr learned my addr 172.21.15.100:0/1720180621 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e71fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e722750 0x55986e720230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 -- 172.21.15.100:0/1720180621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e722750 msgr2=0x55986e720230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e722750 0x55986e720230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 -- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 msgr2=0x55986e71fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e71fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 -- 172.21.15.100:0/1720180621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48009e00 con 0x55986e719250 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e71fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 0x55986e71f7b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f6c48001eb0 tx=0x7f6c48005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4ec112d193c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c417fa640 1 -- 172.21.15.100:0/1720180621 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48018070 con 0x55986e719250 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c417fa640 1 -- 172.21.15.100:0/1720180621 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6c48000e40 con 0x55986e719250 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55986e3807e0 con 0x55986e719250 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c417fa640 1 -- 172.21.15.100:0/1720180621 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c480054f0 con 0x55986e719250 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c62e10740 10 client.19296 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55986e67f6a0 con 0x55986e719250 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.765+0000 7f6c417fa640 1 -- 172.21.15.100:0/1720180621 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55986e3807e0 con 0x55986e719250 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c417fa640 1 -- 172.21.15.100:0/1720180621 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c48004d60 con 0x55986e719250 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c62e10740 10 client.19296 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c62e10740 10 client.19296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c62e10740 20 client.19296 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e734ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c62e10740 4 client.19296 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55986e3bd720 con 0x55986e732ae0 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e734ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c417fa640 10 client.19296 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:16.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.766+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e734ed0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0f72b13c280c48f server_cookie=e44be5025423f4e3 in_seq=0 out_seq=0 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.767+0000 7f6c417fa640 1 -- 172.21.15.100:0/1720180621 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55986e3bd720 con 0x55986e732ae0 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.767+0000 7f6c417fa640 10 client.19296 handle_command_reply: tid=0 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.767+0000 7f6c62e10740 1 client.19296 shutdown 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 msgr2=0x55986e71f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 0x55986e71f7b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f6c48001eb0 tx=0x7f6c48005270 comp rx=0 tx=0).stop 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 shutdown_connections 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e734ed0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e722750 0x55986e720230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c417fa640 0 client.19296 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e719b70 0x55986e71fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1720180621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719250 0x55986e71f7b0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 >> 172.21.15.100:0/1720180621 conn(0x55986e713980 msgr2=0x55986e7156d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 shutdown_connections 2024-03-12T05:30:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 1 -- 172.21.15.100:0/1720180621 wait complete. 2024-03-12T05:30:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.768+0000 7f6c62e10740 20 client.19296 trim_cache size 0 max 16384 2024-03-12T05:30:16.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 Processor -- start 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 -- start start 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e721ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e71ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e71cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e606130 con 0x55986e719b70 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e718650 con 0x55986e722750 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c62e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e71a5f0 con 0x55986e719250 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e71ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e71ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60054/0 (socket says 172.21.15.100:60054) 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 -- 172.21.15.100:0/1225674195 learned_addr learned my addr 172.21.15.100:0/1225674195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e721ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5d706640 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e71cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 -- 172.21.15.100:0/1225674195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 msgr2=0x55986e721ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e721ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 -- 172.21.15.100:0/1225674195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 msgr2=0x55986e71cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e71cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.781+0000 7f6c5df07640 1 -- 172.21.15.100:0/1225674195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55986e61bc20 con 0x55986e719b70 2024-03-12T05:30:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.782+0000 7f6c5df07640 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e71ca10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6c54000c00 tx=0x7f6c5402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569feef76185197f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.782+0000 7f6c437fe640 1 -- 172.21.15.100:0/1225674195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540044e0 con 0x55986e719b70 2024-03-12T05:30:16.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.782+0000 7f6c437fe640 1 -- 172.21.15.100:0/1225674195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c54046860 con 0x55986e719b70 2024-03-12T05:30:16.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.782+0000 7f6c437fe640 1 -- 172.21.15.100:0/1225674195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540349c0 con 0x55986e719b70 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.782+0000 7f6c62e10740 1 -- 172.21.15.100:0/1225674195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 msgr2=0x55986e71ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.782+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e71ca10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6c54000c00 tx=0x7f6c5402e040 comp rx=0 tx=0).stop 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- 172.21.15.100:0/1225674195 shutdown_connections 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e71cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e71ca10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 --2- 172.21.15.100:0/1225674195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e721ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- 172.21.15.100:0/1225674195 >> 172.21.15.100:0/1225674195 conn(0x55986e713980 msgr2=0x55986e71e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- 172.21.15.100:0/1225674195 shutdown_connections 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- 172.21.15.100:0/1225674195 wait complete. 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- start start 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 Processor -- start 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e7286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e728c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e788f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e729350 con 0x55986e719b70 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e7894d0 con 0x55986e722750 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986e789660 con 0x55986e719250 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c62e10740 10 client.0 _ll_register_callbacks cb 0x7f6c480097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c5df07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e728c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c5df07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e728c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60064/0 (socket says 172.21.15.100:60064) 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c5df07640 1 -- 172.21.15.100:0/3523872434 learned_addr learned my addr 172.21.15.100:0/3523872434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.783+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e7286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e788f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5df07640 1 -- 172.21.15.100:0/3523872434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 msgr2=0x55986e7286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5df07640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e7286d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5df07640 1 -- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 msgr2=0x55986e788f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5df07640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e788f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5df07640 1 -- 172.21.15.100:0/3523872434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5810d710 con 0x55986e719b70 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c5df07640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e728c10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6c54005e00 tx=0x7f6c54032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1543c9c0d5d66c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:16.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c417fa640 1 -- 172.21.15.100:0/3523872434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5405a070 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c417fa640 1 -- 172.21.15.100:0/3523872434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6c54043040 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c4800f960 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c417fa640 1 -- 172.21.15.100:0/3523872434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c540044f0 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c62e10740 10 client.19302 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55986e6418c0 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.784+0000 7f6c417fa640 1 -- 172.21.15.100:0/3523872434 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c4800f960 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c417fa640 1 -- 172.21.15.100:0/3523872434 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c5405d030 con 0x55986e719b70 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c62e10740 10 client.19302 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c62e10740 10 client.19302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:30:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c62e10740 20 client.19302 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e78ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c62e10740 4 client.19302 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55986e40c890 con 0x55986e732ae0 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e78ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c5d706640 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e78ca20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c4496f90516d6b0 server_cookie=f71331e53e18ac6a in_seq=0 out_seq=0 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.785+0000 7f6c417fa640 10 client.19302 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.788+0000 7f6c417fa640 1 -- 172.21.15.100:0/3523872434 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55986e40c890 con 0x55986e732ae0 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.788+0000 7f6c417fa640 10 client.19302 handle_command_reply: tid=0 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.788+0000 7f6c62e10740 1 client.19302 shutdown 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.788+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 msgr2=0x55986e728c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.788+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e728c10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6c54005e00 tx=0x7f6c54032460 comp rx=0 tx=0).stop 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 shutdown_connections 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55986e732ae0 0x55986e78ca20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55986e722750 0x55986e788f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55986e719b70 0x55986e728c10 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c417fa640 0 client.19302 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 --2- 172.21.15.100:0/3523872434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55986e719250 0x55986e7286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 >> 172.21.15.100:0/3523872434 conn(0x55986e713980 msgr2=0x55986e71f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 shutdown_connections 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 1 -- 172.21.15.100:0/3523872434 wait complete. 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c62e10740 20 client.19302 trim_cache size 0 max 16384 2024-03-12T05:30:16.790 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:30:16.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6c440a1c50 msgr2=0x7f6c440a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c6016b640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6c440a1c50 0x7f6c440a4110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6c5009a270 tx=0x7f6c5006a380 comp rx=0 tx=0).stop 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 msgr2=0x7f6c581a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.789+0000 7f6c6016b640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 0x7f6c581a5a00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000efc0 tx=0x7f6c5000c800 comp rx=0 tx=0).stop 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 shutdown_connections 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6c440a1c50 0x7f6c440a4110 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c58102770 0x7f6c581a5a00 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58105330 0x7f6c58102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 --2- 172.21.15.100:0/1687355795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c58100260 0x7f6c58101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 >> 172.21.15.100:0/1687355795 conn(0x7f6c58077180 msgr2=0x7f6c580686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 shutdown_connections 2024-03-12T05:30:16.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.790+0000 7f6c6016b640 1 -- 172.21.15.100:0/1687355795 wait complete. 2024-03-12T05:30:16.795 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:30:16.795 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:30:16.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 -- 172.21.15.100:0/524506626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788101380 msgr2=0x7fe788101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 --2- 172.21.15.100:0/524506626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788101380 0x7fe788101760 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe778009930 tx=0x7fe77802bc30 comp rx=0 tx=0).stop 2024-03-12T05:30:16.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 -- 172.21.15.100:0/524506626 shutdown_connections 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 --2- 172.21.15.100:0/524506626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788106450 0x7fe78810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 --2- 172.21.15.100:0/524506626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788101d30 0x7fe788105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 --2- 172.21.15.100:0/524506626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788101380 0x7fe788101760 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 -- 172.21.15.100:0/524506626 >> 172.21.15.100:0/524506626 conn(0x7fe788078540 msgr2=0x7fe788078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 -- 172.21.15.100:0/524506626 shutdown_connections 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.922+0000 7fe78de0e640 1 -- 172.21.15.100:0/524506626 wait complete. 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 Processor -- start 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 -- start start 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788101380 0x7fe78819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 0x7fe78819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788197630 0x7fe788197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788198100 con 0x7fe788106450 2024-03-12T05:30:16.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788198270 con 0x7fe788101380 2024-03-12T05:30:16.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe78de0e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788198430 con 0x7fe788197630 2024-03-12T05:30:16.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.923+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 0x7fe78819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788197630 0x7fe788197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 0x7fe78819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60070/0 (socket says 172.21.15.100:60070) 2024-03-12T05:30:16.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe786ffd640 1 -- 172.21.15.100:0/1189030093 learned_addr learned my addr 172.21.15.100:0/1189030093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe7877fe640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788101380 0x7fe78819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 -- 172.21.15.100:0/1189030093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788101380 msgr2=0x7fe78819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788101380 0x7fe78819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 -- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 msgr2=0x7fe78819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 0x7fe78819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 -- 172.21.15.100:0/1189030093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780009840 con 0x7fe788197630 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe786ffd640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 0x7fe78819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe787fff640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788197630 0x7fe788197ab0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe78000ec50 tx=0x7fe78000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe784ff9640 1 -- 172.21.15.100:0/1189030093 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780003b70 con 0x7fe788197630 2024-03-12T05:30:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe784ff9640 1 -- 172.21.15.100:0/1189030093 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe78000ce70 con 0x7fe788197630 2024-03-12T05:30:16.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe778009560 con 0x7fe788197630 2024-03-12T05:30:16.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe784ff9640 1 -- 172.21.15.100:0/1189030093 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780010830 con 0x7fe788197630 2024-03-12T05:30:16.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.924+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7881a46d0 con 0x7fe788197630 2024-03-12T05:30:16.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.927+0000 7fe784ff9640 1 -- 172.21.15.100:0/1189030093 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe780004020 con 0x7fe788197630 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.927+0000 7fe7909e3740 1 Processor -- start 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.927+0000 7fe7909e3740 1 -- start start 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.927+0000 7fe784ff9640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe7700a1ba0 0x7fe7700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.927+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50a250 0x556b3e50a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50ab70 0x556b3e513210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e515b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7909e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e4c5960 con 0x556b3e50ab70 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7909e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e479310 con 0x556b3e513750 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7909e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e476840 con 0x556b3e50a250 2024-03-12T05:30:16.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe7700a1ba0 0x7fe7700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50a250 0x556b3e50a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50a250 0x556b3e50a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42966/0 (socket says 172.21.15.100:42966) 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe7877fe640 1 -- 172.21.15.100:0/425198955 learned_addr learned my addr 172.21.15.100:0/425198955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50ab70 0x556b3e513210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe787fff640 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e515b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe784ff9640 1 -- 172.21.15.100:0/1189030093 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe780014070 con 0x7fe788197630 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 -- 172.21.15.100:0/425198955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50a250 msgr2=0x556b3e50a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50a250 0x556b3e50a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 -- 172.21.15.100:0/425198955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 msgr2=0x556b3e515b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e515b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 -- 172.21.15.100:0/425198955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b3e434060 con 0x556b3e50ab70 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe7700a1ba0 0x7fe7700a4060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009600 tx=0x7fe77c011440 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe786ffd640 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50ab70 0x556b3e513210 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe788198e80 tx=0x7fe77c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd06a8ba0c47047f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.928+0000 7fe76dffb640 1 -- 172.21.15.100:0/425198955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c009c10 con 0x556b3e50ab70 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe76dffb640 1 -- 172.21.15.100:0/425198955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe77c011e00 con 0x556b3e50ab70 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe76dffb640 1 -- 172.21.15.100:0/425198955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c01e3d0 con 0x556b3e50ab70 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 -- 172.21.15.100:0/425198955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50ab70 msgr2=0x556b3e513210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50ab70 0x556b3e513210 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe788198e80 tx=0x7fe77c01d000 comp rx=0 tx=0).stop 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 -- 172.21.15.100:0/425198955 shutdown_connections 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e515b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50ab70 0x556b3e513210 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 --2- 172.21.15.100:0/425198955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50a250 0x556b3e50a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 -- 172.21.15.100:0/425198955 >> 172.21.15.100:0/425198955 conn(0x556b3e504980 msgr2=0x556b3e504d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 -- 172.21.15.100:0/425198955 shutdown_connections 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.929+0000 7fe7909e3740 1 -- 172.21.15.100:0/425198955 wait complete. 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 -- start start 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 Processor -- start 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e510750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e510c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e5111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e472020 con 0x556b3e50a250 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e145430 con 0x556b3e513750 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e101e00 con 0x556b3e50ab70 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7909e3740 10 client.0 _ll_register_callbacks cb 0x556b3e419cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e510750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e510750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60088/0 (socket says 172.21.15.100:60088) 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe7877fe640 1 -- 172.21.15.100:0/3413035570 learned_addr learned my addr 172.21.15.100:0/3413035570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe786ffd640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e510c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.930+0000 7fe787fff640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e5111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7877fe640 1 -- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 msgr2=0x556b3e510c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7877fe640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e510c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7877fe640 1 -- 172.21.15.100:0/3413035570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 msgr2=0x556b3e5111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7877fe640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e5111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7877fe640 1 -- 172.21.15.100:0/3413035570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c019040 con 0x556b3e50a250 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7877fe640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e510750 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe77802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d50db9eb9f3e82 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe766ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe766ffd640 1 -- 172.21.15.100:0/3413035570 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778004040 con 0x556b3e50a250 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe766ffd640 1 -- 172.21.15.100:0/3413035570 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe7780041a0 con 0x556b3e50a250 2024-03-12T05:30:16.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b3e1717e0 con 0x556b3e50a250 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe766ffd640 1 -- 172.21.15.100:0/3413035570 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe778004360 con 0x556b3e50a250 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 10 client.19314 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556b3e4706a0 con 0x556b3e50a250 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe766ffd640 1 -- 172.21.15.100:0/3413035570 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556b3e1717e0 con 0x556b3e50a250 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe766ffd640 1 -- 172.21.15.100:0/3413035570 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe778050070 con 0x556b3e50a250 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 10 client.19314 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 10 client.19314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 20 client.19314 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e523af0 0x556b3e525f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.931+0000 7fe7909e3740 4 client.19314 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.932+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b3e1ae720 con 0x556b3e523af0 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.932+0000 7fe786ffd640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e523af0 0x556b3e525f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.932+0000 7fe786ffd640 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e523af0 0x556b3e525f10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc0a22442a79923 server_cookie=aef5897d385775c9 in_seq=0 out_seq=0 2024-03-12T05:30:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.932+0000 7fe766ffd640 10 client.19314 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe766ffd640 1 -- 172.21.15.100:0/3413035570 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b3e1ae720 con 0x556b3e523af0 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe766ffd640 10 client.19314 handle_command_reply: tid=0 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 client.19314 shutdown 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 msgr2=0x556b3e510750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e510750 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe77802ca10 comp rx=0 tx=0).stop 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 shutdown_connections 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e523af0 0x556b3e525f10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e5111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe766ffd640 0 client.19314 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:16.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e510c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 --2- 172.21.15.100:0/3413035570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e510750 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.933+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 >> 172.21.15.100:0/3413035570 conn(0x556b3e504980 msgr2=0x556b3e5066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.934+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 shutdown_connections 2024-03-12T05:30:16.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.934+0000 7fe7909e3740 1 -- 172.21.15.100:0/3413035570 wait complete. 2024-03-12T05:30:16.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.934+0000 7fe7909e3740 20 client.19314 trim_cache size 0 max 16384 2024-03-12T05:30:16.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.946+0000 7fe7909e3740 1 Processor -- start 2024-03-12T05:30:16.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.946+0000 7fe7909e3740 1 -- start start 2024-03-12T05:30:16.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.946+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e512e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.946+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e50ab70 0x556b3e510ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.946+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e513750 0x556b3e50d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.946+0000 7fe7909e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e3f7130 con 0x556b3e50a250 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7909e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e505780 con 0x556b3e50ab70 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7909e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e5047f0 con 0x556b3e513750 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e512e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e512e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60102/0 (socket says 172.21.15.100:60102) 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 -- 172.21.15.100:0/2338604628 learned_addr learned my addr 172.21.15.100:0/2338604628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe787fff640 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e513750 0x556b3e50d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe786ffd640 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e50ab70 0x556b3e510ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 -- 172.21.15.100:0/2338604628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e513750 msgr2=0x556b3e50d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e513750 0x556b3e50d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 -- 172.21.15.100:0/2338604628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e50ab70 msgr2=0x556b3e510ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e50ab70 0x556b3e510ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 -- 172.21.15.100:0/2338604628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b3e40cc20 con 0x556b3e50a250 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe7877fe640 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e512e80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe778004ae0 tx=0x7fe77802c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b71b3c6de02bbe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe76d7fa640 1 -- 172.21.15.100:0/2338604628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7780035f0 con 0x556b3e50a250 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe76d7fa640 1 -- 172.21.15.100:0/2338604628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe778009780 con 0x556b3e50a250 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.947+0000 7fe76d7fa640 1 -- 172.21.15.100:0/2338604628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778005310 con 0x556b3e50a250 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 -- 172.21.15.100:0/2338604628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 msgr2=0x556b3e512e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e512e80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe778004ae0 tx=0x7fe77802c700 comp rx=0 tx=0).stop 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 -- 172.21.15.100:0/2338604628 shutdown_connections 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e513750 0x556b3e50d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e50ab70 0x556b3e510ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 --2- 172.21.15.100:0/2338604628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e512e80 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 -- 172.21.15.100:0/2338604628 >> 172.21.15.100:0/2338604628 conn(0x556b3e504980 msgr2=0x556b3e52a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 -- 172.21.15.100:0/2338604628 shutdown_connections 2024-03-12T05:30:16.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 -- 172.21.15.100:0/2338604628 wait complete. 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.948+0000 7fe7909e3740 1 -- start start 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 Processor -- start 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e5191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e519720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e519c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e57a190 con 0x556b3e50a250 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e57a300 con 0x556b3e513750 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3e57a4c0 con 0x556b3e50ab70 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7909e3740 10 client.0 _ll_register_callbacks cb 0x7fe77c011950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e5191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e5191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60110/0 (socket says 172.21.15.100:60110) 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 -- 172.21.15.100:0/295275933 learned_addr learned my addr 172.21.15.100:0/295275933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe786ffd640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e519720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe787fff640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e519c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 -- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 msgr2=0x556b3e519720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e519720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 -- 172.21.15.100:0/295275933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 msgr2=0x556b3e519c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e519c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.949+0000 7fe7877fe640 1 -- 172.21.15.100:0/295275933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b3e4706a0 con 0x556b3e50a250 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7877fe640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e5191e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe77802c170 tx=0x7fe778000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75791c7fee997dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe766ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe766ffd640 1 -- 172.21.15.100:0/295275933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7780035f0 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe766ffd640 1 -- 172.21.15.100:0/295275933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe778000e40 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe77c011e00 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe766ffd640 1 -- 172.21.15.100:0/295275933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe778003870 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 10 client.19320 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556b3e4328c0 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe766ffd640 1 -- 172.21.15.100:0/295275933 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe77c011e00 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe766ffd640 1 -- 172.21.15.100:0/295275933 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe778069070 con 0x556b3e50a250 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 10 client.19320 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 10 client.19320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 20 client.19320 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e57b7c0 0x556b3e57dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 4 client.19320 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556b3e1fd890 con 0x556b3e57b7c0 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.950+0000 7fe786ffd640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e57b7c0 0x556b3e57dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe766ffd640 10 client.19320 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe786ffd640 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e57b7c0 0x556b3e57dbb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8322e7444470f7b6 server_cookie=f5ee70c09aa2e78b in_seq=0 out_seq=0 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe766ffd640 1 -- 172.21.15.100:0/295275933 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556b3e1fd890 con 0x556b3e57b7c0 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe766ffd640 10 client.19320 handle_command_reply: tid=0 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 client.19320 shutdown 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 msgr2=0x556b3e5191e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e5191e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe77802c170 tx=0x7fe778000bc0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 shutdown_connections 2024-03-12T05:30:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x556b3e57b7c0 0x556b3e57dbb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe766ffd640 0 client.19320 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b3e513750 0x556b3e519c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b3e50ab70 0x556b3e519720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 --2- 172.21.15.100:0/295275933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b3e50a250 0x556b3e5191e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 >> 172.21.15.100:0/295275933 conn(0x556b3e504980 msgr2=0x556b3e505db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 shutdown_connections 2024-03-12T05:30:16.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 1 -- 172.21.15.100:0/295275933 wait complete. 2024-03-12T05:30:16.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.951+0000 7fe7909e3740 20 client.19320 trim_cache size 0 max 16384 2024-03-12T05:30:16.961 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:30:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe7700a1ba0 msgr2=0x7fe7700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe7700a1ba0 0x7fe7700a4060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009600 tx=0x7fe77c011440 comp rx=0 tx=0).stop 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788197630 msgr2=0x7fe788197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788197630 0x7fe788197ab0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe78000ec50 tx=0x7fe78000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 shutdown_connections 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe7700a1ba0 0x7fe7700a4060 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe788197630 0x7fe788197ab0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe788106450 0x7fe78819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 --2- 172.21.15.100:0/1189030093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788101380 0x7fe78819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 >> 172.21.15.100:0/1189030093 conn(0x7fe788078540 msgr2=0x7fe7880ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 shutdown_connections 2024-03-12T05:30:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:16.952+0000 7fe78de0e640 1 -- 172.21.15.100:0/1189030093 wait complete. 2024-03-12T05:30:16.966 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:30:16.966 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:30:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/3707245716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd4106450 msgr2=0x7fbcd410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/3707245716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd4106450 0x7fbcd410e020 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009cd0 tx=0x7fbccc02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/3707245716 shutdown_connections 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/3707245716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd4106450 0x7fbcd410e020 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/3707245716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4100240 0x7fbcd4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/3707245716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd40ff890 0x7fbcd40ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/3707245716 >> 172.21.15.100:0/3707245716 conn(0x7fbcd40765a0 msgr2=0x7fbcd40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.088+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/3707245716 shutdown_connections 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/3707245716 wait complete. 2024-03-12T05:30:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 Processor -- start 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 -- start start 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd40ff890 0x7fbcd419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4106450 0x7fbcd419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4197600 0x7fbcd4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4197ff0 con 0x7fbcd40ff890 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4198160 con 0x7fbcd4197600 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcdaf0d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4198320 con 0x7fbcd4106450 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcd8ca9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd40ff890 0x7fbcd419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcd8ca9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd40ff890 0x7fbcd419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60118/0 (socket says 172.21.15.100:60118) 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcd8ca9640 1 -- 172.21.15.100:0/821794984 learned_addr learned my addr 172.21.15.100:0/821794984 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.089+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4106450 0x7fbcd419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4197600 0x7fbcd4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd3fff640 1 -- 172.21.15.100:0/821794984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4197600 msgr2=0x7fbcd4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4197600 0x7fbcd4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd3fff640 1 -- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd40ff890 msgr2=0x7fbcd419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd40ff890 0x7fbcd419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd3fff640 1 -- 172.21.15.100:0/821794984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc009900 con 0x7fbcd4106450 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4106450 0x7fbcd419d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e9c0 tx=0x7fbcc000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd1ffb640 1 -- 172.21.15.100:0/821794984 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000ccc0 con 0x7fbcd4106450 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd41a4310 con 0x7fbcd4106450 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd1ffb640 1 -- 172.21.15.100:0/821794984 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcc000ce20 con 0x7fbcd4106450 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcd1ffb640 1 -- 172.21.15.100:0/821794984 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0004a00 con 0x7fbcd4106450 2024-03-12T05:30:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.090+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd41a47d0 con 0x7fbcd4106450 2024-03-12T05:30:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.092+0000 7fbcd1ffb640 1 -- 172.21.15.100:0/821794984 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fbcc0004b60 con 0x7fbcd4106450 2024-03-12T05:30:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcd1ffb640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbcc40a1ba0 0x7fbcc40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 Processor -- start 2024-03-12T05:30:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbcc40a1ba0 0x7fbcc40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcd1ffb640 1 -- 172.21.15.100:0/821794984 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbcc0014070 con 0x7fbcd4106450 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 -- start start 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f41630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 0x559269f4a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f4a750 0x559269f4cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269efc960 con 0x559269f41b70 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269eb0310 con 0x559269f41250 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269ead840 con 0x559269f4a750 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.093+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 0x559269f4a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f41630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f4a750 0x559269f4cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 0x559269f4a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60126/0 (socket says 172.21.15.100:60126) 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f41630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38036/0 (socket says 172.21.15.100:38036) 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd3fff640 1 -- 172.21.15.100:0/1316400752 learned_addr learned my addr 172.21.15.100:0/1316400752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbcc40a1ba0 0x7fbcc40a4060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009ce0 tx=0x7fbcc8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 -- 172.21.15.100:0/1316400752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f4a750 msgr2=0x559269f4cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f4a750 0x559269f4cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 -- 172.21.15.100:0/1316400752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 msgr2=0x559269f4a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 0x559269f4a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 -- 172.21.15.100:0/1316400752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559269e6b060 con 0x559269f41250 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 0x559269f4a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f41630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8015000 tx=0x7fbcc8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0c7b2d6aa44006 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcbeffd640 1 -- 172.21.15.100:0/1316400752 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc801dc30 con 0x559269f41250 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcbeffd640 1 -- 172.21.15.100:0/1316400752 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcc801dd90 con 0x559269f41250 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.094+0000 7fbcbeffd640 1 -- 172.21.15.100:0/1316400752 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc801b7b0 con 0x559269f41250 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 -- 172.21.15.100:0/1316400752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 msgr2=0x559269f41630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f41630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8015000 tx=0x7fbcc8014000 comp rx=0 tx=0).stop 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 -- 172.21.15.100:0/1316400752 shutdown_connections 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f4a750 0x559269f4cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f41b70 0x559269f4a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/1316400752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f41630 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 -- 172.21.15.100:0/1316400752 >> 172.21.15.100:0/1316400752 conn(0x559269f3b980 msgr2=0x559269f3bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 -- 172.21.15.100:0/1316400752 shutdown_connections 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 -- 172.21.15.100:0/1316400752 wait complete. 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 -- start start 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.095+0000 7fbcddbb9740 1 Processor -- start 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f47740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f47c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269ea9020 con 0x559269f4a750 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269b7c430 con 0x559269f41b70 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269b38e00 con 0x559269f41250 2024-03-12T05:30:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcddbb9740 10 client.0 _ll_register_callbacks cb 0x559269e50cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60138/0 (socket says 172.21.15.100:60138) 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd94aa640 1 -- 172.21.15.100:0/3527606373 learned_addr learned my addr 172.21.15.100:0/3527606373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f47c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f47740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 -- 172.21.15.100:0/3527606373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 msgr2=0x559269f47740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f47740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 -- 172.21.15.100:0/3527606373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 msgr2=0x559269f481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 -- 172.21.15.100:0/3527606373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc8017050 con 0x559269f41b70 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f47740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f47c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0096790 tx=0x7fbcc0099dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ddcec77af0458a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.096+0000 7fbcb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcb3fff640 1 -- 172.21.15.100:0/3527606373 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0094ea0 con 0x559269f41b70 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcb3fff640 1 -- 172.21.15.100:0/3527606373 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbcc009e480 con 0x559269f41b70 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559269ba87e0 con 0x559269f41b70 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcb3fff640 1 -- 172.21.15.100:0/3527606373 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcc00a2ad0 con 0x559269f41b70 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 10 client.28679 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559269ea76a0 con 0x559269f41b70 2024-03-12T05:30:17.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcb3fff640 1 -- 172.21.15.100:0/3527606373 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcc00b6030 con 0x559269f41b70 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcb3fff640 1 -- 172.21.15.100:0/3527606373 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbcc0099780 con 0x559269f41b70 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 10 client.28679 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 10 client.28679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 20 client.28679 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269f5aba0 0x559269f5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 4 client.28679 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559269be5720 con 0x559269f5aba0 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.097+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269f5aba0 0x559269f5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.098+0000 7fbcb3fff640 10 client.28679 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.098+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269f5aba0 0x559269f5cfc0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8bd4be224bcb36 server_cookie=3d4858873c37afd4 in_seq=0 out_seq=0 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.098+0000 7fbcb3fff640 1 -- 172.21.15.100:0/3527606373 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559269be5720 con 0x559269f5aba0 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.098+0000 7fbcb3fff640 10 client.28679 handle_command_reply: tid=0 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.098+0000 7fbcddbb9740 1 client.28679 shutdown 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 msgr2=0x559269f47c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f47c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0096790 tx=0x7fbcc0099dc0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 shutdown_connections 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269f5aba0 0x559269f5cfc0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcb3fff640 0 client.28679 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f47c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/3527606373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f47740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 >> 172.21.15.100:0/3527606373 conn(0x559269f3b980 msgr2=0x559269f3d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 shutdown_connections 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 1 -- 172.21.15.100:0/3527606373 wait complete. 2024-03-12T05:30:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.099+0000 7fbcddbb9740 20 client.28679 trim_cache size 0 max 16384 2024-03-12T05:30:17.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 Processor -- start 2024-03-12T05:30:17.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 -- start start 2024-03-12T05:30:17.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f49e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41b70 0x559269f47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f42a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269e2e130 con 0x559269f4a750 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269f621d0 con 0x559269f41250 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269f425c0 con 0x559269f41b70 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f42a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f42a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60144/0 (socket says 172.21.15.100:60144) 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 -- 172.21.15.100:0/6603487 learned_addr learned my addr 172.21.15.100:0/6603487 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41b70 0x559269f47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f49e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 -- 172.21.15.100:0/6603487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41b70 msgr2=0x559269f47fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41b70 0x559269f47fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 -- 172.21.15.100:0/6603487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 msgr2=0x559269f49e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f49e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 -- 172.21.15.100:0/6603487 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559269e43c20 con 0x559269f4a750 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f49e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.112+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f42a90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbccc032650 tx=0x7fbccc032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77da1a85c09fffeb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcbe7fc640 1 -- 172.21.15.100:0/6603487 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc034700 con 0x559269f4a750 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcbe7fc640 1 -- 172.21.15.100:0/6603487 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbccc0326b0 con 0x559269f4a750 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcbe7fc640 1 -- 172.21.15.100:0/6603487 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc004040 con 0x559269f4a750 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 -- 172.21.15.100:0/6603487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 msgr2=0x559269f42a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f42a90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbccc032650 tx=0x7fbccc032e20 comp rx=0 tx=0).stop 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 -- 172.21.15.100:0/6603487 shutdown_connections 2024-03-12T05:30:17.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269f42a90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41b70 0x559269f47fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/6603487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41250 0x559269f49e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 -- 172.21.15.100:0/6603487 >> 172.21.15.100:0/6603487 conn(0x559269f3b980 msgr2=0x559269f36a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 -- 172.21.15.100:0/6603487 shutdown_connections 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.113+0000 7fbcddbb9740 1 -- 172.21.15.100:0/6603487 wait complete. 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 -- start start 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 Processor -- start 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f50e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269fab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269fab5c0 con 0x559269f4a750 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269fab730 con 0x559269f41b70 2024-03-12T05:30:17.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559269fab8f0 con 0x559269f41250 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcddbb9740 10 client.0 _ll_register_callbacks cb 0x7fbcc80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcd8ca9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269fab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcd94aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269fab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60146/0 (socket says 172.21.15.100:60146) 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcd94aa640 1 -- 172.21.15.100:0/2933876036 learned_addr learned my addr 172.21.15.100:0/2933876036 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcd8ca9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43056/0 (socket says 172.21.15.100:43056) 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.114+0000 7fbcd3fff640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f50e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd94aa640 1 -- 172.21.15.100:0/2933876036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 msgr2=0x559269f508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd94aa640 1 -- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 msgr2=0x559269f50e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:17.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f50e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd94aa640 1 -- 172.21.15.100:0/2933876036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd410e830 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcd94aa640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269fab080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006010 tx=0x7fbccc032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846e18819e5fad51 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcb3fff640 1 -- 172.21.15.100:0/2933876036 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc034ea0 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcb3fff640 1 -- 172.21.15.100:0/2933876036 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbccc051570 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcc801dd90 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcb3fff640 1 -- 172.21.15.100:0/2933876036 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbccc051730 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcddbb9740 10 client.19338 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559269e698c0 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.115+0000 7fbcb3fff640 1 -- 172.21.15.100:0/2933876036 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbccc065080 con 0x559269f4a750 2024-03-12T05:30:17.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcb3fff640 1 -- 172.21.15.100:0/2933876036 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbccc0619a0 con 0x559269f4a750 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcddbb9740 10 client.19338 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcddbb9740 10 client.19338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcddbb9740 20 client.19338 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269fb2f50 0x559269fb5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcddbb9740 4 client.19338 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559269c34890 con 0x559269fb2f50 2024-03-12T05:30:17.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.116+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269fb2f50 0x559269fb5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:17.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcd8ca9640 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269fb2f50 0x559269fb5370 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb3a3bbb8b6e770 server_cookie=f5b074435083f42a in_seq=0 out_seq=0 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcb3fff640 10 client.19338 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcb3fff640 1 -- 172.21.15.100:0/2933876036 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559269c34890 con 0x559269fb2f50 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcb3fff640 10 client.19338 handle_command_reply: tid=0 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 client.19338 shutdown 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 msgr2=0x559269fab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269fab080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006010 tx=0x7fbccc032a50 comp rx=0 tx=0).stop 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 shutdown_connections 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x559269fb2f50 0x559269fb5370 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559269f4a750 0x559269fab080 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559269f41b70 0x559269f50e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcb3fff640 0 client.19338 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:17.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 --2- 172.21.15.100:0/2933876036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559269f41250 0x559269f508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 >> 172.21.15.100:0/2933876036 conn(0x559269f3b980 msgr2=0x559269f42a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:17.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 shutdown_connections 2024-03-12T05:30:17.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.117+0000 7fbcddbb9740 1 -- 172.21.15.100:0/2933876036 wait complete. 2024-03-12T05:30:17.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcddbb9740 20 client.19338 trim_cache size 0 max 16384 2024-03-12T05:30:17.119 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbcc40a1ba0 msgr2=0x7fbcc40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbcc40a1ba0 0x7fbcc40a4060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009ce0 tx=0x7fbcc8005f90 comp rx=0 tx=0).stop 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4106450 msgr2=0x7fbcd419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4106450 0x7fbcd419d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e9c0 tx=0x7fbcc000ee80 comp rx=0 tx=0).stop 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 shutdown_connections 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fbcc40a1ba0 0x7fbcc40a4060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4197600 0x7fbcd4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcd4106450 0x7fbcd419d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 --2- 172.21.15.100:0/821794984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcd40ff890 0x7fbcd419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:17.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 >> 172.21.15.100:0/821794984 conn(0x7fbcd40765a0 msgr2=0x7fbcd4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:17.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 shutdown_connections 2024-03-12T05:30:17.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:17.118+0000 7fbcdaf0d640 1 -- 172.21.15.100:0/821794984 wait complete. 2024-03-12T05:30:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 167722 28.03 MB/sec execute 180 sec latency 177.945 ms 2024-03-12T05:30:17.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:17 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3052964706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:30:17.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:17 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3052964706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:30:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:17 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3052964706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:30:18.124 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:30:18.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:30:18.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 -- 172.21.15.100:0/3844102156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460100c10 msgr2=0x7fd460104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 --2- 172.21.15.100:0/3844102156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460100c10 0x7fd460104c50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd454009930 tx=0x7fd45402bc30 comp rx=0 tx=0).stop 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 -- 172.21.15.100:0/3844102156 shutdown_connections 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 --2- 172.21.15.100:0/3844102156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd460105330 0x7fd46010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 --2- 172.21.15.100:0/3844102156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460100c10 0x7fd460104c50 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 --2- 172.21.15.100:0/3844102156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460100260 0x7fd460100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 -- 172.21.15.100:0/3844102156 >> 172.21.15.100:0/3844102156 conn(0x7fd460077180 msgr2=0x7fd460077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.242+0000 7fd465864640 1 -- 172.21.15.100:0/3844102156 shutdown_connections 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.243+0000 7fd465864640 1 -- 172.21.15.100:0/3844102156 wait complete. 2024-03-12T05:30:18.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.243+0000 7fd465864640 1 Processor -- start 2024-03-12T05:30:18.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.243+0000 7fd465864640 1 -- start start 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.243+0000 7fd465864640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460100260 0x7fd46019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 0x7fd46019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd460199850 0x7fd460199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019a300 con 0x7fd460105330 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019a470 con 0x7fd460199850 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019a5e0 con 0x7fd460100260 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 0x7fd46019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 0x7fd46019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60160/0 (socket says 172.21.15.100:60160) 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2341099279 learned_addr learned my addr 172.21.15.100:0/2341099279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45effd640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460100260 0x7fd46019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd460199850 0x7fd460199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2341099279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460100260 msgr2=0x7fd46019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460100260 0x7fd46019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2341099279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd460199850 msgr2=0x7fd460199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd460199850 0x7fd460199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2341099279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009560 con 0x7fd460105330 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 0x7fd46019fc20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd454009230 tx=0x7fd45402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd464862640 1 -- 172.21.15.100:0/2341099279 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454004350 con 0x7fd460105330 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd464862640 1 -- 172.21.15.100:0/2341099279 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4540044b0 con 0x7fd460105330 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4601a6530 con 0x7fd460105330 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd464862640 1 -- 172.21.15.100:0/2341099279 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454003da0 con 0x7fd460105330 2024-03-12T05:30:18.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.244+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4601a6a40 con 0x7fd460105330 2024-03-12T05:30:18.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 168181 28.01 MB/sec execute 181 sec latency 216.474 ms 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.247+0000 7fd464862640 1 -- 172.21.15.100:0/2341099279 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd454004df0 con 0x7fd460105330 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 Processor -- start 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 -- start start 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd464862640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd4480a1c20 0x7fd4480a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x558543660760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585435cef90 con 0x5585436698b0 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585435cc4c0 con 0x558543660d30 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd468433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585435c7ca0 con 0x558543660380 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x558543660760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd4480a1c20 0x7fd4480a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x558543660760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43076/0 (socket says 172.21.15.100:43076) 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 -- 172.21.15.100:0/3371612371 learned_addr learned my addr 172.21.15.100:0/3371612371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd464862640 1 -- 172.21.15.100:0/2341099279 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd4540ef7a0 con 0x7fd460105330 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 -- 172.21.15.100:0/3371612371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 msgr2=0x5585436692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 -- 172.21.15.100:0/3371612371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 msgr2=0x55854366bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45effd640 1 -- 172.21.15.100:0/3371612371 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55854358a0b0 con 0x558543660380 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.248+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd45effd640 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x558543660760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd450009cf0 tx=0x7fd450011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e5aa2cb07aacb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd4480a1c20 0x7fd4480a40e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd458005fd0 tx=0x7fd458009e90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd445ffb640 1 -- 172.21.15.100:0/3371612371 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450004030 con 0x558543660380 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd445ffb640 1 -- 172.21.15.100:0/3371612371 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd450004190 con 0x558543660380 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd445ffb640 1 -- 172.21.15.100:0/3371612371 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450019750 con 0x558543660380 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd468433740 1 -- 172.21.15.100:0/3371612371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 msgr2=0x558543660760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.249+0000 7fd468433740 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x558543660760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd450009cf0 tx=0x7fd450011810 comp rx=0 tx=0).stop 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- 172.21.15.100:0/3371612371 shutdown_connections 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 --2- 172.21.15.100:0/3371612371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x558543660760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- 172.21.15.100:0/3371612371 >> 172.21.15.100:0/3371612371 conn(0x55854365a840 msgr2=0x55854365ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- 172.21.15.100:0/3371612371 shutdown_connections 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- 172.21.15.100:0/3371612371 wait complete. 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- start start 2024-03-12T05:30:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 Processor -- start 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543666780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x558543666cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x558543667200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854356cf40 con 0x558543660d30 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854329b4b0 con 0x558543660380 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.250+0000 7fd468433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854354d670 con 0x5585436698b0 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd468433740 10 client.0 _ll_register_callbacks cb 0x5585435baf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543666780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x558543667200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x558543666cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543666780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:38102/0 (socket says 172.21.15.100:38102) 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x558543666cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60164/0 (socket says 172.21.15.100:60164) 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x558543667200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43090/0 (socket says 172.21.15.100:43090) 2024-03-12T05:30:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45effd640 1 -- 172.21.15.100:0/1228650714 learned_addr learned my addr 172.21.15.100:0/1228650714 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 -- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 msgr2=0x558543666780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543666780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 -- 172.21.15.100:0/1228650714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 msgr2=0x558543666cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x558543666cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 -- 172.21.15.100:0/1228650714 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd450009950 con 0x5585436698b0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x558543666cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x558543667200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd45800f850 tx=0x7fd45801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2e6dd5b3a2597f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd43effd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd43effd640 1 -- 172.21.15.100:0/1228650714 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458016040 con 0x5585436698b0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585435a1f40 con 0x5585436698b0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd43effd640 1 -- 172.21.15.100:0/1228650714 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd458003e00 con 0x5585436698b0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd43effd640 1 -- 172.21.15.100:0/1228650714 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd458004790 con 0x5585436698b0 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd468433740 10 client.28630 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.251+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5585435c6320 con 0x5585436698b0 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd43effd640 1 -- 172.21.15.100:0/1228650714 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5585435a1f40 con 0x5585436698b0 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd43effd640 1 -- 172.21.15.100:0/1228650714 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd458026030 con 0x5585436698b0 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd468433740 10 client.28630 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd468433740 10 client.28630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd468433740 20 client.28630 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd468433740 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558543679e00 0x55854367c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd468433740 4 client.28630 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585435be310 con 0x558543679e00 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd45effd640 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558543679e00 0x55854367c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd45effd640 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558543679e00 0x55854367c220 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272e6eb9161557f7 server_cookie=ca87831e6347e28d in_seq=0 out_seq=0 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.252+0000 7fd43effd640 10 client.28630 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd43effd640 1 -- 172.21.15.100:0/1228650714 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585435be310 con 0x558543679e00 2024-03-12T05:30:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd43effd640 10 client.28630 handle_command_reply: tid=0 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 client.28630 shutdown 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 msgr2=0x558543667200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x558543667200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd45800f850 tx=0x7fd45801d000 comp rx=0 tx=0).stop 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 shutdown_connections 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558543679e00 0x55854367c220 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x558543667200 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd43effd640 0 client.28630 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x558543666cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 --2- 172.21.15.100:0/1228650714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543666780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 >> 172.21.15.100:0/1228650714 conn(0x55854365a840 msgr2=0x55854365c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 shutdown_connections 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 1 -- 172.21.15.100:0/1228650714 wait complete. 2024-03-12T05:30:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.253+0000 7fd468433740 20 client.28630 trim_cache size 0 max 16384 2024-03-12T05:30:18.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 Processor -- start 2024-03-12T05:30:18.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 -- start start 2024-03-12T05:30:18.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543667020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x55854365c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x5585436708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558543661670 con 0x558543660d30 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558543654da0 con 0x558543660380 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd468433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558543664cd0 con 0x5585436698b0 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x55854365c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x55854365c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60180/0 (socket says 172.21.15.100:60180) 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2326789281 learned_addr learned my addr 172.21.15.100:0/2326789281 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45effd640 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543667020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x5585436708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2326789281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 msgr2=0x5585436708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x5585436708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2326789281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 msgr2=0x558543667020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543667020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.270+0000 7fd45e7fc640 1 -- 172.21.15.100:0/2326789281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558543562c00 con 0x558543660d30 2024-03-12T05:30:18.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x55854365c4e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd4540b8900 tx=0x7fd454031c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e428eecce83138 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd4457fa640 1 -- 172.21.15.100:0/2326789281 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4540bbe00 con 0x558543660d30 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd4457fa640 1 -- 172.21.15.100:0/2326789281 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4540d53b0 con 0x558543660d30 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd4457fa640 1 -- 172.21.15.100:0/2326789281 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4540bc4a0 con 0x558543660d30 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd468433740 1 -- 172.21.15.100:0/2326789281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 msgr2=0x55854365c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd468433740 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x55854365c4e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd4540b8900 tx=0x7fd454031c20 comp rx=0 tx=0).stop 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd468433740 1 -- 172.21.15.100:0/2326789281 shutdown_connections 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd468433740 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585436698b0 0x5585436708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.271+0000 7fd468433740 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558543660d30 0x55854365c4e0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 --2- 172.21.15.100:0/2326789281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660380 0x558543667020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- 172.21.15.100:0/2326789281 >> 172.21.15.100:0/2326789281 conn(0x55854365a840 msgr2=0x55854365bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- 172.21.15.100:0/2326789281 shutdown_connections 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- 172.21.15.100:0/2326789281 wait complete. 2024-03-12T05:30:18.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- start start 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 Processor -- start 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x5585436cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558543670590 con 0x5585436698b0 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558543670700 con 0x558543660d30 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585436708c0 con 0x558543660380 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd468433740 10 client.0 _ll_register_callbacks cb 0x7fd4500097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.272+0000 7fd45effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x5585436cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60188/0 (socket says 172.21.15.100:60188) 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 -- 172.21.15.100:0/2802706956 learned_addr learned my addr 172.21.15.100:0/2802706956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x5585436cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43104/0 (socket says 172.21.15.100:43104) 2024-03-12T05:30:18.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45e7fc640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 -- 172.21.15.100:0/2802706956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 msgr2=0x5585436cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x5585436cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 -- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 msgr2=0x5585436cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 -- 172.21.15.100:0/2802706956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4601a6530 con 0x5585436698b0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45effd640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x5585436cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd45f7fe640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366ffd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd4580058d0 tx=0x7fd458011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392a4fc1ad0c1204 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd43effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd43effd640 1 -- 172.21.15.100:0/2802706956 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458016040 con 0x5585436698b0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd43effd640 1 -- 172.21.15.100:0/2802706956 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd458003e00 con 0x5585436698b0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585435ed900 con 0x5585436698b0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd43effd640 1 -- 172.21.15.100:0/2802706956 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd458009a50 con 0x5585436698b0 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd468433740 10 client.19356 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558543588910 con 0x5585436698b0 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.273+0000 7fd43effd640 1 -- 172.21.15.100:0/2802706956 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5585435ed900 con 0x5585436698b0 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd43effd640 1 -- 172.21.15.100:0/2802706956 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd458036070 con 0x5585436698b0 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd468433740 10 client.19356 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd468433740 10 client.19356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd468433740 20 client.19356 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd468433740 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585436d2990 0x5585436d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd468433740 4 client.19356 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5585432c7860 con 0x5585436d2990 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd45effd640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585436d2990 0x5585436d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd45effd640 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585436d2990 0x5585436d4db0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a6c3ab36581cac server_cookie=6062c4ab6732058a in_seq=0 out_seq=0 2024-03-12T05:30:18.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.274+0000 7fd43effd640 10 client.19356 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd43effd640 1 -- 172.21.15.100:0/2802706956 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5585432c7860 con 0x5585436d2990 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd43effd640 10 client.19356 handle_command_reply: tid=0 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 client.19356 shutdown 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 msgr2=0x55854366ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366ffd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd4580058d0 tx=0x7fd458011960 comp rx=0 tx=0).stop 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 shutdown_connections 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5585436d2990 0x5585436d4db0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585436698b0 0x55854366ffd0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd43effd640 0 client.19356 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558543660d30 0x5585436cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 --2- 172.21.15.100:0/2802706956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558543660380 0x5585436cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 >> 172.21.15.100:0/2802706956 conn(0x55854365a840 msgr2=0x5585436598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 shutdown_connections 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.275+0000 7fd468433740 1 -- 172.21.15.100:0/2802706956 wait complete. 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd468433740 20 client.19356 trim_cache size 0 max 16384 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "5d23324a-8173-4d75-9eca-3ba722ed38b9", 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:30:18.276 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:30:18.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd4480a1c20 msgr2=0x7fd4480a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd4480a1c20 0x7fd4480a40e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd458005fd0 tx=0x7fd458009e90 comp rx=0 tx=0).stop 2024-03-12T05:30:18.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 msgr2=0x7fd46019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 0x7fd46019fc20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd454009230 tx=0x7fd45402c680 comp rx=0 tx=0).stop 2024-03-12T05:30:18.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 shutdown_connections 2024-03-12T05:30:18.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd4480a1c20 0x7fd4480a40e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd460199850 0x7fd460199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460105330 0x7fd46019fc20 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 --2- 172.21.15.100:0/2341099279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd460100260 0x7fd46019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 >> 172.21.15.100:0/2341099279 conn(0x7fd460077180 msgr2=0x7fd460068880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 shutdown_connections 2024-03-12T05:30:18.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.276+0000 7fd465864640 1 -- 172.21.15.100:0/2341099279 wait complete. 2024-03-12T05:30:18.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:30:18.281 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.405+0000 7f2fb8de9640 1 -- 172.21.15.100:0/2009096545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 msgr2=0x7f2fb410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.405+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/2009096545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb410db80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009ca0 tx=0x7f2fac02c180 comp rx=0 tx=0).stop 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 -- 172.21.15.100:0/2009096545 shutdown_connections 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/2009096545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb410db80 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/2009096545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb4102cc0 0x7f2fb410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/2009096545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb41023a0 0x7f2fb4102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 -- 172.21.15.100:0/2009096545 >> 172.21.15.100:0/2009096545 conn(0x7f2fb4077230 msgr2=0x7f2fb4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 -- 172.21.15.100:0/2009096545 shutdown_connections 2024-03-12T05:30:18.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 -- 172.21.15.100:0/2009096545 wait complete. 2024-03-12T05:30:18.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 Processor -- start 2024-03-12T05:30:18.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.406+0000 7f2fb8de9640 1 -- start start 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb41023a0 0x7f2fb419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4102cc0 0x7f2fb419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4199ef0 con 0x7f2fb410b6e0 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419a060 con 0x7f2fb4102cc0 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419a220 con 0x7f2fb41023a0 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60206/0 (socket says 172.21.15.100:60206) 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 -- 172.21.15.100:0/1967987997 learned_addr learned my addr 172.21.15.100:0/1967987997 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb259c640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb41023a0 0x7f2fb419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fabfff640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4102cc0 0x7f2fb419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 -- 172.21.15.100:0/1967987997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb41023a0 msgr2=0x7f2fb419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb41023a0 0x7f2fb419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 -- 172.21.15.100:0/1967987997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4102cc0 msgr2=0x7f2fb419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4102cc0 0x7f2fb419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 -- 172.21.15.100:0/1967987997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009900 con 0x7f2fb410b6e0 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb2d9d640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb4199760 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2fac02c790 tx=0x7f2fac034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fab7fe640 1 -- 172.21.15.100:0/1967987997 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003b70 con 0x7f2fb410b6e0 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fab7fe640 1 -- 172.21.15.100:0/1967987997 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fac003cd0 con 0x7f2fb410b6e0 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb419a4a0 con 0x7f2fb410b6e0 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fab7fe640 1 -- 172.21.15.100:0/1967987997 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0046c0 con 0x7f2fb410b6e0 2024-03-12T05:30:18.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.407+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb41a6620 con 0x7f2fb410b6e0 2024-03-12T05:30:18.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.410+0000 7f2fab7fe640 1 -- 172.21.15.100:0/1967987997 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2fac00a040 con 0x7f2fb410b6e0 2024-03-12T05:30:18.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.410+0000 7f2fbb9c1740 1 Processor -- start 2024-03-12T05:30:18.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 -- start start 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b910690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b919270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b91bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b87f280 con 0x56213b9102b0 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b87c7b0 con 0x56213b9197b0 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b877f90 con 0x56213b910bd0 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fab7fe640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fa00a1ba0 0x7f2fa00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b910690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b919270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b910690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60216/0 (socket says 172.21.15.100:60216) 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 -- 172.21.15.100:0/1782706417 learned_addr learned my addr 172.21.15.100:0/1782706417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b919270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:43136/0 (socket says 172.21.15.100:43136) 2024-03-12T05:30:18.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fb2d9d640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fa00a1ba0 0x7f2fa00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b91bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fab7fe640 1 -- 172.21.15.100:0/1967987997 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2fac0f0260 con 0x7f2fb410b6e0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 -- 172.21.15.100:0/1782706417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 msgr2=0x56213b919270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b919270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 -- 172.21.15.100:0/1782706417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 msgr2=0x56213b91bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b91bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fabfff640 1 -- 172.21.15.100:0/1782706417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213b836570 con 0x56213b9102b0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.411+0000 7f2fb259c640 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b919270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.412+0000 7f2fb2d9d640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fa00a1ba0 0x7f2fa00a4060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0b93c0 tx=0x7f2fac0883c0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.412+0000 7f2fabfff640 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b910690 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0137a0 tx=0x7f2f9c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d137a19e26826 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.412+0000 7f2fa8ff9640 1 -- 172.21.15.100:0/1782706417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c013d20 con 0x56213b9102b0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.412+0000 7f2fa8ff9640 1 -- 172.21.15.100:0/1782706417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c004510 con 0x56213b9102b0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fa8ff9640 1 -- 172.21.15.100:0/1782706417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004a00 con 0x56213b9102b0 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/1782706417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 msgr2=0x56213b910690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b910690 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0137a0 tx=0x7f2f9c005f90 comp rx=0 tx=0).stop 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/1782706417 shutdown_connections 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b91bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b919270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/1782706417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b910690 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/1782706417 >> 172.21.15.100:0/1782706417 conn(0x56213b90a990 msgr2=0x56213b90ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/1782706417 shutdown_connections 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/1782706417 wait complete. 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- start start 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 Processor -- start 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b9190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b916460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b9169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b54b430 con 0x56213b9102b0 2024-03-12T05:30:18.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b507e00 con 0x56213b9197b0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b7fcfd0 con 0x56213b910bd0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fbb9c1740 10 client.0 _ll_register_callbacks cb 0x56213b852fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b9190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b9190d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60222/0 (socket says 172.21.15.100:60222) 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.413+0000 7f2fabfff640 1 -- 172.21.15.100:0/3333414396 learned_addr learned my addr 172.21.15.100:0/3333414396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fb259c640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b916460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b9169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 -- 172.21.15.100:0/3333414396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 msgr2=0x56213b916460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b916460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 -- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 msgr2=0x56213b9169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b9169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 -- 172.21.15.100:0/3333414396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c01b040 con 0x56213b9102b0 2024-03-12T05:30:18.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b9190d0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005470 tx=0x7f2f9c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbfb987e95ced97 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2f99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3333414396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0119f0 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3333414396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2f9c011b50 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56213b5777e0 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3333414396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f9c011d10 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fbb9c1740 10 client.19368 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56213b876610 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.414+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3333414396 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56213b5777e0 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3333414396 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f9c0205e0 con 0x56213b9102b0 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fbb9c1740 10 client.19368 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fbb9c1740 10 client.19368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fbb9c1740 20 client.19368 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b92bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fbb9c1740 4 client.19368 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56213b5b4720 con 0x56213b929770 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fb259c640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b92bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2fb259c640 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b92bb90 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7448e308e96b4b2 server_cookie=c0b82beb73b94919 in_seq=0 out_seq=0 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.415+0000 7f2f99ffb640 10 client.19368 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3333414396 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56213b5b4720 con 0x56213b929770 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2f99ffb640 10 client.19368 handle_command_reply: tid=0 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 client.19368 shutdown 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 msgr2=0x56213b9190d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b9190d0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005470 tx=0x7f2f9c005540 comp rx=0 tx=0).stop 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 shutdown_connections 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b92bb90 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b9169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2f99ffb640 0 client.19368 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b916460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3333414396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b9190d0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 >> 172.21.15.100:0/3333414396 conn(0x56213b90a990 msgr2=0x56213b90c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.416+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 shutdown_connections 2024-03-12T05:30:18.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.417+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3333414396 wait complete. 2024-03-12T05:30:18.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.417+0000 7f2fbb9c1740 20 client.19368 trim_cache size 0 max 16384 2024-03-12T05:30:18.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.432+0000 7f2fbb9c1740 1 Processor -- start 2024-03-12T05:30:18.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.432+0000 7f2fbb9c1740 1 -- start start 2024-03-12T05:30:18.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b918b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b9190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b916420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b91c170 con 0x56213b9102b0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b926430 con 0x56213b9197b0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b90fa50 con 0x56213b910bd0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b918b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b918b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60234/0 (socket says 172.21.15.100:60234) 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 -- 172.21.15.100:0/2853860246 learned_addr learned my addr 172.21.15.100:0/2853860246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fb259c640 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b9190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b916420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 -- 172.21.15.100:0/2853860246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 msgr2=0x56213b9190d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b9190d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 -- 172.21.15.100:0/2853860246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 msgr2=0x56213b916420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b916420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 -- 172.21.15.100:0/2853860246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213b812ac0 con 0x56213b9102b0 2024-03-12T05:30:18.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2fabfff640 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b918b90 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c023040 tx=0x7f2f9c020c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71721c34af6cc149 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2f9bfff640 1 -- 172.21.15.100:0/2853860246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c009780 con 0x56213b9102b0 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2f9bfff640 1 -- 172.21.15.100:0/2853860246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c009b70 con 0x56213b9102b0 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.433+0000 7f2f9bfff640 1 -- 172.21.15.100:0/2853860246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c020690 con 0x56213b9102b0 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/2853860246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 msgr2=0x56213b918b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b918b90 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c023040 tx=0x7f2f9c020c40 comp rx=0 tx=0).stop 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/2853860246 shutdown_connections 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9197b0 0x56213b916420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b910bd0 0x56213b9190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/2853860246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b9102b0 0x56213b918b90 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/2853860246 >> 172.21.15.100:0/2853860246 conn(0x56213b90a990 msgr2=0x56213b911770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/2853860246 shutdown_connections 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.434+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/2853860246 wait complete. 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 -- start start 2024-03-12T05:30:18.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 Processor -- start 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9102b0 0x56213b9207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 0x56213b91cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b9197b0 0x56213b91d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b91d760 con 0x56213b910bd0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b91d8d0 con 0x56213b9102b0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213b91da90 con 0x56213b9197b0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fbb9c1740 10 client.0 _ll_register_callbacks cb 0x7f2f9c009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 0x56213b91cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 0x56213b91cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60250/0 (socket says 172.21.15.100:60250) 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 -- 172.21.15.100:0/3459946036 learned_addr learned my addr 172.21.15.100:0/3459946036 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b9197b0 0x56213b91d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9102b0 0x56213b9207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 -- 172.21.15.100:0/3459946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b9197b0 msgr2=0x56213b91d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b9197b0 0x56213b91d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 -- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9102b0 msgr2=0x56213b9207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9102b0 0x56213b9207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 -- 172.21.15.100:0/3459946036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56213b876610 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2fb259c640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 0x56213b91cce0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x56213b92c4a0 tx=0x7f2fa4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68609a979b09e99a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.435+0000 7f2f99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3459946036 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4017040 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3459946036 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2fa4005220 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56213b916290 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3459946036 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa40053e0 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 10 client.19374 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56213b8386a0 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3459946036 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56213b916290 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3459946036 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fa4027510 con 0x56213b910bd0 2024-03-12T05:30:18.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 10 client.19374 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 10 client.19374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 20 client.19374 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b983140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 4 client.19374 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f2f9c004510 con 0x56213b929770 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.436+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b983140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fabfff640 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b983140 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966c97848ceeffa4 server_cookie=148b92035c10563d in_seq=0 out_seq=0 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2f99ffb640 10 client.19374 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2f99ffb640 1 -- 172.21.15.100:0/3459946036 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f2f9c004510 con 0x56213b929770 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2f99ffb640 10 client.19374 handle_command_reply: tid=0 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 client.19374 shutdown 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 msgr2=0x56213b91cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 0x56213b91cce0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x56213b92c4a0 tx=0x7f2fa4010800 comp rx=0 tx=0).stop 2024-03-12T05:30:18.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 shutdown_connections 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56213b929770 0x56213b983140 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56213b9197b0 0x56213b91d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2f99ffb640 0 client.19374 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56213b910bd0 0x56213b91cce0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 --2- 172.21.15.100:0/3459946036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213b9102b0 0x56213b9207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 >> 172.21.15.100:0/3459946036 conn(0x56213b90a990 msgr2=0x56213b916420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 shutdown_connections 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 1 -- 172.21.15.100:0/3459946036 wait complete. 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.437+0000 7f2fbb9c1740 20 client.19374 trim_cache size 0 max 16384 2024-03-12T05:30:18.438 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: "5d23324a-8173-4d75-9eca-3ba722ed38b9": { 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "5d23324a-8173-4d75-9eca-3ba722ed38b9", 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:30:18.439 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fa00a1ba0 msgr2=0x7f2fa00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fa00a1ba0 0x7f2fa00a4060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0b93c0 tx=0x7f2fac0883c0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 msgr2=0x7f2fb4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb4199760 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2fac02c790 tx=0x7f2fac034670 comp rx=0 tx=0).stop 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 shutdown_connections 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2fa00a1ba0 0x7f2fa00a4060 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fb410b6e0 0x7f2fb4199760 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4102cc0 0x7f2fb419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 --2- 172.21.15.100:0/1967987997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb41023a0 0x7f2fb419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 >> 172.21.15.100:0/1967987997 conn(0x7f2fb4077230 msgr2=0x7f2fb41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 shutdown_connections 2024-03-12T05:30:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:18.438+0000 7f2fb8de9640 1 -- 172.21.15.100:0/1967987997 wait complete. 2024-03-12T05:30:18.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:18 smithi100 ceph-mon[32210]: pgmap v573: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-03-12T05:30:18.444 INFO:tasks.cephfs.filesystem:scrub status for tag:5d23324a-8173-4d75-9eca-3ba722ed38b9 - {'path': '/', 'tag': '5d23324a-8173-4d75-9eca-3ba722ed38b9', 'options': 'recursive,force'} 2024-03-12T05:30:18.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:18 smithi156 ceph-mon[36270]: pgmap v573: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-03-12T05:30:18.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:18 smithi161 ceph-mon[36043]: pgmap v573: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-03-12T05:30:19.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 168609 28.00 MB/sec execute 182 sec latency 410.902 ms 2024-03-12T05:30:19.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:30:19.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:19.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:19 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:30:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:19 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:30:19.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:19.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:19.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:19.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:19.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:19.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:19 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 169218 28.03 MB/sec execute 183 sec latency 184.921 ms 2024-03-12T05:30:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:20 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:20 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:20 smithi156 ceph-mon[36270]: pgmap v574: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-03-12T05:30:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:20 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:20 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:20 smithi161 ceph-mon[36043]: pgmap v574: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-03-12T05:30:20.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:20 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:20 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:20.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:20 smithi100 ceph-mon[32210]: pgmap v574: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-03-12T05:30:21.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 169633 28.01 MB/sec execute 184 sec latency 327.633 ms 2024-03-12T05:30:22.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 170052 27.95 MB/sec execute 185 sec latency 398.286 ms 2024-03-12T05:30:22.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:22 smithi156 ceph-mon[36270]: pgmap v575: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 749 op/s 2024-03-12T05:30:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:22 smithi161 ceph-mon[36043]: pgmap v575: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 749 op/s 2024-03-12T05:30:22.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:22 smithi100 ceph-mon[32210]: pgmap v575: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 749 op/s 2024-03-12T05:30:23.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 170576 27.96 MB/sec execute 186 sec latency 165.956 ms 2024-03-12T05:30:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 171057 27.93 MB/sec execute 187 sec latency 144.192 ms 2024-03-12T05:30:24.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:24 smithi156 ceph-mon[36270]: pgmap v576: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 517 op/s 2024-03-12T05:30:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:24 smithi161 ceph-mon[36043]: pgmap v576: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 517 op/s 2024-03-12T05:30:24.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:24 smithi100 ceph-mon[32210]: pgmap v576: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 517 op/s 2024-03-12T05:30:25.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 171633 27.97 MB/sec execute 188 sec latency 145.282 ms 2024-03-12T05:30:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 171999 27.94 MB/sec execute 189 sec latency 173.299 ms 2024-03-12T05:30:26.365 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:26 smithi100 ceph-mon[32210]: pgmap v577: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 493 op/s 2024-03-12T05:30:26.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:26 smithi156 ceph-mon[36270]: pgmap v577: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 493 op/s 2024-03-12T05:30:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:26 smithi161 ceph-mon[36043]: pgmap v577: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 493 op/s 2024-03-12T05:30:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 172351 27.86 MB/sec execute 190 sec latency 313.945 ms 2024-03-12T05:30:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 172950 27.90 MB/sec execute 191 sec latency 164.986 ms 2024-03-12T05:30:28.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:28 smithi156 ceph-mon[36270]: pgmap v578: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-03-12T05:30:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:28 smithi161 ceph-mon[36043]: pgmap v578: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-03-12T05:30:28.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:28 smithi100 ceph-mon[32210]: pgmap v578: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-03-12T05:30:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 173562 27.95 MB/sec execute 192 sec latency 145.060 ms 2024-03-12T05:30:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 174017 27.95 MB/sec execute 193 sec latency 155.939 ms 2024-03-12T05:30:30.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:30 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:30:30.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:30 smithi100 ceph-mon[32210]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.5 MiB/s wr, 475 op/s 2024-03-12T05:30:30.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:30 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:30:30.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:30 smithi156 ceph-mon[36270]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.5 MiB/s wr, 475 op/s 2024-03-12T05:30:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:30 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:30:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:30 smithi161 ceph-mon[36043]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.5 MiB/s wr, 475 op/s 2024-03-12T05:30:31.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 174427 27.89 MB/sec execute 194 sec latency 134.217 ms 2024-03-12T05:30:31.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:31 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:30:31.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:31 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:30:31.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:31 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:30:31.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:31 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:30:31.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:31 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:30:31.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:31 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:30:32.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 174840 27.81 MB/sec execute 195 sec latency 204.262 ms 2024-03-12T05:30:32.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:32 smithi100 ceph-mon[32210]: pgmap v580: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 657 op/s 2024-03-12T05:30:32.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:32 smithi156 ceph-mon[36270]: pgmap v580: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 657 op/s 2024-03-12T05:30:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:32 smithi161 ceph-mon[36043]: pgmap v580: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 657 op/s 2024-03-12T05:30:33.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 175327 27.85 MB/sec execute 196 sec latency 202.633 ms 2024-03-12T05:30:33.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:33 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:30:33.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:33 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:30:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:33 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:30:34.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 175808 27.85 MB/sec execute 197 sec latency 170.049 ms 2024-03-12T05:30:34.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:34 smithi100 ceph-mon[32210]: pgmap v581: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.5 MiB/s wr, 425 op/s 2024-03-12T05:30:34.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:34 smithi156 ceph-mon[36270]: pgmap v581: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.5 MiB/s wr, 425 op/s 2024-03-12T05:30:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:34 smithi161 ceph-mon[36043]: pgmap v581: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.5 MiB/s wr, 425 op/s 2024-03-12T05:30:35.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 176317 27.86 MB/sec execute 198 sec latency 157.972 ms 2024-03-12T05:30:35.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:35 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:30:35.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:35 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:30:35.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:35 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:30:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:35 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:30:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:35 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:30:35.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:35 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:30:35.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:35 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:30:35.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:35 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:30:35.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:35 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:30:36.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 176712 27.77 MB/sec execute 199 sec latency 174.503 ms 2024-03-12T05:30:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:36 smithi156 ceph-mon[36270]: pgmap v582: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.8 MiB/s wr, 398 op/s 2024-03-12T05:30:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:36 smithi100 ceph-mon[32210]: pgmap v582: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.8 MiB/s wr, 398 op/s 2024-03-12T05:30:36.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:36 smithi161 ceph-mon[36043]: pgmap v582: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.8 MiB/s wr, 398 op/s 2024-03-12T05:30:37.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 177129 27.74 MB/sec execute 200 sec latency 242.516 ms 2024-03-12T05:30:38.240 INFO:tasks.workunit.client.0.smithi100.stdout: 10 177673 27.78 MB/sec execute 201 sec latency 152.986 ms 2024-03-12T05:30:38.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:38 smithi156 ceph-mon[36270]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-03-12T05:30:38.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:38 smithi100 ceph-mon[32210]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-03-12T05:30:38.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:38 smithi161 ceph-mon[36043]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-03-12T05:30:39.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 178176 27.78 MB/sec execute 202 sec latency 140.172 ms 2024-03-12T05:30:40.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 178784 27.79 MB/sec execute 203 sec latency 128.160 ms 2024-03-12T05:30:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:40 smithi156 ceph-mon[36270]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.2 MiB/s wr, 361 op/s 2024-03-12T05:30:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:40 smithi161 ceph-mon[36043]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.2 MiB/s wr, 361 op/s 2024-03-12T05:30:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:40 smithi100 ceph-mon[32210]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.2 MiB/s wr, 361 op/s 2024-03-12T05:30:41.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 179299 27.82 MB/sec execute 204 sec latency 160.931 ms 2024-03-12T05:30:42.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 179623 27.76 MB/sec execute 205 sec latency 271.333 ms 2024-03-12T05:30:42.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:42 smithi156 ceph-mon[36270]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 529 op/s 2024-03-12T05:30:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:42 smithi161 ceph-mon[36043]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 529 op/s 2024-03-12T05:30:42.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:42 smithi100 ceph-mon[32210]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 529 op/s 2024-03-12T05:30:43.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 180010 27.74 MB/sec execute 206 sec latency 209.967 ms 2024-03-12T05:30:44.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 180423 27.68 MB/sec execute 207 sec latency 235.344 ms 2024-03-12T05:30:44.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:44 smithi156 ceph-mon[36270]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.0 MiB/s wr, 380 op/s 2024-03-12T05:30:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:44 smithi161 ceph-mon[36043]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.0 MiB/s wr, 380 op/s 2024-03-12T05:30:44.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:44 smithi100 ceph-mon[32210]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.0 MiB/s wr, 380 op/s 2024-03-12T05:30:45.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 180925 27.70 MB/sec execute 208 sec latency 139.591 ms 2024-03-12T05:30:46.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 181395 27.64 MB/sec execute 209 sec latency 162.235 ms 2024-03-12T05:30:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:46 smithi156 ceph-mon[36270]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.8 MiB/s wr, 367 op/s 2024-03-12T05:30:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:46 smithi161 ceph-mon[36043]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.8 MiB/s wr, 367 op/s 2024-03-12T05:30:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:46 smithi100 ceph-mon[32210]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.8 MiB/s wr, 367 op/s 2024-03-12T05:30:47.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 181802 27.62 MB/sec execute 210 sec latency 399.400 ms 2024-03-12T05:30:48.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 182453 27.67 MB/sec execute 211 sec latency 136.772 ms 2024-03-12T05:30:48.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:30:48.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:30:48.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:48 smithi156 ceph-mon[36270]: pgmap v588: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 581 op/s 2024-03-12T05:30:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:48 smithi161 ceph-mon[36043]: pgmap v588: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 581 op/s 2024-03-12T05:30:48.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:48 smithi100 ceph-mon[32210]: pgmap v588: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 581 op/s 2024-03-12T05:30:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 -- 172.21.15.100:0/107365155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0584105370 msgr2=0x7f058410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 --2- 172.21.15.100:0/107365155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0584105370 0x7f058410cf40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f057c009cd0 tx=0x7f057c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 -- 172.21.15.100:0/107365155 shutdown_connections 2024-03-12T05:30:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 --2- 172.21.15.100:0/107365155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0584105370 0x7f058410cf40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 --2- 172.21.15.100:0/107365155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0584100c50 0x7f0584104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 --2- 172.21.15.100:0/107365155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05841002a0 0x7f0584100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 -- 172.21.15.100:0/107365155 >> 172.21.15.100:0/107365155 conn(0x7f0584077210 msgr2=0x7f0584077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 -- 172.21.15.100:0/107365155 shutdown_connections 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.565+0000 7f05888d0640 1 -- 172.21.15.100:0/107365155 wait complete. 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 Processor -- start 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 -- start start 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05841002a0 0x7f058410d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 0x7f058410dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f058410e2a0 0x7f05841a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a5ed0 con 0x7f058410e2a0 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a6040 con 0x7f0584105370 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f05888d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a6200 con 0x7f05841002a0 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f058410e2a0 0x7f05841a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 0x7f058410dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 0x7f058410dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40274/0 (socket says 172.21.15.100:40274) 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0581d9b640 1 -- 172.21.15.100:0/2056903171 learned_addr learned my addr 172.21.15.100:0/2056903171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f058259c640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05841002a0 0x7f058410d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 -- 172.21.15.100:0/2056903171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05841002a0 msgr2=0x7f058410d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05841002a0 0x7f058410d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 -- 172.21.15.100:0/2056903171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 msgr2=0x7f058410dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 0x7f058410dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 -- 172.21.15.100:0/2056903171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c009900 con 0x7f058410e2a0 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0581d9b640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 0x7f058410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.566+0000 7f0582d9d640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f058410e2a0 0x7f05841a5900 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f057c037040 tx=0x7f057c034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.567+0000 7f05737fe640 1 -- 172.21.15.100:0/2056903171 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0328b0 con 0x7f058410e2a0 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.567+0000 7f05737fe640 1 -- 172.21.15.100:0/2056903171 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f057c003e50 con 0x7f058410e2a0 2024-03-12T05:30:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.567+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05841a6480 con 0x7f058410e2a0 2024-03-12T05:30:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.567+0000 7f05737fe640 1 -- 172.21.15.100:0/2056903171 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c004780 con 0x7f058410e2a0 2024-03-12T05:30:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.567+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05841a6950 con 0x7f058410e2a0 2024-03-12T05:30:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.569+0000 7f05737fe640 1 -- 172.21.15.100:0/2056903171 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f057c00a040 con 0x7f058410e2a0 2024-03-12T05:30:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.569+0000 7f058b577740 1 Processor -- start 2024-03-12T05:30:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.569+0000 7f058b577740 1 -- start start 2024-03-12T05:30:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058b577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba39690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058b577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba39bd0 0x56146ba42270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058b577740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba427b0 0x56146ba44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058b577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146b9a8280 con 0x56146ba392b0 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058b577740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146b9a57b0 con 0x56146ba427b0 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058b577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146b9a0f90 con 0x56146ba39bd0 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba39690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba39690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56058/0 (socket says 172.21.15.100:56058) 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 -- 172.21.15.100:0/1356018178 learned_addr learned my addr 172.21.15.100:0/1356018178 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f058259c640 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba39bd0 0x56146ba42270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba427b0 0x56146ba44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f05737fe640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f056c0a1c50 0x7f056c0a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 -- 172.21.15.100:0/1356018178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba39bd0 msgr2=0x56146ba42270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba39bd0 0x56146ba42270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 -- 172.21.15.100:0/1356018178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba427b0 msgr2=0x56146ba44ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba427b0 0x56146ba44ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 -- 172.21.15.100:0/1356018178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56146b95f570 con 0x56146ba392b0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0582d9d640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f056c0a1c50 0x7f056c0a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0581d9b640 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba39690 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f0578012b90 tx=0x7f057800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6569236e205f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f05737fe640 1 -- 172.21.15.100:0/2056903171 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f057c0f02b0 con 0x7f058410e2a0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.570+0000 7f0570ff9640 1 -- 172.21.15.100:0/1356018178 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578012c80 con 0x56146ba392b0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f0570ff9640 1 -- 172.21.15.100:0/1356018178 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0578012de0 con 0x56146ba392b0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f0570ff9640 1 -- 172.21.15.100:0/1356018178 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801d370 con 0x56146ba392b0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 -- 172.21.15.100:0/1356018178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 msgr2=0x56146ba39690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba39690 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f0578012b90 tx=0x7f057800bb90 comp rx=0 tx=0).stop 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f0582d9d640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f056c0a1c50 0x7f056c0a4110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f057c009430 tx=0x7f057c08f2d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 -- 172.21.15.100:0/1356018178 shutdown_connections 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba427b0 0x56146ba44ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba39bd0 0x56146ba42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 --2- 172.21.15.100:0/1356018178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba39690 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 -- 172.21.15.100:0/1356018178 >> 172.21.15.100:0/1356018178 conn(0x56146ba33990 msgr2=0x56146ba33d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 -- 172.21.15.100:0/1356018178 shutdown_connections 2024-03-12T05:30:48.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.571+0000 7f058b577740 1 -- 172.21.15.100:0/1356018178 wait complete. 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 -- start start 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 Processor -- start 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba3f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba427b0 0x56146ba3fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146b674430 con 0x56146ba392b0 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146b630e00 con 0x56146ba39bd0 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146b925fd0 con 0x56146ba427b0 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058b577740 10 client.0 _ll_register_callbacks cb 0x56146b97bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba3f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56070/0 (socket says 172.21.15.100:56070) 2024-03-12T05:30:48.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f0581d9b640 1 -- 172.21.15.100:0/809901382 learned_addr learned my addr 172.21.15.100:0/809901382 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f0581d9b640 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba427b0 0x56146ba3fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058259c640 1 -- 172.21.15.100:0/809901382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba427b0 msgr2=0x56146ba3fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058259c640 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba427b0 0x56146ba3fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058259c640 1 -- 172.21.15.100:0/809901382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 msgr2=0x56146ba421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.572+0000 7f058259c640 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f058259c640 1 -- 172.21.15.100:0/809901382 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057801a040 con 0x56146ba39bd0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f058259c640 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba3f5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f05740126d0 tx=0x7f0574012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e254f840e7a8cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f0569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f0569ffb640 1 -- 172.21.15.100:0/809901382 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574017070 con 0x56146ba39bd0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56146b6a07e0 con 0x56146ba39bd0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f0569ffb640 1 -- 172.21.15.100:0/809901382 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f057400f8c0 con 0x56146ba39bd0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f0569ffb640 1 -- 172.21.15.100:0/809901382 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057400fa80 con 0x56146ba39bd0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f058b577740 10 client.28733 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56146b99f610 con 0x56146ba39bd0 2024-03-12T05:30:48.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.573+0000 7f0569ffb640 1 -- 172.21.15.100:0/809901382 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0574027020 con 0x56146ba39bd0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f0569ffb640 1 -- 172.21.15.100:0/809901382 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f057401c420 con 0x56146ba39bd0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f058b577740 10 client.28733 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f058b577740 10 client.28733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f058b577740 20 client.28733 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f058b577740 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146ba528e0 0x56146ba54d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f058b577740 4 client.28733 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56146b6dd720 con 0x56146ba528e0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f0581d9b640 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146ba528e0 0x56146ba54d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f0569ffb640 10 client.28733 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.574+0000 7f0581d9b640 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146ba528e0 0x56146ba54d00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ad234de51e70b8 server_cookie=20cb9e2f2cce6b86 in_seq=0 out_seq=0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f0569ffb640 1 -- 172.21.15.100:0/809901382 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56146b6dd720 con 0x56146ba528e0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f0569ffb640 10 client.28733 handle_command_reply: tid=0 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 client.28733 shutdown 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 msgr2=0x56146ba3f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba3f5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f05740126d0 tx=0x7f0574012b90 comp rx=0 tx=0).stop 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 shutdown_connections 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146ba528e0 0x56146ba54d00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba427b0 0x56146ba3fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba3f5a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 --2- 172.21.15.100:0/809901382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba392b0 0x56146ba421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f0569ffb640 0 client.28733 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 >> 172.21.15.100:0/809901382 conn(0x56146ba33990 msgr2=0x56146ba35730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 shutdown_connections 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 1 -- 172.21.15.100:0/809901382 wait complete. 2024-03-12T05:30:48.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.575+0000 7f058b577740 20 client.28733 trim_cache size 0 max 16384 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 Processor -- start 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 -- start start 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba49b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba3c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146ba4f420 con 0x56146ba427b0 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146ba2de20 con 0x56146ba39bd0 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058b577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146ba38f10 con 0x56146ba392b0 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba3c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f058259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba49b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba3c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56076/0 (socket says 172.21.15.100:56076) 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f058259c640 1 -- 172.21.15.100:0/3013081543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 msgr2=0x56146ba495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f058259c640 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f058259c640 1 -- 172.21.15.100:0/3013081543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 msgr2=0x56146ba3c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f058259c640 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba3c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f058259c640 1 -- 172.21.15.100:0/3013081543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56146b93bac0 con 0x56146ba39bd0 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f058259c640 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba49b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0574009e00 tx=0x7f05740096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=179a4ca8c7d4fdb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.592+0000 7f0581d9b640 1 -- 172.21.15.100:0/3013081543 learned_addr learned my addr 172.21.15.100:0/3013081543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f0581d9b640 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba3c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f056bfff640 1 -- 172.21.15.100:0/3013081543 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574017070 con 0x56146ba39bd0 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f056bfff640 1 -- 172.21.15.100:0/3013081543 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0574027070 con 0x56146ba39bd0 2024-03-12T05:30:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.593+0000 7f056bfff640 1 -- 172.21.15.100:0/3013081543 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400f790 con 0x56146ba39bd0 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 -- 172.21.15.100:0/3013081543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 msgr2=0x56146ba49b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba49b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0574009e00 tx=0x7f05740096c0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 -- 172.21.15.100:0/3013081543 shutdown_connections 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba3c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba49b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 --2- 172.21.15.100:0/3013081543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 -- 172.21.15.100:0/3013081543 >> 172.21.15.100:0/3013081543 conn(0x56146ba33990 msgr2=0x56146ba3deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 -- 172.21.15.100:0/3013081543 shutdown_connections 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 -- 172.21.15.100:0/3013081543 wait complete. 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 -- start start 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.594+0000 7f058b577740 1 Processor -- start 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba46a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba46fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146ba474f0 con 0x56146ba427b0 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146ba47660 con 0x56146ba39bd0 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146ba47820 con 0x56146ba392b0 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058b577740 10 client.0 _ll_register_callbacks cb 0x7f057800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba46a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba46fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba46a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40322/0 (socket says 172.21.15.100:40322) 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058259c640 1 -- 172.21.15.100:0/2310695800 learned_addr learned my addr 172.21.15.100:0/2310695800 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 -- 172.21.15.100:0/2310695800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 msgr2=0x56146ba496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 -- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 msgr2=0x56146ba46a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba46a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 -- 172.21.15.100:0/2310695800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56146b99f610 con 0x56146ba427b0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f058259c640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba46a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0581d9b640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba46fb0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x56146ba555f0 tx=0x7f0578012c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c83b8c405b8e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.595+0000 7f0569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f0569ffb640 1 -- 172.21.15.100:0/2310695800 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801f7a0 con 0x56146ba427b0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f0569ffb640 1 -- 172.21.15.100:0/2310695800 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0578037060 con 0x56146ba427b0 2024-03-12T05:30:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56146ba4edb0 con 0x56146ba427b0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f0569ffb640 1 -- 172.21.15.100:0/2310695800 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057801f900 con 0x56146ba427b0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 10 client.19389 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56146b9616a0 con 0x56146ba427b0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f0569ffb640 1 -- 172.21.15.100:0/2310695800 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f057803e020 con 0x56146ba427b0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f0569ffb640 1 -- 172.21.15.100:0/2310695800 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f057802d040 con 0x56146ba427b0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 10 client.19389 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 10 client.19389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 20 client.19389 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146bab39e0 0x56146bab5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 4 client.19389 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f0578012de0 con 0x56146bab39e0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.596+0000 7f058259c640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146bab39e0 0x56146bab5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f0569ffb640 10 client.19389 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058259c640 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146bab39e0 0x56146bab5e00 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f83fc33063ece5 server_cookie=7f91cb947b5424e in_seq=0 out_seq=0 2024-03-12T05:30:48.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f0569ffb640 1 -- 172.21.15.100:0/2310695800 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f0578012de0 con 0x56146bab39e0 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f0569ffb640 10 client.19389 handle_command_reply: tid=0 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 client.19389 shutdown 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 msgr2=0x56146ba46fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba46fb0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x56146ba555f0 tx=0x7f0578012c80 comp rx=0 tx=0).stop 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 shutdown_connections 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56146bab39e0 0x56146bab5e00 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56146ba427b0 0x56146ba46fb0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56146ba39bd0 0x56146ba46a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.597+0000 7f058b577740 1 --2- 172.21.15.100:0/2310695800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146ba392b0 0x56146ba496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 >> 172.21.15.100:0/2310695800 conn(0x56146ba33990 msgr2=0x56146ba321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f0569ffb640 0 client.19389 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 shutdown_connections 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f058b577740 1 -- 172.21.15.100:0/2310695800 wait complete. 2024-03-12T05:30:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f058b577740 20 client.19389 trim_cache size 0 max 16384 2024-03-12T05:30:48.599 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:30:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:30:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:30:48.599 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f056c0a1c50 msgr2=0x7f056c0a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f056c0a1c50 0x7f056c0a4110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f057c009430 tx=0x7f057c08f2d0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f058410e2a0 msgr2=0x7f05841a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f058410e2a0 0x7f05841a5900 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f057c037040 tx=0x7f057c034a90 comp rx=0 tx=0).stop 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 shutdown_connections 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f056c0a1c50 0x7f056c0a4110 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f058410e2a0 0x7f05841a5900 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0584105370 0x7f058410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 --2- 172.21.15.100:0/2056903171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05841002a0 0x7f058410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.598+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 >> 172.21.15.100:0/2056903171 conn(0x7f0584077210 msgr2=0x7f05840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.599+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 shutdown_connections 2024-03-12T05:30:48.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.599+0000 7f05888d0640 1 -- 172.21.15.100:0/2056903171 wait complete. 2024-03-12T05:30:48.605 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:30:48.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 -- 172.21.15.100:0/3677838339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 msgr2=0x7f06ac1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3677838339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac1006d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f069c009930 tx=0x7f069c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 -- 172.21.15.100:0/3677838339 shutdown_connections 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3677838339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06ac1053c0 0x7f06ac10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3677838339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ac100ca0 0x7f06ac104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3677838339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac1006d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 -- 172.21.15.100:0/3677838339 >> 172.21.15.100:0/3677838339 conn(0x7f06ac077180 msgr2=0x7f06ac077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 -- 172.21.15.100:0/3677838339 shutdown_connections 2024-03-12T05:30:48.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 -- 172.21.15.100:0/3677838339 wait complete. 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 Processor -- start 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.763+0000 7f06b18b5640 1 -- start start 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06b18b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06b18b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ac1053c0 0x7f06ac198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06b18b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06ac1991e0 0x7f06ac1a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06b18b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a3e30 con 0x7f06ac1002f0 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06b18b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a3fa0 con 0x7f06ac1991e0 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06b18b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a4160 con 0x7f06ac1053c0 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56100/0 (socket says 172.21.15.100:56100) 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 -- 172.21.15.100:0/3886263140 learned_addr learned my addr 172.21.15.100:0/3886263140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06ab7fe640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06ac1991e0 0x7f06ac1a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06a27fc640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ac1053c0 0x7f06ac198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 -- 172.21.15.100:0/3886263140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ac1053c0 msgr2=0x7f06ac198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ac1053c0 0x7f06ac198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 -- 172.21.15.100:0/3886263140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06ac1991e0 msgr2=0x7f06ac1a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06ac1991e0 0x7f06ac1a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 -- 172.21.15.100:0/3886263140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c009560 con 0x7f06ac1002f0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.764+0000 7f06aaffd640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac198710 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f069c000c00 tx=0x7f069c02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.765+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0042d0 con 0x7f06ac1002f0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.765+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac1a43e0 con 0x7f06ac1002f0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.765+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069c004430 con 0x7f06ac1002f0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.765+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c003c20 con 0x7f06ac1002f0 2024-03-12T05:30:48.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.765+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac1a4940 con 0x7f06ac1002f0 2024-03-12T05:30:48.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.766+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac067f70 con 0x7f06ac1002f0 2024-03-12T05:30:48.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.767+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f069c004df0 con 0x7f06ac1002f0 2024-03-12T05:30:48.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.767+0000 7f06b08b3640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06980a1ba0 0x7f06980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:48.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.768+0000 7f06a27fc640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06980a1ba0 0x7f06980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:48.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.768+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f069c0f0120 con 0x7f06ac1002f0 2024-03-12T05:30:48.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.768+0000 7f06a27fc640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06980a1ba0 0x7f06980a4060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f06ac068e30 tx=0x7f0694005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:48.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:48.771+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f069c0b91e0 con 0x7f06ac1002f0 2024-03-12T05:30:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.016+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06ac1085f0 con 0x7f06ac1002f0 2024-03-12T05:30:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.017+0000 7f06b08b3640 1 -- 172.21.15.100:0/3886263140 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f069c043020 con 0x7f06ac1002f0 2024-03-12T05:30:49.012 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:30:49.012 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:30:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06980a1ba0 msgr2=0x7f06980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06980a1ba0 0x7f06980a4060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f06ac068e30 tx=0x7f0694005f90 comp rx=0 tx=0).stop 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 msgr2=0x7f06ac198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac198710 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f069c000c00 tx=0x7f069c02c5f0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 shutdown_connections 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f06980a1ba0 0x7f06980a4060 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06ac1991e0 0x7f06ac1a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ac1053c0 0x7f06ac198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 --2- 172.21.15.100:0/3886263140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac1002f0 0x7f06ac198710 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 >> 172.21.15.100:0/3886263140 conn(0x7f06ac077180 msgr2=0x7f06ac0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 shutdown_connections 2024-03-12T05:30:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.019+0000 7f06b18b5640 1 -- 172.21.15.100:0/3886263140 wait complete. 2024-03-12T05:30:49.025 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:30:49.026 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:30:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 -- 172.21.15.100:0/1020560259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100c10 msgr2=0x7f4748104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 --2- 172.21.15.100:0/1020560259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100c10 0x7f4748104c50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f4730009930 tx=0x7f473002bc30 comp rx=0 tx=0).stop 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 -- 172.21.15.100:0/1020560259 shutdown_connections 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 --2- 172.21.15.100:0/1020560259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748105330 0x7f474810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 --2- 172.21.15.100:0/1020560259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100c10 0x7f4748104c50 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 --2- 172.21.15.100:0/1020560259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4748100260 0x7f4748100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 -- 172.21.15.100:0/1020560259 >> 172.21.15.100:0/1020560259 conn(0x7f4748077180 msgr2=0x7f4748077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 -- 172.21.15.100:0/1020560259 shutdown_connections 2024-03-12T05:30:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 -- 172.21.15.100:0/1020560259 wait complete. 2024-03-12T05:30:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.146+0000 7f474f24d640 1 Processor -- start 2024-03-12T05:30:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 -- start start 2024-03-12T05:30:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 0x7f4748198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4748105330 0x7f4748198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47481990f0 0x7f47481a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481a3d70 con 0x7f4748100260 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481a3ee0 con 0x7f47481990f0 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474f24d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481a40a0 con 0x7f4748105330 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 0x7f4748198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 0x7f4748198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56108/0 (socket says 172.21.15.100:56108) 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474d7ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47481990f0 0x7f47481a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f4747fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4748105330 0x7f4748198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 -- 172.21.15.100:0/68513972 learned_addr learned my addr 172.21.15.100:0/68513972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 -- 172.21.15.100:0/68513972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4748105330 msgr2=0x7f4748198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4748105330 0x7f4748198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 -- 172.21.15.100:0/68513972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47481990f0 msgr2=0x7f47481a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47481990f0 0x7f47481a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 -- 172.21.15.100:0/68513972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730009560 con 0x7f4748100260 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f474cfe9640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 0x7f4748198620 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f474000eb90 tx=0x7f474000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.147+0000 7f4745ffb640 1 -- 172.21.15.100:0/68513972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400099b0 con 0x7f4748100260 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.148+0000 7f4745ffb640 1 -- 172.21.15.100:0/68513972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f474000ee90 con 0x7f4748100260 2024-03-12T05:30:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.148+0000 7f4745ffb640 1 -- 172.21.15.100:0/68513972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400107a0 con 0x7f4748100260 2024-03-12T05:30:49.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.148+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47481a4380 con 0x7f4748100260 2024-03-12T05:30:49.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.148+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481a4810 con 0x7f4748100260 2024-03-12T05:30:49.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.150+0000 7f4745ffb640 1 -- 172.21.15.100:0/68513972 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4740005020 con 0x7f4748100260 2024-03-12T05:30:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4745ffb640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47340a1ba0 0x7f47340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 Processor -- start 2024-03-12T05:30:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4745ffb640 1 -- 172.21.15.100:0/68513972 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4740014070 con 0x7f4748100260 2024-03-12T05:30:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4747fff640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47340a1ba0 0x7f47340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 -- start start 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f7d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f7db70 0x55d5c2f86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 0x55d5c2f88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2f38960 con 0x55d5c2f86750 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2eec310 con 0x55d5c2f7d250 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4751e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2ee9840 con 0x55d5c2f7db70 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474d7ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 0x55d5c2f88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f7d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474d7ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 0x55d5c2f88b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56110/0 (socket says 172.21.15.100:56110) 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474d7ea640 1 -- 172.21.15.100:0/1258588194 learned_addr learned my addr 172.21.15.100:0/1258588194 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4747fff640 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f7db70 0x55d5c2f86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f4747fff640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47340a1ba0 0x7f47340a4060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f473002c170 tx=0x7f4730000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474cfe9640 1 -- 172.21.15.100:0/1258588194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f7db70 msgr2=0x55d5c2f86210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474cfe9640 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f7db70 0x55d5c2f86210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474cfe9640 1 -- 172.21.15.100:0/1258588194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 msgr2=0x55d5c2f88b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474cfe9640 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 0x55d5c2f88b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474cfe9640 1 -- 172.21.15.100:0/1258588194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c2ea7060 con 0x55d5c2f7d250 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.151+0000 7f474d7ea640 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 0x55d5c2f88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.152+0000 7f474cfe9640 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f7d630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f47400cb720 tx=0x7f474009a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c08b75d10710ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.152+0000 7f473effd640 1 -- 172.21.15.100:0/1258588194 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400a33a0 con 0x55d5c2f7d250 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.152+0000 7f473effd640 1 -- 172.21.15.100:0/1258588194 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4740098f00 con 0x55d5c2f7d250 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.152+0000 7f473effd640 1 -- 172.21.15.100:0/1258588194 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474009a250 con 0x55d5c2f7d250 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- 172.21.15.100:0/1258588194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 msgr2=0x55d5c2f7d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f7d630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f47400cb720 tx=0x7f474009a620 comp rx=0 tx=0).stop 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- 172.21.15.100:0/1258588194 shutdown_connections 2024-03-12T05:30:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f86750 0x55d5c2f88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f7db70 0x55d5c2f86210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1258588194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f7d630 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- 172.21.15.100:0/1258588194 >> 172.21.15.100:0/1258588194 conn(0x55d5c2f77980 msgr2=0x55d5c2f77d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- 172.21.15.100:0/1258588194 shutdown_connections 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- 172.21.15.100:0/1258588194 wait complete. 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- start start 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 Processor -- start 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2ee5020 con 0x55d5c2f7d250 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2bb8430 con 0x55d5c2f7db70 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.153+0000 7f4751e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2b74e00 con 0x55d5c2f86750 2024-03-12T05:30:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4751e1b740 10 client.0 _ll_register_callbacks cb 0x55d5c2e8ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f837a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56116/0 (socket says 172.21.15.100:56116) 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 -- 172.21.15.100:0/3230148600 learned_addr learned my addr 172.21.15.100:0/3230148600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4747fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474d7ea640 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 -- 172.21.15.100:0/3230148600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 msgr2=0x55d5c2f84220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 -- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 msgr2=0x55d5c2f83ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 -- 172.21.15.100:0/3230148600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481a4810 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f474cfe9640 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f837a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f47400cbd20 tx=0x7f474009ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4448f1b210c54018 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4727fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4727fff640 1 -- 172.21.15.100:0/3230148600 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400b9070 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4727fff640 1 -- 172.21.15.100:0/3230148600 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f47400a33a0 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5c2be47e0 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4727fff640 1 -- 172.21.15.100:0/3230148600 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4740097450 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4751e1b740 10 client.19407 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d5c2ee36a0 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.154+0000 7f4727fff640 1 -- 172.21.15.100:0/3230148600 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5c2be47e0 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4727fff640 1 -- 172.21.15.100:0/3230148600 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f474001f060 con 0x55d5c2f7d250 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4751e1b740 10 client.19407 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4751e1b740 10 client.19407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4751e1b740 20 client.19407 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4751e1b740 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2f966d0 0x55d5c2f98ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4751e1b740 4 client.19407 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5c2c21720 con 0x55d5c2f966d0 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4747fff640 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2f966d0 0x55d5c2f98ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4727fff640 10 client.19407 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.155+0000 7f4747fff640 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2f966d0 0x55d5c2f98ac0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2fa5eae5ad56eb8 server_cookie=ee0b8066320b15cb in_seq=0 out_seq=0 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4727fff640 1 -- 172.21.15.100:0/3230148600 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5c2c21720 con 0x55d5c2f966d0 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4727fff640 10 client.19407 handle_command_reply: tid=0 2024-03-12T05:30:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 client.19407 shutdown 2024-03-12T05:30:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 msgr2=0x55d5c2f837a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f837a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f47400cbd20 tx=0x7f474009ee10 comp rx=0 tx=0).stop 2024-03-12T05:30:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 shutdown_connections 2024-03-12T05:30:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2f966d0 0x55d5c2f98ac0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 --2- 172.21.15.100:0/3230148600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f837a0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 >> 172.21.15.100:0/3230148600 conn(0x55d5c2f77980 msgr2=0x55d5c2f796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4727fff640 0 client.19407 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 shutdown_connections 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 1 -- 172.21.15.100:0/3230148600 wait complete. 2024-03-12T05:30:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.157+0000 7f4751e1b740 20 client.19407 trim_cache size 0 max 16384 2024-03-12T05:30:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 Processor -- start 2024-03-12T05:30:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 -- start start 2024-03-12T05:30:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f85ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2e6a130 con 0x55d5c2f7d250 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2f9dc10 con 0x55d5c2f7db70 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f4751e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2f7cc20 con 0x55d5c2f86750 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f85ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f85ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56120/0 (socket says 172.21.15.100:56120) 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f474cfe9640 1 -- 172.21.15.100:0/1279885068 learned_addr learned my addr 172.21.15.100:0/1279885068 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.169+0000 7f474d7ea640 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f4747fff640 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f474cfe9640 1 -- 172.21.15.100:0/1279885068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 msgr2=0x55d5c2f84230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f474cfe9640 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f474cfe9640 1 -- 172.21.15.100:0/1279885068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 msgr2=0x55d5c2f83cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f474cfe9640 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f474cfe9640 1 -- 172.21.15.100:0/1279885068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c2e7fc20 con 0x55d5c2f7d250 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f474cfe9640 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f85ed0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f474000cba0 tx=0x7f474009a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffbb9a08ec16e00 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f473e7fc640 1 -- 172.21.15.100:0/1279885068 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400b9040 con 0x55d5c2f7d250 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f473e7fc640 1 -- 172.21.15.100:0/1279885068 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47400a33a0 con 0x55d5c2f7d250 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.170+0000 7f473e7fc640 1 -- 172.21.15.100:0/1279885068 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740005550 con 0x55d5c2f7d250 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 -- 172.21.15.100:0/1279885068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 msgr2=0x55d5c2f85ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f85ed0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f474000cba0 tx=0x7f474009a620 comp rx=0 tx=0).stop 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 -- 172.21.15.100:0/1279885068 shutdown_connections 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f84230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7db70 0x55d5c2f83cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 --2- 172.21.15.100:0/1279885068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7d250 0x55d5c2f85ed0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 -- 172.21.15.100:0/1279885068 >> 172.21.15.100:0/1279885068 conn(0x55d5c2f77980 msgr2=0x55d5c2f7ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 -- 172.21.15.100:0/1279885068 shutdown_connections 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 -- 172.21.15.100:0/1279885068 wait complete. 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 -- start start 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.171+0000 7f4751e1b740 1 Processor -- start 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 0x55d5c2f8bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f8c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2f8c6f0 con 0x55d5c2f7db70 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2f8c860 con 0x55d5c2f7d250 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2f8ca20 con 0x55d5c2f86750 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4751e1b740 10 client.0 _ll_register_callbacks cb 0x7f4740004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:49.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 0x55d5c2f8bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f474cfe9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 0x55d5c2f8bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56130/0 (socket says 172.21.15.100:56130) 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 -- 172.21.15.100:0/2900561311 learned_addr learned my addr 172.21.15.100:0/2900561311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f474d7ea640 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f8c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 -- 172.21.15.100:0/2900561311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 msgr2=0x55d5c2f8c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f8c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 -- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 msgr2=0x55d5c2f8d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f8d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 -- 172.21.15.100:0/2900561311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5c2ee36a0 con 0x55d5c2f7db70 2024-03-12T05:30:49.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4747fff640 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 0x55d5c2f8bc20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4730009900 tx=0x7f4730041030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ed4f3c8825841d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.172+0000 7f4727fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4727fff640 1 -- 172.21.15.100:0/2900561311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730053040 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4727fff640 1 -- 172.21.15.100:0/2900561311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4730004250 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4740098f00 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4727fff640 1 -- 172.21.15.100:0/2900561311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4730004410 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 10 client.19413 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d5c2ea58c0 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4727fff640 1 -- 172.21.15.100:0/2900561311 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4740098f00 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4727fff640 1 -- 172.21.15.100:0/2900561311 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f473004a040 con 0x55d5c2f7db70 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 10 client.19413 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 10 client.19413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 20 client.19413 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2fee170 0x55d5c2ff0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 4 client.19413 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5c2c70890 con 0x55d5c2fee170 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.173+0000 7f474cfe9640 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2fee170 0x55d5c2ff0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.174+0000 7f474cfe9640 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2fee170 0x55d5c2ff0590 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e386543781f07a10 server_cookie=92becb518d33dc20 in_seq=0 out_seq=0 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.174+0000 7f4727fff640 10 client.19413 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4727fff640 1 -- 172.21.15.100:0/2900561311 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5c2c70890 con 0x55d5c2fee170 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4727fff640 10 client.19413 handle_command_reply: tid=0 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 client.19413 shutdown 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 msgr2=0x55d5c2f8bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 0x55d5c2f8bc20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4730009900 tx=0x7f4730041030 comp rx=0 tx=0).stop 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 shutdown_connections 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55d5c2fee170 0x55d5c2ff0590 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5c2f86750 0x55d5c2f8c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5c2f7db70 0x55d5c2f8bc20 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4727fff640 0 client.19413 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:30:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 --2- 172.21.15.100:0/2900561311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c2f7d250 0x55d5c2f8d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 >> 172.21.15.100:0/2900561311 conn(0x55d5c2f77980 msgr2=0x55d5c2f762f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 shutdown_connections 2024-03-12T05:30:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 1 -- 172.21.15.100:0/2900561311 wait complete. 2024-03-12T05:30:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.175+0000 7f4751e1b740 20 client.19413 trim_cache size 0 max 16384 2024-03-12T05:30:49.184 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47340a1ba0 msgr2=0x7f47340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47340a1ba0 0x7f47340a4060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f473002c170 tx=0x7f4730000bc0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 msgr2=0x7f4748198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 0x7f4748198620 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f474000eb90 tx=0x7f474000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 shutdown_connections 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f47340a1ba0 0x7f47340a4060 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47481990f0 0x7f47481a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4748105330 0x7f4748198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 --2- 172.21.15.100:0/68513972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4748100260 0x7f4748198620 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 >> 172.21.15.100:0/68513972 conn(0x7f4748077180 msgr2=0x7f47480fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 shutdown_connections 2024-03-12T05:30:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.176+0000 7f474f24d640 1 -- 172.21.15.100:0/68513972 wait complete. 2024-03-12T05:30:49.187 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:30:49.187 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:30:49.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 183033 27.71 MB/sec execute 212 sec latency 161.425 ms 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.314+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/2461439883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 msgr2=0x7f6f041040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.314+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/2461439883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f041040c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009cd0 tx=0x7f6efc02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/2461439883 shutdown_connections 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/2461439883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f041040c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/2461439883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 0x7f6f04068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/2461439883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f04068380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/2461439883 >> 172.21.15.100:0/2461439883 conn(0x7f6f04077180 msgr2=0x7f6f04077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/2461439883 shutdown_connections 2024-03-12T05:30:49.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.315+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/2461439883 wait complete. 2024-03-12T05:30:49.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 Processor -- start 2024-03-12T05:30:49.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 -- start start 2024-03-12T05:30:49.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 0x7f6f0419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f0419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f041a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f041a2bb0 con 0x7f6f04103c40 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f041a2d20 con 0x7f6f040688c0 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.316+0000 7f6f0c5c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f041a2ee0 con 0x7f6f041060d0 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f041a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f041a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51590/0 (socket says 172.21.15.100:51590) 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 -- 172.21.15.100:0/696403090 learned_addr learned my addr 172.21.15.100:0/696403090 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f09b62640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f0419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0a363640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 0x7f6f0419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 -- 172.21.15.100:0/696403090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 msgr2=0x7f6f0419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 0x7f6f0419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 -- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 msgr2=0x7f6f0419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f0419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 -- 172.21.15.100:0/696403090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc009900 con 0x7f6f041060d0 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f09b62640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f0419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0a363640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 0x7f6f0419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.317+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f041a25e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005e30 tx=0x7f6efc034a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.318+0000 7f6efb7fe640 1 -- 172.21.15.100:0/696403090 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0328b0 con 0x7f6f041060d0 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.318+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f041a31c0 con 0x7f6f041060d0 2024-03-12T05:30:49.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.318+0000 7f6efb7fe640 1 -- 172.21.15.100:0/696403090 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6efc003e50 con 0x7f6f041060d0 2024-03-12T05:30:49.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.318+0000 7f6efb7fe640 1 -- 172.21.15.100:0/696403090 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc004760 con 0x7f6f041060d0 2024-03-12T05:30:49.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.318+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f04106770 con 0x7f6f041060d0 2024-03-12T05:30:49.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:49 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3886263140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:30:49.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.322+0000 7f6efb7fe640 1 -- 172.21.15.100:0/696403090 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6efc00a040 con 0x7f6f041060d0 2024-03-12T05:30:49.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.323+0000 7f6f0f199740 1 Processor -- start 2024-03-12T05:30:49.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.324+0000 7f6efb7fe640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6ef00a1c50 0x7f6ef00a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.324+0000 7f6f0f199740 1 -- start start 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f0f199740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc893960 con 0x558bfc8d8b70 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f0f199740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc847310 con 0x558bfc8e1750 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f0f199740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc844840 con 0x558bfc8d8250 2024-03-12T05:30:49.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6ef00a1c50 0x7f6ef00a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56166/0 (socket says 172.21.15.100:56166) 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 -- 172.21.15.100:0/1177830587 learned_addr learned my addr 172.21.15.100:0/1177830587 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 -- 172.21.15.100:0/1177830587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 msgr2=0x558bfc8d8630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8d8630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 -- 172.21.15.100:0/1177830587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 msgr2=0x558bfc8e3b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.325+0000 7f6f09b62640 1 -- 172.21.15.100:0/1177830587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bfc802060 con 0x558bfc8d8b70 2024-03-12T05:30:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.326+0000 7f6efb7fe640 1 -- 172.21.15.100:0/696403090 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6efc0f0810 con 0x7f6f041060d0 2024-03-12T05:30:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.326+0000 7f6f09b62640 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8e1210 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4015850 tx=0x7f6ef4015d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9974d72d25e4e49b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.326+0000 7f6ef8bfe640 1 -- 172.21.15.100:0/1177830587 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4013e50 con 0x558bfc8d8b70 2024-03-12T05:30:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.327+0000 7f6ef8bfe640 1 -- 172.21.15.100:0/1177830587 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef4021c20 con 0x558bfc8d8b70 2024-03-12T05:30:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.327+0000 7f6ef8bfe640 1 -- 172.21.15.100:0/1177830587 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40163d0 con 0x558bfc8d8b70 2024-03-12T05:30:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.327+0000 7f6f09b62640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6ef00a1c50 0x7f6ef00a4110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4020000 tx=0x7f6ef401d000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.328+0000 7f6f0f199740 1 -- 172.21.15.100:0/1177830587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 msgr2=0x558bfc8e1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.328+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8e1210 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4015850 tx=0x7f6ef4015d10 comp rx=0 tx=0).stop 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.328+0000 7f6f0f199740 1 -- 172.21.15.100:0/1177830587 shutdown_connections 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.328+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.328+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8e1210 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.328+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1177830587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.329+0000 7f6f0f199740 1 -- 172.21.15.100:0/1177830587 >> 172.21.15.100:0/1177830587 conn(0x558bfc8d2980 msgr2=0x558bfc8d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.329+0000 7f6f0f199740 1 -- 172.21.15.100:0/1177830587 shutdown_connections 2024-03-12T05:30:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.329+0000 7f6f0f199740 1 -- 172.21.15.100:0/1177830587 wait complete. 2024-03-12T05:30:49.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.330+0000 7f6f0f199740 1 -- start start 2024-03-12T05:30:49.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.330+0000 7f6f0f199740 1 Processor -- start 2024-03-12T05:30:49.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.330+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8e1750 0x558bfc8df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0a363640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8dec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56168/0 (socket says 172.21.15.100:56168) 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0a363640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8de6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40424/0 (socket says 172.21.15.100:40424) 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f09b62640 1 -- 172.21.15.100:0/3013506904 learned_addr learned my addr 172.21.15.100:0/3013506904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8e1750 0x558bfc8df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc840020 con 0x558bfc8d8b70 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc513430 con 0x558bfc8d8250 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc4cfe00 con 0x558bfc8e1750 2024-03-12T05:30:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.331+0000 7f6f0f199740 10 client.0 _ll_register_callbacks cb 0x558bfc7e7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f09b62640 1 -- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8e1750 msgr2=0x558bfc8df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f09b62640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8e1750 0x558bfc8df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f09b62640 1 -- 172.21.15.100:0/3013506904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 msgr2=0x558bfc8de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f09b62640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8de6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f09b62640 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef4015480 con 0x558bfc8d8b70 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f0a363640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f09b62640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8dec20 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40136b0 tx=0x7f6ef4017e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa80533013ad6ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6eedffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6eedffb640 1 -- 172.21.15.100:0/3013506904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4013e50 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6eedffb640 1 -- 172.21.15.100:0/3013506904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6ef4017650 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.332+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bfc53f7e0 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6eedffb640 1 -- 172.21.15.100:0/3013506904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef4017810 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 10 client.19425 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558bfc83e6a0 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6eedffb640 1 -- 172.21.15.100:0/3013506904 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558bfc53f7e0 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6eedffb640 1 -- 172.21.15.100:0/3013506904 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6ef4006660 con 0x558bfc8d8b70 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 10 client.19425 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 10 client.19425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 20 client.19425 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc8f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 4 client.19425 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bfc57c720 con 0x558bfc8f1880 2024-03-12T05:30:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.333+0000 7f6f0a363640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc8f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.334+0000 7f6eedffb640 10 client.19425 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.334+0000 7f6f0a363640 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc8f3ca0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4de70cb3c07d61b server_cookie=42405b681561ee9e in_seq=0 out_seq=0 2024-03-12T05:30:49.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6eedffb640 1 -- 172.21.15.100:0/3013506904 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558bfc57c720 con 0x558bfc8f1880 2024-03-12T05:30:49.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6eedffb640 10 client.19425 handle_command_reply: tid=0 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 client.19425 shutdown 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 msgr2=0x558bfc8dec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8dec20 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40136b0 tx=0x7f6ef4017e40 comp rx=0 tx=0).stop 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 shutdown_connections 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8e1750 0x558bfc8df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8dec20 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc8f3ca0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6eedffb640 0 client.19425 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 --2- 172.21.15.100:0/3013506904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 >> 172.21.15.100:0/3013506904 conn(0x558bfc8d2980 msgr2=0x558bfc8d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.336+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 shutdown_connections 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.337+0000 7f6f0f199740 1 -- 172.21.15.100:0/3013506904 wait complete. 2024-03-12T05:30:49.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.337+0000 7f6f0f199740 20 client.19425 trim_cache size 0 max 16384 2024-03-12T05:30:49.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 Processor -- start 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 -- start start 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8def80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc7c5130 con 0x558bfc8d8b70 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc8d7f10 con 0x558bfc8e1750 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0f199740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc8e4420 con 0x558bfc8d8250 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8d44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56176/0 (socket says 172.21.15.100:56176) 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 -- 172.21.15.100:0/1861999532 learned_addr learned my addr 172.21.15.100:0/1861999532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0a363640 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 -- 172.21.15.100:0/1861999532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 msgr2=0x558bfc8def80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8def80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 -- 172.21.15.100:0/1861999532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 msgr2=0x558bfc8e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.352+0000 7f6f09b62640 1 -- 172.21.15.100:0/1861999532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bfc7dac20 con 0x558bfc8d8b70 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.353+0000 7f6f09b62640 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8d44e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4015f90 tx=0x7f6ef4015fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f370d2a71a85885 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.353+0000 7f6eeffff640 1 -- 172.21.15.100:0/1861999532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4013e50 con 0x558bfc8d8b70 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.353+0000 7f6eeffff640 1 -- 172.21.15.100:0/1861999532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef40069c0 con 0x558bfc8d8b70 2024-03-12T05:30:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.353+0000 7f6eeffff640 1 -- 172.21.15.100:0/1861999532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef402eda0 con 0x558bfc8d8b70 2024-03-12T05:30:49.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- 172.21.15.100:0/1861999532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 msgr2=0x558bfc8d44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8d44e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4015f90 tx=0x7f6ef4015fc0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- 172.21.15.100:0/1861999532 shutdown_connections 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8e1750 0x558bfc8e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8d8b70 0x558bfc8d44e0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- 172.21.15.100:0/1861999532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8250 0x558bfc8def80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- 172.21.15.100:0/1861999532 >> 172.21.15.100:0/1861999532 conn(0x558bfc8d2980 msgr2=0x558bfc8d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- 172.21.15.100:0/1861999532 shutdown_connections 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- 172.21.15.100:0/1861999532 wait complete. 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- start start 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 Processor -- start 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8e7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 0x558bfc8e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8e1750 0x558bfc947d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc8e8270 con 0x558bfc8e1750 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc9482d0 con 0x558bfc8d8250 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfc948410 con 0x558bfc8d8b70 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.354+0000 7f6f0f199740 10 client.0 _ll_register_callbacks cb 0x7f6ef4009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 0x558bfc8e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8e1750 0x558bfc947d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0a363640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f09b62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 0x558bfc8e79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:51612/0 (socket says 172.21.15.100:51612) 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f09b62640 1 -- 172.21.15.100:0/79553794 learned_addr learned my addr 172.21.15.100:0/79553794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 -- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 msgr2=0x558bfc8e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 0x558bfc8e79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 -- 172.21.15.100:0/79553794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 msgr2=0x558bfc8e7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8e7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 -- 172.21.15.100:0/79553794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bfc83e6a0 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f09b62640 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 0x558bfc8e79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0ab64640 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8e1750 0x558bfc947d90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0edfd0 tx=0x7f6efc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2061ce5e68e065af server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6eedffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6eedffb640 1 -- 172.21.15.100:0/79553794 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0bdbb0 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6eedffb640 1 -- 172.21.15.100:0/79553794 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6efc0a80b0 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ef4021c20 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6eedffb640 1 -- 172.21.15.100:0/79553794 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6efc0bd4b0 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0f199740 10 client.19431 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.355+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558bfc8008c0 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6eedffb640 1 -- 172.21.15.100:0/79553794 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef4021c20 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6eedffb640 1 -- 172.21.15.100:0/79553794 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6efc0bd7a0 con 0x558bfc8e1750 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0f199740 10 client.19431 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:49.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0f199740 10 client.19431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0f199740 20 client.19431 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0f199740 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc94b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0f199740 4 client.19431 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558bfc5cb890 con 0x558bfc8f1880 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0a363640 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc94b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6f0a363640 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc94b9e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=846e639944e2689c server_cookie=30ce0ae20b2180b0 in_seq=0 out_seq=0 2024-03-12T05:30:49.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.356+0000 7f6eedffb640 10 client.19431 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:49.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6eedffb640 1 -- 172.21.15.100:0/79553794 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558bfc5cb890 con 0x558bfc8f1880 2024-03-12T05:30:49.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6eedffb640 10 client.19431 handle_command_reply: tid=0 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 client.19431 shutdown 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8e1750 msgr2=0x558bfc947d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8e1750 0x558bfc947d90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0edfd0 tx=0x7f6efc0095c0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 shutdown_connections 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bfc8e1750 0x558bfc947d90 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558bfc8d8b70 0x558bfc8e79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558bfc8f1880 0x558bfc94b9e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 --2- 172.21.15.100:0/79553794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558bfc8d8250 0x558bfc8e7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6eedffb640 0 client.19431 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.358+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 >> 172.21.15.100:0/79553794 conn(0x558bfc8d2980 msgr2=0x558bfc8d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 shutdown_connections 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0f199740 1 -- 172.21.15.100:0/79553794 wait complete. 2024-03-12T05:30:49.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0f199740 20 client.19431 trim_cache size 0 max 16384 2024-03-12T05:30:49.359 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6ef00a1c50 msgr2=0x7f6ef00a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6ef00a1c50 0x7f6ef00a4110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4020000 tx=0x7f6ef401d000 comp rx=0 tx=0).stop 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 msgr2=0x7f6f041a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f041a25e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005e30 tx=0x7f6efc034a90 comp rx=0 tx=0).stop 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 shutdown_connections 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6ef00a1c50 0x7f6ef00a4110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f041060d0 0x7f6f041a25e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04103c40 0x7f6f0419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 --2- 172.21.15.100:0/696403090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f040688c0 0x7f6f0419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 >> 172.21.15.100:0/696403090 conn(0x7f6f04077180 msgr2=0x7f6f040fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 shutdown_connections 2024-03-12T05:30:49.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.359+0000 7f6f0c5c7640 1 -- 172.21.15.100:0/696403090 wait complete. 2024-03-12T05:30:49.366 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:30:49.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:30:49.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:49 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3886263140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:30:49.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:49 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3886263140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:30:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 -- 172.21.15.100:0/2504456074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0100c10 msgr2=0x7f0ed0104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/2504456074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0100c10 0x7f0ed0104c50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009930 tx=0x7f0ebc02bc30 comp rx=0 tx=0).stop 2024-03-12T05:30:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 -- 172.21.15.100:0/2504456074 shutdown_connections 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/2504456074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ed0105330 0x7f0ed010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/2504456074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0100c10 0x7f0ed0104c50 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/2504456074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0100260 0x7f0ed0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 -- 172.21.15.100:0/2504456074 >> 172.21.15.100:0/2504456074 conn(0x7f0ed0077180 msgr2=0x7f0ed0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 -- 172.21.15.100:0/2504456074 shutdown_connections 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.484+0000 7f0ed6b99640 1 -- 172.21.15.100:0/2504456074 wait complete. 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 Processor -- start 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 -- start start 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0100260 0x7f0ed019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 0x7f0ed019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ed0197640 0x7f0ed0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0198030 con 0x7f0ed0105330 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed01981a0 con 0x7f0ed0197640 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed6b99640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0198360 con 0x7f0ed0100260 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 0x7f0ed019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 0x7f0ed019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56198/0 (socket says 172.21.15.100:56198) 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ecffff640 1 -- 172.21.15.100:0/3808774174 learned_addr learned my addr 172.21.15.100:0/3808774174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed5136640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ed0197640 0x7f0ed0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.485+0000 7f0ed4935640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0100260 0x7f0ed019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecffff640 1 -- 172.21.15.100:0/3808774174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0100260 msgr2=0x7f0ed019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecffff640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0100260 0x7f0ed019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecffff640 1 -- 172.21.15.100:0/3808774174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ed0197640 msgr2=0x7f0ed0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecffff640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ed0197640 0x7f0ed0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecffff640 1 -- 172.21.15.100:0/3808774174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009560 con 0x7f0ed0105330 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecffff640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 0x7f0ed019d980 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009230 tx=0x7f0ebc02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecdffb640 1 -- 172.21.15.100:0/3808774174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc004350 con 0x7f0ed0105330 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecdffb640 1 -- 172.21.15.100:0/3808774174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc0044b0 con 0x7f0ed0105330 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ecdffb640 1 -- 172.21.15.100:0/3808774174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc003da0 con 0x7f0ed0105330 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed01a4300 con 0x7f0ed0105330 2024-03-12T05:30:49.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.486+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed01a4760 con 0x7f0ed0105330 2024-03-12T05:30:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecdffb640 1 -- 172.21.15.100:0/3808774174 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0ebc004df0 con 0x7f0ed0105330 2024-03-12T05:30:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 Processor -- start 2024-03-12T05:30:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 -- start start 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68250 0x5620dcb68630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb71210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb73b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcb23960 con 0x5620dcb71750 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcad7310 con 0x5620dcb68250 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed9768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcad4840 con 0x5620dcb68b70 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb73b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed4935640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb71210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb73b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56202/0 (socket says 172.21.15.100:56202) 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 -- 172.21.15.100:0/2859455151 learned_addr learned my addr 172.21.15.100:0/2859455151 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed5136640 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68250 0x5620dcb68630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecdffb640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ec00a1c20 0x7f0ec00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 -- 172.21.15.100:0/2859455151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 msgr2=0x5620dcb71210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb71210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 -- 172.21.15.100:0/2859455151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68250 msgr2=0x5620dcb68630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68250 0x5620dcb68630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ecffff640 1 -- 172.21.15.100:0/2859455151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620dca92060 con 0x5620dcb71750 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.489+0000 7f0ed5136640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ec00a1c20 0x7f0ec00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.490+0000 7f0ecffff640 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb73b40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0ece80 tx=0x7f0ebc0b84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8406d46928ad7452 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.490+0000 7f0ecdffb640 1 -- 172.21.15.100:0/3808774174 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ebc0ef6f0 con 0x7f0ed0105330 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.490+0000 7f0ebaffd640 1 -- 172.21.15.100:0/2859455151 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0bbf30 con 0x5620dcb71750 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.490+0000 7f0ebaffd640 1 -- 172.21.15.100:0/2859455151 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc0bbd50 con 0x5620dcb71750 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.490+0000 7f0ed5136640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ec00a1c20 0x7f0ec00a40e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005c40 tx=0x7f0ec8004a40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.490+0000 7f0ebaffd640 1 -- 172.21.15.100:0/2859455151 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0a7030 con 0x5620dcb71750 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 -- 172.21.15.100:0/2859455151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 msgr2=0x5620dcb73b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb73b40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0ece80 tx=0x7f0ebc0b84c0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 -- 172.21.15.100:0/2859455151 shutdown_connections 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb73b40 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb71210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2859455151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68250 0x5620dcb68630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 -- 172.21.15.100:0/2859455151 >> 172.21.15.100:0/2859455151 conn(0x5620dcb62980 msgr2=0x5620dcb62d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 -- 172.21.15.100:0/2859455151 shutdown_connections 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 -- 172.21.15.100:0/2859455151 wait complete. 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 -- start start 2024-03-12T05:30:49.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.491+0000 7f0ed9768740 1 Processor -- start 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68250 0x5620dcb6e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68b70 0x5620dcb6ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb6f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcad0020 con 0x5620dcb71750 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc7a3430 con 0x5620dcb68b70 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc75fe00 con 0x5620dcb68250 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed9768740 10 client.0 _ll_register_callbacks cb 0x5620dca77cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb6f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68b70 0x5620dcb6ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed4935640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68250 0x5620dcb6e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb6f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56208/0 (socket says 172.21.15.100:56208) 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 -- 172.21.15.100:0/204285562 learned_addr learned my addr 172.21.15.100:0/204285562 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68b70 0x5620dcb6ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40478/0 (socket says 172.21.15.100:40478) 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 -- 172.21.15.100:0/204285562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68250 msgr2=0x5620dcb6e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68250 0x5620dcb6e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 -- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68b70 msgr2=0x5620dcb6ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68b70 0x5620dcb6ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 -- 172.21.15.100:0/204285562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed01a4760 con 0x5620dcb71750 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0ed5136640 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb6f1f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800fac0 tx=0x7f0ec8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe217b5a97fcea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0eaffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0eaffff640 1 -- 172.21.15.100:0/204285562 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8022070 con 0x5620dcb71750 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.492+0000 7f0eaffff640 1 -- 172.21.15.100:0/204285562 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ec801d5f0 con 0x5620dcb71750 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620dc7cf7e0 con 0x5620dcb71750 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0eaffff640 1 -- 172.21.15.100:0/204285562 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec801d7b0 con 0x5620dcb71750 2024-03-12T05:30:49.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 10 client.19443 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5620dcace6a0 con 0x5620dcb71750 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0eaffff640 1 -- 172.21.15.100:0/204285562 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5620dc7cf7e0 con 0x5620dcb71750 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0eaffff640 1 -- 172.21.15.100:0/204285562 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ec8026a90 con 0x5620dcb71750 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 10 client.19443 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 10 client.19443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 20 client.19443 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcb81820 0x5620dcb83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 4 client.19443 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620dc80c720 con 0x5620dcb81820 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.493+0000 7f0ed4935640 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcb81820 0x5620dcb83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0ed4935640 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcb81820 0x5620dcb83c10 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabd1270db1a881d server_cookie=4016c1b1abb31a25 in_seq=0 out_seq=0 2024-03-12T05:30:49.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0eaffff640 10 client.19443 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0eaffff640 1 -- 172.21.15.100:0/204285562 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620dc80c720 con 0x5620dcb81820 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0eaffff640 10 client.19443 handle_command_reply: tid=0 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0ed9768740 1 client.19443 shutdown 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 msgr2=0x5620dcb6f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.494+0000 7f0ed9768740 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb6f1f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800fac0 tx=0x7f0ec8004ab0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 shutdown_connections 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcb81820 0x5620dcb83c10 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb71750 0x5620dcb6f1f0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb68b70 0x5620dcb6ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0eaffff640 0 client.19443 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 --2- 172.21.15.100:0/204285562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68250 0x5620dcb6e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 >> 172.21.15.100:0/204285562 conn(0x5620dcb62980 msgr2=0x5620dcb646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 shutdown_connections 2024-03-12T05:30:49.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 1 -- 172.21.15.100:0/204285562 wait complete. 2024-03-12T05:30:49.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.495+0000 7f0ed9768740 20 client.19443 trim_cache size 0 max 16384 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 Processor -- start 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 -- start start 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcb70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb6f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb6b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dca55130 con 0x5620dcb68250 2024-03-12T05:30:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcb7d280 con 0x5620dcb71750 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.508+0000 7f0ed9768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcb5e5c0 con 0x5620dcb68b70 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcb70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcb70ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56224/0 (socket says 172.21.15.100:56224) 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 -- 172.21.15.100:0/2446409336 learned_addr learned my addr 172.21.15.100:0/2446409336 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ecffff640 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb6f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed5136640 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb6b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 -- 172.21.15.100:0/2446409336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 msgr2=0x5620dcb6f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb6f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 -- 172.21.15.100:0/2446409336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 msgr2=0x5620dcb6b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb6b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 -- 172.21.15.100:0/2446409336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620dca6ac20 con 0x5620dcb68250 2024-03-12T05:30:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0ed4935640 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcb70ea0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x5620dcb844f0 tx=0x7f0ec4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8971f9690dce5b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0eba7fc640 1 -- 172.21.15.100:0/2446409336 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4017070 con 0x5620dcb68250 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0eba7fc640 1 -- 172.21.15.100:0/2446409336 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec4018040 con 0x5620dcb68250 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.509+0000 7f0eba7fc640 1 -- 172.21.15.100:0/2446409336 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4010990 con 0x5620dcb68250 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 -- 172.21.15.100:0/2446409336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 msgr2=0x5620dcb70ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcb70ea0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x5620dcb844f0 tx=0x7f0ec4005f90 comp rx=0 tx=0).stop 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 -- 172.21.15.100:0/2446409336 shutdown_connections 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb6b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcb6f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 --2- 172.21.15.100:0/2446409336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcb70ea0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 -- 172.21.15.100:0/2446409336 >> 172.21.15.100:0/2446409336 conn(0x5620dcb62980 msgr2=0x5620dcb69a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 -- 172.21.15.100:0/2446409336 shutdown_connections 2024-03-12T05:30:49.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 -- 172.21.15.100:0/2446409336 wait complete. 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 -- start start 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.510+0000 7f0ed9768740 1 Processor -- start 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcbd2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcbd2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcb779f0 con 0x5620dcb68250 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcb77b60 con 0x5620dcb71750 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dcb77d20 con 0x5620dcb68b70 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 10 client.0 _ll_register_callbacks cb 0x7f0ebc0f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcbd2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcbd2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56226/0 (socket says 172.21.15.100:56226) 2024-03-12T05:30:49.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 -- 172.21.15.100:0/462004201 learned_addr learned my addr 172.21.15.100:0/462004201 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ecffff640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcbd2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed5136640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 -- 172.21.15.100:0/462004201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 msgr2=0x5620dcbd2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcbd2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 -- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 msgr2=0x5620dcb77430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb77430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 -- 172.21.15.100:0/462004201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec4009b00 con 0x5620dcb68250 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed4935640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcbd2030 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009dd0 tx=0x7f0ec401f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27f9287c0de658a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0eaffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0eaffff640 1 -- 172.21.15.100:0/462004201 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4017070 con 0x5620dcb68250 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ebc0bbd50 con 0x5620dcb68250 2024-03-12T05:30:49.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.511+0000 7f0eaffff640 1 -- 172.21.15.100:0/462004201 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ec4010b20 con 0x5620dcb68250 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0eaffff640 1 -- 172.21.15.100:0/462004201 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec4003de0 con 0x5620dcb68250 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 10 client.19449 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5620dca908c0 con 0x5620dcb68250 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0eaffff640 1 -- 172.21.15.100:0/462004201 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc0bbd50 con 0x5620dcb68250 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0eaffff640 1 -- 172.21.15.100:0/462004201 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ec4038070 con 0x5620dcb68250 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 10 client.19449 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 10 client.19449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 20 client.19449 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcbd9ed0 0x5620dcbdc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 4 client.19449 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620dc85b890 con 0x5620dcbd9ed0 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ecffff640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcbd9ed0 0x5620dcbdc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0ecffff640 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcbd9ed0 0x5620dcbdc2f0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22fa06120d96720 server_cookie=d52c537bcd8c00ff in_seq=0 out_seq=0 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.512+0000 7f0eaffff640 10 client.19449 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0eaffff640 1 -- 172.21.15.100:0/462004201 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620dc85b890 con 0x5620dcbd9ed0 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0eaffff640 10 client.19449 handle_command_reply: tid=0 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 client.19449 shutdown 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 msgr2=0x5620dcbd2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcbd2030 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009dd0 tx=0x7f0ec401f9b0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 shutdown_connections 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5620dcbd9ed0 0x5620dcbdc2f0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620dcb71750 0x5620dcb77430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dcb68b70 0x5620dcbd2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0eaffff640 0 client.19449 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:49.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 --2- 172.21.15.100:0/462004201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dcb68250 0x5620dcbd2030 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 >> 172.21.15.100:0/462004201 conn(0x5620dcb62980 msgr2=0x5620dcb612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 shutdown_connections 2024-03-12T05:30:49.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 1 -- 172.21.15.100:0/462004201 wait complete. 2024-03-12T05:30:49.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed9768740 20 client.19449 trim_cache size 0 max 16384 2024-03-12T05:30:49.516 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:30:49.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ec00a1c20 msgr2=0x7f0ec00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ec00a1c20 0x7f0ec00a40e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005c40 tx=0x7f0ec8004a40 comp rx=0 tx=0).stop 2024-03-12T05:30:49.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 msgr2=0x7f0ed019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:49.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 0x7f0ed019d980 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009230 tx=0x7f0ebc02c680 comp rx=0 tx=0).stop 2024-03-12T05:30:49.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.513+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 shutdown_connections 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0ec00a1c20 0x7f0ec00a40e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ed0197640 0x7f0ed0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed0105330 0x7f0ed019d980 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 --2- 172.21.15.100:0/3808774174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0100260 0x7f0ed019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 >> 172.21.15.100:0/3808774174 conn(0x7f0ed0077180 msgr2=0x7f0ed0068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 shutdown_connections 2024-03-12T05:30:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:49.514+0000 7f0ed6b99640 1 -- 172.21.15.100:0/3808774174 wait complete. 2024-03-12T05:30:50.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 183513 27.73 MB/sec execute 213 sec latency 178.535 ms 2024-03-12T05:30:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:50 smithi156 ceph-mon[36270]: pgmap v589: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 414 op/s 2024-03-12T05:30:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:50 smithi161 ceph-mon[36043]: pgmap v589: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 414 op/s 2024-03-12T05:30:50.520 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:30:50.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:30:50.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:30:50.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:50 smithi100 ceph-mon[32210]: pgmap v589: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 414 op/s 2024-03-12T05:30:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 -- 172.21.15.100:0/572494622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700105330 msgr2=0x7fd70010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 --2- 172.21.15.100:0/572494622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700105330 0x7fd70010cf00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009cd0 tx=0x7fd6f802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 -- 172.21.15.100:0/572494622 shutdown_connections 2024-03-12T05:30:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 --2- 172.21.15.100:0/572494622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700105330 0x7fd70010cf00 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 --2- 172.21.15.100:0/572494622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700100c10 0x7fd700104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 --2- 172.21.15.100:0/572494622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd700100260 0x7fd700100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.656+0000 7fd708476640 1 -- 172.21.15.100:0/572494622 >> 172.21.15.100:0/572494622 conn(0x7fd700077180 msgr2=0x7fd700077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 -- 172.21.15.100:0/572494622 shutdown_connections 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 -- 172.21.15.100:0/572494622 wait complete. 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 Processor -- start 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 -- start start 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 0x7fd70019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd700105330 0x7fd70019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700197630 0x7fd700197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700198020 con 0x7fd700100260 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700198190 con 0x7fd700197630 2024-03-12T05:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd708476640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700198350 con 0x7fd700105330 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd706212640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 0x7fd70019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd706212640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 0x7fd70019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50288/0 (socket says 172.21.15.100:50288) 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd706212640 1 -- 172.21.15.100:0/1048754939 learned_addr learned my addr 172.21.15.100:0/1048754939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd705a11640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd700105330 0x7fd70019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.657+0000 7fd706a13640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700197630 0x7fd700197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd706212640 1 -- 172.21.15.100:0/1048754939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd700105330 msgr2=0x7fd70019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd706212640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd700105330 0x7fd70019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd706212640 1 -- 172.21.15.100:0/1048754939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700197630 msgr2=0x7fd700197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd706212640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700197630 0x7fd700197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd706212640 1 -- 172.21.15.100:0/1048754939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8009900 con 0x7fd700100260 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd706212640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 0x7fd70019d430 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e9f0 tx=0x7fd6f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd6ff7fe640 1 -- 172.21.15.100:0/1048754939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0009720 con 0x7fd700100260 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd6ff7fe640 1 -- 172.21.15.100:0/1048754939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6f000cd80 con 0x7fd700100260 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7001a42f0 con 0x7fd700100260 2024-03-12T05:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd6ff7fe640 1 -- 172.21.15.100:0/1048754939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0010690 con 0x7fd700100260 2024-03-12T05:30:50.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.658+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7001a4800 con 0x7fd700100260 2024-03-12T05:30:50.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.660+0000 7fd6ff7fe640 1 -- 172.21.15.100:0/1048754939 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd6f00046c0 con 0x7fd700100260 2024-03-12T05:30:50.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.661+0000 7fd70b049740 1 Processor -- start 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.661+0000 7fd70b049740 1 -- start start 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.661+0000 7fd70b049740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf3367760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.661+0000 7fd70b049740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf33702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.661+0000 7fd70b049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf33708b0 0x55daf3372d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.661+0000 7fd6ff7fe640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6e80a1c20 0x7fd6e80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd70b049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf32d5f90 con 0x55daf33708b0 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd70b049740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf32d34c0 con 0x55daf3367d30 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd70b049740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf32ceca0 con 0x55daf3367380 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf33702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706a13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf3367760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706a13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf3367760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:47996/0 (socket says 172.21.15.100:47996) 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706a13640 1 -- 172.21.15.100:0/1631082614 learned_addr learned my addr 172.21.15.100:0/1631082614 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd705a11640 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf33708b0 0x55daf3372d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd6ff7fe640 1 -- 172.21.15.100:0/1048754939 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6f0014070 con 0x7fd700100260 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706a13640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6e80a1c20 0x7fd6e80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 -- 172.21.15.100:0/1631082614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 msgr2=0x55daf3367760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf3367760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 -- 172.21.15.100:0/1631082614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf33708b0 msgr2=0x55daf3372d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf33708b0 0x55daf3372d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 -- 172.21.15.100:0/1631082614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf32910b0 con 0x55daf3367d30 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706212640 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf33702e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00cb260 tx=0x7fd6f009a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26efa9efe5726e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd706a13640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6e80a1c20 0x7fd6e80a40e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8033000 tx=0x7fd6f802c0f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd6fcbfe640 1 -- 172.21.15.100:0/1631082614 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f009a740 con 0x55daf3367d30 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd6fcbfe640 1 -- 172.21.15.100:0/1631082614 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6f009e890 con 0x55daf3367d30 2024-03-12T05:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.662+0000 7fd6fcbfe640 1 -- 172.21.15.100:0/1631082614 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00850f0 con 0x55daf3367d30 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 -- 172.21.15.100:0/1631082614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 msgr2=0x55daf33702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf33702e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00cb260 tx=0x7fd6f009a5d0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 -- 172.21.15.100:0/1631082614 shutdown_connections 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf33708b0 0x55daf3372d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf33702e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 --2- 172.21.15.100:0/1631082614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf3367760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 -- 172.21.15.100:0/1631082614 >> 172.21.15.100:0/1631082614 conn(0x55daf3361840 msgr2=0x55daf3361c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 -- 172.21.15.100:0/1631082614 shutdown_connections 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.663+0000 7fd70b049740 1 -- 172.21.15.100:0/1631082614 wait complete. 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 -- start start 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 Processor -- start 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf336d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 0x55daf336dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf33708b0 0x55daf336e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf3273f40 con 0x55daf3367d30 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf2fa24b0 con 0x55daf33708b0 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf3254670 con 0x55daf3367380 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd70b049740 10 client.0 _ll_register_callbacks cb 0x55daf32c1f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 0x55daf336dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 0x55daf336dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50314/0 (socket says 172.21.15.100:50314) 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 -- 172.21.15.100:0/1217572496 learned_addr learned my addr 172.21.15.100:0/1217572496 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd706a13640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf33708b0 0x55daf336e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd706212640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf336d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 -- 172.21.15.100:0/1217572496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 msgr2=0x55daf336d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf336d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 -- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf33708b0 msgr2=0x55daf336e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf33708b0 0x55daf336e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.664+0000 7fd705a11640 1 -- 172.21.15.100:0/1217572496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7001a4800 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd705a11640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 0x55daf336dd30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4012730 tx=0x7fd6f4012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f68da0cfbf5e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd6e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/1217572496 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4017070 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/1217572496 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6f4004030 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daf32a8f40 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/1217572496 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6f40041f0 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 10 client.19461 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55daf32cd320 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/1217572496 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55daf32a8f40 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/1217572496 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6f4022070 con 0x55daf3367d30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 10 client.19461 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 10 client.19461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 20 client.19461 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf3380ed0 0x55daf33832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 4 client.19461 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daf32c5310 con 0x55daf3380ed0 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.665+0000 7fd706212640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf3380ed0 0x55daf33832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd6e5ffb640 10 client.19461 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd706212640 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf3380ed0 0x55daf33832f0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228e287bbbcf8cf1 server_cookie=e176f76e0d048934 in_seq=0 out_seq=0 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/1217572496 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55daf32c5310 con 0x55daf3380ed0 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd6e5ffb640 10 client.19461 handle_command_reply: tid=0 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 client.19461 shutdown 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 msgr2=0x55daf336dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 0x55daf336dd30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4012730 tx=0x7fd6f4012bf0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 shutdown_connections 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf3380ed0 0x55daf33832f0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf33708b0 0x55daf336e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3367d30 0x55daf336dd30 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd6e5ffb640 0 client.19461 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 --2- 172.21.15.100:0/1217572496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf336d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.666+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 >> 172.21.15.100:0/1217572496 conn(0x55daf3361840 msgr2=0x55daf33637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.667+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 shutdown_connections 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.667+0000 7fd70b049740 1 -- 172.21.15.100:0/1217572496 wait complete. 2024-03-12T05:30:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.667+0000 7fd70b049740 20 client.19461 trim_cache size 0 max 16384 2024-03-12T05:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 Processor -- start 2024-03-12T05:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 -- start start 2024-03-12T05:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf336e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf33778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf3377e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf337d8e0 con 0x55daf3361840 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf335bcf0 con 0x55daf3367d30 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.683+0000 7fd70b049740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf33625d0 con 0x55daf3367380 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf336e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf336e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50328/0 (socket says 172.21.15.100:50328) 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 -- 172.21.15.100:0/1197042391 learned_addr learned my addr 172.21.15.100:0/1197042391 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706a13640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf3377e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd705a11640 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf33778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 -- 172.21.15.100:0/1197042391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 msgr2=0x55daf33778e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf33778e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 -- 172.21.15.100:0/1197042391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 msgr2=0x55daf3377e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf3377e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 -- 172.21.15.100:0/1197042391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf3269c00 con 0x55daf3361840 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706a13640 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf3377e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd706212640 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf336e090 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd6f009a440 tx=0x7fd6f00cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660598d65ad560c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd6e7fff640 1 -- 172.21.15.100:0/1197042391 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a4030 con 0x55daf3361840 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd6e7fff640 1 -- 172.21.15.100:0/1197042391 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6f00b3260 con 0x55daf3361840 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.684+0000 7fd6e7fff640 1 -- 172.21.15.100:0/1197042391 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a8a90 con 0x55daf3361840 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 -- 172.21.15.100:0/1197042391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 msgr2=0x55daf336e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf336e090 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd6f009a440 tx=0x7fd6f00cd040 comp rx=0 tx=0).stop 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 -- 172.21.15.100:0/1197042391 shutdown_connections 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367d30 0x55daf3377e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367380 0x55daf33778e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 --2- 172.21.15.100:0/1197042391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf336e090 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 -- 172.21.15.100:0/1197042391 >> 172.21.15.100:0/1197042391 conn(0x55daf3368480 msgr2=0x55daf3360050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 -- 172.21.15.100:0/1197042391 shutdown_connections 2024-03-12T05:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 -- 172.21.15.100:0/1197042391 wait complete. 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 -- start start 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.685+0000 7fd70b049740 1 Processor -- start 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf33d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367380 0x55daf33d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367d30 0x55daf3376c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf3377340 con 0x55daf3361840 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf33774b0 con 0x55daf3367380 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf3377670 con 0x55daf3367d30 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf33d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd70b049740 10 client.0 _ll_register_callbacks cb 0x7fd6f4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf33d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50340/0 (socket says 172.21.15.100:50340) 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 -- 172.21.15.100:0/4196598453 learned_addr learned my addr 172.21.15.100:0/4196598453 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd705a11640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367380 0x55daf33d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706a13640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367d30 0x55daf3376c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 -- 172.21.15.100:0/4196598453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367d30 msgr2=0x55daf3376c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367d30 0x55daf3376c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 -- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367380 msgr2=0x55daf33d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367380 0x55daf33d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 -- 172.21.15.100:0/4196598453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7000682e0 con 0x55daf3361840 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd706212640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf33d0740 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c6a0 tx=0x7fd6f009a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f17636bf4aa4f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.686+0000 7fd6e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/4196598453 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00b7070 con 0x55daf3361840 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/4196598453 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6f00a5070 con 0x55daf3361840 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daf32f4900 con 0x55daf3361840 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/4196598453 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6f00a4070 con 0x55daf3361840 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 10 client.19467 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:50.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55daf32910b0 con 0x55daf3361840 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/4196598453 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55daf32f4900 con 0x55daf3361840 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/4196598453 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6f0096030 con 0x55daf3361840 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 10 client.19467 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 20 client.19467 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf33708b0 0x55daf33db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 4 client.19467 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55daf2fce860 con 0x55daf33708b0 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.687+0000 7fd705a11640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf33708b0 0x55daf33db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.688+0000 7fd6e5ffb640 10 client.19467 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.688+0000 7fd705a11640 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf33708b0 0x55daf33db2a0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f22d5791362e16 server_cookie=aa069d0608463154 in_seq=0 out_seq=0 2024-03-12T05:30:50.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd6e5ffb640 1 -- 172.21.15.100:0/4196598453 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55daf2fce860 con 0x55daf33708b0 2024-03-12T05:30:50.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd6e5ffb640 10 client.19467 handle_command_reply: tid=0 2024-03-12T05:30:50.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 client.19467 shutdown 2024-03-12T05:30:50.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 msgr2=0x55daf33d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf33d0740 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c6a0 tx=0x7fd6f009a340 comp rx=0 tx=0).stop 2024-03-12T05:30:50.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 shutdown_connections 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55daf33708b0 0x55daf33db2a0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55daf3367d30 0x55daf3376c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daf3367380 0x55daf33d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 --2- 172.21.15.100:0/4196598453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55daf3361840 0x55daf33d0740 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd6e5ffb640 0 client.19467 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.689+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 >> 172.21.15.100:0/4196598453 conn(0x55daf3368480 msgr2=0x55daf3363410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 shutdown_connections 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd70b049740 1 -- 172.21.15.100:0/4196598453 wait complete. 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd70b049740 20 client.19467 trim_cache size 0 max 16384 2024-03-12T05:30:50.690 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:30:50.691 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:30:50.691 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "e0546abb-dc9b-409b-95d5-64093a96ba1e", 2024-03-12T05:30:50.691 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:30:50.691 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6e80a1c20 msgr2=0x7fd6e80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6e80a1c20 0x7fd6e80a40e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8033000 tx=0x7fd6f802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 msgr2=0x7fd70019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 0x7fd70019d430 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e9f0 tx=0x7fd6f000edf0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 shutdown_connections 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd6e80a1c20 0x7fd6e80a40e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700197630 0x7fd700197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd700105330 0x7fd70019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 --2- 172.21.15.100:0/1048754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd700100260 0x7fd70019d430 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.690+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 >> 172.21.15.100:0/1048754939 conn(0x7fd700077180 msgr2=0x7fd7000fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.691+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 shutdown_connections 2024-03-12T05:30:50.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.691+0000 7fd708476640 1 -- 172.21.15.100:0/1048754939 wait complete. 2024-03-12T05:30:50.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:30:50.697 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.862+0000 7f1cdd12c640 1 -- 172.21.15.100:0/105568204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8100270 msgr2=0x7f1cd8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.862+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/105568204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8100270 0x7f1cd8100650 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009930 tx=0x7f1cc802bc30 comp rx=0 tx=0).stop 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.862+0000 7f1cdd12c640 1 -- 172.21.15.100:0/105568204 shutdown_connections 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.862+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/105568204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cd8105340 0x7f1cd810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.863+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/105568204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100c20 0x7f1cd8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.863+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/105568204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8100270 0x7f1cd8100650 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.863+0000 7f1cdd12c640 1 -- 172.21.15.100:0/105568204 >> 172.21.15.100:0/105568204 conn(0x7f1cd8077230 msgr2=0x7f1cd8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.863+0000 7f1cdd12c640 1 -- 172.21.15.100:0/105568204 shutdown_connections 2024-03-12T05:30:50.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.863+0000 7f1cdd12c640 1 -- 172.21.15.100:0/105568204 wait complete. 2024-03-12T05:30:50.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.863+0000 7f1cdd12c640 1 Processor -- start 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 -- start start 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100270 0x7f1cd8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 0x7f1cd8102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cd8102780 0x7f1cd81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd81a5fc0 con 0x7f1cd8105340 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd81a6130 con 0x7f1cd8100270 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cdd12c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd81a62f0 con 0x7f1cd8102780 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cce59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 0x7f1cd8102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100270 0x7f1cd8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cce59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 0x7f1cd8102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50354/0 (socket says 172.21.15.100:50354) 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cce59c640 1 -- 172.21.15.100:0/2236315765 learned_addr learned my addr 172.21.15.100:0/2236315765 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd759e640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cd8102780 0x7f1cd81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/2236315765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cd8102780 msgr2=0x7f1cd81a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cd8102780 0x7f1cd81a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 msgr2=0x7f1cd8102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 0x7f1cd8102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/2236315765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0009490 con 0x7f1cd8100270 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cce59c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 0x7f1cd8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.864+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100270 0x7f1cd8101d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009230 tx=0x7f1cc802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.865+0000 7f1cd4d99640 1 -- 172.21.15.100:0/2236315765 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80042d0 con 0x7f1cd8100270 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.865+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc8009560 con 0x7f1cd8100270 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.865+0000 7f1cd4d99640 1 -- 172.21.15.100:0/2236315765 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cc8004430 con 0x7f1cd8100270 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.865+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd81a6910 con 0x7f1cd8100270 2024-03-12T05:30:50.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.865+0000 7f1cd4d99640 1 -- 172.21.15.100:0/2236315765 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8003bd0 con 0x7f1cd8100270 2024-03-12T05:30:50.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.867+0000 7f1cd4d99640 1 -- 172.21.15.100:0/2236315765 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1cc8003d30 con 0x7f1cd8100270 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.867+0000 7f1cdfdd8740 1 Processor -- start 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.867+0000 7f1cdfdd8740 1 -- start start 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb1534690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb153d7b0 0x560cb153fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb14a3280 con 0x560cb15342b0 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb14a07b0 con 0x560cb153d7b0 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb149bf90 con 0x560cb1534bd0 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb1534690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb1534690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50356/0 (socket says 172.21.15.100:50356) 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 -- 172.21.15.100:0/1581947359 learned_addr learned my addr 172.21.15.100:0/1581947359 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cce59c640 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb153d7b0 0x560cb153fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd4d99640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1cc40a1bd0 0x7f1cc40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 -- 172.21.15.100:0/1581947359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 msgr2=0x560cb153d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 -- 172.21.15.100:0/1581947359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb153d7b0 msgr2=0x560cb153fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb153d7b0 0x560cb153fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 -- 172.21.15.100:0/1581947359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb145a570 con 0x560cb15342b0 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cce59c640 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb153d7b0 0x560cb153fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd4d99640 1 -- 172.21.15.100:0/2236315765 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1cc80ef600 con 0x7f1cd8100270 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1cc40a1bd0 0x7f1cc40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.868+0000 7f1cd759e640 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb1534690 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0011590 tx=0x7f1cd0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c25b7bb6ff4a20 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.869+0000 7f1ccd59a640 1 -- 172.21.15.100:0/1581947359 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd001bb90 con 0x560cb15342b0 2024-03-12T05:30:50.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.869+0000 7f1cd759e640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1cc40a1bd0 0x7f1cc40a4090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0022000 tx=0x7f1cd0023000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.869+0000 7f1ccd59a640 1 -- 172.21.15.100:0/1581947359 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cd001f070 con 0x560cb15342b0 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.869+0000 7f1ccd59a640 1 -- 172.21.15.100:0/1581947359 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000ee70 con 0x560cb15342b0 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1581947359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 msgr2=0x560cb1534690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb1534690 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0011590 tx=0x7f1cd0011a50 comp rx=0 tx=0).stop 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1581947359 shutdown_connections 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb153d7b0 0x560cb153fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1581947359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb1534690 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1581947359 >> 172.21.15.100:0/1581947359 conn(0x560cb152e990 msgr2=0x560cb152ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1581947359 shutdown_connections 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1581947359 wait complete. 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 -- start start 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.870+0000 7f1cdfdd8740 1 Processor -- start 2024-03-12T05:30:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb153a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb153ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb153b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb116f430 con 0x560cb15342b0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb112be00 con 0x560cb1534bd0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb1420fd0 con 0x560cb153d7b0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cdfdd8740 10 client.0 _ll_register_callbacks cb 0x560cb1476fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb153a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb153a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50370/0 (socket says 172.21.15.100:50370) 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/558201882 learned_addr learned my addr 172.21.15.100:0/558201882 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cce59c640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb153ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd759e640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb153b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/558201882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 msgr2=0x560cb153b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb153b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 msgr2=0x560cb153ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb153ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/558201882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0009c50 con 0x560cb15342b0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb153a610 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ece80 tx=0x7f1cc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ba7af66ad71973 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.871+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/558201882 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80f5040 con 0x560cb15342b0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/558201882 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1cc80bca00 con 0x560cb15342b0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cb119b7e0 con 0x560cb15342b0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/558201882 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cc8005640 con 0x560cb15342b0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 10 client.19479 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560cb149a610 con 0x560cb15342b0 2024-03-12T05:30:50.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/558201882 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560cb119b7e0 con 0x560cb15342b0 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/558201882 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cc803c070 con 0x560cb15342b0 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 10 client.19479 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 10 client.19479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 20 client.19479 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb154fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 4 client.19479 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cb11d8720 con 0x560cb154d830 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.872+0000 7f1cce59c640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb154fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.873+0000 7f1cce59c640 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb154fc20 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88afa5dbe4a7dd58 server_cookie=13bcc64f4e0c5761 in_seq=0 out_seq=0 2024-03-12T05:30:50.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.873+0000 7f1cbe7fc640 10 client.19479 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/558201882 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cb11d8720 con 0x560cb154d830 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cbe7fc640 10 client.19479 handle_command_reply: tid=0 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 client.19479 shutdown 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 msgr2=0x560cb153a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb153a610 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ece80 tx=0x7f1cc8009260 comp rx=0 tx=0).stop 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 shutdown_connections 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb154fc20 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb153b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cbe7fc640 0 client.19479 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb153ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/558201882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb153a610 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 >> 172.21.15.100:0/558201882 conn(0x560cb152e990 msgr2=0x560cb1530730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 shutdown_connections 2024-03-12T05:30:50.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/558201882 wait complete. 2024-03-12T05:30:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.874+0000 7f1cdfdd8740 20 client.19479 trim_cache size 0 max 16384 2024-03-12T05:30:50.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 Processor -- start 2024-03-12T05:30:50.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 -- start start 2024-03-12T05:30:50.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb15342b0 0x560cb1539770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 0x560cb1544660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb153ffa0 con 0x560cb153d7b0 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb1533f10 con 0x560cb15342b0 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.890+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb1539ce0 con 0x560cb1534bd0 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 0x560cb1544660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cce59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb15342b0 0x560cb1539770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cce59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:48046/0 (socket says 172.21.15.100:48046) 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cce59c640 1 -- 172.21.15.100:0/306829459 learned_addr learned my addr 172.21.15.100:0/306829459 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 0x560cb1544660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50372/0 (socket says 172.21.15.100:50372) 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 -- 172.21.15.100:0/306829459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 msgr2=0x560cb153aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 -- 172.21.15.100:0/306829459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb15342b0 msgr2=0x560cb1539770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb15342b0 0x560cb1539770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 -- 172.21.15.100:0/306829459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb1436ac0 con 0x560cb153d7b0 2024-03-12T05:30:50.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb15342b0 0x560cb1539770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:30:50.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cd759e640 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 0x560cb1544660 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000ffd0 tx=0x7f1cd000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdd9a973c2aa90b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cccd99640 1 -- 172.21.15.100:0/306829459 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000f5c0 con 0x560cb153d7b0 2024-03-12T05:30:50.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cccd99640 1 -- 172.21.15.100:0/306829459 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cd000ec80 con 0x560cb153d7b0 2024-03-12T05:30:50.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.891+0000 7f1cccd99640 1 -- 172.21.15.100:0/306829459 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd001bbf0 con 0x560cb153d7b0 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/306829459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 msgr2=0x560cb1544660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 0x560cb1544660 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000ffd0 tx=0x7f1cd000ea10 comp rx=0 tx=0).stop 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/306829459 shutdown_connections 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb153d7b0 0x560cb1544660 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb1534bd0 0x560cb153aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/306829459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb15342b0 0x560cb1539770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/306829459 >> 172.21.15.100:0/306829459 conn(0x560cb152e990 msgr2=0x560cb152fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/306829459 shutdown_connections 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/306829459 wait complete. 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.892+0000 7f1cdfdd8740 1 -- start start 2024-03-12T05:30:50.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 Processor -- start 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb15430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb1543610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb1543b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb15a3d10 con 0x560cb15342b0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb15a3e50 con 0x560cb1534bd0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb15a3f90 con 0x560cb153d7b0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 10 client.0 _ll_register_callbacks cb 0x7f1cd0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb15430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb15430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50386/0 (socket says 172.21.15.100:50386) 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/1440037075 learned_addr learned my addr 172.21.15.100:0/1440037075 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cce59c640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb1543610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd759e640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb1543b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/1440037075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 msgr2=0x560cb1543b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb1543b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 msgr2=0x560cb1543610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb1543610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 -- 172.21.15.100:0/1440037075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cb149a610 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cd6d9d640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb15430d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ed310 tx=0x7f1cc80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39e4f7e6e748218 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/1440037075 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80c4610 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cd001f070 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.893+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/1440037075 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1cc80bca00 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/1440037075 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cc80d8020 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 10 client.19485 fetch_fsmap learned FSMap version 30 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560cb145a570 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/1440037075 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cd001f070 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/1440037075 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cc803c070 con 0x560cb15342b0 2024-03-12T05:30:50.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 10 client.19485 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 20 client.19485 populate_metadata read hostname 'smithi100' 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb15a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 4 client.19485 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cb1227890 con 0x560cb154d830 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cce59c640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb15a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cbe7fc640 10 client.19485 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.894+0000 7f1cce59c640 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb15a7250 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6ca558dc0712bb server_cookie=a25bc111e2547749 in_seq=0 out_seq=0 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cbe7fc640 1 -- 172.21.15.100:0/1440037075 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560cb1227890 con 0x560cb154d830 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cbe7fc640 10 client.19485 handle_command_reply: tid=0 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 client.19485 shutdown 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 msgr2=0x560cb15430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb15430d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ed310 tx=0x7f1cc80035f0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 shutdown_connections 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560cb154d830 0x560cb15a7250 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cb153d7b0 0x560cb1543b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb1534bd0 0x560cb1543610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cbe7fc640 0 client.19485 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 --2- 172.21.15.100:0/1440037075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560cb15342b0 0x560cb15430d0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 >> 172.21.15.100:0/1440037075 conn(0x560cb152e990 msgr2=0x560cb152da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 shutdown_connections 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 1 -- 172.21.15.100:0/1440037075 wait complete. 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdfdd8740 20 client.19485 trim_cache size 0 max 16384 2024-03-12T05:30:50.895 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: "e0546abb-dc9b-409b-95d5-64093a96ba1e": { 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "e0546abb-dc9b-409b-95d5-64093a96ba1e", 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:30:50.896 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1cc40a1bd0 msgr2=0x7f1cc40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1cc40a1bd0 0x7f1cc40a4090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0022000 tx=0x7f1cd0023000 comp rx=0 tx=0).stop 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100270 msgr2=0x7f1cd8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100270 0x7f1cd8101d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009230 tx=0x7f1cc802c5d0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 shutdown_connections 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1cc40a1bd0 0x7f1cc40a4090 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.895+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cd8102780 0x7f1cd81a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.896+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cd8105340 0x7f1cd8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.896+0000 7f1cdd12c640 1 --2- 172.21.15.100:0/2236315765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cd8100270 0x7f1cd8101d00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:30:50.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.896+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 >> 172.21.15.100:0/2236315765 conn(0x7f1cd8077230 msgr2=0x7f1cd8068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:30:50.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.896+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 shutdown_connections 2024-03-12T05:30:50.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:30:50.896+0000 7f1cdd12c640 1 -- 172.21.15.100:0/2236315765 wait complete. 2024-03-12T05:30:50.901 INFO:tasks.cephfs.filesystem:scrub status for tag:e0546abb-dc9b-409b-95d5-64093a96ba1e - {'path': '/', 'tag': 'e0546abb-dc9b-409b-95d5-64093a96ba1e', 'options': 'recursive,force'} 2024-03-12T05:30:51.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 183865 27.67 MB/sec execute 214 sec latency 171.728 ms 2024-03-12T05:30:51.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:51 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:30:51.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:51 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:51 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:51.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:51 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:51 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:51.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:51 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:51 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:30:51.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:51 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:51 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:51 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:51 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:51.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:51 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:51 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:30:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:51 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:51 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:51 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:51 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:51 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 184186 27.57 MB/sec execute 215 sec latency 339.860 ms 2024-03-12T05:30:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:52 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:52 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:30:52.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:52 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:52 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:52 smithi156 ceph-mon[36270]: pgmap v590: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 638 op/s 2024-03-12T05:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:52 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:52 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:52 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:52 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:52 smithi161 ceph-mon[36043]: pgmap v590: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 638 op/s 2024-03-12T05:30:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:52 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:52 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:30:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:52 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:52 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:30:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:52 smithi100 ceph-mon[32210]: pgmap v590: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 638 op/s 2024-03-12T05:30:53.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 184768 27.63 MB/sec execute 216 sec latency 195.077 ms 2024-03-12T05:30:54.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 185148 27.63 MB/sec execute 217 sec latency 272.526 ms 2024-03-12T05:30:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:54 smithi156 ceph-mon[36270]: pgmap v591: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.6 MiB/s wr, 511 op/s 2024-03-12T05:30:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:54 smithi161 ceph-mon[36043]: pgmap v591: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.6 MiB/s wr, 511 op/s 2024-03-12T05:30:54.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:54 smithi100 ceph-mon[32210]: pgmap v591: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.6 MiB/s wr, 511 op/s 2024-03-12T05:30:55.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 185542 27.55 MB/sec execute 218 sec latency 128.927 ms 2024-03-12T05:30:56.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 186103 27.52 MB/sec execute 219 sec latency 145.043 ms 2024-03-12T05:30:56.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:56 smithi100 ceph-mon[32210]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.9 MiB/s wr, 479 op/s 2024-03-12T05:30:56.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:56 smithi156 ceph-mon[36270]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.9 MiB/s wr, 479 op/s 2024-03-12T05:30:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:56 smithi161 ceph-mon[36043]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.9 MiB/s wr, 479 op/s 2024-03-12T05:30:57.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 186575 27.54 MB/sec execute 220 sec latency 348.724 ms 2024-03-12T05:30:58.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 187140 27.61 MB/sec execute 221 sec latency 167.622 ms 2024-03-12T05:30:58.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:30:58 smithi156 ceph-mon[36270]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 678 op/s 2024-03-12T05:30:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:30:58 smithi161 ceph-mon[36043]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 678 op/s 2024-03-12T05:30:58.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:30:58 smithi100 ceph-mon[32210]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 678 op/s 2024-03-12T05:30:59.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 187569 27.57 MB/sec execute 222 sec latency 152.878 ms 2024-03-12T05:31:00.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 188041 27.55 MB/sec execute 223 sec latency 157.633 ms 2024-03-12T05:31:00.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:00 smithi156 ceph-mon[36270]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.0 MiB/s wr, 464 op/s 2024-03-12T05:31:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:00 smithi161 ceph-mon[36043]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.0 MiB/s wr, 464 op/s 2024-03-12T05:31:00.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:00 smithi100 ceph-mon[32210]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.0 MiB/s wr, 464 op/s 2024-03-12T05:31:01.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 188338 27.49 MB/sec execute 224 sec latency 269.365 ms 2024-03-12T05:31:02.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 188584 27.42 MB/sec execute 225 sec latency 249.924 ms 2024-03-12T05:31:02.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:02 smithi156 ceph-mon[36270]: pgmap v595: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-03-12T05:31:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:02 smithi161 ceph-mon[36043]: pgmap v595: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-03-12T05:31:02.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:02 smithi100 ceph-mon[32210]: pgmap v595: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-03-12T05:31:03.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 188968 27.40 MB/sec execute 226 sec latency 300.769 ms 2024-03-12T05:31:04.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 189363 27.38 MB/sec execute 227 sec latency 139.450 ms 2024-03-12T05:31:04.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:04 smithi156 ceph-mon[36270]: pgmap v596: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.6 MiB/s wr, 517 op/s 2024-03-12T05:31:04.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:04 smithi161 ceph-mon[36043]: pgmap v596: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.6 MiB/s wr, 517 op/s 2024-03-12T05:31:04.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:04 smithi100 ceph-mon[32210]: pgmap v596: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.6 MiB/s wr, 517 op/s 2024-03-12T05:31:05.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 189768 27.34 MB/sec execute 228 sec latency 129.896 ms 2024-03-12T05:31:05.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:05 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:31:05.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:05 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:31:05.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:05 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:31:06.241 INFO:tasks.workunit.client.0.smithi100.stdout: 10 190337 27.30 MB/sec execute 229 sec latency 143.120 ms 2024-03-12T05:31:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:06 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:31:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:06 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:31:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:06 smithi156 ceph-mon[36270]: osdmap e114: 12 total, 12 up, 12 in 2024-03-12T05:31:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:06 smithi156 ceph-mon[36270]: pgmap v598: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-03-12T05:31:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:06 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:31:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:06 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:31:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:06 smithi161 ceph-mon[36043]: osdmap e114: 12 total, 12 up, 12 in 2024-03-12T05:31:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:06 smithi161 ceph-mon[36043]: pgmap v598: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-03-12T05:31:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:06 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:31:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:06 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:31:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:06 smithi100 ceph-mon[32210]: osdmap e114: 12 total, 12 up, 12 in 2024-03-12T05:31:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:06 smithi100 ceph-mon[32210]: pgmap v598: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-03-12T05:31:07.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 190961 27.40 MB/sec execute 230 sec latency 193.168 ms 2024-03-12T05:31:08.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 191358 27.40 MB/sec execute 231 sec latency 386.632 ms 2024-03-12T05:31:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:08 smithi156 ceph-mon[36270]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 654 op/s 2024-03-12T05:31:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:08 smithi161 ceph-mon[36043]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 654 op/s 2024-03-12T05:31:08.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:08 smithi100 ceph-mon[32210]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 654 op/s 2024-03-12T05:31:09.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 191717 27.35 MB/sec execute 232 sec latency 263.157 ms 2024-03-12T05:31:10.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 192167 27.33 MB/sec execute 233 sec latency 165.869 ms 2024-03-12T05:31:10.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:10 smithi156 ceph-mon[36270]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 654 op/s 2024-03-12T05:31:10.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:10 smithi161 ceph-mon[36043]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 654 op/s 2024-03-12T05:31:10.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:10 smithi100 ceph-mon[32210]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 10 MiB/s wr, 654 op/s 2024-03-12T05:31:11.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 192584 27.32 MB/sec execute 234 sec latency 198.508 ms 2024-03-12T05:31:12.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:11 smithi156 ceph-mon[36270]: osdmap e115: 12 total, 12 up, 12 in 2024-03-12T05:31:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:11 smithi161 ceph-mon[36043]: osdmap e115: 12 total, 12 up, 12 in 2024-03-12T05:31:12.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 193016 27.27 MB/sec execute 235 sec latency 120.373 ms 2024-03-12T05:31:12.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:11 smithi100 ceph-mon[32210]: osdmap e115: 12 total, 12 up, 12 in 2024-03-12T05:31:13.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:12 smithi156 ceph-mon[36270]: pgmap v602: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 574 op/s 2024-03-12T05:31:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:12 smithi161 ceph-mon[36043]: pgmap v602: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 574 op/s 2024-03-12T05:31:13.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 193423 27.23 MB/sec execute 236 sec latency 329.235 ms 2024-03-12T05:31:13.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:12 smithi100 ceph-mon[32210]: pgmap v602: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 574 op/s 2024-03-12T05:31:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 193850 27.25 MB/sec execute 237 sec latency 260.910 ms 2024-03-12T05:31:14.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:14 smithi156 ceph-mon[36270]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 631 op/s 2024-03-12T05:31:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:14 smithi161 ceph-mon[36043]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 631 op/s 2024-03-12T05:31:14.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:14 smithi100 ceph-mon[32210]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 631 op/s 2024-03-12T05:31:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 194374 27.25 MB/sec execute 238 sec latency 172.653 ms 2024-03-12T05:31:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 194795 27.21 MB/sec execute 239 sec latency 127.675 ms 2024-03-12T05:31:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:16 smithi156 ceph-mon[36270]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 562 op/s 2024-03-12T05:31:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:16 smithi161 ceph-mon[36043]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 562 op/s 2024-03-12T05:31:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:16 smithi100 ceph-mon[32210]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 562 op/s 2024-03-12T05:31:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 195321 27.18 MB/sec execute 240 sec latency 173.960 ms 2024-03-12T05:31:18.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 195723 27.19 MB/sec execute 241 sec latency 269.072 ms 2024-03-12T05:31:18.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:18 smithi156 ceph-mon[36270]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 514 op/s 2024-03-12T05:31:18.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:18 smithi161 ceph-mon[36043]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 514 op/s 2024-03-12T05:31:18.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:18 smithi100 ceph-mon[32210]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 514 op/s 2024-03-12T05:31:19.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 196345 27.24 MB/sec execute 242 sec latency 166.460 ms 2024-03-12T05:31:20.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 196805 27.25 MB/sec execute 243 sec latency 181.100 ms 2024-03-12T05:31:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:20 smithi156 ceph-mon[36270]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 514 op/s 2024-03-12T05:31:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:20 smithi161 ceph-mon[36043]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 514 op/s 2024-03-12T05:31:20.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:20 smithi100 ceph-mon[32210]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 514 op/s 2024-03-12T05:31:20.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:31:20.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:31:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 -- 172.21.15.100:0/499399824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4100270 msgr2=0x7f63d4100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 --2- 172.21.15.100:0/499399824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4100270 0x7f63d4100650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009890 tx=0x7f63c402bb70 comp rx=0 tx=0).stop 2024-03-12T05:31:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 -- 172.21.15.100:0/499399824 shutdown_connections 2024-03-12T05:31:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 --2- 172.21.15.100:0/499399824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4105340 0x7f63d410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 --2- 172.21.15.100:0/499399824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63d4100c20 0x7f63d4104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 --2- 172.21.15.100:0/499399824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4100270 0x7f63d4100650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 -- 172.21.15.100:0/499399824 >> 172.21.15.100:0/499399824 conn(0x7f63d4077230 msgr2=0x7f63d4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 -- 172.21.15.100:0/499399824 shutdown_connections 2024-03-12T05:31:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 -- 172.21.15.100:0/499399824 wait complete. 2024-03-12T05:31:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 Processor -- start 2024-03-12T05:31:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.027+0000 7f63da5d7640 1 -- start start 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63da5d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 0x7f63d41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63da5d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4105340 0x7f63d4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63da5d7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63d4199090 0x7f63d41a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63da5d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41a3d10 con 0x7f63d4100270 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63da5d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41a3e80 con 0x7f63d4105340 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63da5d7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41a4040 con 0x7f63d4199090 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 0x7f63d41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 0x7f63d41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49542/0 (socket says 172.21.15.100:49542) 2024-03-12T05:31:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 -- 172.21.15.100:0/2924034504 learned_addr learned my addr 172.21.15.100:0/2924034504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4105340 0x7f63d4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d8b74640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63d4199090 0x7f63d41a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 -- 172.21.15.100:0/2924034504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63d4199090 msgr2=0x7f63d41a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63d4199090 0x7f63d41a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 -- 172.21.15.100:0/2924034504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4105340 msgr2=0x7f63d4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4105340 0x7f63d4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 -- 172.21.15.100:0/2924034504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc009900 con 0x7f63d4100270 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.028+0000 7f63d3fff640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 0x7f63d41985c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f63c4000c00 tx=0x7f63c4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.029+0000 7f63d17fa640 1 -- 172.21.15.100:0/2924034504 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4034950 con 0x7f63d4100270 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.029+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c4009490 con 0x7f63d4100270 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.029+0000 7f63d17fa640 1 -- 172.21.15.100:0/2924034504 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63c402c9f0 con 0x7f63d4100270 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.029+0000 7f63d17fa640 1 -- 172.21.15.100:0/2924034504 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4032600 con 0x7f63d4100270 2024-03-12T05:31:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.029+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41a44f0 con 0x7f63d4100270 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.031+0000 7f63d17fa640 1 -- 172.21.15.100:0/2924034504 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f63c403b040 con 0x7f63d4100270 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.031+0000 7f63dd1ab740 1 Processor -- start 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.031+0000 7f63dd1ab740 1 -- start start 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bec2b0 0x5580c2bec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2becbd0 0x5580c2bf5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2b5b280 con 0x5580c2bf57b0 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2b587b0 con 0x5580c2becbd0 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2b53f90 con 0x5580c2bec2b0 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2becbd0 0x5580c2bf5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d8b74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bec2b0 0x5580c2bec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d17fa640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f63bc0a1ba0 0x7f63bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49550/0 (socket says 172.21.15.100:49550) 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d8b74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bec2b0 0x5580c2bec690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:48956/0 (socket says 172.21.15.100:48956) 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2becbd0 0x5580c2bf5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40732/0 (socket says 172.21.15.100:40732) 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 -- 172.21.15.100:0/2945779039 learned_addr learned my addr 172.21.15.100:0/2945779039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 -- 172.21.15.100:0/2945779039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bec2b0 msgr2=0x5580c2bec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bec2b0 0x5580c2bec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d8b74640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f63bc0a1ba0 0x7f63bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d17fa640 1 -- 172.21.15.100:0/2924034504 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f63c40ef190 con 0x7f63d4100270 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 -- 172.21.15.100:0/2945779039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2becbd0 msgr2=0x5580c2bf5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2becbd0 0x5580c2bf5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 -- 172.21.15.100:0/2945779039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580c2b12570 con 0x5580c2bf57b0 2024-03-12T05:31:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.032+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf7ba0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009c60 tx=0x7f63c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea44182f1012cd05 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.033+0000 7f63ba7fc640 1 -- 172.21.15.100:0/2945779039 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004230 con 0x5580c2bf57b0 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.033+0000 7f63d8b74640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f63bc0a1ba0 0x7f63bc0a4060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0049d0 tx=0x7f63cc004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.033+0000 7f63ba7fc640 1 -- 172.21.15.100:0/2945779039 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63c8009e10 con 0x5580c2bf57b0 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.033+0000 7f63ba7fc640 1 -- 172.21.15.100:0/2945779039 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004c20 con 0x5580c2bf57b0 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.033+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2945779039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 msgr2=0x5580c2bf7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.033+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf7ba0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009c60 tx=0x7f63c8011810 comp rx=0 tx=0).stop 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2945779039 shutdown_connections 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf7ba0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2becbd0 0x5580c2bf5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2945779039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bec2b0 0x5580c2bec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2945779039 >> 172.21.15.100:0/2945779039 conn(0x5580c2be6990 msgr2=0x5580c2be6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2945779039 shutdown_connections 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2945779039 wait complete. 2024-03-12T05:31:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 -- start start 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 Processor -- start 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 0x5580c2bf24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.034+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bf57b0 0x5580c2bf2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2827430 con 0x5580c2becbd0 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c27e3e00 con 0x5580c2bec2b0 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2ad8fd0 con 0x5580c2bf57b0 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63dd1ab740 10 client.0 _ll_register_callbacks cb 0x5580c2b2efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 0x5580c2bf24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 0x5580c2bf24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49554/0 (socket says 172.21.15.100:49554) 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d8b74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bf57b0 0x5580c2bf2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 -- 172.21.15.100:0/2947564755 learned_addr learned my addr 172.21.15.100:0/2947564755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 -- 172.21.15.100:0/2947564755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bf57b0 msgr2=0x5580c2bf2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bf57b0 0x5580c2bf2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 -- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 msgr2=0x5580c2bf5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 -- 172.21.15.100:0/2947564755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c8009890 con 0x5580c2becbd0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63d37fe640 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 0x5580c2bf24e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000c00 tx=0x7f63c8019b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd694cf0a2d366bd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63b37fe640 1 -- 172.21.15.100:0/2947564755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8005680 con 0x5580c2becbd0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63b37fe640 1 -- 172.21.15.100:0/2947564755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f63c8024470 con 0x5580c2becbd0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.035+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580c28537e0 con 0x5580c2becbd0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63b37fe640 1 -- 172.21.15.100:0/2947564755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c8024630 con 0x5580c2becbd0 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 10 client.19497 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580c2b52610 con 0x5580c2becbd0 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63b37fe640 1 -- 172.21.15.100:0/2947564755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5580c28537e0 con 0x5580c2becbd0 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63b37fe640 1 -- 172.21.15.100:0/2947564755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f63c8023730 con 0x5580c2becbd0 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 10 client.19497 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 10 client.19497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 20 client.19497 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c09c80 0x5580c2c0c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 4 client.19497 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580c2890720 con 0x5580c2c09c80 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.036+0000 7f63d3fff640 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c09c80 0x5580c2c0c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.037+0000 7f63b37fe640 10 client.19497 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.037+0000 7f63d3fff640 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c09c80 0x5580c2c0c0a0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e47760a92346a3 server_cookie=8b3bb9decf62f02d in_seq=0 out_seq=0 2024-03-12T05:31:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63b37fe640 1 -- 172.21.15.100:0/2947564755 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580c2890720 con 0x5580c2c09c80 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63b37fe640 10 client.19497 handle_command_reply: tid=0 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 client.19497 shutdown 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 msgr2=0x5580c2bf24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 0x5580c2bf24e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000c00 tx=0x7f63c8019b50 comp rx=0 tx=0).stop 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 shutdown_connections 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c09c80 0x5580c2c0c0a0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2bf57b0 0x5580c2bf2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2becbd0 0x5580c2bf24e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63b37fe640 0 client.19497 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/2947564755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 >> 172.21.15.100:0/2947564755 conn(0x5580c2be6990 msgr2=0x5580c2be8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 shutdown_connections 2024-03-12T05:31:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 1 -- 172.21.15.100:0/2947564755 wait complete. 2024-03-12T05:31:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.038+0000 7f63dd1ab740 20 client.19497 trim_cache size 0 max 16384 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.055+0000 7f63dd1ab740 1 Processor -- start 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.055+0000 7f63dd1ab740 1 -- start start 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.055+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.055+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2bf5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.055+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2be2120 con 0x5580c2bf57b0 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2bebf10 con 0x5580c2bec2b0 2024-03-12T05:31:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2be79e0 con 0x5580c2becbd0 2024-03-12T05:31:21.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2bf5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40738/0 (socket says 172.21.15.100:40738) 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d3fff640 1 -- 172.21.15.100:0/616660735 learned_addr learned my addr 172.21.15.100:0/616660735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 -- 172.21.15.100:0/616660735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 msgr2=0x5580c2bf5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2bf5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 -- 172.21.15.100:0/616660735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 msgr2=0x5580c2bf4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 -- 172.21.15.100:0/616660735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580c2aeeac0 con 0x5580c2bf57b0 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d37fe640 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2bf5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d3fff640 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.056+0000 7f63d8b74640 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf1d70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ffd0 tx=0x7f63cc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f069f6abb7ded8d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.057+0000 7f63b9ffb640 1 -- 172.21.15.100:0/616660735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc023050 con 0x5580c2bf57b0 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.057+0000 7f63b9ffb640 1 -- 172.21.15.100:0/616660735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63cc024040 con 0x5580c2bf57b0 2024-03-12T05:31:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.057+0000 7f63b9ffb640 1 -- 172.21.15.100:0/616660735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc01cb10 con 0x5580c2bf57b0 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.057+0000 7f63dd1ab740 1 -- 172.21.15.100:0/616660735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 msgr2=0x5580c2bf1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.057+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf1d70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ffd0 tx=0x7f63cc01a000 comp rx=0 tx=0).stop 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 -- 172.21.15.100:0/616660735 shutdown_connections 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bf57b0 0x5580c2bf1d70 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2bf5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/616660735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bec2b0 0x5580c2bf4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 -- 172.21.15.100:0/616660735 >> 172.21.15.100:0/616660735 conn(0x5580c2be6990 msgr2=0x5580c2c10a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 -- 172.21.15.100:0/616660735 shutdown_connections 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 -- 172.21.15.100:0/616660735 wait complete. 2024-03-12T05:31:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 -- start start 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.058+0000 7f63dd1ab740 1 Processor -- start 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 0x5580c2c5a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2c5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bf57b0 0x5580c2bf9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2bf9e40 con 0x5580c2bec2b0 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2bf9fb0 con 0x5580c2bf57b0 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2bfa170 con 0x5580c2becbd0 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63dd1ab740 10 client.0 _ll_register_callbacks cb 0x7f63c80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 0x5580c2c5a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2c5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 0x5580c2c5a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49566/0 (socket says 172.21.15.100:49566) 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 -- 172.21.15.100:0/140582334 learned_addr learned my addr 172.21.15.100:0/140582334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d8b74640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bf57b0 0x5580c2bf9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 -- 172.21.15.100:0/140582334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 msgr2=0x5580c2c5aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2c5aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 -- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bf57b0 msgr2=0x5580c2bf9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bf57b0 0x5580c2bf9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d3fff640 1 -- 172.21.15.100:0/140582334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41a44f0 con 0x5580c2bec2b0 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d37fe640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2c5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.059+0000 7f63d8b74640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bf57b0 0x5580c2bf9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63d3fff640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 0x5580c2c5a540 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f63c40bbed0 tx=0x7f63c402c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771915b6435e6caa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63b37fe640 1 -- 172.21.15.100:0/140582334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40bffe0 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63b37fe640 1 -- 172.21.15.100:0/140582334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f63c40c4630 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c8009e10 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63b37fe640 1 -- 172.21.15.100:0/140582334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c402cb60 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63dd1ab740 10 client.19503 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580c2b52610 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63b37fe640 1 -- 172.21.15.100:0/140582334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63c8009e10 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63b37fe640 1 -- 172.21.15.100:0/140582334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f63c40b85b0 con 0x5580c2bec2b0 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63dd1ab740 10 client.19503 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63dd1ab740 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.060+0000 7f63dd1ab740 20 client.19503 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c62050 0x5580c2c64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63dd1ab740 4 client.19503 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:31:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5580c28df890 con 0x5580c2c62050 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63d37fe640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c62050 0x5580c2c64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63b37fe640 10 client.19503 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63d37fe640 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c62050 0x5580c2c64470 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91619ff2fb4c1ef server_cookie=f046ff5cc936307c in_seq=0 out_seq=0 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63b37fe640 1 -- 172.21.15.100:0/140582334 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5580c28df890 con 0x5580c2c62050 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63b37fe640 10 client.19503 handle_command_reply: tid=0 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.061+0000 7f63dd1ab740 1 client.19503 shutdown 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 msgr2=0x5580c2c5a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 0x5580c2c5a540 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f63c40bbed0 tx=0x7f63c402c130 comp rx=0 tx=0).stop 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 shutdown_connections 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5580c2c62050 0x5580c2c64470 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580c2bf57b0 0x5580c2bf9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580c2becbd0 0x5580c2c5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 --2- 172.21.15.100:0/140582334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c2bec2b0 0x5580c2c5a540 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63b37fe640 0 client.19503 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 >> 172.21.15.100:0/140582334 conn(0x5580c2be6990 msgr2=0x5580c2bf1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 shutdown_connections 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 1 -- 172.21.15.100:0/140582334 wait complete. 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.062+0000 7f63dd1ab740 20 client.19503 trim_cache size 0 max 16384 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:31:21.061 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:31:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f63bc0a1ba0 msgr2=0x7f63bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f63bc0a1ba0 0x7f63bc0a4060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0049d0 tx=0x7f63cc004ab0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 msgr2=0x7f63d41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 0x7f63d41985c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f63c4000c00 tx=0x7f63c4004530 comp rx=0 tx=0).stop 2024-03-12T05:31:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 shutdown_connections 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f63bc0a1ba0 0x7f63bc0a4060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63d4199090 0x7f63d41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63d4105340 0x7f63d4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 --2- 172.21.15.100:0/2924034504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d4100270 0x7f63d41985c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 >> 172.21.15.100:0/2924034504 conn(0x7f63d4077230 msgr2=0x7f63d40fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 shutdown_connections 2024-03-12T05:31:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.063+0000 7f63da5d7640 1 -- 172.21.15.100:0/2924034504 wait complete. 2024-03-12T05:31:21.070 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:31:21.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:31:21.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 -- 172.21.15.100:0/1681357106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001053c0 msgr2=0x7fa80010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 --2- 172.21.15.100:0/1681357106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001053c0 0x7fa80010cf90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009be0 tx=0x7fa7f802bef0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 -- 172.21.15.100:0/1681357106 shutdown_connections 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 --2- 172.21.15.100:0/1681357106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001053c0 0x7fa80010cf90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 --2- 172.21.15.100:0/1681357106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa800100ca0 0x7fa800104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 --2- 172.21.15.100:0/1681357106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001002f0 0x7fa8001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 -- 172.21.15.100:0/1681357106 >> 172.21.15.100:0/1681357106 conn(0x7fa800077180 msgr2=0x7fa800077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 -- 172.21.15.100:0/1681357106 shutdown_connections 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.196+0000 7fa8048d4640 1 -- 172.21.15.100:0/1681357106 wait complete. 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 Processor -- start 2024-03-12T05:31:21.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 -- start start 2024-03-12T05:31:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 0x7fa800198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001053c0 0x7fa800198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001990e0 0x7fa8001a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8001a3d30 con 0x7fa8001002f0 2024-03-12T05:31:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8001a3ea0 con 0x7fa8001053c0 2024-03-12T05:31:21.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa8048d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8001a4060 con 0x7fa8001990e0 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa7fe59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 0x7fa800198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001053c0 0x7fa800198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa7fe59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 0x7fa800198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49588/0 (socket says 172.21.15.100:49588) 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa7fe59c640 1 -- 172.21.15.100:0/2776326978 learned_addr learned my addr 172.21.15.100:0/2776326978 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.197+0000 7fa7fed9d640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001990e0 0x7fa8001a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7fe59c640 1 -- 172.21.15.100:0/2776326978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001990e0 msgr2=0x7fa8001a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7fe59c640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001990e0 0x7fa8001a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7fe59c640 1 -- 172.21.15.100:0/2776326978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001053c0 msgr2=0x7fa800198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7fe59c640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001053c0 0x7fa800198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7fe59c640 1 -- 172.21.15.100:0/2776326978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8009840 con 0x7fa8001002f0 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7f7fff640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001053c0 0x7fa800198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7fe59c640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 0x7fa800198610 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e9f0 tx=0x7fa7f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0009720 con 0x7fa8001002f0 2024-03-12T05:31:21.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa800196460 con 0x7fa8001002f0 2024-03-12T05:31:21.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7f0005500 con 0x7fa8001002f0 2024-03-12T05:31:21.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0010430 con 0x7fa8001002f0 2024-03-12T05:31:21.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.198+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa800196920 con 0x7fa8001002f0 2024-03-12T05:31:21.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.199+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800067f70 con 0x7fa8001002f0 2024-03-12T05:31:21.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.201+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fa7f0005000 con 0x7fa8001002f0 2024-03-12T05:31:21.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.201+0000 7fa7f77fe640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa7ec0a1ba0 0x7fa7ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.202+0000 7fa7f7fff640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa7ec0a1ba0 0x7fa7ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.202+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7f0014070 con 0x7fa8001002f0 2024-03-12T05:31:21.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.202+0000 7fa7f7fff640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa7ec0a1ba0 0x7fa7ec0a4060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa800068e30 tx=0x7fa7e8012040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.206+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7f0096ca0 con 0x7fa8001002f0 2024-03-12T05:31:21.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 197282 27.23 MB/sec execute 244 sec latency 101.533 ms 2024-03-12T05:31:21.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.490+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa800108650 con 0x7fa8001002f0 2024-03-12T05:31:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.491+0000 7fa7f77fe640 1 -- 172.21.15.100:0/2776326978 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fa7f00ba030 con 0x7fa8001002f0 2024-03-12T05:31:21.485 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:31:21.485 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:31:21.485 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa7ec0a1ba0 msgr2=0x7fa7ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa7ec0a1ba0 0x7fa7ec0a4060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa800068e30 tx=0x7fa7e8012040 comp rx=0 tx=0).stop 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 msgr2=0x7fa800198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 0x7fa800198610 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e9f0 tx=0x7fa7f000edf0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 shutdown_connections 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fa7ec0a1ba0 0x7fa7ec0a4060 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8001990e0 0x7fa8001a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8001053c0 0x7fa800198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 --2- 172.21.15.100:0/2776326978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8001002f0 0x7fa800198610 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 >> 172.21.15.100:0/2776326978 conn(0x7fa800077180 msgr2=0x7fa8000fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 shutdown_connections 2024-03-12T05:31:21.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.493+0000 7fa8048d4640 1 -- 172.21.15.100:0/2776326978 wait complete. 2024-03-12T05:31:21.499 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:31:21.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:31:21.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 -- 172.21.15.100:0/1833225391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20101d40 msgr2=0x7fac20105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 --2- 172.21.15.100:0/1833225391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20101d40 0x7fac20105d80 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fac14009930 tx=0x7fac1402bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 -- 172.21.15.100:0/1833225391 shutdown_connections 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 --2- 172.21.15.100:0/1833225391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 0x7fac2010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 --2- 172.21.15.100:0/1833225391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20101d40 0x7fac20105d80 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 --2- 172.21.15.100:0/1833225391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac20101390 0x7fac20101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 -- 172.21.15.100:0/1833225391 >> 172.21.15.100:0/1833225391 conn(0x7fac200fcbb0 msgr2=0x7fac200fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 -- 172.21.15.100:0/1833225391 shutdown_connections 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.620+0000 7fac26283640 1 -- 172.21.15.100:0/1833225391 wait complete. 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 Processor -- start 2024-03-12T05:31:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 -- start start 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac20101390 0x7fac2019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 0x7fac2019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 0x7fac20197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20198030 con 0x7fac20197640 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201981a0 con 0x7fac20106460 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac26283640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20198360 con 0x7fac20101390 2024-03-12T05:31:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac24820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 0x7fac20197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac24820640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 0x7fac20197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49600/0 (socket says 172.21.15.100:49600) 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 0x7fac2019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac24820640 1 -- 172.21.15.100:0/4103867511 learned_addr learned my addr 172.21.15.100:0/4103867511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.621+0000 7fac1ffff640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac20101390 0x7fac2019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac24820640 1 -- 172.21.15.100:0/4103867511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac20101390 msgr2=0x7fac2019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac24820640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac20101390 0x7fac2019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac24820640 1 -- 172.21.15.100:0/4103867511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 msgr2=0x7fac2019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac24820640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 0x7fac2019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac24820640 1 -- 172.21.15.100:0/4103867511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009560 con 0x7fac20197640 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 0x7fac2019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac24820640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 0x7fac20197af0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fac1800eb10 tx=0x7fac1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac1d7fa640 1 -- 172.21.15.100:0/4103867511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004b20 con 0x7fac20197640 2024-03-12T05:31:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac201a4300 con 0x7fac20197640 2024-03-12T05:31:21.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac1d7fa640 1 -- 172.21.15.100:0/4103867511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18004c80 con 0x7fac20197640 2024-03-12T05:31:21.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac1d7fa640 1 -- 172.21.15.100:0/4103867511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18010650 con 0x7fac20197640 2024-03-12T05:31:21.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.622+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201a4790 con 0x7fac20197640 2024-03-12T05:31:21.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.624+0000 7fac1d7fa640 1 -- 172.21.15.100:0/4103867511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fac18005020 con 0x7fac20197640 2024-03-12T05:31:21.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 Processor -- start 2024-03-12T05:31:21.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 -- start start 2024-03-12T05:31:21.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 0x564aa850c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa8515210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8517b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1d7fa640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac080a1ba0 0x7fac080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa84c7960 con 0x564aa850cb70 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa847b310 con 0x564aa850c250 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac28e4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa8478840 con 0x564aa8515750 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 0x564aa850c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac24820640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac080a1ba0 0x7fac080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa8515210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 0x564aa850c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40798/0 (socket says 172.21.15.100:40798) 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1f7fe640 1 -- 172.21.15.100:0/1281927427 learned_addr learned my addr 172.21.15.100:0/1281927427 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8517b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.625+0000 7fac1d7fa640 1 -- 172.21.15.100:0/4103867511 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fac18014070 con 0x7fac20197640 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1f7fe640 1 -- 172.21.15.100:0/1281927427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 msgr2=0x564aa8517b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8517b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1f7fe640 1 -- 172.21.15.100:0/1281927427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 msgr2=0x564aa8515210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa8515210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1f7fe640 1 -- 172.21.15.100:0/1281927427 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aa8436060 con 0x564aa850c250 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1ffff640 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa8515210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac24820640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac080a1ba0 0x7fac080a4060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fac180cbaa0 tx=0x7fac1806cb80 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 0x564aa850c630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac14005ec0 tx=0x7fac14005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675bc4665608a9bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac067fc640 1 -- 172.21.15.100:0/1281927427 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1402cbf0 con 0x564aa850c250 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac067fc640 1 -- 172.21.15.100:0/1281927427 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac14004500 con 0x564aa850c250 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.626+0000 7fac067fc640 1 -- 172.21.15.100:0/1281927427 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14004df0 con 0x564aa850c250 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 -- 172.21.15.100:0/1281927427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 msgr2=0x564aa850c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 0x564aa850c630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac14005ec0 tx=0x7fac14005e50 comp rx=0 tx=0).stop 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 -- 172.21.15.100:0/1281927427 shutdown_connections 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8517b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa8515210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 --2- 172.21.15.100:0/1281927427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850c250 0x564aa850c630 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 -- 172.21.15.100:0/1281927427 >> 172.21.15.100:0/1281927427 conn(0x564aa8506980 msgr2=0x564aa8506d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 -- 172.21.15.100:0/1281927427 shutdown_connections 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 -- 172.21.15.100:0/1281927427 wait complete. 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 -- start start 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.627+0000 7fac28e4b740 1 Processor -- start 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 0x564aa85127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa8512d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8513250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa8474020 con 0x564aa850c250 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa8147430 con 0x564aa850cb70 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa8103e00 con 0x564aa8515750 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 10 client.0 _ll_register_callbacks cb 0x564aa841bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 0x564aa85127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 0x564aa85127d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49632/0 (socket says 172.21.15.100:49632) 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 -- 172.21.15.100:0/3765354058 learned_addr learned my addr 172.21.15.100:0/3765354058 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1ffff640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa8512d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8513250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 -- 172.21.15.100:0/3765354058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 msgr2=0x564aa8513250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8513250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 -- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 msgr2=0x564aa8512d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa8512d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 -- 172.21.15.100:0/3765354058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2010e840 con 0x564aa850c250 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1ffff640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa8512d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 0x564aa85127d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fac14036040 tx=0x7fac14004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4070a8bc471c863 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fabff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fabff7fe640 1 -- 172.21.15.100:0/3765354058 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140037f0 con 0x564aa850c250 2024-03-12T05:31:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fabff7fe640 1 -- 172.21.15.100:0/3765354058 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fac14004330 con 0x564aa850c250 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.628+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564aa81b0720 con 0x564aa850c250 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fabff7fe640 1 -- 172.21.15.100:0/3765354058 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac14003950 con 0x564aa850c250 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 10 client.19521 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564aa84726a0 con 0x564aa850c250 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fabff7fe640 1 -- 172.21.15.100:0/3765354058 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1405e020 con 0x564aa850c250 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fabff7fe640 1 -- 172.21.15.100:0/3765354058 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac1404b070 con 0x564aa850c250 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 10 client.19521 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 10 client.19521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 20 client.19521 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa8525850 0x564aa8527c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 4 client.19521 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564aa81ff890 con 0x564aa8525850 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.629+0000 7fac1ffff640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa8525850 0x564aa8527c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.630+0000 7fac1ffff640 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa8525850 0x564aa8527c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf8fbc0a8cc961a server_cookie=75b05f97c5e82ae7 in_seq=0 out_seq=0 2024-03-12T05:31:21.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.630+0000 7fabff7fe640 10 client.19521 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fabff7fe640 1 -- 172.21.15.100:0/3765354058 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564aa81ff890 con 0x564aa8525850 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fabff7fe640 10 client.19521 handle_command_reply: tid=0 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 client.19521 shutdown 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 msgr2=0x564aa85127d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 0x564aa85127d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fac14036040 tx=0x7fac14004be0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 shutdown_connections 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa8525850 0x564aa8527c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa8515750 0x564aa8513250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa8512d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 --2- 172.21.15.100:0/3765354058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850c250 0x564aa85127d0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 >> 172.21.15.100:0/3765354058 conn(0x564aa8506980 msgr2=0x564aa85086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fabff7fe640 0 client.19521 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:21.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 shutdown_connections 2024-03-12T05:31:21.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 1 -- 172.21.15.100:0/3765354058 wait complete. 2024-03-12T05:31:21.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.632+0000 7fac28e4b740 20 client.19521 trim_cache size 0 max 16384 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 Processor -- start 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 -- start start 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa8514f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa85110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa8515750 0x564aa85115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa83f9130 con 0x564aa8515750 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa850b340 con 0x564aa850cb70 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac28e4b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa85064b0 con 0x564aa850c250 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa85110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa85110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40824/0 (socket says 172.21.15.100:40824) 2024-03-12T05:31:21.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.652+0000 7fac1ffff640 1 -- 172.21.15.100:0/2356268224 learned_addr learned my addr 172.21.15.100:0/2356268224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa8515750 0x564aa85115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa8514f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1ffff640 1 -- 172.21.15.100:0/2356268224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 msgr2=0x564aa8514f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1ffff640 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa8514f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1ffff640 1 -- 172.21.15.100:0/2356268224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa8515750 msgr2=0x564aa85115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1ffff640 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa8515750 0x564aa85115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.653+0000 7fac1ffff640 1 -- 172.21.15.100:0/2356268224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aa840ec20 con 0x564aa850cb70 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.654+0000 7fac1ffff640 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa85110a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x564aa8512030 tx=0x7fac10005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4431af4524a2ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.654+0000 7fac05ffb640 1 -- 172.21.15.100:0/2356268224 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10017040 con 0x564aa850cb70 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.654+0000 7fac05ffb640 1 -- 172.21.15.100:0/2356268224 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac10018040 con 0x564aa850cb70 2024-03-12T05:31:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.654+0000 7fac05ffb640 1 -- 172.21.15.100:0/2356268224 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10004a00 con 0x564aa850cb70 2024-03-12T05:31:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.655+0000 7fac28e4b740 1 -- 172.21.15.100:0/2356268224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 msgr2=0x564aa85110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.655+0000 7fac28e4b740 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa85110a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x564aa8512030 tx=0x7fac10005f90 comp rx=0 tx=0).stop 2024-03-12T05:31:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.657+0000 7fac28e4b740 1 -- 172.21.15.100:0/2356268224 shutdown_connections 2024-03-12T05:31:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.657+0000 7fac28e4b740 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa8515750 0x564aa85115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.657+0000 7fac28e4b740 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa850cb70 0x564aa85110a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.657+0000 7fac28e4b740 1 --2- 172.21.15.100:0/2356268224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa8514f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.657+0000 7fac28e4b740 1 -- 172.21.15.100:0/2356268224 >> 172.21.15.100:0/2356268224 conn(0x564aa8506980 msgr2=0x564aa852c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.657+0000 7fac28e4b740 1 -- 172.21.15.100:0/2356268224 shutdown_connections 2024-03-12T05:31:21.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.658+0000 7fac28e4b740 1 -- 172.21.15.100:0/2356268224 wait complete. 2024-03-12T05:31:21.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.658+0000 7fac28e4b740 1 -- start start 2024-03-12T05:31:21.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.658+0000 7fac28e4b740 1 Processor -- start 2024-03-12T05:31:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa851ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa851adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac28e4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa8515750 0x564aa851b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa851adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa8515750 0x564aa851b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa851ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa851adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49652/0 (socket says 172.21.15.100:49652) 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa851ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:49076/0 (socket says 172.21.15.100:49076) 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac1ffff640 1 -- 172.21.15.100:0/133633792 learned_addr learned my addr 172.21.15.100:0/133633792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa851bb30 con 0x564aa850cb70 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.659+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa857bd60 con 0x564aa8515750 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa857bea0 con 0x564aa850c250 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac28e4b740 10 client.0 _ll_register_callbacks cb 0x7fac10005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac1ffff640 1 -- 172.21.15.100:0/133633792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 msgr2=0x564aa851ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac1ffff640 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa851ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac1ffff640 1 -- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa8515750 msgr2=0x564aa851b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac1ffff640 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa8515750 0x564aa851b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac1ffff640 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac10005c50 con 0x564aa850cb70 2024-03-12T05:31:21.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac1ffff640 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa851adb0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fac1000a010 tx=0x7fac10012280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e5cc3077ec8cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fabff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fabff7fe640 1 -- 172.21.15.100:0/133633792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10017040 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fabff7fe640 1 -- 172.21.15.100:0/133633792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fac10004050 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.660+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564aa81737e0 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fabff7fe640 1 -- 172.21.15.100:0/133633792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac10004210 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fac28e4b740 10 client.19527 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564aa84348c0 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fabff7fe640 1 -- 172.21.15.100:0/133633792 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564aa81737e0 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fabff7fe640 1 -- 172.21.15.100:0/133633792 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac100036f0 con 0x564aa850cb70 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fac28e4b740 10 client.19527 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fac28e4b740 10 client.19527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fac28e4b740 20 client.19527 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.661+0000 7fac28e4b740 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa857d670 0x564aa857fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.662+0000 7fac28e4b740 4 client.19527 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:31:21.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.662+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564aa811f700 con 0x564aa857d670 2024-03-12T05:31:21.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.662+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa857d670 0x564aa857fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.663+0000 7fac1f7fe640 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa857d670 0x564aa857fa90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cce5791ea777297 server_cookie=a31b80bb38d59b8d in_seq=0 out_seq=0 2024-03-12T05:31:21.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.663+0000 7fabff7fe640 10 client.19527 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:21.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.666+0000 7fabff7fe640 1 -- 172.21.15.100:0/133633792 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564aa811f700 con 0x564aa857d670 2024-03-12T05:31:21.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.666+0000 7fabff7fe640 10 client.19527 handle_command_reply: tid=0 2024-03-12T05:31:21.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.666+0000 7fac28e4b740 1 client.19527 shutdown 2024-03-12T05:31:21.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.667+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 msgr2=0x564aa851adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.667+0000 7fac28e4b740 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa851adb0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fac1000a010 tx=0x7fac10012280 comp rx=0 tx=0).stop 2024-03-12T05:31:21.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 shutdown_connections 2024-03-12T05:31:21.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x564aa857d670 0x564aa857fa90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564aa8515750 0x564aa851b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564aa850cb70 0x564aa851adb0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 --2- 172.21.15.100:0/133633792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564aa850c250 0x564aa851ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 >> 172.21.15.100:0/133633792 conn(0x564aa8506980 msgr2=0x564aa8511020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fabff7fe640 0 client.19527 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.668+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 shutdown_connections 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.669+0000 7fac28e4b740 1 -- 172.21.15.100:0/133633792 wait complete. 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.669+0000 7fac28e4b740 20 client.19527 trim_cache size 0 max 16384 2024-03-12T05:31:21.663 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.670+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac080a1ba0 msgr2=0x7fac080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.670+0000 7fac26283640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac080a1ba0 0x7fac080a4060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fac180cbaa0 tx=0x7fac1806cb80 comp rx=0 tx=0).stop 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.670+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 msgr2=0x7fac20197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.670+0000 7fac26283640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 0x7fac20197af0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fac1800eb10 tx=0x7fac1800c2d0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 shutdown_connections 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac080a1ba0 0x7fac080a4060 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac20197640 0x7fac20197af0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac20106460 0x7fac2019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 --2- 172.21.15.100:0/4103867511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac20101390 0x7fac2019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 >> 172.21.15.100:0/4103867511 conn(0x7fac200fcbb0 msgr2=0x7fac200fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 shutdown_connections 2024-03-12T05:31:21.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.671+0000 7fac26283640 1 -- 172.21.15.100:0/4103867511 wait complete. 2024-03-12T05:31:21.682 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:31:21.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 -- 172.21.15.100:0/2015063455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc1009e0 msgr2=0x7f99dc10cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 --2- 172.21.15.100:0/2015063455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc1009e0 0x7f99dc10cea0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009ca0 tx=0x7f99d402c180 comp rx=0 tx=0).stop 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 -- 172.21.15.100:0/2015063455 shutdown_connections 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 --2- 172.21.15.100:0/2015063455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc1009e0 0x7f99dc10cea0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 --2- 172.21.15.100:0/2015063455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc100040 0x7f99dc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 --2- 172.21.15.100:0/2015063455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc106100 0x7f99dc1064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 -- 172.21.15.100:0/2015063455 >> 172.21.15.100:0/2015063455 conn(0x7f99dc077110 msgr2=0x7f99dc077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 -- 172.21.15.100:0/2015063455 shutdown_connections 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.812+0000 7f99e298c640 1 -- 172.21.15.100:0/2015063455 wait complete. 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 Processor -- start 2024-03-12T05:31:21.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 -- start start 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc100040 0x7f99dc19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1009e0 0x7f99dc19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc106100 0x7f99dc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc199ea0 con 0x7f99dc100040 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc19a010 con 0x7f99dc106100 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.813+0000 7f99e298c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc19a1d0 con 0x7f99dc1009e0 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99dbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc100040 0x7f99dc19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc106100 0x7f99dc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99dbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc100040 0x7f99dc19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49670/0 (socket says 172.21.15.100:49670) 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99dbfff640 1 -- 172.21.15.100:0/3489832530 learned_addr learned my addr 172.21.15.100:0/3489832530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1009e0 0x7f99dc19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 -- 172.21.15.100:0/3489832530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1009e0 msgr2=0x7f99dc19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1009e0 0x7f99dc19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 -- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc100040 msgr2=0x7f99dc19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc100040 0x7f99dc19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 -- 172.21.15.100:0/3489832530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4009900 con 0x7f99dc106100 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc106100 0x7f99dc199710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99d402c790 tx=0x7f99d40346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99d97fa640 1 -- 172.21.15.100:0/3489832530 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4003d80 con 0x7f99dc106100 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc19a450 con 0x7f99dc106100 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99d97fa640 1 -- 172.21.15.100:0/3489832530 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99d40348e0 con 0x7f99dc106100 2024-03-12T05:31:21.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99d97fa640 1 -- 172.21.15.100:0/3489832530 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4004790 con 0x7f99dc106100 2024-03-12T05:31:21.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.814+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc1a65d0 con 0x7f99dc106100 2024-03-12T05:31:21.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99d97fa640 1 -- 172.21.15.100:0/3489832530 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f99d400a040 con 0x7f99dc106100 2024-03-12T05:31:21.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 Processor -- start 2024-03-12T05:31:21.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 -- start start 2024-03-12T05:31:21.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99d97fa640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99c40a1ba0 0x7f99c40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 0x55defe2ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ceb70 0x55defe2d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe289960 con 0x55defe2d7750 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe23d310 con 0x55defe2ce250 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.817+0000 7f99e5564740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe23a840 con 0x55defe2ceb70 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99c40a1ba0 0x7f99c40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 0x55defe2ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 0x55defe2ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40868/0 (socket says 172.21.15.100:40868) 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 -- 172.21.15.100:0/288942350 learned_addr learned my addr 172.21.15.100:0/288942350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99d97fa640 1 -- 172.21.15.100:0/3489832530 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f99d40f0250 con 0x7f99dc106100 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99db7fe640 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ceb70 0x55defe2d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 -- 172.21.15.100:0/288942350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ceb70 msgr2=0x55defe2d7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ceb70 0x55defe2d7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 -- 172.21.15.100:0/288942350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 msgr2=0x55defe2d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 -- 172.21.15.100:0/288942350 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55defe1f8060 con 0x55defe2ce250 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99db7fe640 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ceb70 0x55defe2d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99dbfff640 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 0x55defe2ce630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009e10 tx=0x7f99cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec56c089359d8729 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99c40a1ba0 0x7f99c40a4060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55defe2cdd20 tx=0x7f99d0009840 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99c27fc640 1 -- 172.21.15.100:0/288942350 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc019630 con 0x55defe2ce250 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99c27fc640 1 -- 172.21.15.100:0/288942350 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99cc019c70 con 0x55defe2ce250 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.818+0000 7f99c27fc640 1 -- 172.21.15.100:0/288942350 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00fd20 con 0x55defe2ce250 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 -- 172.21.15.100:0/288942350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 msgr2=0x55defe2ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 0x55defe2ce630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009e10 tx=0x7f99cc011810 comp rx=0 tx=0).stop 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 -- 172.21.15.100:0/288942350 shutdown_connections 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ceb70 0x55defe2d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 --2- 172.21.15.100:0/288942350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ce250 0x55defe2ce630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 -- 172.21.15.100:0/288942350 >> 172.21.15.100:0/288942350 conn(0x55defe2c8980 msgr2=0x55defe2c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 -- 172.21.15.100:0/288942350 shutdown_connections 2024-03-12T05:31:21.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 -- 172.21.15.100:0/288942350 wait complete. 2024-03-12T05:31:21.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.819+0000 7f99e5564740 1 -- start start 2024-03-12T05:31:21.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 Processor -- start 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe236020 con 0x55defe2d7750 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defdf09430 con 0x55defe2ceb70 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defdec5e00 con 0x55defe2ce250 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e5564740 10 client.0 _ll_register_callbacks cb 0x55defe1ddcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40876/0 (socket says 172.21.15.100:40876) 2024-03-12T05:31:21.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99db7fe640 1 -- 172.21.15.100:0/3628202516 learned_addr learned my addr 172.21.15.100:0/3628202516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49686/0 (socket says 172.21.15.100:49686) 2024-03-12T05:31:21.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.820+0000 7f99dbfff640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:21 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2776326978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99db7fe640 1 -- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 msgr2=0x55defe2d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99db7fe640 1 -- 172.21.15.100:0/3628202516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 msgr2=0x55defe2d5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99db7fe640 1 -- 172.21.15.100:0/3628202516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc009a70 con 0x55defe2ceb70 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d4cc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99d000f8f0 tx=0x7f99d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632ed21ea110064c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3628202516 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0004260 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3628202516 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f99d00043c0 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55defdf357e0 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3628202516 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d0010930 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99e5564740 10 client.28883 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55defe2346a0 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.821+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3628202516 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55defdf357e0 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3628202516 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99d0005030 con 0x55defe2ceb70 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99e5564740 10 client.28883 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99e5564740 10 client.28883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99e5564740 20 client.28883 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99e5564740 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe2ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99e5564740 4 client.28883 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55defdf72720 con 0x55defe2e7c40 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99dbfff640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe2ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99bb7fe640 10 client.28883 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.822+0000 7f99dbfff640 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe2ea060 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9c1bee4f6b3bca3 server_cookie=db1d4f09d5ee6823 in_seq=0 out_seq=0 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3628202516 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55defdf72720 con 0x55defe2e7c40 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99bb7fe640 10 client.28883 handle_command_reply: tid=0 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 client.28883 shutdown 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 msgr2=0x55defe2d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d4cc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99d000f8f0 tx=0x7f99d0005f90 comp rx=0 tx=0).stop 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 shutdown_connections 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe2ea060 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99bb7fe640 0 client.28883 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:21.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d4cc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 --2- 172.21.15.100:0/3628202516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 >> 172.21.15.100:0/3628202516 conn(0x55defe2c8980 msgr2=0x55defe2ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 shutdown_connections 2024-03-12T05:31:21.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 1 -- 172.21.15.100:0/3628202516 wait complete. 2024-03-12T05:31:21.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.824+0000 7f99e5564740 20 client.28883 trim_cache size 0 max 16384 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 Processor -- start 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 -- start start 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe1bb130 con 0x55defe2d7750 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe2cd5d0 con 0x55defe2ceb70 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e5564740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe2c9980 con 0x55defe2ce250 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49694/0 (socket says 172.21.15.100:49694) 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99e0f29640 1 -- 172.21.15.100:0/446641636 learned_addr learned my addr 172.21.15.100:0/446641636 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.837+0000 7f99dbfff640 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99e0f29640 1 -- 172.21.15.100:0/446641636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 msgr2=0x55defe2d6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99e0f29640 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99e0f29640 1 -- 172.21.15.100:0/446641636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 msgr2=0x55defe2d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99e0f29640 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99e0f29640 1 -- 172.21.15.100:0/446641636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55defe1d0c20 con 0x55defe2d7750 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99db7fe640 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99e0f29640 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d19e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f99d40edf60 tx=0x7f99d404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a9748b37a128e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99c1ffb640 1 -- 172.21.15.100:0/446641636 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40bcb00 con 0x55defe2d7750 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99c1ffb640 1 -- 172.21.15.100:0/446641636 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99d40bc920 con 0x55defe2d7750 2024-03-12T05:31:21.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.838+0000 7f99c1ffb640 1 -- 172.21.15.100:0/446641636 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40b9630 con 0x55defe2d7750 2024-03-12T05:31:21.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 -- 172.21.15.100:0/446641636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 msgr2=0x55defe2d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d19e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f99d40edf60 tx=0x7f99d404b000 comp rx=0 tx=0).stop 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 -- 172.21.15.100:0/446641636 shutdown_connections 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2d19e0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 --2- 172.21.15.100:0/446641636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 -- 172.21.15.100:0/446641636 >> 172.21.15.100:0/446641636 conn(0x55defe2c8980 msgr2=0x55defe2cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 -- 172.21.15.100:0/446641636 shutdown_connections 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.839+0000 7f99e5564740 1 -- 172.21.15.100:0/446641636 wait complete. 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 -- start start 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 Processor -- start 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe2ddce0 con 0x55defe2d7750 2024-03-12T05:31:21.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe33e170 con 0x55defe2ceb70 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55defe33e2b0 con 0x55defe2ce250 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e5564740 10 client.0 _ll_register_callbacks cb 0x7f99cc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e0f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2dd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49708/0 (socket says 172.21.15.100:49708) 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2dcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40886/0 (socket says 172.21.15.100:40886) 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99e0f29640 1 -- 172.21.15.100:0/3692476868 learned_addr learned my addr 172.21.15.100:0/3692476868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.840+0000 7f99dbfff640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e0f29640 1 -- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 msgr2=0x55defe2deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e0f29640 1 -- 172.21.15.100:0/3692476868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 msgr2=0x55defe2dcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2dcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e0f29640 1 -- 172.21.15.100:0/3692476868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55defe2346a0 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99db7fe640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e0f29640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2dd5e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f99d402c760 tx=0x7f99d40bcb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc798c02cdcb2273 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3692476868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40c10a0 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3692476868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f99d40dce40 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99cc019c70 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3692476868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d40bd650 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e5564740 10 client.19545 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f99dc1a65d0 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.841+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3692476868 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99cc019c70 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3692476868 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99d40bc410 con 0x55defe2d7750 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99e5564740 10 client.19545 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99e5564740 10 client.19545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:31:21.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99e5564740 20 client.19545 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99e5564740 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe3416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99e5564740 4 client.19545 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:31:21.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55defdfc1890 con 0x55defe2e7c40 2024-03-12T05:31:21.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99dbfff640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe3416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99bb7fe640 10 client.19545 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:21.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.842+0000 7f99dbfff640 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe3416e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fced69cb99c7ce8c server_cookie=763f7ec9ac3dc5ee in_seq=0 out_seq=0 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.844+0000 7f99bb7fe640 1 -- 172.21.15.100:0/3692476868 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55defdfc1890 con 0x55defe2e7c40 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.844+0000 7f99bb7fe640 10 client.19545 handle_command_reply: tid=0 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.844+0000 7f99e5564740 1 client.19545 shutdown 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 msgr2=0x55defe2dd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2dd5e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f99d402c760 tx=0x7f99d40bcb80 comp rx=0 tx=0).stop 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 shutdown_connections 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55defe2d7750 0x55defe2dd5e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55defe2e7c40 0x55defe3416e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55defe2ceb70 0x55defe2dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 --2- 172.21.15.100:0/3692476868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55defe2ce250 0x55defe2deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99bb7fe640 0 client.19545 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 >> 172.21.15.100:0/3692476868 conn(0x55defe2c8980 msgr2=0x55defe2c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 shutdown_connections 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 1 -- 172.21.15.100:0/3692476868 wait complete. 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e5564740 20 client.19545 trim_cache size 0 max 16384 2024-03-12T05:31:21.842 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99c40a1ba0 msgr2=0x7f99c40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e298c640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99c40a1ba0 0x7f99c40a4060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55defe2cdd20 tx=0x7f99d0009840 comp rx=0 tx=0).stop 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc106100 msgr2=0x7f99dc199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.845+0000 7f99e298c640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc106100 0x7f99dc199710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99d402c790 tx=0x7f99d40346f0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 shutdown_connections 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f99c40a1ba0 0x7f99c40a4060 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99dc106100 0x7f99dc199710 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1009e0 0x7f99dc19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 --2- 172.21.15.100:0/3489832530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99dc100040 0x7f99dc19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 >> 172.21.15.100:0/3489832530 conn(0x7f99dc077110 msgr2=0x7f99dc0692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 shutdown_connections 2024-03-12T05:31:21.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.846+0000 7f99e298c640 1 -- 172.21.15.100:0/3489832530 wait complete. 2024-03-12T05:31:21.852 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:31:21.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:31:21.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:21 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2776326978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:31:21.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:21 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2776326978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 -- 172.21.15.100:0/4261257735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8181057a0 msgr2=0x7fb818105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 --2- 172.21.15.100:0/4261257735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8181057a0 0x7fb818105b80 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb808009930 tx=0x7fb80802bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 -- 172.21.15.100:0/4261257735 shutdown_connections 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 --2- 172.21.15.100:0/4261257735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb818100150 0x7fb81810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 --2- 172.21.15.100:0/4261257735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8180ff790 0x7fb8180ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 --2- 172.21.15.100:0/4261257735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8181057a0 0x7fb818105b80 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 -- 172.21.15.100:0/4261257735 >> 172.21.15.100:0/4261257735 conn(0x7fb8180765a0 msgr2=0x7fb8180fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 -- 172.21.15.100:0/4261257735 shutdown_connections 2024-03-12T05:31:21.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.984+0000 7fb81d281640 1 -- 172.21.15.100:0/4261257735 wait complete. 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 Processor -- start 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 -- start start 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 0x7fb81819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 0x7fb81819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8181057a0 0x7fb818197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818197c60 con 0x7fb818100150 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818197dd0 con 0x7fb8180ff790 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb81d281640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818197f90 con 0x7fb8181057a0 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 0x7fb81819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 0x7fb81819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 0x7fb81819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40900/0 (socket says 172.21.15.100:40900) 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 0x7fb81819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49722/0 (socket says 172.21.15.100:49722) 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb816ffd640 1 -- 172.21.15.100:0/3315578503 learned_addr learned my addr 172.21.15.100:0/3315578503 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.985+0000 7fb8177fe640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8181057a0 0x7fb818197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb8167fc640 1 -- 172.21.15.100:0/3315578503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8181057a0 msgr2=0x7fb818197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb8167fc640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8181057a0 0x7fb818197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb8167fc640 1 -- 172.21.15.100:0/3315578503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 msgr2=0x7fb81819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb8167fc640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 0x7fb81819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb8167fc640 1 -- 172.21.15.100:0/3315578503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb808009560 con 0x7fb818100150 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb816ffd640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 0x7fb81819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb8167fc640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 0x7fb81819d9a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0099c0 tx=0x7fb80c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb7fffff640 1 -- 172.21.15.100:0/3315578503 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c014070 con 0x7fb818100150 2024-03-12T05:31:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818198270 con 0x7fb818100150 2024-03-12T05:31:21.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb818106810 con 0x7fb818100150 2024-03-12T05:31:21.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb7fffff640 1 -- 172.21.15.100:0/3315578503 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb80c007da0 con 0x7fb818100150 2024-03-12T05:31:21.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.986+0000 7fb7fffff640 1 -- 172.21.15.100:0/3315578503 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004b60 con 0x7fb818100150 2024-03-12T05:31:21.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 Processor -- start 2024-03-12T05:31:21.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 -- start start 2024-03-12T05:31:21.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb7fffff640 1 -- 172.21.15.100:0/3315578503 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb80c018070 con 0x7fb818100150 2024-03-12T05:31:21.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3d4b70 0x56149e3dd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 0x56149e3dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e38f960 con 0x56149e3dd750 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e343310 con 0x56149e3d4250 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e340840 con 0x56149e3d4b70 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 0x56149e3dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb8177fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3d4b70 0x56149e3dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb8177fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3d4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40904/0 (socket says 172.21.15.100:40904) 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb8177fe640 1 -- 172.21.15.100:0/2143159358 learned_addr learned my addr 172.21.15.100:0/2143159358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.989+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 0x56149e3dfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49738/0 (socket says 172.21.15.100:49738) 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8177fe640 1 -- 172.21.15.100:0/2143159358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3d4b70 msgr2=0x56149e3dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8177fe640 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3d4b70 0x56149e3dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8177fe640 1 -- 172.21.15.100:0/2143159358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 msgr2=0x56149e3dfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8177fe640 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 0x56149e3dfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8177fe640 1 -- 172.21.15.100:0/2143159358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56149e2fe060 con 0x56149e3d4250 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb7fffff640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8000a1c20 0x7fb8000a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8167fc640 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 0x56149e3dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb8177fe640 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3d4630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb810009f90 tx=0x7fb810010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee1d0fc7b390669 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb7fffff640 1 -- 172.21.15.100:0/3315578503 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb80c0d08c0 con 0x7fb818100150 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb816ffd640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8000a1c20 0x7fb8000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb7fd7fa640 1 -- 172.21.15.100:0/2143159358 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810011700 con 0x56149e3d4250 2024-03-12T05:31:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.990+0000 7fb7fd7fa640 1 -- 172.21.15.100:0/2143159358 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb810015070 con 0x56149e3d4250 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb7fd7fa640 1 -- 172.21.15.100:0/2143159358 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810019cd0 con 0x56149e3d4250 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2143159358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 msgr2=0x56149e3d4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3d4630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb810009f90 tx=0x7fb810010860 comp rx=0 tx=0).stop 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb816ffd640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8000a1c20 0x7fb8000a40e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb808000c00 tx=0x7fb8080058d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2143159358 shutdown_connections 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3dd750 0x56149e3dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3d4b70 0x56149e3dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2143159358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3d4630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2143159358 >> 172.21.15.100:0/2143159358 conn(0x56149e3ce980 msgr2=0x56149e3ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2143159358 shutdown_connections 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.991+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2143159358 wait complete. 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 -- start start 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 Processor -- start 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4250 0x56149e3da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 0x56149e3daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e33c020 con 0x56149e3d4250 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e00f430 con 0x56149e3d4b70 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149dfcbe00 con 0x56149e3dd750 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb81fe4f740 10 client.0 _ll_register_callbacks cb 0x56149e2e3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 0x56149e3daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 0x56149e3daaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40912/0 (socket says 172.21.15.100:40912) 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb8167fc640 1 -- 172.21.15.100:0/4187753319 learned_addr learned my addr 172.21.15.100:0/4187753319 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.992+0000 7fb8177fe640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb816ffd640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4250 0x56149e3da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb8167fc640 1 -- 172.21.15.100:0/4187753319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 msgr2=0x56149e3db030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb8167fc640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3db030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb8167fc640 1 -- 172.21.15.100:0/4187753319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4250 msgr2=0x56149e3da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb8167fc640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4250 0x56149e3da5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb8167fc640 1 -- 172.21.15.100:0/4187753319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb810009bf0 con 0x56149e3d4b70 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb8167fc640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 0x56149e3daaf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009fd0 tx=0x7fb80c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d431ea3a614d0e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb7faff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb7faff5640 1 -- 172.21.15.100:0/4187753319 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c09d2d0 con 0x56149e3d4b70 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb7faff5640 1 -- 172.21.15.100:0/4187753319 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb80c09d500 con 0x56149e3d4b70 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56149e03b7e0 con 0x56149e3d4b70 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.993+0000 7fb7faff5640 1 -- 172.21.15.100:0/4187753319 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80c09d040 con 0x56149e3d4b70 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 10 client.28901 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56149e33a6a0 con 0x56149e3d4b70 2024-03-12T05:31:21.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb7faff5640 1 -- 172.21.15.100:0/4187753319 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb80c0b5070 con 0x56149e3d4b70 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb7faff5640 1 -- 172.21.15.100:0/4187753319 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb80c0a1160 con 0x56149e3d4b70 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 10 client.28901 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 10 client.28901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 20 client.28901 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e3edb20 0x56149e3eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 4 client.28901 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56149e078720 con 0x56149e3edb20 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.994+0000 7fb816ffd640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e3edb20 0x56149e3eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.995+0000 7fb816ffd640 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e3edb20 0x56149e3eff40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b07a4d83991d40e server_cookie=14bc3200cbfd4e36 in_seq=0 out_seq=0 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.995+0000 7fb7faff5640 10 client.28901 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb7faff5640 1 -- 172.21.15.100:0/4187753319 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56149e078720 con 0x56149e3edb20 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb7faff5640 10 client.28901 handle_command_reply: tid=0 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 client.28901 shutdown 2024-03-12T05:31:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 msgr2=0x56149e3daaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 0x56149e3daaf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009fd0 tx=0x7fb80c004730 comp rx=0 tx=0).stop 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 shutdown_connections 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e3edb20 0x56149e3eff40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb7faff5640 0 client.28901 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4b70 0x56149e3daaf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/4187753319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4250 0x56149e3da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 >> 172.21.15.100:0/4187753319 conn(0x56149e3ce980 msgr2=0x56149e3d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 shutdown_connections 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 1 -- 172.21.15.100:0/4187753319 wait complete. 2024-03-12T05:31:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:21.996+0000 7fb81fe4f740 20 client.28901 trim_cache size 0 max 16384 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 Processor -- start 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 -- start start 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e2c1130 con 0x56149e3d4b70 2024-03-12T05:31:22.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e3d3eb0 con 0x56149e3d4250 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb81fe4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e3ce7b0 con 0x56149e3dd750 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49754/0 (socket says 172.21.15.100:49754) 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.010+0000 7fb8167fc640 1 -- 172.21.15.100:0/3311924301 learned_addr learned my addr 172.21.15.100:0/3311924301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb8177fe640 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb816ffd640 1 -- 172.21.15.100:0/3311924301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 msgr2=0x56149e3d9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb816ffd640 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3d9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb816ffd640 1 -- 172.21.15.100:0/3311924301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 msgr2=0x56149e3dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb816ffd640 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb816ffd640 1 -- 172.21.15.100:0/3311924301 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56149e2d6c20 con 0x56149e3d4250 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb8167fc640 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb816ffd640 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3dcce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb808009900 tx=0x7fb808045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a1c4ceea7c9bda server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:22.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb7fcff9640 1 -- 172.21.15.100:0/3311924301 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808055ca0 con 0x56149e3d4250 2024-03-12T05:31:22.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb7fcff9640 1 -- 172.21.15.100:0/3311924301 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb808055e00 con 0x56149e3d4250 2024-03-12T05:31:22.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.011+0000 7fb7fcff9640 1 -- 172.21.15.100:0/3311924301 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808003cc0 con 0x56149e3d4250 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 -- 172.21.15.100:0/3311924301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 msgr2=0x56149e3dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3dcce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb808009900 tx=0x7fb808045000 comp rx=0 tx=0).stop 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 -- 172.21.15.100:0/3311924301 shutdown_connections 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e3d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/3311924301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3dcce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 -- 172.21.15.100:0/3311924301 >> 172.21.15.100:0/3311924301 conn(0x56149e3ce980 msgr2=0x56149e3d8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 -- 172.21.15.100:0/3311924301 shutdown_connections 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 -- 172.21.15.100:0/3311924301 wait complete. 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.012+0000 7fb81fe4f740 1 -- start start 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 Processor -- start 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3e3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40920/0 (socket says 172.21.15.100:40920) 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3e3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49764/0 (socket says 172.21.15.100:49764) 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb816ffd640 1 -- 172.21.15.100:0/2118733114 learned_addr learned my addr 172.21.15.100:0/2118733114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e444030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e444570 con 0x56149e3d4b70 2024-03-12T05:31:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e4446b0 con 0x56149e3d4250 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.013+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149e4447f0 con 0x56149e3dd750 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb816ffd640 1 -- 172.21.15.100:0/2118733114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 msgr2=0x56149e444030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb816ffd640 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e444030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb816ffd640 1 -- 172.21.15.100:0/2118733114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 msgr2=0x56149e3e3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb816ffd640 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3e3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb816ffd640 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56149e33a6a0 con 0x56149e3d4250 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb8167fc640 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb81fe4f740 10 client.0 _ll_register_callbacks cb 0x7fb810009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb810015070 con 0x56149e3d4250 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb816ffd640 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3e3810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb80802bc70 tx=0x7fb808055d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463a1bef8918f05d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb7faff5640 10 client.28907 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb7faff5640 1 -- 172.21.15.100:0/2118733114 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808064070 con 0x56149e3d4250 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb7faff5640 1 -- 172.21.15.100:0/2118733114 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb808052050 con 0x56149e3d4250 2024-03-12T05:31:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.014+0000 7fb7faff5640 1 -- 172.21.15.100:0/2118733114 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb808003e80 con 0x56149e3d4250 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 10 client.28907 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x7fb81810cf60 con 0x56149e3d4250 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb7faff5640 1 -- 172.21.15.100:0/2118733114 <== mon.2 v2:172.21.15.156:3300/0 4 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb808067070 con 0x56149e3d4250 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 10 client.28907 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 10 client.28907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 20 client.28907 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e4459f0 0x56149e447e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 4 client.28907 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56149e0c7890 con 0x56149e4459f0 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.015+0000 7fb8167fc640 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e4459f0 0x56149e447e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb7faff5640 10 client.28907 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:22.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb8167fc640 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e4459f0 0x56149e447e90 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e229b2b436837e27 server_cookie=63dccbd35a2192ab in_seq=0 out_seq=0 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb7faff5640 1 -- 172.21.15.100:0/2118733114 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56149e0c7890 con 0x56149e4459f0 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb7faff5640 10 client.28907 handle_command_reply: tid=0 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 client.28907 shutdown 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 msgr2=0x56149e3e3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3e3810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb80802bc70 tx=0x7fb808055d30 comp rx=0 tx=0).stop 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 shutdown_connections 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56149e4459f0 0x56149e447e90 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56149e3dd750 0x56149e444030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149e3d4b70 0x56149e3e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 --2- 172.21.15.100:0/2118733114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56149e3d4250 0x56149e3e3810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 >> 172.21.15.100:0/2118733114 conn(0x56149e3ce980 msgr2=0x56149e3cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb7faff5640 0 client.28907 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 shutdown_connections 2024-03-12T05:31:22.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 1 -- 172.21.15.100:0/2118733114 wait complete. 2024-03-12T05:31:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.016+0000 7fb81fe4f740 20 client.28907 trim_cache size 0 max 16384 2024-03-12T05:31:22.016 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8000a1c20 msgr2=0x7fb8000a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8000a1c20 0x7fb8000a40e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb808000c00 tx=0x7fb8080058d0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 msgr2=0x7fb81819d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 0x7fb81819d9a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0099c0 tx=0x7fb80c009e80 comp rx=0 tx=0).stop 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 shutdown_connections 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb8000a1c20 0x7fb8000a40e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8181057a0 0x7fb818197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb818100150 0x7fb81819d9a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 --2- 172.21.15.100:0/3315578503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8180ff790 0x7fb81819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 >> 172.21.15.100:0/3315578503 conn(0x7fb8180765a0 msgr2=0x7fb818101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 shutdown_connections 2024-03-12T05:31:22.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:22.017+0000 7fb81d281640 1 -- 172.21.15.100:0/3315578503 wait complete. 2024-03-12T05:31:22.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 197712 27.21 MB/sec execute 245 sec latency 135.054 ms 2024-03-12T05:31:22.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:22 smithi100 ceph-mon[32210]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 611 op/s 2024-03-12T05:31:22.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:22 smithi156 ceph-mon[36270]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 611 op/s 2024-03-12T05:31:22.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:22 smithi161 ceph-mon[36043]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 611 op/s 2024-03-12T05:31:23.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:31:23.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:31:23.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:31:23.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 -- 172.21.15.100:0/1506595361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 msgr2=0x7f0c2010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 --2- 172.21.15.100:0/1506595361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c2010e000 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009cd0 tx=0x7f0c1802c180 comp rx=0 tx=0).stop 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 -- 172.21.15.100:0/1506595361 shutdown_connections 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 --2- 172.21.15.100:0/1506595361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c2010e000 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 --2- 172.21.15.100:0/1506595361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c200ffa40 0x7f0c200ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 --2- 172.21.15.100:0/1506595361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c20107380 0x7f0c20107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 -- 172.21.15.100:0/1506595361 >> 172.21.15.100:0/1506595361 conn(0x7f0c200753c0 msgr2=0x7f0c200757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 -- 172.21.15.100:0/1506595361 shutdown_connections 2024-03-12T05:31:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.146+0000 7f0c24e96640 1 -- 172.21.15.100:0/1506595361 wait complete. 2024-03-12T05:31:23.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 Processor -- start 2024-03-12T05:31:23.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 -- start start 2024-03-12T05:31:23.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c200ffa40 0x7f0c20100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c20101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c20107380 0x7f0c201016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a5b50 con 0x7f0c20100400 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a5cc0 con 0x7f0c20107380 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.147+0000 7f0c24e96640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a5e80 con 0x7f0c200ffa40 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c20101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c20101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49778/0 (socket says 172.21.15.100:49778) 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2884238502 learned_addr learned my addr 172.21.15.100:0/2884238502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c200ffa40 0x7f0c20100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c20107380 0x7f0c201016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2884238502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c200ffa40 msgr2=0x7f0c20100c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c200ffa40 0x7f0c20100c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2884238502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c20107380 msgr2=0x7f0c201016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c20107380 0x7f0c201016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2884238502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009900 con 0x7f0c20100400 2024-03-12T05:31:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c20101160 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400e9f0 tx=0x7f0c1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c0f7fe640 1 -- 172.21.15.100:0/2884238502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14009700 con 0x7f0c20100400 2024-03-12T05:31:23.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c0f7fe640 1 -- 172.21.15.100:0/2884238502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c1400cd80 con 0x7f0c20100400 2024-03-12T05:31:23.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c201a6160 con 0x7f0c20100400 2024-03-12T05:31:23.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c0f7fe640 1 -- 172.21.15.100:0/2884238502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14018920 con 0x7f0c20100400 2024-03-12T05:31:23.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.148+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c201a66c0 con 0x7f0c20100400 2024-03-12T05:31:23.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c0f7fe640 1 -- 172.21.15.100:0/2884238502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0c14018a80 con 0x7f0c20100400 2024-03-12T05:31:23.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 Processor -- start 2024-03-12T05:31:23.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 -- start start 2024-03-12T05:31:23.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb454f90 con 0x563ceb4e6d30 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb4524c0 con 0x563ceb4e6380 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c27a62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb44dca0 con 0x563ceb4ef8b0 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c0f7fe640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c080a1c20 0x7f0c080a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c1e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c1e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49786/0 (socket says 172.21.15.100:49786) 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c1e59c640 1 -- 172.21.15.100:0/821556369 learned_addr learned my addr 172.21.15.100:0/821556369 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.151+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c080a1c20 0x7f0c080a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c0f7fe640 1 -- 172.21.15.100:0/2884238502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c14014070 con 0x7f0c20100400 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1e59c640 1 -- 172.21.15.100:0/821556369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 msgr2=0x563ceb4f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1e59c640 1 -- 172.21.15.100:0/821556369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 msgr2=0x563ceb4e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1e59c640 1 -- 172.21.15.100:0/821556369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ceb4100b0 con 0x563ceb4e6d30 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4ef2e0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009c70 tx=0x7f0c10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4caa5edb5802b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c080a1c20 0x7f0c080a40e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005cc0 tx=0x7f0c18005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c0cff9640 1 -- 172.21.15.100:0/821556369 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10004230 con 0x563ceb4e6d30 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c0cff9640 1 -- 172.21.15.100:0/821556369 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c10004390 con 0x563ceb4e6d30 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.152+0000 7f0c0cff9640 1 -- 172.21.15.100:0/821556369 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100197a0 con 0x563ceb4e6d30 2024-03-12T05:31:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 -- 172.21.15.100:0/821556369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 msgr2=0x563ceb4ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4ef2e0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009c70 tx=0x7f0c10011810 comp rx=0 tx=0).stop 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 -- 172.21.15.100:0/821556369 shutdown_connections 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4ef2e0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 --2- 172.21.15.100:0/821556369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 -- 172.21.15.100:0/821556369 >> 172.21.15.100:0/821556369 conn(0x563ceb4e0840 msgr2=0x563ceb4e0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 -- 172.21.15.100:0/821556369 shutdown_connections 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 -- 172.21.15.100:0/821556369 wait complete. 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 -- start start 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.153+0000 7f0c27a62740 1 Processor -- start 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 0x563ceb4ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6d30 0x563ceb4ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb3f2f40 con 0x563ceb4e6380 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb1214b0 con 0x563ceb4e6d30 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb3d3670 con 0x563ceb4ef8b0 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c27a62740 10 client.0 _ll_register_callbacks cb 0x563ceb440f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 0x563ceb4ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 0x563ceb4ec7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49794/0 (socket says 172.21.15.100:49794) 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1e59c640 1 -- 172.21.15.100:0/2885301878 learned_addr learned my addr 172.21.15.100:0/2885301878 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6d30 0x563ceb4ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2885301878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 msgr2=0x563ceb4ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2885301878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 msgr2=0x563ceb4ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 0x563ceb4ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/2885301878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c100098d0 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 0x563ceb4ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6d30 0x563ceb4ecd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c140cb250 tx=0x7f0c1406a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cbe9e825a39058c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.154+0000 7f0c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c05ffb640 1 -- 172.21.15.100:0/2885301878 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140b8ea0 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ceb427f40 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c05ffb640 1 -- 172.21.15.100:0/2885301878 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c1409a7e0 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c05ffb640 1 -- 172.21.15.100:0/2885301878 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c1409e880 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 10 client.28919 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563ceb44c320 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c05ffb640 1 -- 172.21.15.100:0/2885301878 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c14068080 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c05ffb640 1 -- 172.21.15.100:0/2885301878 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0c140ba0b0 con 0x563ceb4e6d30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 10 client.28919 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 10 client.28919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:23.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 20 client.28919 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb4ffe20 0x563ceb502230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 4 client.28919 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.155+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ceb444310 con 0x563ceb4ffe20 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.156+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb4ffe20 0x563ceb502230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.156+0000 7f0c05ffb640 10 client.28919 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.156+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb4ffe20 0x563ceb502230 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ec4d5381e3e307 server_cookie=502f9f5d0c5200bb in_seq=0 out_seq=0 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c05ffb640 1 -- 172.21.15.100:0/2885301878 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ceb444310 con 0x563ceb4ffe20 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c05ffb640 10 client.28919 handle_command_reply: tid=0 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 client.28919 shutdown 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6d30 msgr2=0x563ceb4ecd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6d30 0x563ceb4ecd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c140cb250 tx=0x7f0c1406a940 comp rx=0 tx=0).stop 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 shutdown_connections 2024-03-12T05:31:23.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb4ffe20 0x563ceb502230 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4ef8b0 0x563ceb4ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6d30 0x563ceb4ecd00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c05ffb640 0 client.28919 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 --2- 172.21.15.100:0/2885301878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6380 0x563ceb4ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 >> 172.21.15.100:0/2885301878 conn(0x563ceb4e0840 msgr2=0x563ceb4e27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 shutdown_connections 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 1 -- 172.21.15.100:0/2885301878 wait complete. 2024-03-12T05:31:23.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.157+0000 7f0c27a62740 20 client.28919 trim_cache size 0 max 16384 2024-03-12T05:31:23.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.176+0000 7f0c27a62740 1 Processor -- start 2024-03-12T05:31:23.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 -- start start 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6380 0x563ceb4eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4ef8b0 0x563ceb4e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb4f1ca0 con 0x563ceb4e6d30 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb4e15d0 con 0x563ceb4ef8b0 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c27a62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb4ead10 con 0x563ceb4e6380 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4e9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49798/0 (socket says 172.21.15.100:49798) 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/4057259090 learned_addr learned my addr 172.21.15.100:0/4057259090 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4ef8b0 0x563ceb4e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6380 0x563ceb4eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/4057259090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6380 msgr2=0x563ceb4eeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6380 0x563ceb4eeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/4057259090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4ef8b0 msgr2=0x563ceb4e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4ef8b0 0x563ceb4e9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 -- 172.21.15.100:0/4057259090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ceb3e8c00 con 0x563ceb4e6d30 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.177+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4e9a20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0c140a29a0 tx=0x7f0c14010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fa8ab98569bfc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c07fff640 1 -- 172.21.15.100:0/4057259090 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140b8ea0 con 0x563ceb4e6d30 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c07fff640 1 -- 172.21.15.100:0/4057259090 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563ceb427f40 con 0x563ceb4e6d30 2024-03-12T05:31:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c07fff640 1 -- 172.21.15.100:0/4057259090 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14099df0 con 0x563ceb4e6d30 2024-03-12T05:31:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 -- 172.21.15.100:0/4057259090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 msgr2=0x563ceb4e9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4e9a20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0c140a29a0 tx=0x7f0c14010f50 comp rx=0 tx=0).stop 2024-03-12T05:31:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 -- 172.21.15.100:0/4057259090 shutdown_connections 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4ef8b0 0x563ceb4e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4e6d30 0x563ceb4e9a20 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 --2- 172.21.15.100:0/4057259090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6380 0x563ceb4eeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 -- 172.21.15.100:0/4057259090 >> 172.21.15.100:0/4057259090 conn(0x563ceb4e0840 msgr2=0x563ceb4eb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 -- 172.21.15.100:0/4057259090 shutdown_connections 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.178+0000 7f0c27a62740 1 -- 172.21.15.100:0/4057259090 wait complete. 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 -- start start 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 Processor -- start 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6d30 0x563ceb4f6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 0x563ceb5503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb5508e0 con 0x563ceb4ef8b0 2024-03-12T05:31:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb550a50 con 0x563ceb4e6380 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ceb550c10 con 0x563ceb4e6d30 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c27a62740 10 client.0 _ll_register_callbacks cb 0x7f0c100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 0x563ceb5503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40966/0 (socket says 172.21.15.100:40966) 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 0x563ceb5503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49812/0 (socket says 172.21.15.100:49812) 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1dd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6d30 0x563ceb4f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 -- 172.21.15.100:0/3996281213 learned_addr learned my addr 172.21.15.100:0/3996281213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 -- 172.21.15.100:0/3996281213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6d30 msgr2=0x563ceb4f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6d30 0x563ceb4f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 -- 172.21.15.100:0/3996281213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 msgr2=0x563ceb5503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 0x563ceb5503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1e59c640 1 -- 172.21.15.100:0/3996281213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ceb44c320 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.179+0000 7f0c1ed9d640 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 0x563ceb5503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c1e59c640 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4f5bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c10005f40 tx=0x7f0c10009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379492ab1800a2bd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c05ffb640 1 -- 172.21.15.100:0/3996281213 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10003a00 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c05ffb640 1 -- 172.21.15.100:0/3996281213 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0c10003b60 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ceb473900 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c05ffb640 1 -- 172.21.15.100:0/3996281213 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c10003d20 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 10 client.28925 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563ceb40e910 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c05ffb640 1 -- 172.21.15.100:0/3996281213 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563ceb473900 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c05ffb640 1 -- 172.21.15.100:0/3996281213 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0c10019b80 con 0x563ceb4e6380 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 10 client.28925 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 10 client.28925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 20 client.28925 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.180+0000 7f0c27a62740 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb5583a0 0x563ceb55a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.181+0000 7f0c27a62740 4 client.28925 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.181+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563ceb14d860 con 0x563ceb5583a0 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.181+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb5583a0 0x563ceb55a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.181+0000 7f0c05ffb640 10 client.28925 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.181+0000 7f0c1dd9b640 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb5583a0 0x563ceb55a7c0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341d51f2116bcf4 server_cookie=ad35e0f3a3b25fbf in_seq=0 out_seq=0 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.182+0000 7f0c05ffb640 1 -- 172.21.15.100:0/3996281213 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563ceb14d860 con 0x563ceb5583a0 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.182+0000 7f0c05ffb640 10 client.28925 handle_command_reply: tid=0 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 client.28925 shutdown 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 msgr2=0x563ceb4f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4f5bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c10005f40 tx=0x7f0c10009eb0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 shutdown_connections 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x563ceb5583a0 0x563ceb55a7c0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ceb4ef8b0 0x563ceb5503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ceb4e6d30 0x563ceb4f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c05ffb640 0 client.28925 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 --2- 172.21.15.100:0/3996281213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563ceb4e6380 0x563ceb4f5bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 >> 172.21.15.100:0/3996281213 conn(0x563ceb4e0840 msgr2=0x563ceb4ec780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 shutdown_connections 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 1 -- 172.21.15.100:0/3996281213 wait complete. 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c27a62740 20 client.28925 trim_cache size 0 max 16384 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:31:23.182 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "fe04054f-3a7b-4c52-a159-37ce7fb95964", 2024-03-12T05:31:23.183 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:31:23.183 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:31:23.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c080a1c20 msgr2=0x7f0c080a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c24e96640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c080a1c20 0x7f0c080a40e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005cc0 tx=0x7f0c18005c50 comp rx=0 tx=0).stop 2024-03-12T05:31:23.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 msgr2=0x7f0c20101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c24e96640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c20101160 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400e9f0 tx=0x7f0c1400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.183+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 shutdown_connections 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f0c080a1c20 0x7f0c080a40e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c20107380 0x7f0c201016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20100400 0x7f0c20101160 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 --2- 172.21.15.100:0/2884238502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c200ffa40 0x7f0c20100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 >> 172.21.15.100:0/2884238502 conn(0x7f0c200753c0 msgr2=0x7f0c200fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 shutdown_connections 2024-03-12T05:31:23.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.184+0000 7f0c24e96640 1 -- 172.21.15.100:0/2884238502 wait complete. 2024-03-12T05:31:23.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:31:23.190 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:31:23.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 198078 27.15 MB/sec execute 246 sec latency 363.516 ms 2024-03-12T05:31:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 -- 172.21.15.100:0/2455907980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8100c20 msgr2=0x7f00c8104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2455907980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8100c20 0x7f00c8104c60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009930 tx=0x7f00b402bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 -- 172.21.15.100:0/2455907980 shutdown_connections 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2455907980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8105340 0x7f00c810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2455907980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8100c20 0x7f00c8104c60 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2455907980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c8100270 0x7f00c8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 -- 172.21.15.100:0/2455907980 >> 172.21.15.100:0/2455907980 conn(0x7f00c8077230 msgr2=0x7f00c8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 -- 172.21.15.100:0/2455907980 shutdown_connections 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 -- 172.21.15.100:0/2455907980 wait complete. 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.312+0000 7f00ceb13640 1 Processor -- start 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 -- start start 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8100270 0x7f00c8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c8105340 0x7f00c8102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102780 0x7f00c81a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c81a5ef0 con 0x7f00c8102780 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c81a6060 con 0x7f00c8100270 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c81a6220 con 0x7f00c8105340 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102780 0x7f00c81a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8100270 0x7f00c8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8100270 0x7f00c8101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:40984/0 (socket says 172.21.15.100:40984) 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c8105340 0x7f00c8102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cc8af640 1 -- 172.21.15.100:0/2143280115 learned_addr learned my addr 172.21.15.100:0/2143280115 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 -- 172.21.15.100:0/2143280115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c8105340 msgr2=0x7f00c8102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c8105340 0x7f00c8102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 -- 172.21.15.100:0/2143280115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8100270 msgr2=0x7f00c8101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8100270 0x7f00c8101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 -- 172.21.15.100:0/2143280115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4009560 con 0x7f00c8102780 2024-03-12T05:31:23.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102780 0x7f00c81a5920 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009fd0 tx=0x7f00c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00c5ffb640 1 -- 172.21.15.100:0/2143280115 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000ce20 con 0x7f00c8102780 2024-03-12T05:31:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00c5ffb640 1 -- 172.21.15.100:0/2143280115 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00c0005020 con 0x7f00c8102780 2024-03-12T05:31:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c81a6500 con 0x7f00c8102780 2024-03-12T05:31:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.313+0000 7f00c5ffb640 1 -- 172.21.15.100:0/2143280115 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0010690 con 0x7f00c8102780 2024-03-12T05:31:23.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.314+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c81a6990 con 0x7f00c8102780 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00c5ffb640 1 -- 172.21.15.100:0/2143280115 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f00c0005500 con 0x7f00c8102780 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00c5ffb640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f00b80a1bd0 0x7f00b80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00d17c8740 1 Processor -- start 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00d17c8740 1 -- start start 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00c5ffb640 1 -- 172.21.15.100:0/2143280115 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00c0014070 con 0x7f00c8102780 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00cc8af640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f00b80a1bd0 0x7f00b80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.316+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9511e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f9511ebd0 0x560f95127270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f951277b0 0x560f95129ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00d17c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9508d280 con 0x560f9511e2b0 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00d17c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9508a7b0 con 0x560f9511ebd0 2024-03-12T05:31:23.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00d17c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f95085f90 con 0x560f951277b0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9511e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9511e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49836/0 (socket says 172.21.15.100:49836) 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00cc8af640 1 -- 172.21.15.100:0/785961465 learned_addr learned my addr 172.21.15.100:0/785961465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f951277b0 0x560f95129ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f9511ebd0 0x560f95127270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 -- 172.21.15.100:0/785961465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f951277b0 msgr2=0x560f95129ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f951277b0 0x560f95129ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 -- 172.21.15.100:0/785961465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 msgr2=0x560f9511e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9511e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00cc8af640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f00b80a1bd0 0x7f00b80a4090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00bc012640 tx=0x7f00bc00b900 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 -- 172.21.15.100:0/785961465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f95044570 con 0x560f9511ebd0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.317+0000 7f00c7fff640 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f9511ebd0 0x560f95127270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00b402c170 tx=0x7f00b4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a40585acbd029ecc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.318+0000 7f00b2ffd640 1 -- 172.21.15.100:0/785961465 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004800 con 0x560f9511ebd0 2024-03-12T05:31:23.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.318+0000 7f00b2ffd640 1 -- 172.21.15.100:0/785961465 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00b4004960 con 0x560f9511ebd0 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.318+0000 7f00b2ffd640 1 -- 172.21.15.100:0/785961465 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4005050 con 0x560f9511ebd0 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 -- 172.21.15.100:0/785961465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f9511ebd0 msgr2=0x560f95127270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f9511ebd0 0x560f95127270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00b402c170 tx=0x7f00b4005c00 comp rx=0 tx=0).stop 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 -- 172.21.15.100:0/785961465 shutdown_connections 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f951277b0 0x560f95129ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f9511ebd0 0x560f95127270 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 --2- 172.21.15.100:0/785961465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9511e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 -- 172.21.15.100:0/785961465 >> 172.21.15.100:0/785961465 conn(0x560f95118990 msgr2=0x560f95118d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 -- 172.21.15.100:0/785961465 shutdown_connections 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 -- 172.21.15.100:0/785961465 wait complete. 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 -- start start 2024-03-12T05:31:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.319+0000 7f00d17c8740 1 Processor -- start 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511e2b0 0x560f95127200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 0x560f95124590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95124cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f94d59430 con 0x560f9511ebd0 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f94d15e00 con 0x560f951277b0 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9500afd0 con 0x560f9511e2b0 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00d17c8740 10 client.0 _ll_register_callbacks cb 0x560f95060fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 0x560f95124590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 0x560f95124590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49840/0 (socket says 172.21.15.100:49840) 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 -- 172.21.15.100:0/2065299467 learned_addr learned my addr 172.21.15.100:0/2065299467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00cc8af640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511e2b0 0x560f95127200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95124cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 -- 172.21.15.100:0/2065299467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511e2b0 msgr2=0x560f95127200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511e2b0 0x560f95127200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 -- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 msgr2=0x560f95124cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95124cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 -- 172.21.15.100:0/2065299467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c810d720 con 0x560f9511ebd0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00c7fff640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 0x560f95124590 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009930 tx=0x7f00b4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c35bcbbc71e37f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.320+0000 7f00a7fff640 1 -- 172.21.15.100:0/2065299467 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4051070 con 0x560f9511ebd0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f94d857e0 con 0x560f9511ebd0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00a7fff640 1 -- 172.21.15.100:0/2065299467 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00b4005450 con 0x560f9511ebd0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00a7fff640 1 -- 172.21.15.100:0/2065299467 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00b4004800 con 0x560f9511ebd0 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 10 client.19593 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560f95084610 con 0x560f9511ebd0 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00a7fff640 1 -- 172.21.15.100:0/2065299467 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560f94d857e0 con 0x560f9511ebd0 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00a7fff640 1 -- 172.21.15.100:0/2065299467 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00b4054020 con 0x560f9511ebd0 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 10 client.19593 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 10 client.19593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 20 client.19593 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f95137510 0x560f95139930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 4 client.19593 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f94dc2720 con 0x560f95137510 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.321+0000 7f00cc8af640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f95137510 0x560f95139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.322+0000 7f00a7fff640 10 client.19593 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.322+0000 7f00cc8af640 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f95137510 0x560f95139930 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdd9108b7470e22 server_cookie=a99f716e9f25109f in_seq=0 out_seq=0 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00a7fff640 1 -- 172.21.15.100:0/2065299467 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f94dc2720 con 0x560f95137510 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00a7fff640 10 client.19593 handle_command_reply: tid=0 2024-03-12T05:31:23.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 client.19593 shutdown 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 msgr2=0x560f95124590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 0x560f95124590 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009930 tx=0x7f00b4005020 comp rx=0 tx=0).stop 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 shutdown_connections 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f95137510 0x560f95139930 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95124cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511ebd0 0x560f95124590 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 --2- 172.21.15.100:0/2065299467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511e2b0 0x560f95127200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 >> 172.21.15.100:0/2065299467 conn(0x560f95118990 msgr2=0x560f9511a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00a7fff640 0 client.19593 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 shutdown_connections 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 1 -- 172.21.15.100:0/2065299467 wait complete. 2024-03-12T05:31:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.323+0000 7f00d17c8740 20 client.19593 trim_cache size 0 max 16384 2024-03-12T05:31:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 Processor -- start 2024-03-12T05:31:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 -- start start 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95126cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f94d857e0 con 0x560f9511e2b0 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9511dcf0 con 0x560f951277b0 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.339+0000 7f00d17c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f951201a0 con 0x560f9511ebd0 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49852/0 (socket says 172.21.15.100:49852) 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00c7fff640 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95126cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 -- 172.21.15.100:0/4085648070 learned_addr learned my addr 172.21.15.100:0/4085648070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 -- 172.21.15.100:0/4085648070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 msgr2=0x560f9512eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 -- 172.21.15.100:0/4085648070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 msgr2=0x560f95126cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95126cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 -- 172.21.15.100:0/4085648070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f95020ac0 con 0x560f9511e2b0 2024-03-12T05:31:23.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00cc8af640 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512e600 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f00bc012ed0 tx=0x7f00bc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5788f03260badcb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00b27fc640 1 -- 172.21.15.100:0/4085648070 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc017040 con 0x560f9511e2b0 2024-03-12T05:31:23.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00b27fc640 1 -- 172.21.15.100:0/4085648070 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00bc02a3b0 con 0x560f9511e2b0 2024-03-12T05:31:23.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.340+0000 7f00b27fc640 1 -- 172.21.15.100:0/4085648070 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc025040 con 0x560f9511e2b0 2024-03-12T05:31:23.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 -- 172.21.15.100:0/4085648070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 msgr2=0x560f9512e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512e600 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f00bc012ed0 tx=0x7f00bc01d000 comp rx=0 tx=0).stop 2024-03-12T05:31:23.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 -- 172.21.15.100:0/4085648070 shutdown_connections 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f95126cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 --2- 172.21.15.100:0/4085648070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512e600 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 -- 172.21.15.100:0/4085648070 >> 172.21.15.100:0/4085648070 conn(0x560f95118990 msgr2=0x560f9513e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 -- 172.21.15.100:0/4085648070 shutdown_connections 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 -- 172.21.15.100:0/4085648070 wait complete. 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.341+0000 7f00d17c8740 1 -- start start 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 Processor -- start 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f9512da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9512ef20 con 0x560f9511e2b0 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9518da40 con 0x560f951277b0 2024-03-12T05:31:23.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9518dc00 con 0x560f9511ebd0 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00d17c8740 10 client.0 _ll_register_callbacks cb 0x7f00b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49866/0 (socket says 172.21.15.100:49866) 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 -- 172.21.15.100:0/346157238 learned_addr learned my addr 172.21.15.100:0/346157238 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00c7fff640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f9512da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 -- 172.21.15.100:0/346157238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 msgr2=0x560f9512d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 -- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 msgr2=0x560f9512da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f9512da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cc8af640 1 -- 172.21.15.100:0/346157238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc00e5f0 con 0x560f9511e2b0 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.342+0000 7f00cd0b0640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f9512da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00cc8af640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512cf80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f00bc012f70 tx=0x7f00bc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee3f3e1d3b1e923 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00a7fff640 1 -- 172.21.15.100:0/346157238 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc025040 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00a7fff640 1 -- 172.21.15.100:0/346157238 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00bc02a730 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00b4004960 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00a7fff640 1 -- 172.21.15.100:0/346157238 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00bc02a8f0 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 10 client.19599 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560f950466a0 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00a7fff640 1 -- 172.21.15.100:0/346157238 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00b4004960 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00a7fff640 1 -- 172.21.15.100:0/346157238 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00bc032070 con 0x560f9511e2b0 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 10 client.19599 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 10 client.19599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 20 client.19599 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f9518f450 0x560f95191870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 4 client.19599 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.343+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f94e11890 con 0x560f9518f450 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00c7fff640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f9518f450 0x560f95191870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00a7fff640 10 client.19599 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00c7fff640 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f9518f450 0x560f95191870 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27cc8d4cad6fa9bc server_cookie=b1649421bee2fafb in_seq=0 out_seq=0 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00a7fff640 1 -- 172.21.15.100:0/346157238 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560f94e11890 con 0x560f9518f450 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00a7fff640 10 client.19599 handle_command_reply: tid=0 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00d17c8740 1 client.19599 shutdown 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 msgr2=0x560f9512cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.344+0000 7f00d17c8740 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512cf80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f00bc012f70 tx=0x7f00bc00ee20 comp rx=0 tx=0).stop 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 shutdown_connections 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560f9518f450 0x560f95191870 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f951277b0 0x560f9512da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f9511ebd0 0x560f9512d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 --2- 172.21.15.100:0/346157238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560f9511e2b0 0x560f9512cf80 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 >> 172.21.15.100:0/346157238 conn(0x560f95118990 msgr2=0x560f95122e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00a7fff640 0 client.19599 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 shutdown_connections 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 1 -- 172.21.15.100:0/346157238 wait complete. 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00d17c8740 20 client.19599 trim_cache size 0 max 16384 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:31:23.344 INFO:teuthology.orchestra.run.smithi100.stdout: "fe04054f-3a7b-4c52-a159-37ce7fb95964": { 2024-03-12T05:31:23.345 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:31:23.345 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "fe04054f-3a7b-4c52-a159-37ce7fb95964", 2024-03-12T05:31:23.345 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:31:23.345 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:31:23.345 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:31:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f00b80a1bd0 msgr2=0x7f00b80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f00b80a1bd0 0x7f00b80a4090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00bc012640 tx=0x7f00bc00b900 comp rx=0 tx=0).stop 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102780 msgr2=0x7f00c81a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102780 0x7f00c81a5920 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009fd0 tx=0x7f00c000ef10 comp rx=0 tx=0).stop 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 shutdown_connections 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f00b80a1bd0 0x7f00b80a4090 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102780 0x7f00c81a5920 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c8105340 0x7f00c8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 --2- 172.21.15.100:0/2143280115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00c8100270 0x7f00c8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.345+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 >> 172.21.15.100:0/2143280115 conn(0x7f00c8077230 msgr2=0x7f00c80686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.346+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 shutdown_connections 2024-03-12T05:31:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:23.346+0000 7f00ceb13640 1 -- 172.21.15.100:0/2143280115 wait complete. 2024-03-12T05:31:23.350 INFO:tasks.cephfs.filesystem:scrub status for tag:fe04054f-3a7b-4c52-a159-37ce7fb95964 - {'path': '/', 'tag': 'fe04054f-3a7b-4c52-a159-37ce7fb95964', 'options': 'recursive,force'} 2024-03-12T05:31:23.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:23 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:23.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:23 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:23.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:23 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:23.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:23 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:23.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:23 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:23.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:23 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 198495 27.18 MB/sec execute 247 sec latency 344.940 ms 2024-03-12T05:31:24.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:31:24.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:24.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:24.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:24.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:24 smithi100 ceph-mon[32210]: pgmap v608: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 523 op/s 2024-03-12T05:31:24.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:31:24.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:24.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:24.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:24.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:24 smithi156 ceph-mon[36270]: pgmap v608: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 523 op/s 2024-03-12T05:31:24.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:31:24.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:24.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:24.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:24.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:24.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:24 smithi161 ceph-mon[36043]: pgmap v608: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 523 op/s 2024-03-12T05:31:25.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 198832 27.11 MB/sec execute 248 sec latency 149.852 ms 2024-03-12T05:31:25.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:25 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:25.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:25 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:25.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:25 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 199278 27.07 MB/sec execute 249 sec latency 159.596 ms 2024-03-12T05:31:26.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:26 smithi100 ceph-mon[32210]: pgmap v609: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 438 op/s 2024-03-12T05:31:26.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:26 smithi156 ceph-mon[36270]: pgmap v609: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 438 op/s 2024-03-12T05:31:26.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:26 smithi161 ceph-mon[36043]: pgmap v609: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 438 op/s 2024-03-12T05:31:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 199905 27.10 MB/sec execute 250 sec latency 119.209 ms 2024-03-12T05:31:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 200390 27.13 MB/sec execute 251 sec latency 347.577 ms 2024-03-12T05:31:28.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:28 smithi156 ceph-mon[36270]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 695 op/s 2024-03-12T05:31:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:28 smithi161 ceph-mon[36043]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 695 op/s 2024-03-12T05:31:28.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:28 smithi100 ceph-mon[32210]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 695 op/s 2024-03-12T05:31:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 200982 27.15 MB/sec execute 252 sec latency 179.552 ms 2024-03-12T05:31:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 201404 27.15 MB/sec execute 253 sec latency 137.738 ms 2024-03-12T05:31:30.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:30 smithi156 ceph-mon[36270]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 467 op/s 2024-03-12T05:31:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:30 smithi161 ceph-mon[36043]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 467 op/s 2024-03-12T05:31:30.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:30 smithi100 ceph-mon[32210]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 467 op/s 2024-03-12T05:31:31.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 201933 27.14 MB/sec execute 254 sec latency 126.932 ms 2024-03-12T05:31:32.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 202394 27.12 MB/sec execute 255 sec latency 180.080 ms 2024-03-12T05:31:32.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:32 smithi156 ceph-mon[36270]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 583 op/s 2024-03-12T05:31:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:32 smithi161 ceph-mon[36043]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 583 op/s 2024-03-12T05:31:32.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:32 smithi100 ceph-mon[32210]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 583 op/s 2024-03-12T05:31:33.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 202842 27.13 MB/sec execute 256 sec latency 202.060 ms 2024-03-12T05:31:34.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 203277 27.11 MB/sec execute 257 sec latency 209.831 ms 2024-03-12T05:31:34.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:34 smithi156 ceph-mon[36270]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 545 op/s 2024-03-12T05:31:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:34 smithi161 ceph-mon[36043]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 545 op/s 2024-03-12T05:31:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:34 smithi100 ceph-mon[32210]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 545 op/s 2024-03-12T05:31:35.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 203721 27.08 MB/sec execute 258 sec latency 132.181 ms 2024-03-12T05:31:35.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:35 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:31:35.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:35 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:31:35.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:35 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:31:36.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 204257 27.06 MB/sec execute 259 sec latency 159.127 ms 2024-03-12T05:31:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:36 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:31:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:36 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:31:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:36 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:31:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:36 smithi156 ceph-mon[36270]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 469 op/s 2024-03-12T05:31:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:36 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:31:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:36 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:31:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:36 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:31:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:36 smithi161 ceph-mon[36043]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 469 op/s 2024-03-12T05:31:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:36 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:31:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:36 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:31:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:36 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:31:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:36 smithi100 ceph-mon[32210]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 469 op/s 2024-03-12T05:31:37.242 INFO:tasks.workunit.client.0.smithi100.stdout: 10 204752 27.10 MB/sec execute 260 sec latency 179.223 ms 2024-03-12T05:31:37.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:37 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:31:37.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:37 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:31:37.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:37 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:31:37.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:37 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:31:37.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:37 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:31:37.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:37 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:31:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:37 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:31:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:37 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:31:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:37 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:31:38.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 205140 27.06 MB/sec execute 261 sec latency 273.809 ms 2024-03-12T05:31:38.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:38 smithi100 ceph-mon[32210]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 640 op/s 2024-03-12T05:31:38.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:38 smithi156 ceph-mon[36270]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 640 op/s 2024-03-12T05:31:38.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:38 smithi161 ceph-mon[36043]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 640 op/s 2024-03-12T05:31:39.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 205571 27.07 MB/sec execute 262 sec latency 360.630 ms 2024-03-12T05:31:40.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 206141 27.10 MB/sec execute 263 sec latency 129.925 ms 2024-03-12T05:31:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:40 smithi156 ceph-mon[36270]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.7 MiB/s wr, 384 op/s 2024-03-12T05:31:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:40 smithi161 ceph-mon[36043]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.7 MiB/s wr, 384 op/s 2024-03-12T05:31:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:40 smithi100 ceph-mon[32210]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.7 MiB/s wr, 384 op/s 2024-03-12T05:31:41.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 206518 27.06 MB/sec execute 264 sec latency 188.927 ms 2024-03-12T05:31:42.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 206940 27.02 MB/sec execute 265 sec latency 162.849 ms 2024-03-12T05:31:42.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:42 smithi156 ceph-mon[36270]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 502 op/s 2024-03-12T05:31:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:42 smithi161 ceph-mon[36043]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 502 op/s 2024-03-12T05:31:42.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:42 smithi100 ceph-mon[32210]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 502 op/s 2024-03-12T05:31:43.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 207272 26.99 MB/sec execute 266 sec latency 295.178 ms 2024-03-12T05:31:44.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 207694 26.95 MB/sec execute 267 sec latency 255.145 ms 2024-03-12T05:31:44.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:44 smithi156 ceph-mon[36270]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 480 op/s 2024-03-12T05:31:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:44 smithi161 ceph-mon[36043]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 480 op/s 2024-03-12T05:31:44.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:44 smithi100 ceph-mon[32210]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 480 op/s 2024-03-12T05:31:45.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 208354 26.99 MB/sec execute 268 sec latency 118.985 ms 2024-03-12T05:31:46.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 209013 27.05 MB/sec execute 269 sec latency 115.495 ms 2024-03-12T05:31:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:46 smithi156 ceph-mon[36270]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 382 op/s 2024-03-12T05:31:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:46 smithi161 ceph-mon[36043]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 382 op/s 2024-03-12T05:31:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:46 smithi100 ceph-mon[32210]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 382 op/s 2024-03-12T05:31:47.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 209541 27.07 MB/sec execute 270 sec latency 169.871 ms 2024-03-12T05:31:48.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 209910 27.04 MB/sec execute 271 sec latency 183.009 ms 2024-03-12T05:31:48.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:48 smithi156 ceph-mon[36270]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 576 op/s 2024-03-12T05:31:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:48 smithi161 ceph-mon[36043]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 576 op/s 2024-03-12T05:31:48.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:48 smithi100 ceph-mon[32210]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 576 op/s 2024-03-12T05:31:49.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 210276 27.01 MB/sec execute 272 sec latency 244.131 ms 2024-03-12T05:31:50.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 210732 26.99 MB/sec execute 273 sec latency 127.720 ms 2024-03-12T05:31:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:50 smithi156 ceph-mon[36270]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.2 MiB/s wr, 405 op/s 2024-03-12T05:31:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:50 smithi161 ceph-mon[36043]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.2 MiB/s wr, 405 op/s 2024-03-12T05:31:50.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:50 smithi100 ceph-mon[32210]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.2 MiB/s wr, 405 op/s 2024-03-12T05:31:51.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 211165 26.97 MB/sec execute 274 sec latency 146.172 ms 2024-03-12T05:31:52.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 211691 26.96 MB/sec execute 275 sec latency 191.232 ms 2024-03-12T05:31:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:52 smithi156 ceph-mon[36270]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 466 op/s 2024-03-12T05:31:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:52 smithi161 ceph-mon[36043]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 466 op/s 2024-03-12T05:31:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:52 smithi100 ceph-mon[32210]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 466 op/s 2024-03-12T05:31:53.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 212074 26.95 MB/sec execute 276 sec latency 210.359 ms 2024-03-12T05:31:53.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:31:53.352 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.468+0000 7f8465fa0640 1 -- 172.21.15.100:0/2415926926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460105150 msgr2=0x7f846010cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.468+0000 7f8465fa0640 1 --2- 172.21.15.100:0/2415926926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460105150 0x7f846010cd20 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8458009cd0 tx=0x7f845802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- 172.21.15.100:0/2415926926 shutdown_connections 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 --2- 172.21.15.100:0/2415926926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460105150 0x7f846010cd20 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 --2- 172.21.15.100:0/2415926926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460100a30 0x7f8460104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 --2- 172.21.15.100:0/2415926926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460100080 0x7f8460100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- 172.21.15.100:0/2415926926 >> 172.21.15.100:0/2415926926 conn(0x7f8460076c30 msgr2=0x7f84600fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- 172.21.15.100:0/2415926926 shutdown_connections 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- 172.21.15.100:0/2415926926 wait complete. 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 Processor -- start 2024-03-12T05:31:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- start start 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460100080 0x7f8460198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460105150 0x7f8460198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 0x7f84601a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a3ad0 con 0x7f8460198e80 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a3c40 con 0x7f8460105150 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8465fa0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a3e00 con 0x7f8460100080 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f846579f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 0x7f84601a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f846579f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 0x7f84601a3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55068/0 (socket says 172.21.15.100:55068) 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f846579f640 1 -- 172.21.15.100:0/3884101187 learned_addr learned my addr 172.21.15.100:0/3884101187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8457fff640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460105150 0x7f8460198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.469+0000 7f8464f9e640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460100080 0x7f8460198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f846579f640 1 -- 172.21.15.100:0/3884101187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460100080 msgr2=0x7f8460198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f846579f640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460100080 0x7f8460198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f846579f640 1 -- 172.21.15.100:0/3884101187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460105150 msgr2=0x7f8460198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f846579f640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460105150 0x7f8460198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f846579f640 1 -- 172.21.15.100:0/3884101187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009900 con 0x7f8460198e80 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f8457fff640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460105150 0x7f8460198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f846579f640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 0x7f84601a3480 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8458004af0 tx=0x7f8458032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f8455ffb640 1 -- 172.21.15.100:0/3884101187 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580328b0 con 0x7f8460198e80 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84601a4080 con 0x7f8460198e80 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f8455ffb640 1 -- 172.21.15.100:0/3884101187 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8458003e50 con 0x7f8460198e80 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f8455ffb640 1 -- 172.21.15.100:0/3884101187 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458004740 con 0x7f8460198e80 2024-03-12T05:31:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.470+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84601a4530 con 0x7f8460198e80 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.472+0000 7f8455ffb640 1 -- 172.21.15.100:0/3884101187 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f845800a040 con 0x7f8460198e80 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 Processor -- start 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8455ffb640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84480a1ba0 0x7f84480a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 -- start start 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8457fff640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84480a1ba0 0x7f84480a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6652f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8455ffb640 1 -- 172.21.15.100:0/3884101187 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f84580f02a0 con 0x7f8460198e80 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66538490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6649e280 con 0x560a6652fdf0 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6649b7b0 con 0x560a665389d0 2024-03-12T05:31:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8469dd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a66496f90 con 0x560a6652f4d0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8457fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66538490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6652f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8457fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66538490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55076/0 (socket says 172.21.15.100:55076) 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8457fff640 1 -- 172.21.15.100:0/110045002 learned_addr learned my addr 172.21.15.100:0/110045002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6652f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54314/0 (socket says 172.21.15.100:54314) 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f846579f640 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8457fff640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84480a1ba0 0x7f84480a4060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8450013840 tx=0x7f8450013c40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 -- 172.21.15.100:0/110045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 msgr2=0x560a6653adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 -- 172.21.15.100:0/110045002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 msgr2=0x560a66538490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66538490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.473+0000 7f8464f9e640 1 -- 172.21.15.100:0/110045002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a66455570 con 0x560a6652f4d0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.474+0000 7f8464f9e640 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6652f8b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f845c009c20 tx=0x7f845c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcd389efd4be7e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.474+0000 7f8446ffd640 1 -- 172.21.15.100:0/110045002 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c004050 con 0x560a6652f4d0 2024-03-12T05:31:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.474+0000 7f8446ffd640 1 -- 172.21.15.100:0/110045002 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f845c005500 con 0x560a6652f4d0 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.474+0000 7f8446ffd640 1 -- 172.21.15.100:0/110045002 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c004cd0 con 0x560a6652f4d0 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- 172.21.15.100:0/110045002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 msgr2=0x560a6652f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6652f8b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f845c009c20 tx=0x7f845c011810 comp rx=0 tx=0).stop 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- 172.21.15.100:0/110045002 shutdown_connections 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66538490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- 172.21.15.100:0/110045002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6652f8b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- 172.21.15.100:0/110045002 >> 172.21.15.100:0/110045002 conn(0x560a66529c00 msgr2=0x560a66529fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- 172.21.15.100:0/110045002 shutdown_connections 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- 172.21.15.100:0/110045002 wait complete. 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- start start 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 Processor -- start 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652f4d0 0x560a665383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66535770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a665389d0 0x560a66535d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6616a430 con 0x560a6652fdf0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a66126e00 con 0x560a6652f4d0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6641bfd0 con 0x560a665389d0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.475+0000 7f8469dd0740 10 client.0 _ll_register_callbacks cb 0x560a66471fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66535770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66535770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55084/0 (socket says 172.21.15.100:55084) 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 -- 172.21.15.100:0/793052598 learned_addr learned my addr 172.21.15.100:0/793052598 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f846579f640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a665389d0 0x560a66535d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8464f9e640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652f4d0 0x560a665383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 -- 172.21.15.100:0/793052598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a665389d0 msgr2=0x560a66535d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a665389d0 0x560a66535d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 -- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652f4d0 msgr2=0x560a665383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652f4d0 0x560a665383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 -- 172.21.15.100:0/793052598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c009850 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8464f9e640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652f4d0 0x560a665383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8457fff640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66535770 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f8450009ee0 tx=0x7f8450004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc36643cdd1c168 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f843ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f843ffff640 1 -- 172.21.15.100:0/793052598 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450018040 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f843ffff640 1 -- 172.21.15.100:0/793052598 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8450003d00 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a661967e0 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f843ffff640 1 -- 172.21.15.100:0/793052598 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8450010430 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8469dd0740 10 client.19611 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.476+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560a66495610 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f843ffff640 1 -- 172.21.15.100:0/793052598 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a661967e0 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f843ffff640 1 -- 172.21.15.100:0/793052598 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8450026020 con 0x560a6652fdf0 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8469dd0740 10 client.19611 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8469dd0740 10 client.19611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:53.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8469dd0740 20 client.19611 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8469dd0740 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a66548c00 0x560a6654b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8469dd0740 4 client.19611 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a661d3720 con 0x560a66548c00 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8464f9e640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a66548c00 0x560a6654b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f843ffff640 10 client.19611 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.477+0000 7f8464f9e640 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a66548c00 0x560a6654b020 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c197cb2aa301068 server_cookie=5eff18a87f5cc23f in_seq=0 out_seq=0 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f843ffff640 1 -- 172.21.15.100:0/793052598 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a661d3720 con 0x560a66548c00 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f843ffff640 10 client.19611 handle_command_reply: tid=0 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 client.19611 shutdown 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 msgr2=0x560a66535770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66535770 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f8450009ee0 tx=0x7f8450004730 comp rx=0 tx=0).stop 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 shutdown_connections 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a66548c00 0x560a6654b020 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a665389d0 0x560a66535d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a66535770 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f843ffff640 0 client.19611 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 --2- 172.21.15.100:0/793052598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652f4d0 0x560a665383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.478+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 >> 172.21.15.100:0/793052598 conn(0x560a66529c00 msgr2=0x560a6652b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.479+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 shutdown_connections 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.479+0000 7f8469dd0740 1 -- 172.21.15.100:0/793052598 wait complete. 2024-03-12T05:31:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.479+0000 7f8469dd0740 20 client.19611 trim_cache size 0 max 16384 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 Processor -- start 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 -- start start 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a66537ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652fdf0 0x560a665383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 0x560a66535000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a66545030 con 0x560a665389d0 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6652ef10 con 0x560a6652fdf0 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8469dd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a66544790 con 0x560a6652f4d0 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f846579f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 0x560a66535000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f846579f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 0x560a66535000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55090/0 (socket says 172.21.15.100:55090) 2024-03-12T05:31:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f846579f640 1 -- 172.21.15.100:0/538854456 learned_addr learned my addr 172.21.15.100:0/538854456 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8464f9e640 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a66537ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.499+0000 7f8457fff640 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652fdf0 0x560a665383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f846579f640 1 -- 172.21.15.100:0/538854456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 msgr2=0x560a66537ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f846579f640 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a66537ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f846579f640 1 -- 172.21.15.100:0/538854456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652fdf0 msgr2=0x560a665383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f846579f640 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652fdf0 0x560a665383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f846579f640 1 -- 172.21.15.100:0/538854456 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a66431ac0 con 0x560a665389d0 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f846579f640 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 0x560a66535000 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f84580bcca0 tx=0x7f8458004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9271457e4b5033d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f84467fc640 1 -- 172.21.15.100:0/538854456 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580bd6a0 con 0x560a665389d0 2024-03-12T05:31:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f84467fc640 1 -- 172.21.15.100:0/538854456 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84580bcfa0 con 0x560a665389d0 2024-03-12T05:31:53.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.500+0000 7f84467fc640 1 -- 172.21.15.100:0/538854456 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580b9610 con 0x560a665389d0 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 -- 172.21.15.100:0/538854456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 msgr2=0x560a66535000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 0x560a66535000 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f84580bcca0 tx=0x7f8458004920 comp rx=0 tx=0).stop 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 -- 172.21.15.100:0/538854456 shutdown_connections 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a665389d0 0x560a66535000 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a6652fdf0 0x560a665383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 --2- 172.21.15.100:0/538854456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a66537ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.501+0000 7f8469dd0740 1 -- 172.21.15.100:0/538854456 >> 172.21.15.100:0/538854456 conn(0x560a66529c00 msgr2=0x560a66530a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.502+0000 7f8469dd0740 1 -- 172.21.15.100:0/538854456 shutdown_connections 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.502+0000 7f8469dd0740 1 -- 172.21.15.100:0/538854456 wait complete. 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.502+0000 7f8469dd0740 1 -- start start 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.502+0000 7f8469dd0740 1 Processor -- start 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6653f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a6653fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6653df80 con 0x560a6652fdf0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6653e0f0 con 0x560a665389d0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a6653e2b0 con 0x560a6652f4d0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8469dd0740 10 client.0 _ll_register_callbacks cb 0x7f845c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a6653fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a6653fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55098/0 (socket says 172.21.15.100:55098) 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 -- 172.21.15.100:0/1090985107 learned_addr learned my addr 172.21.15.100:0/1090985107 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8464f9e640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6653f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f846579f640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 -- 172.21.15.100:0/1090985107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 msgr2=0x560a6653f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6653f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 -- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 msgr2=0x560a6653d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.503+0000 7f8457fff640 1 -- 172.21.15.100:0/1090985107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84601a4530 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8457fff640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a6653fdc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f84500058c0 tx=0x7f8450009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6024363a563251 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f843ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f843ffff640 1 -- 172.21.15.100:0/1090985107 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450018030 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f843ffff640 1 -- 172.21.15.100:0/1090985107 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f845000fd50 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f845c005500 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f843ffff640 1 -- 172.21.15.100:0/1090985107 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8450020dd0 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 10 client.19617 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560a664576a0 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f843ffff640 1 -- 172.21.15.100:0/1090985107 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f845c005500 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f843ffff640 1 -- 172.21.15.100:0/1090985107 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8450026070 con 0x560a6652fdf0 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 10 client.19617 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 10 client.19617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:53.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 20 client.19617 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a665a56b0 0x560a665a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 4 client.19617 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.504+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a66222890 con 0x560a665a56b0 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f8464f9e640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a665a56b0 0x560a665a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f843ffff640 10 client.19617 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f8464f9e640 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a665a56b0 0x560a665a7ad0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33ab88ddb6cc815 server_cookie=4779a037e6295bf in_seq=0 out_seq=0 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f843ffff640 1 -- 172.21.15.100:0/1090985107 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560a66222890 con 0x560a665a56b0 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f843ffff640 10 client.19617 handle_command_reply: tid=0 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f8469dd0740 1 client.19617 shutdown 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 msgr2=0x560a6653fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f8469dd0740 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a6653fdc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f84500058c0 tx=0x7f8450009b30 comp rx=0 tx=0).stop 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.505+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 shutdown_connections 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560a665a56b0 0x560a665a7ad0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a665389d0 0x560a6653d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a6652fdf0 0x560a6653fdc0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f843ffff640 0 client.19617 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 --2- 172.21.15.100:0/1090985107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a6652f4d0 0x560a6653f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 >> 172.21.15.100:0/1090985107 conn(0x560a66529c00 msgr2=0x560a66535000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 shutdown_connections 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 1 -- 172.21.15.100:0/1090985107 wait complete. 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8469dd0740 20 client.19617 trim_cache size 0 max 16384 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:31:53.504 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:31:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84480a1ba0 msgr2=0x7f84480a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.506+0000 7f8465fa0640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84480a1ba0 0x7f84480a4060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8450013840 tx=0x7f8450013c40 comp rx=0 tx=0).stop 2024-03-12T05:31:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 msgr2=0x7f84601a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 0x7f84601a3480 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8458004af0 tx=0x7f8458032570 comp rx=0 tx=0).stop 2024-03-12T05:31:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 shutdown_connections 2024-03-12T05:31:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f84480a1ba0 0x7f84480a4060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8460198e80 0x7f84601a3480 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8460105150 0x7f8460198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 --2- 172.21.15.100:0/3884101187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460100080 0x7f8460198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 >> 172.21.15.100:0/3884101187 conn(0x7f8460076c30 msgr2=0x7f8460103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 shutdown_connections 2024-03-12T05:31:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.507+0000 7f8465fa0640 1 -- 172.21.15.100:0/3884101187 wait complete. 2024-03-12T05:31:53.513 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:31:53.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 -- 172.21.15.100:0/3845044834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 msgr2=0x7f758810dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 --2- 172.21.15.100:0/3845044834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758810dbc0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f7580009be0 tx=0x7f758002bf80 comp rx=0 tx=0).stop 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 -- 172.21.15.100:0/3845044834 shutdown_connections 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 --2- 172.21.15.100:0/3845044834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758810dbc0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 --2- 172.21.15.100:0/3845044834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102ca0 0x7f758810b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 --2- 172.21.15.100:0/3845044834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102380 0x7f7588102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 -- 172.21.15.100:0/3845044834 >> 172.21.15.100:0/3845044834 conn(0x7f7588076e30 msgr2=0x7f75880fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 -- 172.21.15.100:0/3845044834 shutdown_connections 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.662+0000 7f758e513640 1 -- 172.21.15.100:0/3845044834 wait complete. 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 Processor -- start 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 -- start start 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 0x7f7588197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 0x7f7588197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758819c600 con 0x7f758810b750 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758819c770 con 0x7f7588102380 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758e513640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758819c930 con 0x7f7588102ca0 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f7587fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 0x7f7588197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f75877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 0x7f7588197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f7587fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 0x7f7588197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34608/0 (socket says 172.21.15.100:34608) 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f758cab0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f75877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 0x7f7588197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54358/0 (socket says 172.21.15.100:54358) 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.663+0000 7f7587fff640 1 -- 172.21.15.100:0/676109930 learned_addr learned my addr 172.21.15.100:0/676109930 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f7587fff640 1 -- 172.21.15.100:0/676109930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 msgr2=0x7f7588197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f7587fff640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 0x7f7588197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f7587fff640 1 -- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 msgr2=0x7f758819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f7587fff640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758819c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f7587fff640 1 -- 172.21.15.100:0/676109930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580009840 con 0x7f7588102380 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f75877fe640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 0x7f7588197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f758cab0640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f7587fff640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 0x7f7588197740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f757800e9f0 tx=0x7f757800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578009720 con 0x7f7588102380 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758819cc10 con 0x7f7588102380 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7578005500 con 0x7f7588102380 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75881a4490 con 0x7f7588102380 2024-03-12T05:31:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.664+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578010430 con 0x7f7588102380 2024-03-12T05:31:53.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.665+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75880676e0 con 0x7f7588102380 2024-03-12T05:31:53.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.667+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7578005000 con 0x7f7588102380 2024-03-12T05:31:53.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.668+0000 7f75857fa640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75700a1ba0 0x7f75700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:53.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.668+0000 7f75877fe640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75700a1ba0 0x7f75700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:53.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.668+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7578014070 con 0x7f7588102380 2024-03-12T05:31:53.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.669+0000 7f75877fe640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75700a1ba0 0x7f75700a4060 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75880ff610 tx=0x7f757c009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:53.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.672+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7578096ca0 con 0x7f7588102380 2024-03-12T05:31:53.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.953+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7588198a90 con 0x7f7588102380 2024-03-12T05:31:53.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.953+0000 7f75857fa640 1 -- 172.21.15.100:0/676109930 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f75780ba030 con 0x7f7588102380 2024-03-12T05:31:53.947 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:31:53.947 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:31:53.948 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:31:53.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75700a1ba0 msgr2=0x7f75700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75700a1ba0 0x7f75700a4060 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75880ff610 tx=0x7f757c009260 comp rx=0 tx=0).stop 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 msgr2=0x7f7588197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 0x7f7588197740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f757800e9f0 tx=0x7f757800edf0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 shutdown_connections 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75700a1ba0 0x7f75700a4060 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758810b750 0x7f758819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7588102ca0 0x7f7588197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 --2- 172.21.15.100:0/676109930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7588102380 0x7f7588197740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 >> 172.21.15.100:0/676109930 conn(0x7f7588076e30 msgr2=0x7f75880fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 shutdown_connections 2024-03-12T05:31:53.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:53.956+0000 7f758e513640 1 -- 172.21.15.100:0/676109930 wait complete. 2024-03-12T05:31:53.960 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:31:53.960 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:31:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 -- 172.21.15.100:0/2650113563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d010b790 msgr2=0x7ff5d010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2650113563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d010b790 0x7ff5d010dc10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009ca0 tx=0x7ff5c802c180 comp rx=0 tx=0).stop 2024-03-12T05:31:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 -- 172.21.15.100:0/2650113563 shutdown_connections 2024-03-12T05:31:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2650113563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d010b790 0x7ff5d010dc10 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2650113563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 0x7ff5d010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2650113563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d0102450 0x7ff5d0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 -- 172.21.15.100:0/2650113563 >> 172.21.15.100:0/2650113563 conn(0x7ff5d00771e0 msgr2=0x7ff5d00775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 -- 172.21.15.100:0/2650113563 shutdown_connections 2024-03-12T05:31:54.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.087+0000 7ff5d549d640 1 -- 172.21.15.100:0/2650113563 wait complete. 2024-03-12T05:31:54.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 Processor -- start 2024-03-12T05:31:54.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 -- start start 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d0102450 0x7ff5d019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 0x7ff5d019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 0x7ff5d0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d0197ca0 con 0x7ff5d0102450 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d0197e10 con 0x7ff5d0102d70 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5d549d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d0197fd0 con 0x7ff5d010b790 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 0x7ff5d0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d0102450 0x7ff5d019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 0x7ff5d019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 0x7ff5d0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54372/0 (socket says 172.21.15.100:54372) 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 -- 172.21.15.100:0/2273455063 learned_addr learned my addr 172.21.15.100:0/2273455063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.088+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 0x7ff5d019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34626/0 (socket says 172.21.15.100:34626) 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 -- 172.21.15.100:0/2273455063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 msgr2=0x7ff5d019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 0x7ff5d019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 -- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d0102450 msgr2=0x7ff5d019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d0102450 0x7ff5d019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 -- 172.21.15.100:0/2273455063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009900 con 0x7ff5d010b790 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d0102450 0x7ff5d019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5cf7fe640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 0x7ff5d0197650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005e30 tx=0x7ff5c8034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5ccff9640 1 -- 172.21.15.100:0/2273455063 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8003b70 con 0x7ff5d010b790 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5ccff9640 1 -- 172.21.15.100:0/2273455063 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8003cd0 con 0x7ff5d010b790 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d0198250 con 0x7ff5d010b790 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5ccff9640 1 -- 172.21.15.100:0/2273455063 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8004680 con 0x7ff5d010b790 2024-03-12T05:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.089+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d01001b0 con 0x7ff5d010b790 2024-03-12T05:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.092+0000 7ff5ccff9640 1 -- 172.21.15.100:0/2273455063 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff5c800a040 con 0x7ff5d010b790 2024-03-12T05:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.092+0000 7ff5d8073740 1 Processor -- start 2024-03-12T05:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 -- start start 2024-03-12T05:31:54.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8ccb70 0x560a5e8d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e887960 con 0x560a5e8cc250 2024-03-12T05:31:54.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e83b310 con 0x560a5e8ccb70 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5d8073740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e838840 con 0x560a5e8d5750 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5ccff9640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5bc0a1ba0 0x7ff5bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5ceffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8ccb70 0x560a5e8d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55136/0 (socket says 172.21.15.100:55136) 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5c7fff640 1 -- 172.21.15.100:0/2737190933 learned_addr learned my addr 172.21.15.100:0/2737190933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.093+0000 7ff5cf7fe640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5bc0a1ba0 0x7ff5bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ccff9640 1 -- 172.21.15.100:0/2273455063 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5c80f0460 con 0x7ff5d010b790 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ceffd640 1 -- 172.21.15.100:0/2737190933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 msgr2=0x560a5e8d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ceffd640 1 -- 172.21.15.100:0/2737190933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 msgr2=0x560a5e8cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ceffd640 1 -- 172.21.15.100:0/2737190933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5e7f6060 con 0x560a5e8ccb70 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5cf7fe640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5bc0a1ba0 0x7ff5bc0a4060 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c53d0 tx=0x7ff5c808d200 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8ccb70 0x560a5e8d5210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009fd0 tx=0x7ff5c00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e865990caab363a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5c57fa640 1 -- 172.21.15.100:0/2737190933 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004310 con 0x560a5e8ccb70 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5c57fa640 1 -- 172.21.15.100:0/2737190933 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5c0004470 con 0x560a5e8ccb70 2024-03-12T05:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.094+0000 7ff5c57fa640 1 -- 172.21.15.100:0/2737190933 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00035f0 con 0x560a5e8ccb70 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 -- 172.21.15.100:0/2737190933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8ccb70 msgr2=0x560a5e8d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8ccb70 0x560a5e8d5210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009fd0 tx=0x7ff5c00118d0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 -- 172.21.15.100:0/2737190933 shutdown_connections 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8ccb70 0x560a5e8d5210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 --2- 172.21.15.100:0/2737190933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 -- 172.21.15.100:0/2737190933 >> 172.21.15.100:0/2737190933 conn(0x560a5e8c6980 msgr2=0x560a5e8c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 -- 172.21.15.100:0/2737190933 shutdown_connections 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.095+0000 7ff5d8073740 1 -- 172.21.15.100:0/2737190933 wait complete. 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 -- start start 2024-03-12T05:31:54.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 Processor -- start 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8cc250 0x560a5e8d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 0x560a5e8d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e834020 con 0x560a5e8ccb70 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e507430 con 0x560a5e8cc250 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e4c3e00 con 0x560a5e8d5750 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5d8073740 10 client.0 _ll_register_callbacks cb 0x560a5e7dbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 0x560a5e8d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 0x560a5e8d2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55144/0 (socket says 172.21.15.100:55144) 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 -- 172.21.15.100:0/1228301897 learned_addr learned my addr 172.21.15.100:0/1228301897 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8cc250 0x560a5e8d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 -- 172.21.15.100:0/1228301897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 msgr2=0x560a5e8d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 -- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8cc250 msgr2=0x560a5e8d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8cc250 0x560a5e8d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.096+0000 7ff5ceffd640 1 -- 172.21.15.100:0/1228301897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c0009c00 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 0x560a5e8d2cc0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004620 tx=0x7ff5c0004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0d9c97e713dc13 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5b67fc640 1 -- 172.21.15.100:0/1228301897 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0003b00 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5b67fc640 1 -- 172.21.15.100:0/1228301897 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5c0003c60 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a5e5337e0 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5b67fc640 1 -- 172.21.15.100:0/1228301897 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5c0003e20 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 10 client.19635 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560a5e8326a0 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5b67fc640 1 -- 172.21.15.100:0/1228301897 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a5e5337e0 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5b67fc640 1 -- 172.21.15.100:0/1228301897 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5c0021040 con 0x560a5e8ccb70 2024-03-12T05:31:54.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 10 client.19635 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 10 client.19635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 20 client.19635 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e8e5a10 0x560a5e8e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 4 client.19635 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a5e570720 con 0x560a5e8e5a10 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.097+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e8e5a10 0x560a5e8e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.098+0000 7ff5b67fc640 10 client.19635 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.098+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e8e5a10 0x560a5e8e7e30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dba4f2d5e5f525a6 server_cookie=e0982403c66bec06 in_seq=0 out_seq=0 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5b67fc640 1 -- 172.21.15.100:0/1228301897 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a5e570720 con 0x560a5e8e5a10 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5b67fc640 10 client.19635 handle_command_reply: tid=0 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 client.19635 shutdown 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 msgr2=0x560a5e8d2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 0x560a5e8d2cc0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004620 tx=0x7ff5c0004400 comp rx=0 tx=0).stop 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 shutdown_connections 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e8e5a10 0x560a5e8e7e30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8d5750 0x560a5e8d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8ccb70 0x560a5e8d2cc0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5b67fc640 0 client.19635 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 --2- 172.21.15.100:0/1228301897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8cc250 0x560a5e8d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.100+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 >> 172.21.15.100:0/1228301897 conn(0x560a5e8c6980 msgr2=0x560a5e8c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.101+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 shutdown_connections 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.101+0000 7ff5d8073740 1 -- 172.21.15.100:0/1228301897 wait complete. 2024-03-12T05:31:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.101+0000 7ff5d8073740 20 client.19635 trim_cache size 0 max 16384 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 Processor -- start 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 -- start start 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e7b9130 con 0x560a5e8cc250 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e8c7b40 con 0x560a5e8d5750 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5d8073740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e8d1760 con 0x560a5e8ccb70 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8d4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55152/0 (socket says 172.21.15.100:55152) 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5c7fff640 1 -- 172.21.15.100:0/3988798216 learned_addr learned my addr 172.21.15.100:0/3988798216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.113+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 -- 172.21.15.100:0/3988798216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 msgr2=0x560a5e8d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 -- 172.21.15.100:0/3988798216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 msgr2=0x560a5e8ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8ec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 -- 172.21.15.100:0/3988798216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5e7cec20 con 0x560a5e8cc250 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8d4eb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009f20 tx=0x7ff5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d301445a245b38f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c4ff9640 1 -- 172.21.15.100:0/3988798216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80035f0 con 0x560a5e8cc250 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c4ff9640 1 -- 172.21.15.100:0/3988798216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8020040 con 0x560a5e8cc250 2024-03-12T05:31:54.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.114+0000 7ff5c4ff9640 1 -- 172.21.15.100:0/3988798216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8028560 con 0x560a5e8cc250 2024-03-12T05:31:54.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 -- 172.21.15.100:0/3988798216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 msgr2=0x560a5e8d4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8d4eb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009f20 tx=0x7ff5b80046c0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 -- 172.21.15.100:0/3988798216 shutdown_connections 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3988798216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8d4eb0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 -- 172.21.15.100:0/3988798216 >> 172.21.15.100:0/3988798216 conn(0x560a5e8c6980 msgr2=0x560a5e8d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 -- 172.21.15.100:0/3988798216 shutdown_connections 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.115+0000 7ff5d8073740 1 -- 172.21.15.100:0/3988798216 wait complete. 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 -- start start 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 Processor -- start 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8daf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e8dbd40 con 0x560a5e8cc250 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e944770 con 0x560a5e8d5750 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5e9448b0 con 0x560a5e8ccb70 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5d8073740 10 client.0 _ll_register_callbacks cb 0x7ff5c0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8dcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55164/0 (socket says 172.21.15.100:55164) 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5ceffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5c7fff640 1 -- 172.21.15.100:0/3946159933 learned_addr learned my addr 172.21.15.100:0/3946159933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5ceffd640 1 -- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 msgr2=0x560a5e8db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5ceffd640 1 -- 172.21.15.100:0/3946159933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 msgr2=0x560a5e8dcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8dcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5ceffd640 1 -- 172.21.15.100:0/3946159933 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b8004470 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.116+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5ceffd640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8daf30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0011dd0 tx=0x7ff5c0003660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7eda111a420c6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5b67fc640 1 -- 172.21.15.100:0/3946159933 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0005420 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5b67fc640 1 -- 172.21.15.100:0/3946159933 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5c0004b10 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5c0004470 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5b67fc640 1 -- 172.21.15.100:0/3946159933 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5c0005650 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5d8073740 10 client.28915 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560a5e7f48c0 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.117+0000 7ff5b67fc640 1 -- 172.21.15.100:0/3946159933 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5c0044020 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5b67fc640 1 -- 172.21.15.100:0/3946159933 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5c000f670 con 0x560a5e8ccb70 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5d8073740 10 client.28915 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:54.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5d8073740 10 client.28915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5d8073740 20 client.28915 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e945cb0 0x560a5e9480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5d8073740 4 client.28915 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a5e5bf890 con 0x560a5e945cb0 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e945cb0 0x560a5e9480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5b67fc640 10 client.28915 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:54.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.118+0000 7ff5c7fff640 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e945cb0 0x560a5e9480a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1465121e1d7ced3e server_cookie=9065caee5447141a in_seq=0 out_seq=0 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.122+0000 7ff5b67fc640 1 -- 172.21.15.100:0/3946159933 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a5e5bf890 con 0x560a5e945cb0 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.122+0000 7ff5b67fc640 10 client.28915 handle_command_reply: tid=0 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.122+0000 7ff5d8073740 1 client.28915 shutdown 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.122+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 msgr2=0x560a5e8daf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.122+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8daf30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0011dd0 tx=0x7ff5c0003660 comp rx=0 tx=0).stop 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 shutdown_connections 2024-03-12T05:31:54.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x560a5e945cb0 0x560a5e9480a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a5e8d5750 0x560a5e8db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a5e8ccb70 0x560a5e8daf30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 --2- 172.21.15.100:0/3946159933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a5e8cc250 0x560a5e8dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5b67fc640 0 client.28915 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 >> 172.21.15.100:0/3946159933 conn(0x560a5e8c6980 msgr2=0x560a5e8ec7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 shutdown_connections 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 1 -- 172.21.15.100:0/3946159933 wait complete. 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d8073740 20 client.28915 trim_cache size 0 max 16384 2024-03-12T05:31:54.118 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5bc0a1ba0 msgr2=0x7ff5bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5bc0a1ba0 0x7ff5bc0a4060 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c53d0 tx=0x7ff5c808d200 comp rx=0 tx=0).stop 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 msgr2=0x7ff5d0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 0x7ff5d0197650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005e30 tx=0x7ff5c8034670 comp rx=0 tx=0).stop 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 shutdown_connections 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5bc0a1ba0 0x7ff5bc0a4060 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d010b790 0x7ff5d0197650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5d0102d70 0x7ff5d019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 --2- 172.21.15.100:0/2273455063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5d0102450 0x7ff5d019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 >> 172.21.15.100:0/2273455063 conn(0x7ff5d00771e0 msgr2=0x7ff5d00fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 shutdown_connections 2024-03-12T05:31:54.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.123+0000 7ff5d549d640 1 -- 172.21.15.100:0/2273455063 wait complete. 2024-03-12T05:31:54.128 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:31:54.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:31:54.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 -- 172.21.15.100:0/2613586513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c100730 msgr2=0x7f044c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 --2- 172.21.15.100:0/2613586513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c100730 0x7f044c100bb0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f0434009900 tx=0x7f043402bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 -- 172.21.15.100:0/2613586513 shutdown_connections 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 --2- 172.21.15.100:0/2613586513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c1010f0 0x7f044c10cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 --2- 172.21.15.100:0/2613586513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c100730 0x7f044c100bb0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 --2- 172.21.15.100:0/2613586513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f044c105050 0x7f044c105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 -- 172.21.15.100:0/2613586513 >> 172.21.15.100:0/2613586513 conn(0x7f044c0765a0 msgr2=0x7f044c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 -- 172.21.15.100:0/2613586513 shutdown_connections 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 -- 172.21.15.100:0/2613586513 wait complete. 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.246+0000 7f0451556640 1 Processor -- start 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 -- start start 2024-03-12T05:31:54.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c100730 0x7f044c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f044c1010f0 0x7f044c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 0x7f044c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c197c30 con 0x7f044c105050 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c197da0 con 0x7f044c1010f0 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0451556640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c197f60 con 0x7f044c100730 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 0x7f044c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 0x7f044c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55184/0 (socket says 172.21.15.100:55184) 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 -- 172.21.15.100:0/3664308183 learned_addr learned my addr 172.21.15.100:0/3664308183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f0443fff640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f044c1010f0 0x7f044c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044affd640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c100730 0x7f044c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 -- 172.21.15.100:0/3664308183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c100730 msgr2=0x7f044c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c100730 0x7f044c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 -- 172.21.15.100:0/3664308183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f044c1010f0 msgr2=0x7f044c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f044c1010f0 0x7f044c19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 -- 172.21.15.100:0/3664308183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434009560 con 0x7f044c105050 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044affd640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c100730 0x7f044c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.247+0000 7f044b7fe640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 0x7f044c1975e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f044400eb10 tx=0x7f044400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.248+0000 7f0448ff9640 1 -- 172.21.15.100:0/3664308183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444004b20 con 0x7f044c105050 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.248+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c198240 con 0x7f044c105050 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.248+0000 7f0448ff9640 1 -- 172.21.15.100:0/3664308183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0444004c80 con 0x7f044c105050 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.248+0000 7f0448ff9640 1 -- 172.21.15.100:0/3664308183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444010650 con 0x7f044c105050 2024-03-12T05:31:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.248+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c1a44d0 con 0x7f044c105050 2024-03-12T05:31:54.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 212381 26.90 MB/sec execute 277 sec latency 413.887 ms 2024-03-12T05:31:54.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.250+0000 7f045420a740 1 Processor -- start 2024-03-12T05:31:54.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.250+0000 7f0448ff9640 1 -- 172.21.15.100:0/3664308183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0444005020 con 0x7f044c105050 2024-03-12T05:31:54.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 -- start start 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f749e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f7459960 con 0x55c6f749e250 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f740d310 con 0x55c6f74a7750 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f045420a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f740a840 con 0x55c6f749eb70 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f749e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f044affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f749e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55196/0 (socket says 172.21.15.100:55196) 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 -- 172.21.15.100:0/1506275337 learned_addr learned my addr 172.21.15.100:0/1506275337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0448ff9640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f04380a1ba0 0x7f04380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 -- 172.21.15.100:0/1506275337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 msgr2=0x55c6f74a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 -- 172.21.15.100:0/1506275337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 msgr2=0x55c6f74a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f0443fff640 1 -- 172.21.15.100:0/1506275337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6f73c8060 con 0x55c6f749e250 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f044affd640 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.251+0000 7f044b7fe640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f04380a1ba0 0x7f04380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.252+0000 7f0448ff9640 1 -- 172.21.15.100:0/3664308183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0444014070 con 0x7f044c105050 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.252+0000 7f0443fff640 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f749e630 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f043402c170 tx=0x7f0434005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc516e38bb26acb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.252+0000 7f04417fa640 1 -- 172.21.15.100:0/1506275337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340046c0 con 0x55c6f749e250 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.252+0000 7f04417fa640 1 -- 172.21.15.100:0/1506275337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0434004510 con 0x55c6f749e250 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.252+0000 7f04417fa640 1 -- 172.21.15.100:0/1506275337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434005180 con 0x55c6f749e250 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- 172.21.15.100:0/1506275337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 msgr2=0x55c6f749e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f044b7fe640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f04380a1ba0 0x7f04380a4060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f044409a950 tx=0x7f044400efd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f749e630 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f043402c170 tx=0x7f0434005b50 comp rx=0 tx=0).stop 2024-03-12T05:31:54.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- 172.21.15.100:0/1506275337 shutdown_connections 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- 172.21.15.100:0/1506275337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f749e630 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- 172.21.15.100:0/1506275337 >> 172.21.15.100:0/1506275337 conn(0x55c6f7498980 msgr2=0x55c6f7498d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- 172.21.15.100:0/1506275337 shutdown_connections 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- 172.21.15.100:0/1506275337 wait complete. 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- start start 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 Processor -- start 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f7406020 con 0x55c6f749e250 2024-03-12T05:31:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f70d9430 con 0x55c6f74a7750 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f7095e00 con 0x55c6f749eb70 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f045420a740 10 client.0 _ll_register_callbacks cb 0x55c6f73adcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55204/0 (socket says 172.21.15.100:55204) 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.253+0000 7f0443fff640 1 -- 172.21.15.100:0/1085441460 learned_addr learned my addr 172.21.15.100:0/1085441460 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f044affd640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 -- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 msgr2=0x55c6f74a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 -- 172.21.15.100:0/1085441460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 msgr2=0x55c6f74a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 -- 172.21.15.100:0/1085441460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c10d750 con 0x55c6f749e250 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f0443fff640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74a47a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0434000c00 tx=0x7f043402ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d583b29e1c85a5d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f04327fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f04327fc640 1 -- 172.21.15.100:0/1085441460 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434053070 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f04327fc640 1 -- 172.21.15.100:0/1085441460 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0434004a70 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6f71057e0 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f04327fc640 1 -- 172.21.15.100:0/1085441460 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0434005310 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f045420a740 10 client.19653 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6f74046a0 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f04327fc640 1 -- 172.21.15.100:0/1085441460 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c6f71057e0 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f04327fc640 1 -- 172.21.15.100:0/1085441460 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f043403c070 con 0x55c6f749e250 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f045420a740 10 client.19653 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.254+0000 7f045420a740 10 client.19653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f045420a740 20 client.19653 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f045420a740 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f74b7990 0x55c6f74b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f045420a740 4 client.19653 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6f7142720 con 0x55c6f74b7990 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f044affd640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f74b7990 0x55c6f74b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f044affd640 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f74b7990 0x55c6f74b9d80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de12b622c4a00366 server_cookie=128caee6da2b1874 in_seq=0 out_seq=0 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.255+0000 7f04327fc640 10 client.19653 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f04327fc640 1 -- 172.21.15.100:0/1085441460 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6f7142720 con 0x55c6f74b7990 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f04327fc640 10 client.19653 handle_command_reply: tid=0 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 client.19653 shutdown 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 msgr2=0x55c6f74a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74a47a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0434000c00 tx=0x7f043402ccf0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 shutdown_connections 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f74b7990 0x55c6f74b9d80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f04327fc640 0 client.19653 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 --2- 172.21.15.100:0/1085441460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74a47a0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 >> 172.21.15.100:0/1085441460 conn(0x55c6f7498980 msgr2=0x55c6f749a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 shutdown_connections 2024-03-12T05:31:54.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 1 -- 172.21.15.100:0/1085441460 wait complete. 2024-03-12T05:31:54.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.257+0000 7f045420a740 20 client.19653 trim_cache size 0 max 16384 2024-03-12T05:31:54.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 Processor -- start 2024-03-12T05:31:54.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 -- start start 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f749e250 0x55c6f74a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f7493a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 0x55c6f749fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f738b130 con 0x55c6f74a7750 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f74a57a0 con 0x55c6f749e250 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f045420a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f749d9f0 con 0x55c6f749eb70 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 0x55c6f749fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 0x55c6f749fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55212/0 (socket says 172.21.15.100:55212) 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 -- 172.21.15.100:0/706781242 learned_addr learned my addr 172.21.15.100:0/706781242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f044affd640 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f7493a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f749e250 0x55c6f74a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 -- 172.21.15.100:0/706781242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 msgr2=0x55c6f7493a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f7493a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 -- 172.21.15.100:0/706781242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f749e250 msgr2=0x55c6f74a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f749e250 0x55c6f74a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.272+0000 7f0443fff640 1 -- 172.21.15.100:0/706781242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6f73a0c20 con 0x55c6f74a7750 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.273+0000 7f0443fff640 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 0x55c6f749fa90 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f043402c420 tx=0x7f043402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6713528caed5f77 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.273+0000 7f0440ff9640 1 -- 172.21.15.100:0/706781242 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434053040 con 0x55c6f74a7750 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.273+0000 7f0440ff9640 1 -- 172.21.15.100:0/706781242 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0434009780 con 0x55c6f74a7750 2024-03-12T05:31:54.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.273+0000 7f0440ff9640 1 -- 172.21.15.100:0/706781242 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340416c0 con 0x55c6f74a7750 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 -- 172.21.15.100:0/706781242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 msgr2=0x55c6f749fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 0x55c6f749fa90 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f043402c420 tx=0x7f043402bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 -- 172.21.15.100:0/706781242 shutdown_connections 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f74a7750 0x55c6f749fa90 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f7493a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 --2- 172.21.15.100:0/706781242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f749e250 0x55c6f74a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 -- 172.21.15.100:0/706781242 >> 172.21.15.100:0/706781242 conn(0x55c6f7498980 msgr2=0x55c6f749ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 -- 172.21.15.100:0/706781242 shutdown_connections 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 -- 172.21.15.100:0/706781242 wait complete. 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 -- start start 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.274+0000 7f045420a740 1 Processor -- start 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f74ad5a0 con 0x55c6f749e250 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f74ad710 con 0x55c6f74a7750 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f74ad8d0 con 0x55c6f749eb70 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 10 client.0 _ll_register_callbacks cb 0x7f04440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55222/0 (socket says 172.21.15.100:55222) 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 -- 172.21.15.100:0/3851680504 learned_addr learned my addr 172.21.15.100:0/3851680504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f044affd640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 -- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 msgr2=0x55c6f74acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 -- 172.21.15.100:0/3851680504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 msgr2=0x55c6f74ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 -- 172.21.15.100:0/3851680504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6f74046a0 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f0443fff640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74ae730 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f0434005010 tx=0x7f0434004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f00570522a8047 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f04327fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f04327fc640 1 -- 172.21.15.100:0/3851680504 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434053070 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f04327fc640 1 -- 172.21.15.100:0/3851680504 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0434000e40 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6f74a45d0 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f04327fc640 1 -- 172.21.15.100:0/3851680504 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043402c810 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 10 client.19659 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.275+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6f73c68c0 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f04327fc640 1 -- 172.21.15.100:0/3851680504 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c6f74a45d0 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f04327fc640 1 -- 172.21.15.100:0/3851680504 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f043403c040 con 0x55c6f749e250 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f045420a740 10 client.19659 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f045420a740 10 client.19659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f045420a740 20 client.19659 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f045420a740 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f750ef60 0x55c6f7511380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f045420a740 4 client.19659 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0434004510 con 0x55c6f750ef60 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f044affd640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f750ef60 0x55c6f7511380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f04327fc640 10 client.19659 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.276+0000 7f044affd640 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f750ef60 0x55c6f7511380 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbde456a4bc9024e server_cookie=24446d967fdc5981 in_seq=0 out_seq=0 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f04327fc640 1 -- 172.21.15.100:0/3851680504 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0434004510 con 0x55c6f750ef60 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f04327fc640 10 client.19659 handle_command_reply: tid=0 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 client.19659 shutdown 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 msgr2=0x55c6f74ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74ae730 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f0434005010 tx=0x7f0434004af0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 shutdown_connections 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55c6f750ef60 0x55c6f7511380 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6f74a7750 0x55c6f74ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f04327fc640 0 client.19659 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6f749eb70 0x55c6f74acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 --2- 172.21.15.100:0/3851680504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f749e250 0x55c6f74ae730 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 >> 172.21.15.100:0/3851680504 conn(0x55c6f7498980 msgr2=0x55c6f7493a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 shutdown_connections 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 1 -- 172.21.15.100:0/3851680504 wait complete. 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f045420a740 20 client.19659 trim_cache size 0 max 16384 2024-03-12T05:31:54.277 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:31:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f04380a1ba0 msgr2=0x7f04380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f04380a1ba0 0x7f04380a4060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f044409a950 tx=0x7f044400efd0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 msgr2=0x7f044c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 0x7f044c1975e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f044400eb10 tx=0x7f044400c2d0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 shutdown_connections 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f04380a1ba0 0x7f04380a4060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044c105050 0x7f044c1975e0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f044c1010f0 0x7f044c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 --2- 172.21.15.100:0/3664308183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f044c100730 0x7f044c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 >> 172.21.15.100:0/3664308183 conn(0x7f044c0765a0 msgr2=0x7f044c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.277+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 shutdown_connections 2024-03-12T05:31:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.278+0000 7f0451556640 1 -- 172.21.15.100:0/3664308183 wait complete. 2024-03-12T05:31:54.281 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/676109930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:31:54.282 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:31:54.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:31:54.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.402+0000 7f2ca440e640 1 -- 172.21.15.100:0/2985589191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 msgr2=0x7f2c9c0689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.402+0000 7f2ca440e640 1 --2- 172.21.15.100:0/2985589191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c0689d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009930 tx=0x7f2c9802bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- 172.21.15.100:0/2985589191 shutdown_connections 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 --2- 172.21.15.100:0/2985589191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c1064c0 0x7f2c9c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 --2- 172.21.15.100:0/2985589191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c068fa0 0x7f2c9c105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 --2- 172.21.15.100:0/2985589191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c0689d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- 172.21.15.100:0/2985589191 >> 172.21.15.100:0/2985589191 conn(0x7f2c9c077180 msgr2=0x7f2c9c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- 172.21.15.100:0/2985589191 shutdown_connections 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- 172.21.15.100:0/2985589191 wait complete. 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 Processor -- start 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- start start 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1064c0 0x7f2c9c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c197630 0x7f2c9c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c198100 con 0x7f2c9c0685f0 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c198270 con 0x7f2c9c197630 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.403+0000 7f2ca440e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c198430 con 0x7f2c9c1064c0 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55228/0 (socket says 172.21.15.100:55228) 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 -- 172.21.15.100:0/3451720427 learned_addr learned my addr 172.21.15.100:0/3451720427 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c197630 0x7f2c9c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1064c0 0x7f2c9c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 -- 172.21.15.100:0/3451720427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1064c0 msgr2=0x7f2c9c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1064c0 0x7f2c9c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 -- 172.21.15.100:0/3451720427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c197630 msgr2=0x7f2c9c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c197630 0x7f2c9c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 -- 172.21.15.100:0/3451720427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98009560 con 0x7f2c9c0685f0 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c19d570 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c9802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2c937fe640 1 -- 172.21.15.100:0/3451720427 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980042d0 con 0x7f2c9c0685f0 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2c937fe640 1 -- 172.21.15.100:0/3451720427 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c98004430 con 0x7f2c9c0685f0 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c1a4370 con 0x7f2c9c0685f0 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2c937fe640 1 -- 172.21.15.100:0/3451720427 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98003c20 con 0x7f2c9c0685f0 2024-03-12T05:31:54.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.404+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c1a4850 con 0x7f2c9c0685f0 2024-03-12T05:31:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.407+0000 7f2c937fe640 1 -- 172.21.15.100:0/3451720427 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2c98004df0 con 0x7f2c9c0685f0 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.407+0000 7f2ca6fdd740 1 Processor -- start 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.407+0000 7f2c937fe640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1ba0 0x7f2c880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.407+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1ba0 0x7f2c880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.407+0000 7f2ca6fdd740 1 -- start start 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.407+0000 7f2c937fe640 1 -- 172.21.15.100:0/3451720427 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c980efd20 con 0x7f2c9c0685f0 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9312630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e931b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e931db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e92cd960 con 0x5624e931b750 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e9281310 con 0x5624e9312b70 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e927e840 con 0x5624e9312250 2024-03-12T05:31:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9312630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca19a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e931b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9312630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54466/0 (socket says 172.21.15.100:54466) 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 -- 172.21.15.100:0/1284084094 learned_addr learned my addr 172.21.15.100:0/1284084094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e931db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1ba0 0x7f2c880a4060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005fd0 tx=0x7f2c94011810 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 -- 172.21.15.100:0/1284084094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 msgr2=0x5624e931b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e931b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 -- 172.21.15.100:0/1284084094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 msgr2=0x5624e931db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e931db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 -- 172.21.15.100:0/1284084094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624e923c060 con 0x5624e9312250 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e931db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e931b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9312630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c980bc380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffa05b1f72ff4df3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.408+0000 7f2c90bfe640 1 -- 172.21.15.100:0/1284084094 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980c0b70 con 0x5624e9312250 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2c90bfe640 1 -- 172.21.15.100:0/1284084094 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c980c51c0 con 0x5624e9312250 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2c90bfe640 1 -- 172.21.15.100:0/1284084094 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980ed830 con 0x5624e9312250 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/1284084094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 msgr2=0x5624e9312630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9312630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c980bc380 comp rx=0 tx=0).stop 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/1284084094 shutdown_connections 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e931db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e931b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/1284084094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9312630 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/1284084094 >> 172.21.15.100:0/1284084094 conn(0x5624e930c980 msgr2=0x5624e930cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/1284084094 shutdown_connections 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.409+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/1284084094 wait complete. 2024-03-12T05:31:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 -- start start 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 Processor -- start 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 0x5624e9318760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9318ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e931b750 0x5624e93191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e927a020 con 0x5624e9312250 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e8f4d430 con 0x5624e9312b70 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e8f09e00 con 0x5624e931b750 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 0x5624e9318760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca6fdd740 10 client.0 _ll_register_callbacks cb 0x5624e9221cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 0x5624e9318760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55252/0 (socket says 172.21.15.100:55252) 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 -- 172.21.15.100:0/2134411538 learned_addr learned my addr 172.21.15.100:0/2134411538 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9318ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e931b750 0x5624e93191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 -- 172.21.15.100:0/2134411538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e931b750 msgr2=0x5624e93191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e931b750 0x5624e93191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 -- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 msgr2=0x5624e9318ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9318ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca21aa640 1 -- 172.21.15.100:0/2134411538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c1a4850 con 0x5624e9312250 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.410+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9318ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 0x5624e9318760 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2c980b8e80 tx=0x7f2c980bcba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f0e1c8cf2c9051 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2c85ffb640 1 -- 172.21.15.100:0/2134411538 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980c0b70 con 0x5624e9312250 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2c85ffb640 1 -- 172.21.15.100:0/2134411538 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c980dc040 con 0x5624e9312250 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624e8f797e0 con 0x5624e9312250 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2c85ffb640 1 -- 172.21.15.100:0/2134411538 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c98005520 con 0x5624e9312250 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 10 client.19671 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5624e92786a0 con 0x5624e9312250 2024-03-12T05:31:54.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2c85ffb640 1 -- 172.21.15.100:0/2134411538 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5624e8f797e0 con 0x5624e9312250 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2c85ffb640 1 -- 172.21.15.100:0/2134411538 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c9803c070 con 0x5624e9312250 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 10 client.19671 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 10 client.19671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 20 client.19671 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e932b400 0x5624e932d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 4 client.19671 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624e8fb6720 con 0x5624e932b400 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.411+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e932b400 0x5624e932d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.412+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e932b400 0x5624e932d7f0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d99898b7ff8c4c server_cookie=411859eb470e884c in_seq=0 out_seq=0 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.412+0000 7f2c85ffb640 10 client.19671 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2c85ffb640 1 -- 172.21.15.100:0/2134411538 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624e8fb6720 con 0x5624e932b400 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2c85ffb640 10 client.19671 handle_command_reply: tid=0 2024-03-12T05:31:54.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 client.19671 shutdown 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 msgr2=0x5624e9318760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 0x5624e9318760 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2c980b8e80 tx=0x7f2c980bcba0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 shutdown_connections 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e932b400 0x5624e932d7f0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e931b750 0x5624e93191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9318ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2134411538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312250 0x5624e9318760 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 >> 172.21.15.100:0/2134411538 conn(0x5624e930c980 msgr2=0x5624e930e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2c85ffb640 0 client.19671 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 shutdown_connections 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2134411538 wait complete. 2024-03-12T05:31:54.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.413+0000 7f2ca6fdd740 20 client.19671 trim_cache size 0 max 16384 2024-03-12T05:31:54.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 Processor -- start 2024-03-12T05:31:54.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 -- start start 2024-03-12T05:31:54.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e931ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9319000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e93159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e91ff130 con 0x5624e931b750 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e930d980 con 0x5624e9312b70 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e9332980 con 0x5624e9312250 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e93159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e93159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55260/0 (socket says 172.21.15.100:55260) 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 -- 172.21.15.100:0/2864001174 learned_addr learned my addr 172.21.15.100:0/2864001174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e931ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9319000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 -- 172.21.15.100:0/2864001174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 msgr2=0x5624e931ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e931ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 -- 172.21.15.100:0/2864001174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 msgr2=0x5624e9319000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9319000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.430+0000 7f2ca29ab640 1 -- 172.21.15.100:0/2864001174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624e9214c20 con 0x5624e931b750 2024-03-12T05:31:54.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e93159e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2c9401d880 tx=0x7f2c9401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15515febcd9bd5bd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2c87fff640 1 -- 172.21.15.100:0/2864001174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94022040 con 0x5624e931b750 2024-03-12T05:31:54.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2c87fff640 1 -- 172.21.15.100:0/2864001174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c94027910 con 0x5624e931b750 2024-03-12T05:31:54.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2c87fff640 1 -- 172.21.15.100:0/2864001174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940269b0 con 0x5624e931b750 2024-03-12T05:31:54.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2864001174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 msgr2=0x5624e93159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e93159e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2c9401d880 tx=0x7f2c9401d000 comp rx=0 tx=0).stop 2024-03-12T05:31:54.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2864001174 shutdown_connections 2024-03-12T05:31:54.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.431+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e931b750 0x5624e93159e0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e9312b70 0x5624e9319000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/2864001174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e931ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2864001174 >> 172.21.15.100:0/2864001174 conn(0x5624e930c980 msgr2=0x5624e9313a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2864001174 shutdown_connections 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/2864001174 wait complete. 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- start start 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 Processor -- start 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9320f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 0x5624e93214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 0x5624e93219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e93818b0 con 0x5624e9312b70 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e9381a20 con 0x5624e931b750 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.432+0000 7f2ca6fdd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e9381be0 con 0x5624e9312250 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca6fdd740 10 client.0 _ll_register_callbacks cb 0x7f2c980045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca19a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 0x5624e93214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca29ab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 0x5624e93219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca29ab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 0x5624e93219e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34754/0 (socket says 172.21.15.100:34754) 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca29ab640 1 -- 172.21.15.100:0/721581950 learned_addr learned my addr 172.21.15.100:0/721581950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9320f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 0x5624e93214a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55274/0 (socket says 172.21.15.100:55274) 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca19a9640 1 -- 172.21.15.100:0/721581950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 msgr2=0x5624e9320f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9320f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.433+0000 7f2ca19a9640 1 -- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 msgr2=0x5624e93219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 0x5624e93219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2ca19a9640 1 -- 172.21.15.100:0/721581950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9401d5e0 con 0x5624e9312b70 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2ca29ab640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 0x5624e93219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2ca19a9640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 0x5624e93214a0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x5624e9317fc0 tx=0x7f2c8c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05758732b852c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2c85ffb640 1 -- 172.21.15.100:0/721581950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c003ac0 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2c85ffb640 1 -- 172.21.15.100:0/721581950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2c8c003c20 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.434+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c980c51c0 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2c85ffb640 1 -- 172.21.15.100:0/721581950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c8c003de0 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 10 client.19677 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5624e923a8c0 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2c85ffb640 1 -- 172.21.15.100:0/721581950 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c980c51c0 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2c85ffb640 1 -- 172.21.15.100:0/721581950 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c8c0106f0 con 0x5624e9312b70 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 10 client.19677 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 10 client.19677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 20 client.19677 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e9383420 0x5624e9385840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 4 client.19677 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:31:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.435+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624e9005890 con 0x5624e9383420 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.436+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e9383420 0x5624e9385840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.436+0000 7f2c85ffb640 10 client.19677 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.436+0000 7f2ca21aa640 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e9383420 0x5624e9385840 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cbae81ef2db629 server_cookie=63aa220bde9b684c in_seq=0 out_seq=0 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2c85ffb640 1 -- 172.21.15.100:0/721581950 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5624e9005890 con 0x5624e9383420 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2c85ffb640 10 client.19677 handle_command_reply: tid=0 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 client.19677 shutdown 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 msgr2=0x5624e93214a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 0x5624e93214a0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x5624e9317fc0 tx=0x7f2c8c004610 comp rx=0 tx=0).stop 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 shutdown_connections 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x5624e9383420 0x5624e9385840 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624e931b750 0x5624e93219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2c85ffb640 0 client.19677 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624e9312b70 0x5624e93214a0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 --2- 172.21.15.100:0/721581950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624e9312250 0x5624e9320f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.437+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 >> 172.21.15.100:0/721581950 conn(0x5624e930c980 msgr2=0x5624e930b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.438+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 shutdown_connections 2024-03-12T05:31:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.438+0000 7f2ca6fdd740 1 -- 172.21.15.100:0/721581950 wait complete. 2024-03-12T05:31:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.438+0000 7f2ca6fdd740 20 client.19677 trim_cache size 0 max 16384 2024-03-12T05:31:54.437 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:31:54.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1ba0 msgr2=0x7f2c880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1ba0 0x7f2c880a4060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005fd0 tx=0x7f2c94011810 comp rx=0 tx=0).stop 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 msgr2=0x7f2c9c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c19d570 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c9802c5f0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 shutdown_connections 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2c880a1ba0 0x7f2c880a4060 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c9c197630 0x7f2c9c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c1064c0 0x7f2c9c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 --2- 172.21.15.100:0/3451720427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c9c0685f0 0x7f2c9c19d570 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 >> 172.21.15.100:0/3451720427 conn(0x7f2c9c077180 msgr2=0x7f2c9c0fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 shutdown_connections 2024-03-12T05:31:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:54.439+0000 7f2ca440e640 1 -- 172.21.15.100:0/3451720427 wait complete. 2024-03-12T05:31:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/676109930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:31:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/676109930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:31:55.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 212907 26.91 MB/sec execute 278 sec latency 191.358 ms 2024-03-12T05:31:55.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:55 smithi100 ceph-mon[32210]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 496 op/s 2024-03-12T05:31:55.449 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:31:55.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:31:55.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:31:55.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:55 smithi156 ceph-mon[36270]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 496 op/s 2024-03-12T05:31:55.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:55 smithi161 ceph-mon[36043]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 496 op/s 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 -- 172.21.15.100:0/2105224286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf8102ad0 msgr2=0x7f8bf810afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/2105224286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf8102ad0 0x7f8bf810afb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009900 tx=0x7f8be802bcd0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 -- 172.21.15.100:0/2105224286 shutdown_connections 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/2105224286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf810b5d0 0x7f8bf810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/2105224286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf8102ad0 0x7f8bf810afb0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/2105224286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bf81021b0 0x7f8bf8102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 -- 172.21.15.100:0/2105224286 >> 172.21.15.100:0/2105224286 conn(0x7f8bf80fb9d0 msgr2=0x7f8bf80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 -- 172.21.15.100:0/2105224286 shutdown_connections 2024-03-12T05:31:55.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.577+0000 7f8bfeba0640 1 -- 172.21.15.100:0/2105224286 wait complete. 2024-03-12T05:31:55.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 Processor -- start 2024-03-12T05:31:55.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 -- start start 2024-03-12T05:31:55.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 0x7f8bf80ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf8102ad0 0x7f8bf8100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bf810b5d0 0x7f8bf8100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf81a5a60 con 0x7f8bf81021b0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf81a5bd0 con 0x7f8bf8102ad0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfeba0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf81a5d90 con 0x7f8bf810b5d0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 0x7f8bf80ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 0x7f8bf80ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55288/0 (socket says 172.21.15.100:55288) 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfd39d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf8102ad0 0x7f8bf8100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3444950823 learned_addr learned my addr 172.21.15.100:0/3444950823 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bf810b5d0 0x7f8bf8100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3444950823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bf810b5d0 msgr2=0x7f8bf8100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bf810b5d0 0x7f8bf8100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3444950823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf8102ad0 msgr2=0x7f8bf8100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf8102ad0 0x7f8bf8100340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3444950823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8009560 con 0x7f8bf81021b0 2024-03-12T05:31:55.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.578+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf8102ad0 0x7f8bf8100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.579+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 0x7f8bf80ffe00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400e9f0 tx=0x7f8bf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.579+0000 7f8beeffd640 1 -- 172.21.15.100:0/3444950823 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4009720 con 0x7f8bf81021b0 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.579+0000 7f8beeffd640 1 -- 172.21.15.100:0/3444950823 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf400cd80 con 0x7f8bf81021b0 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.579+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf81a6070 con 0x7f8bf81021b0 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.579+0000 7f8beeffd640 1 -- 172.21.15.100:0/3444950823 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4010690 con 0x7f8bf81021b0 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.579+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf81a6500 con 0x7f8bf81021b0 2024-03-12T05:31:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.581+0000 7f8beeffd640 1 -- 172.21.15.100:0/3444950823 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8bf40046c0 con 0x7f8bf81021b0 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.582+0000 7f8beeffd640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8be40a1ba0 0x7f8be40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.582+0000 7f8c029d3740 1 Processor -- start 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.582+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8be40a1ba0 0x7f8be40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.582+0000 7f8beeffd640 1 -- 172.21.15.100:0/3444950823 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8bf4014070 con 0x7f8bf81021b0 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 -- start start 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930977f90 con 0x557930a09f00 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579309754c0 con 0x557930a09550 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8c029d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930970ca0 con 0x557930a12a80 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a09930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34780/0 (socket says 172.21.15.100:34780) 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3639067526 learned_addr learned my addr 172.21.15.100:0/3639067526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8be40a1ba0 0x7f8be40a4060 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006010 tx=0x7f8be8005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3639067526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 msgr2=0x557930a14ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a14ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3639067526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 msgr2=0x557930a124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/3639067526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579309330b0 con 0x557930a09550 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.583+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a09930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4097180 tx=0x7f8bf409a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142b9abe24ca4b43 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8be3fff640 1 -- 172.21.15.100:0/3639067526 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf409e900 con 0x557930a09550 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8be3fff640 1 -- 172.21.15.100:0/3639067526 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf40a2f50 con 0x557930a09550 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8be3fff640 1 -- 172.21.15.100:0/3639067526 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40a2ac0 con 0x557930a09550 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 -- 172.21.15.100:0/3639067526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 msgr2=0x557930a09930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a09930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4097180 tx=0x7f8bf409a110 comp rx=0 tx=0).stop 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 -- 172.21.15.100:0/3639067526 shutdown_connections 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 --2- 172.21.15.100:0/3639067526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a09930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.584+0000 7f8c029d3740 1 -- 172.21.15.100:0/3639067526 >> 172.21.15.100:0/3639067526 conn(0x557930a03a10 msgr2=0x557930a03df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 -- 172.21.15.100:0/3639067526 shutdown_connections 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 -- 172.21.15.100:0/3639067526 wait complete. 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 -- start start 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 Processor -- start 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a0fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a10530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930915f40 con 0x557930a09f00 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579306444b0 con 0x557930a09550 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579308f6670 con 0x557930a12a80 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8c029d3740 10 client.0 _ll_register_callbacks cb 0x557930963f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8bfd39d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.585+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a0fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfd39d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a0fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55306/0 (socket says 172.21.15.100:55306) 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfd39d640 1 -- 172.21.15.100:0/442159356 learned_addr learned my addr 172.21.15.100:0/442159356 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a10530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/442159356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 msgr2=0x557930a10530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a10530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/442159356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 msgr2=0x557930a0fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a0fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/442159356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf81a6500 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a0fab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8bf409a740 tx=0x7f8bf409a930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb8bc2be9b45daa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8be17fa640 1 -- 172.21.15.100:0/442159356 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf409e900 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8be17fa640 1 -- 172.21.15.100:0/442159356 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8bf40a2ac0 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55793094af40 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8be17fa640 1 -- 172.21.15.100:0/442159356 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf40a2c80 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8c029d3740 10 client.29033 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.586+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55793096f320 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8be17fa640 1 -- 172.21.15.100:0/442159356 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55793094af40 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8be17fa640 1 -- 172.21.15.100:0/442159356 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8bf401f060 con 0x557930a09550 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8c029d3740 10 client.29033 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8c029d3740 10 client.29033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8c029d3740 20 client.29033 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:55.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8c029d3740 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a22d50 0x557930a25140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8c029d3740 4 client.29033 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557930967310 con 0x557930a22d50 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a22d50 0x557930a25140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8be17fa640 10 client.29033 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.587+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a22d50 0x557930a25140 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b540f4a4a5f712 server_cookie=2d68e339cc990ea5 in_seq=0 out_seq=0 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8be17fa640 1 -- 172.21.15.100:0/442159356 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557930967310 con 0x557930a22d50 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8be17fa640 10 client.29033 handle_command_reply: tid=0 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 client.29033 shutdown 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 msgr2=0x557930a0fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a0fab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8bf409a740 tx=0x7f8bf409a930 comp rx=0 tx=0).stop 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 shutdown_connections 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a22d50 0x557930a25140 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a12a80 0x557930a10530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09f00 0x557930a0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8be17fa640 0 client.29033 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 --2- 172.21.15.100:0/442159356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a09550 0x557930a0fab0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 >> 172.21.15.100:0/442159356 conn(0x557930a03a10 msgr2=0x557930a05970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 shutdown_connections 2024-03-12T05:31:55.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 1 -- 172.21.15.100:0/442159356 wait complete. 2024-03-12T05:31:55.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.588+0000 7f8c029d3740 20 client.29033 trim_cache size 0 max 16384 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 Processor -- start 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 -- start start 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a10350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930a0b720 con 0x557930a09550 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579309ff2e0 con 0x557930a12a80 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8c029d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930a254a0 con 0x557930a09f00 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8bfe39f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55318/0 (socket says 172.21.15.100:55318) 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/1780391105 learned_addr learned my addr 172.21.15.100:0/1780391105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.606+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a10350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfe39f640 1 -- 172.21.15.100:0/1780391105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 msgr2=0x557930a10350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a10350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfe39f640 1 -- 172.21.15.100:0/1780391105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 msgr2=0x557930a121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfe39f640 1 -- 172.21.15.100:0/1780391105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793090bc00 con 0x557930a12a80 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a0cd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0003a50 tx=0x7f8bf0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b2f24e99bbf0dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8be37fe640 1 -- 172.21.15.100:0/1780391105 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0016070 con 0x557930a12a80 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8be37fe640 1 -- 172.21.15.100:0/1780391105 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf0017040 con 0x557930a12a80 2024-03-12T05:31:55.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.607+0000 7f8be37fe640 1 -- 172.21.15.100:0/1780391105 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf001b850 con 0x557930a12a80 2024-03-12T05:31:55.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 -- 172.21.15.100:0/1780391105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 msgr2=0x557930a0cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a0cd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0003a50 tx=0x7f8bf0003f10 comp rx=0 tx=0).stop 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 -- 172.21.15.100:0/1780391105 shutdown_connections 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a0cd50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a10350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 --2- 172.21.15.100:0/1780391105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 -- 172.21.15.100:0/1780391105 >> 172.21.15.100:0/1780391105 conn(0x557930a03a10 msgr2=0x557930a29ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 -- 172.21.15.100:0/1780391105 shutdown_connections 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 -- 172.21.15.100:0/1780391105 wait complete. 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 -- start start 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.608+0000 7f8c029d3740 1 Processor -- start 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a72a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a72fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930a198e0 con 0x557930a09550 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930a19a50 con 0x557930a12a80 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557930a19c10 con 0x557930a09f00 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 10 client.0 _ll_register_callbacks cb 0x7f8bf4096390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a72a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a72a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55320/0 (socket says 172.21.15.100:55320) 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/2084439996 learned_addr learned my addr 172.21.15.100:0/2084439996 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfe39f640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a72fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/2084439996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 msgr2=0x557930a72fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a72fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 msgr2=0x557930a19320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a19320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 -- 172.21.15.100:0/2084439996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793096f320 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8bfdb9e640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a72a80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4096c10 tx=0x7f8bf409a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b483317b52f6f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8be17fa640 1 -- 172.21.15.100:0/2084439996 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4096a50 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8be17fa640 1 -- 172.21.15.100:0/2084439996 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8bf409aeb0 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.609+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557930996900 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8be17fa640 1 -- 172.21.15.100:0/2084439996 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf409e900 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 10 client.19695 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f8bf0009e70 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8be17fa640 1 -- 172.21.15.100:0/2084439996 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557930996900 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8be17fa640 1 -- 172.21.15.100:0/2084439996 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8bf408b070 con 0x557930a09550 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 10 client.19695 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 10 client.19695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 20 client.19695 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:55.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a7b5f0 0x557930a7da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 4 client.19695 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557930670860 con 0x557930a7b5f0 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a7b5f0 0x557930a7da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.610+0000 7f8be17fa640 10 client.19695 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.611+0000 7f8bfd39d640 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a7b5f0 0x557930a7da10 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c17caa44b94261d server_cookie=e81114af528947c3 in_seq=0 out_seq=0 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8be17fa640 1 -- 172.21.15.100:0/2084439996 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557930670860 con 0x557930a7b5f0 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8be17fa640 10 client.19695 handle_command_reply: tid=0 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 client.19695 shutdown 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 msgr2=0x557930a72a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a72a80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4096c10 tx=0x7f8bf409a240 comp rx=0 tx=0).stop 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 shutdown_connections 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x557930a7b5f0 0x557930a7da10 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557930a12a80 0x557930a19320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557930a09f00 0x557930a72fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 --2- 172.21.15.100:0/2084439996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557930a09550 0x557930a72a80 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8be17fa640 0 client.19695 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 >> 172.21.15.100:0/2084439996 conn(0x557930a03a10 msgr2=0x557930a02400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 shutdown_connections 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 1 -- 172.21.15.100:0/2084439996 wait complete. 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.612+0000 7f8c029d3740 20 client.19695 trim_cache size 0 max 16384 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "9f345ad6-0589-4b4a-b567-5bbc3adbb77f", 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:31:55.611 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:31:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8be40a1ba0 msgr2=0x7f8be40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8be40a1ba0 0x7f8be40a4060 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006010 tx=0x7f8be8005e40 comp rx=0 tx=0).stop 2024-03-12T05:31:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 msgr2=0x7f8bf80ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 0x7f8bf80ffe00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400e9f0 tx=0x7f8bf400edf0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 shutdown_connections 2024-03-12T05:31:55.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f8be40a1ba0 0x7f8be40a4060 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bf810b5d0 0x7f8bf8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bf8102ad0 0x7f8bf8100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 --2- 172.21.15.100:0/3444950823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf81021b0 0x7f8bf80ffe00 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 >> 172.21.15.100:0/3444950823 conn(0x7f8bf80fb9d0 msgr2=0x7f8bf80fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 shutdown_connections 2024-03-12T05:31:55.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.613+0000 7f8bfeba0640 1 -- 172.21.15.100:0/3444950823 wait complete. 2024-03-12T05:31:55.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:31:55.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:31:55.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 -- 172.21.15.100:0/2940744609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258100c20 msgr2=0x7f5258104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 --2- 172.21.15.100:0/2940744609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258100c20 0x7f5258104c60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f524c009930 tx=0x7f524c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 -- 172.21.15.100:0/2940744609 shutdown_connections 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 --2- 172.21.15.100:0/2940744609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5258105340 0x7f525810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 --2- 172.21.15.100:0/2940744609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258100c20 0x7f5258104c60 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 --2- 172.21.15.100:0/2940744609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 -- 172.21.15.100:0/2940744609 >> 172.21.15.100:0/2940744609 conn(0x7f5258077230 msgr2=0x7f5258077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 -- 172.21.15.100:0/2940744609 shutdown_connections 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.746+0000 7f525e042640 1 -- 172.21.15.100:0/2940744609 wait complete. 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 Processor -- start 2024-03-12T05:31:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 -- start start 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258105340 0x7f5258196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f525819af20 0x7f525819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525819b910 con 0x7f5258105340 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525819ba80 con 0x7f5258100270 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.747+0000 7f525e042640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525819bbf0 con 0x7f525819af20 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34834/0 (socket says 172.21.15.100:34834) 2024-03-12T05:31:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 -- 172.21.15.100:0/442650656 learned_addr learned my addr 172.21.15.100:0/442650656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f5257fff640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f525819af20 0x7f525819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f5256ffd640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258105340 0x7f5258196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 -- 172.21.15.100:0/442650656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f525819af20 msgr2=0x7f525819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f525819af20 0x7f525819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 -- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258105340 msgr2=0x7f5258196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258105340 0x7f5258196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 -- 172.21.15.100:0/442650656 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009560 con 0x7f5258100270 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f52577fe640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258196660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f524800e9f0 tx=0x7f524800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f5254ff9640 1 -- 172.21.15.100:0/442650656 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248009700 con 0x7f5258100270 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52581a31d0 con 0x7f5258100270 2024-03-12T05:31:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.748+0000 7f5254ff9640 1 -- 172.21.15.100:0/442650656 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f524800cd80 con 0x7f5258100270 2024-03-12T05:31:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.749+0000 7f5254ff9640 1 -- 172.21.15.100:0/442650656 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248018920 con 0x7f5258100270 2024-03-12T05:31:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.749+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581a36a0 con 0x7f5258100270 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.751+0000 7f5254ff9640 1 -- 172.21.15.100:0/442650656 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5248018a80 con 0x7f5258100270 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.751+0000 7f5260c0b740 1 Processor -- start 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.751+0000 7f5260c0b740 1 -- start start 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5254ff9640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52400a1ba0 0x7f52400a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e3416fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e341705480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341707db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5260c0b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341669280 con 0x55e3416f9c90 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5260c0b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3416667b0 con 0x55e3417059c0 2024-03-12T05:31:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5260c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341661f90 con 0x55e3416fa640 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5257fff640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52400a1ba0 0x7f52400a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e341705480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f52577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e3416fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e341705480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54600/0 (socket says 172.21.15.100:54600) 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 -- 172.21.15.100:0/805301979 learned_addr learned my addr 172.21.15.100:0/805301979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f52577fe640 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e3416fa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55342/0 (socket says 172.21.15.100:55342) 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5254ff9640 1 -- 172.21.15.100:0/442650656 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5248014070 con 0x7f5258100270 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5257fff640 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341707db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 -- 172.21.15.100:0/805301979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 msgr2=0x55e341707db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341707db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 -- 172.21.15.100:0/805301979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 msgr2=0x55e3416fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e3416fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5256ffd640 1 -- 172.21.15.100:0/805301979 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e341620570 con 0x55e3416fa640 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f52577fe640 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e3416fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.752+0000 7f5257fff640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52400a1ba0 0x7f52400a4060 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f5250011860 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.753+0000 7f5256ffd640 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e341705480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f524c005ab0 tx=0x7f524c02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b1a4ced8b14fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.753+0000 7f523dffb640 1 -- 172.21.15.100:0/805301979 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c004780 con 0x55e3416fa640 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.753+0000 7f523dffb640 1 -- 172.21.15.100:0/805301979 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f524c0048e0 con 0x55e3416fa640 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.753+0000 7f523dffb640 1 -- 172.21.15.100:0/805301979 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c005050 con 0x55e3416fa640 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 -- 172.21.15.100:0/805301979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 msgr2=0x55e341705480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e341705480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f524c005ab0 tx=0x7f524c02c5d0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 -- 172.21.15.100:0/805301979 shutdown_connections 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341707db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e341705480 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 --2- 172.21.15.100:0/805301979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e3416fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.754+0000 7f5260c0b740 1 -- 172.21.15.100:0/805301979 >> 172.21.15.100:0/805301979 conn(0x55e3416f4990 msgr2=0x55e3416f4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.755+0000 7f5260c0b740 1 -- 172.21.15.100:0/805301979 shutdown_connections 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.755+0000 7f5260c0b740 1 -- 172.21.15.100:0/805301979 wait complete. 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.755+0000 7f5260c0b740 1 -- start start 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.755+0000 7f5260c0b740 1 Processor -- start 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.755+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 0x55e3417053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.755+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416fa640 0x55e341702740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341702d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5260c0b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341335430 con 0x55e3416fa640 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5260c0b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3412f1e00 con 0x55e3417059c0 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5260c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3415e6fd0 con 0x55e3416f9c90 2024-03-12T05:31:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5260c0b740 10 client.0 _ll_register_callbacks cb 0x55e34163cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 0x55e3417053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 0x55e3417053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54612/0 (socket says 172.21.15.100:54612) 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 -- 172.21.15.100:0/2088455876 learned_addr learned my addr 172.21.15.100:0/2088455876 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5256ffd640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416fa640 0x55e341702740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5257fff640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341702d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 -- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 msgr2=0x55e341702d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341702d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 -- 172.21.15.100:0/2088455876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416fa640 msgr2=0x55e341702740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416fa640 0x55e341702740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 -- 172.21.15.100:0/2088455876 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525810d720 con 0x55e3416f9c90 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f52577fe640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 0x55e3417053b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5248096cc0 tx=0x7f524809a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=214edf41fd0bf48b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.756+0000 7f5236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:31:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5236ffd640 1 -- 172.21.15.100:0/2088455876 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524801cce0 con 0x55e3416f9c90 2024-03-12T05:31:55.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3413617e0 con 0x55e3416f9c90 2024-03-12T05:31:55.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5236ffd640 1 -- 172.21.15.100:0/2088455876 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5248025170 con 0x55e3416f9c90 2024-03-12T05:31:55.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5236ffd640 1 -- 172.21.15.100:0/2088455876 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f524809a860 con 0x55e3416f9c90 2024-03-12T05:31:55.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5260c0b740 10 client.28981 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:55.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e341660610 con 0x55e3416f9c90 2024-03-12T05:31:55.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.757+0000 7f5236ffd640 1 -- 172.21.15.100:0/2088455876 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3413617e0 con 0x55e3416f9c90 2024-03-12T05:31:55.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5236ffd640 1 -- 172.21.15.100:0/2088455876 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52480173d0 con 0x55e3416f9c90 2024-03-12T05:31:55.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5260c0b740 10 client.28981 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:55.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5260c0b740 10 client.28981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:55.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5260c0b740 20 client.28981 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:55.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5260c0b740 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3416fc830 0x55e3416fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5260c0b740 4 client.28981 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e34139e720 con 0x55e3416fc830 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5256ffd640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3416fc830 0x55e3416fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5236ffd640 10 client.28981 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.758+0000 7f5256ffd640 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3416fc830 0x55e3416fcc30 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96138c6762b5bef server_cookie=1cc5245432941d17 in_seq=0 out_seq=0 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.759+0000 7f5236ffd640 1 -- 172.21.15.100:0/2088455876 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e34139e720 con 0x55e3416fc830 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.759+0000 7f5236ffd640 10 client.28981 handle_command_reply: tid=0 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.759+0000 7f5260c0b740 1 client.28981 shutdown 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.759+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 msgr2=0x55e3417053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.759+0000 7f5260c0b740 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 0x55e3417053b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5248096cc0 tx=0x7f524809a490 comp rx=0 tx=0).stop 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 shutdown_connections 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3416fc830 0x55e3416fcc30 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5236ffd640 0 client.28981 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3417059c0 0x55e341702d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416fa640 0x55e341702740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 --2- 172.21.15.100:0/2088455876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416f9c90 0x55e3417053b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 >> 172.21.15.100:0/2088455876 conn(0x55e3416f4990 msgr2=0x55e3416f6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 shutdown_connections 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 1 -- 172.21.15.100:0/2088455876 wait complete. 2024-03-12T05:31:55.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.760+0000 7f5260c0b740 20 client.28981 trim_cache size 0 max 16384 2024-03-12T05:31:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 Processor -- start 2024-03-12T05:31:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 -- start start 2024-03-12T05:31:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e341704e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e3417053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e34170c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341711fd0 con 0x55e3416f9c90 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3416f93a0 con 0x55e3416fc830 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.780+0000 7f5260c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34171ae60 con 0x55e3416fa640 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e341704e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e341704e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55354/0 (socket says 172.21.15.100:55354) 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f5257fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e34170c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 -- 172.21.15.100:0/3835090352 learned_addr learned my addr 172.21.15.100:0/3835090352 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f5256ffd640 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e3417053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 -- 172.21.15.100:0/3835090352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 msgr2=0x55e3417053b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e3417053b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 -- 172.21.15.100:0/3835090352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 msgr2=0x55e34170c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e34170c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 -- 172.21.15.100:0/3835090352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3415fcac0 con 0x55e3416f9c90 2024-03-12T05:31:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f52577fe640 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e341704e70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5248097230 tx=0x7f524801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869a83cafa1d4f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f523d7fa640 1 -- 172.21.15.100:0/3835090352 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524809a630 con 0x55e3416f9c90 2024-03-12T05:31:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f523d7fa640 1 -- 172.21.15.100:0/3835090352 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5248025170 con 0x55e3416f9c90 2024-03-12T05:31:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.781+0000 7f523d7fa640 1 -- 172.21.15.100:0/3835090352 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480a2fe0 con 0x55e3416f9c90 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 -- 172.21.15.100:0/3835090352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 msgr2=0x55e341704e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e341704e70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5248097230 tx=0x7f524801f000 comp rx=0 tx=0).stop 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 -- 172.21.15.100:0/3835090352 shutdown_connections 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e34170c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e3417053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 --2- 172.21.15.100:0/3835090352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3416f9c90 0x55e341704e70 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 -- 172.21.15.100:0/3835090352 >> 172.21.15.100:0/3835090352 conn(0x55e3416f4990 msgr2=0x55e3416fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 -- 172.21.15.100:0/3835090352 shutdown_connections 2024-03-12T05:31:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.782+0000 7f5260c0b740 1 -- 172.21.15.100:0/3835090352 wait complete. 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 -- start start 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 Processor -- start 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e34170ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e341708e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3417059c0 0x55e341709350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341709910 con 0x55e3417059c0 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341709a80 con 0x55e3416fc830 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e341709c40 con 0x55e3416fa640 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5260c0b740 10 client.0 _ll_register_callbacks cb 0x7f52500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e341708e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f52577fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e34170ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f52577fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e34170ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:54632/0 (socket says 172.21.15.100:54632) 2024-03-12T05:31:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e341708e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:34876/0 (socket says 172.21.15.100:34876) 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f52577fe640 1 -- 172.21.15.100:0/1628576115 learned_addr learned my addr 172.21.15.100:0/1628576115 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.783+0000 7f5257fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3417059c0 0x55e341709350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f52577fe640 1 -- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 msgr2=0x55e341708e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f52577fe640 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e341708e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f52577fe640 1 -- 172.21.15.100:0/1628576115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3417059c0 msgr2=0x55e341709350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f52577fe640 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3417059c0 0x55e341709350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f52577fe640 1 -- 172.21.15.100:0/1628576115 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e341660610 con 0x55e3416fa640 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f52577fe640 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e34170ca90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f524800c6a0 tx=0x7f52480963e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c744d472d29f5d7c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5236ffd640 1 -- 172.21.15.100:0/1628576115 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524809a630 con 0x55e3416fa640 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5236ffd640 1 -- 172.21.15.100:0/1628576115 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f524809a860 con 0x55e3416fa640 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f524c0048e0 con 0x55e3416fa640 2024-03-12T05:31:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5236ffd640 1 -- 172.21.15.100:0/1628576115 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52480a2fc0 con 0x55e3416fa640 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5260c0b740 10 client.28987 fetch_fsmap learned FSMap version 30 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e3416226a0 con 0x55e3416fa640 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.784+0000 7f5236ffd640 1 -- 172.21.15.100:0/1628576115 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5248091080 con 0x55e3416fa640 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5236ffd640 1 -- 172.21.15.100:0/1628576115 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52480ae2c0 con 0x55e3416fa640 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5260c0b740 10 client.28987 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5260c0b740 10 client.28987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5260c0b740 20 client.28987 populate_metadata read hostname 'smithi100' 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5260c0b740 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e34176aea0 0x55e34176d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5260c0b740 4 client.28987 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3413ed890 con 0x55e34176aea0 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5256ffd640 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e34176aea0 0x55e34176d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5236ffd640 10 client.28987 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5256ffd640 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e34176aea0 0x55e34176d290 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd5a4d2291ccf5c server_cookie=c6050bfa5299bfdf in_seq=0 out_seq=0 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5236ffd640 1 -- 172.21.15.100:0/1628576115 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e3413ed890 con 0x55e34176aea0 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.785+0000 7f5236ffd640 10 client.28987 handle_command_reply: tid=0 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 client.28987 shutdown 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 msgr2=0x55e34170ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e34170ca90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f524800c6a0 tx=0x7f52480963e0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 shutdown_connections 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e34176aea0 0x55e34176d290 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3417059c0 0x55e341709350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3416fc830 0x55e341708e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 --2- 172.21.15.100:0/1628576115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3416fa640 0x55e34170ca90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5236ffd640 0 client.28987 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 >> 172.21.15.100:0/1628576115 conn(0x55e3416f4990 msgr2=0x55e341700f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 shutdown_connections 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 1 -- 172.21.15.100:0/1628576115 wait complete. 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f5260c0b740 20 client.28987 trim_cache size 0 max 16384 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "9f345ad6-0589-4b4a-b567-5bbc3adbb77f": { 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:31:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "9f345ad6-0589-4b4a-b567-5bbc3adbb77f", 2024-03-12T05:31:55.787 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:31:55.787 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:31:55.787 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:31:55.787 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52400a1ba0 msgr2=0x7f52400a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f525e042640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52400a1ba0 0x7f52400a4060 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f5250011860 comp rx=0 tx=0).stop 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 msgr2=0x7f5258196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.786+0000 7f525e042640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258196660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f524800e9f0 tx=0x7f524800eeb0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 shutdown_connections 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f52400a1ba0 0x7f52400a4060 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f525819af20 0x7f525819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5258105340 0x7f5258196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 --2- 172.21.15.100:0/442650656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5258100270 0x7f5258196660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:31:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 >> 172.21.15.100:0/442650656 conn(0x7f5258077230 msgr2=0x7f52580fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:31:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 shutdown_connections 2024-03-12T05:31:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:31:55.787+0000 7f525e042640 1 -- 172.21.15.100:0/442650656 wait complete. 2024-03-12T05:31:55.791 INFO:tasks.cephfs.filesystem:scrub status for tag:9f345ad6-0589-4b4a-b567-5bbc3adbb77f - {'path': '/', 'tag': '9f345ad6-0589-4b4a-b567-5bbc3adbb77f', 'options': 'recursive,force'} 2024-03-12T05:31:56.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 213426 26.91 MB/sec execute 279 sec latency 210.027 ms 2024-03-12T05:31:57.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 213962 26.91 MB/sec execute 280 sec latency 162.153 ms 2024-03-12T05:31:57.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:31:57.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:57 smithi100 ceph-mon[32210]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.8 MiB/s wr, 403 op/s 2024-03-12T05:31:57.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:31:57.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:57.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:57 smithi156 ceph-mon[36270]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.8 MiB/s wr, 403 op/s 2024-03-12T05:31:57.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:31:57.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:31:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:57 smithi161 ceph-mon[36043]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.8 MiB/s wr, 403 op/s 2024-03-12T05:31:58.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 214577 26.98 MB/sec execute 281 sec latency 82.445 ms 2024-03-12T05:31:59.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 214827 26.92 MB/sec execute 282 sec latency 345.841 ms 2024-03-12T05:31:59.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:31:59 smithi100 ceph-mon[32210]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-03-12T05:31:59.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:31:59 smithi156 ceph-mon[36270]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-03-12T05:31:59.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:31:59 smithi161 ceph-mon[36043]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-03-12T05:32:00.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 215264 26.90 MB/sec execute 283 sec latency 92.840 ms 2024-03-12T05:32:01.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 215536 26.85 MB/sec execute 284 sec latency 450.173 ms 2024-03-12T05:32:01.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:01 smithi100 ceph-mon[32210]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.7 MiB/s wr, 447 op/s 2024-03-12T05:32:01.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:01 smithi156 ceph-mon[36270]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.7 MiB/s wr, 447 op/s 2024-03-12T05:32:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:01 smithi161 ceph-mon[36043]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.7 MiB/s wr, 447 op/s 2024-03-12T05:32:02.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 215838 26.81 MB/sec execute 285 sec latency 547.570 ms 2024-03-12T05:32:03.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 216126 26.78 MB/sec execute 286 sec latency 214.008 ms 2024-03-12T05:32:03.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:03 smithi100 ceph-mon[32210]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-03-12T05:32:03.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:03 smithi156 ceph-mon[36270]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-03-12T05:32:03.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:03 smithi161 ceph-mon[36043]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-03-12T05:32:04.243 INFO:tasks.workunit.client.0.smithi100.stdout: 10 216398 26.73 MB/sec execute 287 sec latency 324.809 ms 2024-03-12T05:32:05.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 216657 26.67 MB/sec execute 288 sec latency 156.664 ms 2024-03-12T05:32:05.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:05 smithi100 ceph-mon[32210]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 640 op/s 2024-03-12T05:32:05.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:05 smithi156 ceph-mon[36270]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 640 op/s 2024-03-12T05:32:05.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:05 smithi161 ceph-mon[36043]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 640 op/s 2024-03-12T05:32:06.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 217133 26.66 MB/sec execute 289 sec latency 172.855 ms 2024-03-12T05:32:06.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:06 smithi100 ceph-mon[32210]: osdmap e116: 12 total, 12 up, 12 in 2024-03-12T05:32:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:06 smithi156 ceph-mon[36270]: osdmap e116: 12 total, 12 up, 12 in 2024-03-12T05:32:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:06 smithi161 ceph-mon[36043]: osdmap e116: 12 total, 12 up, 12 in 2024-03-12T05:32:07.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 217727 26.68 MB/sec execute 290 sec latency 140.528 ms 2024-03-12T05:32:07.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:07 smithi100 ceph-mon[32210]: pgmap v630: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 591 op/s 2024-03-12T05:32:07.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:07 smithi156 ceph-mon[36270]: pgmap v630: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 591 op/s 2024-03-12T05:32:07.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:07 smithi161 ceph-mon[36043]: pgmap v630: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 591 op/s 2024-03-12T05:32:08.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 218315 26.72 MB/sec execute 291 sec latency 143.006 ms 2024-03-12T05:32:08.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:08 smithi100 ceph-mon[32210]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-12T05:32:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:08 smithi156 ceph-mon[36270]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-12T05:32:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:08 smithi161 ceph-mon[36043]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-12T05:32:09.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 218629 26.68 MB/sec execute 292 sec latency 442.333 ms 2024-03-12T05:32:10.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 219063 26.66 MB/sec execute 293 sec latency 145.313 ms 2024-03-12T05:32:10.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:10 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:32:10.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:10 smithi100 ceph-mon[32210]: pgmap v632: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-12T05:32:10.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:10 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:32:10.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:10 smithi156 ceph-mon[36270]: pgmap v632: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-12T05:32:10.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:10 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:32:10.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:10 smithi161 ceph-mon[36043]: pgmap v632: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-12T05:32:11.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 219591 26.68 MB/sec execute 294 sec latency 179.494 ms 2024-03-12T05:32:11.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:11 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:32:11.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:11 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:32:11.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:11 smithi100 ceph-mon[32210]: osdmap e117: 12 total, 12 up, 12 in 2024-03-12T05:32:11.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:11 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:32:11.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:11 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:32:11.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:11 smithi156 ceph-mon[36270]: osdmap e117: 12 total, 12 up, 12 in 2024-03-12T05:32:11.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:11 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:32:11.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:11 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:32:11.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:11 smithi161 ceph-mon[36043]: osdmap e117: 12 total, 12 up, 12 in 2024-03-12T05:32:12.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 220147 26.70 MB/sec execute 295 sec latency 167.303 ms 2024-03-12T05:32:12.370 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:12 smithi161 ceph-mon[36043]: pgmap v634: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 667 op/s 2024-03-12T05:32:12.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:12 smithi156 ceph-mon[36270]: pgmap v634: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 667 op/s 2024-03-12T05:32:12.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:12 smithi100 ceph-mon[32210]: pgmap v634: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 667 op/s 2024-03-12T05:32:13.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 220619 26.71 MB/sec execute 296 sec latency 126.959 ms 2024-03-12T05:32:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 220867 26.66 MB/sec execute 297 sec latency 565.736 ms 2024-03-12T05:32:14.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:14 smithi156 ceph-mon[36270]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 732 op/s 2024-03-12T05:32:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:14 smithi161 ceph-mon[36043]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 732 op/s 2024-03-12T05:32:14.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:14 smithi100 ceph-mon[32210]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 732 op/s 2024-03-12T05:32:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 221237 26.60 MB/sec execute 298 sec latency 246.805 ms 2024-03-12T05:32:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 221794 26.62 MB/sec execute 299 sec latency 124.614 ms 2024-03-12T05:32:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:16 smithi156 ceph-mon[36270]: pgmap v636: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 657 op/s 2024-03-12T05:32:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:16 smithi161 ceph-mon[36043]: pgmap v636: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 657 op/s 2024-03-12T05:32:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:16 smithi100 ceph-mon[32210]: pgmap v636: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 657 op/s 2024-03-12T05:32:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 222232 26.63 MB/sec execute 300 sec latency 157.884 ms 2024-03-12T05:32:18.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 222774 26.61 MB/sec execute 301 sec latency 812.039 ms 2024-03-12T05:32:18.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:18 smithi156 ceph-mon[36270]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 516 op/s 2024-03-12T05:32:18.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:18 smithi161 ceph-mon[36043]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 516 op/s 2024-03-12T05:32:18.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:18 smithi100 ceph-mon[32210]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 516 op/s 2024-03-12T05:32:19.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 223191 26.61 MB/sec execute 302 sec latency 1751.890 ms 2024-03-12T05:32:20.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 223697 26.62 MB/sec execute 303 sec latency 141.319 ms 2024-03-12T05:32:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:20 smithi156 ceph-mon[36270]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 516 op/s 2024-03-12T05:32:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:20 smithi161 ceph-mon[36043]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 516 op/s 2024-03-12T05:32:20.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:20 smithi100 ceph-mon[32210]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 516 op/s 2024-03-12T05:32:21.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 224141 26.63 MB/sec execute 304 sec latency 175.215 ms 2024-03-12T05:32:22.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 224513 26.59 MB/sec execute 305 sec latency 157.619 ms 2024-03-12T05:32:22.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:22 smithi156 ceph-mon[36270]: pgmap v639: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 617 op/s 2024-03-12T05:32:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:22 smithi161 ceph-mon[36043]: pgmap v639: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 617 op/s 2024-03-12T05:32:22.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:22 smithi100 ceph-mon[32210]: pgmap v639: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 617 op/s 2024-03-12T05:32:23.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 224998 26.58 MB/sec execute 306 sec latency 160.289 ms 2024-03-12T05:32:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 225399 26.55 MB/sec execute 307 sec latency 271.751 ms 2024-03-12T05:32:24.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:24 smithi156 ceph-mon[36270]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-03-12T05:32:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:24 smithi161 ceph-mon[36043]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-03-12T05:32:24.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:24 smithi100 ceph-mon[32210]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-03-12T05:32:25.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 225897 26.56 MB/sec execute 308 sec latency 125.652 ms 2024-03-12T05:32:25.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:32:25.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:32:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 -- 172.21.15.100:0/207837772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 msgr2=0x7f6aa010ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 --2- 172.21.15.100:0/207837772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa010ce50 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009ca0 tx=0x7f6a9802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 -- 172.21.15.100:0/207837772 shutdown_connections 2024-03-12T05:32:25.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 --2- 172.21.15.100:0/207837772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa010ce50 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 --2- 172.21.15.100:0/207837772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa01002d0 0x7f6aa0100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 --2- 172.21.15.100:0/207837772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa01060b0 0x7f6aa0106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.919+0000 7f6aa8356640 1 -- 172.21.15.100:0/207837772 >> 172.21.15.100:0/207837772 conn(0x7f6aa0076e20 msgr2=0x7f6aa0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:25.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.920+0000 7f6aa8356640 1 -- 172.21.15.100:0/207837772 shutdown_connections 2024-03-12T05:32:25.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.920+0000 7f6aa8356640 1 -- 172.21.15.100:0/207837772 wait complete. 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.920+0000 7f6aa8356640 1 Processor -- start 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.920+0000 7f6aa8356640 1 -- start start 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa8356640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa01002d0 0x7f6aa0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa8356640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa0102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa8356640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa01060b0 0x7f6aa0102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa8356640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0103c40 con 0x7f6aa0100c70 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa8356640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01aa160 con 0x7f6aa01002d0 2024-03-12T05:32:25.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa8356640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01aa2a0 con 0x7f6aa01060b0 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa0102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa0102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44300/0 (socket says 172.21.15.100:44300) 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa01060b0 0x7f6aa0102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 -- 172.21.15.100:0/1977301014 learned_addr learned my addr 172.21.15.100:0/1977301014 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa01002d0 0x7f6aa0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 -- 172.21.15.100:0/1977301014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa01060b0 msgr2=0x7f6aa0102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa01060b0 0x7f6aa0102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 -- 172.21.15.100:0/1977301014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa01002d0 msgr2=0x7f6aa0101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa01002d0 0x7f6aa0101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.921+0000 7f6aa58f1640 1 -- 172.21.15.100:0/1977301014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98009900 con 0x7f6aa0100c70 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.922+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa0102230 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000b4a0 tx=0x7f6a9000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.922+0000 7f6a977fe640 1 -- 172.21.15.100:0/1977301014 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90011830 con 0x7f6aa0100c70 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.922+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa01aa580 con 0x7f6aa0100c70 2024-03-12T05:32:25.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.922+0000 7f6a977fe640 1 -- 172.21.15.100:0/1977301014 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a90011e70 con 0x7f6aa0100c70 2024-03-12T05:32:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.922+0000 7f6a977fe640 1 -- 172.21.15.100:0/1977301014 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000f2e0 con 0x7f6aa0100c70 2024-03-12T05:32:25.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.922+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01aaae0 con 0x7f6aa0100c70 2024-03-12T05:32:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.925+0000 7f6aab007740 1 Processor -- start 2024-03-12T05:32:25.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.925+0000 7f6aab007740 1 -- start start 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aab007740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f006137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aab007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f006137bd0 0x55f006140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6a977fe640 1 -- 172.21.15.100:0/1977301014 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f6a90010610 con 0x7f6aa0100c70 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aab007740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f006142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aab007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0060a6280 con 0x55f006137bd0 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aab007740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0060a37b0 con 0x55f0061372b0 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aab007740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00609ef90 con 0x55f0061407b0 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f006137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f006142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f006142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40080/0 (socket says 172.21.15.100:40080) 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 -- 172.21.15.100:0/2181688191 learned_addr learned my addr 172.21.15.100:0/2181688191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f006137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49186/0 (socket says 172.21.15.100:49186) 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f006137bd0 0x55f006140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 -- 172.21.15.100:0/2181688191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 msgr2=0x55f006137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f006137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 -- 172.21.15.100:0/2181688191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f006137bd0 msgr2=0x55f006140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f006137bd0 0x55f006140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.926+0000 7f6aa58f1640 1 -- 172.21.15.100:0/2181688191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f00605d570 con 0x55f0061407b0 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f006137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6a977fe640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a8c0a1ba0 0x7f6a8c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f006142ba0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a900ca9b0 tx=0x7f6a900d5780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79761bec0c6f9b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a8c0a1ba0 0x7f6a8c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6a977fe640 1 -- 172.21.15.100:0/1977301014 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6a900cd220 con 0x7f6aa0100c70 2024-03-12T05:32:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6a94bfe640 1 -- 172.21.15.100:0/2181688191 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900092b0 con 0x55f0061407b0 2024-03-12T05:32:25.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.927+0000 7f6a94bfe640 1 -- 172.21.15.100:0/2181688191 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a900961c0 con 0x55f0061407b0 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6a94bfe640 1 -- 172.21.15.100:0/2181688191 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9009a620 con 0x55f0061407b0 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 -- 172.21.15.100:0/2181688191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 msgr2=0x55f006142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f006142ba0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a900ca9b0 tx=0x7f6a900d5780 comp rx=0 tx=0).stop 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a8c0a1ba0 0x7f6a8c0a4060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a980098d0 tx=0x7f6a98005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 -- 172.21.15.100:0/2181688191 shutdown_connections 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f006142ba0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f006137bd0 0x55f006140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 --2- 172.21.15.100:0/2181688191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f006137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 -- 172.21.15.100:0/2181688191 >> 172.21.15.100:0/2181688191 conn(0x55f006131990 msgr2=0x55f006131d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 -- 172.21.15.100:0/2181688191 shutdown_connections 2024-03-12T05:32:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.928+0000 7f6aab007740 1 -- 172.21.15.100:0/2181688191 wait complete. 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 -- start start 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 Processor -- start 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061372b0 0x55f0061400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f006137bd0 0x55f00613d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f00613d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f005d72430 con 0x55f0061372b0 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f005d2ee00 con 0x55f006137bd0 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f006023fd0 con 0x55f0061407b0 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aab007740 10 client.0 _ll_register_callbacks cb 0x55f006079fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aa60f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061372b0 0x55f0061400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.929+0000 7f6aa60f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061372b0 0x55f0061400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44320/0 (socket says 172.21.15.100:44320) 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa60f2640 1 -- 172.21.15.100:0/430392093 learned_addr learned my addr 172.21.15.100:0/430392093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f006137bd0 0x55f00613d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f00613d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 -- 172.21.15.100:0/430392093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 msgr2=0x55f00613d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f00613d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 -- 172.21.15.100:0/430392093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061372b0 msgr2=0x55f0061400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061372b0 0x55f0061400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 -- 172.21.15.100:0/430392093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01aaae0 con 0x55f006137bd0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f006137bd0 0x55f00613d430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000bda0 tx=0x7f6a900d0880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503e42920e114d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6a89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6a89ffb640 1 -- 172.21.15.100:0/430392093 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900092b0 con 0x55f006137bd0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f005d9e7e0 con 0x55f006137bd0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.930+0000 7f6a89ffb640 1 -- 172.21.15.100:0/430392093 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6a9009a620 con 0x55f006137bd0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6a89ffb640 1 -- 172.21.15.100:0/430392093 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a90099cf0 con 0x55f006137bd0 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 10 client.29063 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f00609d610 con 0x55f006137bd0 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6a89ffb640 1 -- 172.21.15.100:0/430392093 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f005d9e7e0 con 0x55f006137bd0 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6a89ffb640 1 -- 172.21.15.100:0/430392093 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a9009d070 con 0x55f006137bd0 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 10 client.29063 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 10 client.29063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 20 client.29063 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f006150560 0x55f006152950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 4 client.29063 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f005ddb720 con 0x55f006150560 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.931+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f006150560 0x55f006152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.932+0000 7f6a89ffb640 10 client.29063 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.932+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f006150560 0x55f006152950 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76314e9d657d1382 server_cookie=873f6ec08e04db6b in_seq=0 out_seq=0 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.932+0000 7f6a89ffb640 1 -- 172.21.15.100:0/430392093 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f005ddb720 con 0x55f006150560 2024-03-12T05:32:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.932+0000 7f6a89ffb640 10 client.29063 handle_command_reply: tid=0 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 client.29063 shutdown 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f006137bd0 msgr2=0x55f00613d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f006137bd0 0x55f00613d430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000bda0 tx=0x7f6a900d0880 comp rx=0 tx=0).stop 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 shutdown_connections 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f006150560 0x55f006152950 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0061407b0 0x55f00613d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f006137bd0 0x55f00613d430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 --2- 172.21.15.100:0/430392093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061372b0 0x55f0061400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6a89ffb640 0 client.29063 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 >> 172.21.15.100:0/430392093 conn(0x55f006131990 msgr2=0x55f006133730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 shutdown_connections 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 1 -- 172.21.15.100:0/430392093 wait complete. 2024-03-12T05:32:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.933+0000 7f6aab007740 20 client.29063 trim_cache size 0 max 16384 2024-03-12T05:32:25.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.949+0000 7f6aab007740 1 Processor -- start 2024-03-12T05:32:25.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.949+0000 7f6aab007740 1 -- start start 2024-03-12T05:32:25.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aab007740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f00613fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aab007740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f0061400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aab007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aab007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00614d450 con 0x55f0061407b0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aab007740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f006136f10 con 0x55f0061372b0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aab007740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0061329e0 con 0x55f006137bd0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44328/0 (socket says 172.21.15.100:44328) 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 -- 172.21.15.100:0/3415033863 learned_addr learned my addr 172.21.15.100:0/3415033863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa60f2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f00613fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f0061400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 -- 172.21.15.100:0/3415033863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 msgr2=0x55f0061400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f0061400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 -- 172.21.15.100:0/3415033863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 msgr2=0x55f00613fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f00613fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 -- 172.21.15.100:0/3415033863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f006039ac0 con 0x55f0061407b0 2024-03-12T05:32:25.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006147770 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009ca0 tx=0x7f6a98043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37889ebdf6b9d92f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:25.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.950+0000 7f6a8bfff640 1 -- 172.21.15.100:0/3415033863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a980328b0 con 0x55f0061407b0 2024-03-12T05:32:25.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6a8bfff640 1 -- 172.21.15.100:0/3415033863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a98051070 con 0x55f0061407b0 2024-03-12T05:32:25.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6a8bfff640 1 -- 172.21.15.100:0/3415033863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004760 con 0x55f0061407b0 2024-03-12T05:32:25.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 -- 172.21.15.100:0/3415033863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 msgr2=0x55f006147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006147770 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009ca0 tx=0x7f6a98043000 comp rx=0 tx=0).stop 2024-03-12T05:32:25.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 -- 172.21.15.100:0/3415033863 shutdown_connections 2024-03-12T05:32:25.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006147770 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f0061400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 --2- 172.21.15.100:0/3415033863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f00613fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 -- 172.21.15.100:0/3415033863 >> 172.21.15.100:0/3415033863 conn(0x55f006131990 msgr2=0x55f006138710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 -- 172.21.15.100:0/3415033863 shutdown_connections 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.951+0000 7f6aab007740 1 -- 172.21.15.100:0/3415033863 wait complete. 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 -- start start 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 Processor -- start 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f0061479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f006143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006144270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0061447e0 con 0x55f0061407b0 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f006144950 con 0x55f0061372b0 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f006144b10 con 0x55f006137bd0 2024-03-12T05:32:25.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aab007740 10 client.0 _ll_register_callbacks cb 0x7f6a900d0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006144270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44334/0 (socket says 172.21.15.100:44334) 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 -- 172.21.15.100:0/2249286012 learned_addr learned my addr 172.21.15.100:0/2249286012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f0061479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa58f1640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f006143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 -- 172.21.15.100:0/2249286012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 msgr2=0x55f006143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f006143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 -- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 msgr2=0x55f0061479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f0061479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 -- 172.21.15.100:0/2249286012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98046070 con 0x55f0061407b0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6aa68f3640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006144270 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6a9802c630 tx=0x7f6a980348b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628d8741e9214ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.952+0000 7f6a89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:25.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6a89ffb640 1 -- 172.21.15.100:0/2249286012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98032980 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6a89ffb640 1 -- 172.21.15.100:0/2249286012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6a98032ae0 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a900961c0 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6a89ffb640 1 -- 172.21.15.100:0/2249286012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a980043a0 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 10 client.19731 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f00605f6a0 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6a89ffb640 1 -- 172.21.15.100:0/2249286012 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a900961c0 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6a89ffb640 1 -- 172.21.15.100:0/2249286012 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a98049040 con 0x55f0061407b0 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 10 client.19731 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 10 client.19731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 20 client.19731 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0061a7d00 0x55f0061aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 4 client.19731 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f005e2a890 con 0x55f0061a7d00 2024-03-12T05:32:25.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.953+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0061a7d00 0x55f0061aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6a89ffb640 10 client.19731 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aa60f2640 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0061a7d00 0x55f0061aa120 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd5058c376c9a78 server_cookie=977e3a85e271e3ac in_seq=0 out_seq=0 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6a89ffb640 1 -- 172.21.15.100:0/2249286012 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f005e2a890 con 0x55f0061a7d00 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6a89ffb640 10 client.19731 handle_command_reply: tid=0 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 client.19731 shutdown 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 msgr2=0x55f006144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006144270 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6a9802c630 tx=0x7f6a980348b0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 shutdown_connections 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55f0061a7d00 0x55f0061aa120 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0061407b0 0x55f006144270 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f006137bd0 0x55f006143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 --2- 172.21.15.100:0/2249286012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0061372b0 0x55f0061479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6a89ffb640 0 client.19731 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 >> 172.21.15.100:0/2249286012 conn(0x55f006131990 msgr2=0x55f00613bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 shutdown_connections 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 1 -- 172.21.15.100:0/2249286012 wait complete. 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.954+0000 7f6aab007740 20 client.19731 trim_cache size 0 max 16384 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:32:25.955 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:32:25.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a8c0a1ba0 msgr2=0x7f6a8c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a8c0a1ba0 0x7f6a8c0a4060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a980098d0 tx=0x7f6a98005c50 comp rx=0 tx=0).stop 2024-03-12T05:32:25.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 msgr2=0x7f6aa0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:25.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa0102230 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000b4a0 tx=0x7f6a9000b8a0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 shutdown_connections 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f6a8c0a1ba0 0x7f6a8c0a4060 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa01060b0 0x7f6aa0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0100c70 0x7f6aa0102230 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 --2- 172.21.15.100:0/1977301014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa01002d0 0x7f6aa0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 >> 172.21.15.100:0/1977301014 conn(0x7f6aa0076e20 msgr2=0x7f6aa00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 shutdown_connections 2024-03-12T05:32:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:25.955+0000 7f6aa8356640 1 -- 172.21.15.100:0/1977301014 wait complete. 2024-03-12T05:32:25.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:32:25.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 -- 172.21.15.100:0/741442220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8981002f0 msgr2=0x7fc8981006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 --2- 172.21.15.100:0/741442220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8981002f0 0x7fc8981006d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc888009930 tx=0x7fc88802bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 -- 172.21.15.100:0/741442220 shutdown_connections 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 --2- 172.21.15.100:0/741442220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981053c0 0x7fc89810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 --2- 172.21.15.100:0/741442220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc898100ca0 0x7fc898104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 --2- 172.21.15.100:0/741442220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8981002f0 0x7fc8981006d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 -- 172.21.15.100:0/741442220 >> 172.21.15.100:0/741442220 conn(0x7fc898077180 msgr2=0x7fc898077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 -- 172.21.15.100:0/741442220 shutdown_connections 2024-03-12T05:32:26.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 -- 172.21.15.100:0/741442220 wait complete. 2024-03-12T05:32:26.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.075+0000 7fc89e18a640 1 Processor -- start 2024-03-12T05:32:26.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 -- start start 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981002f0 0x7fc898196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8981053c0 0x7fc898196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 0x7fc89819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819b9f0 con 0x7fc89819b000 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819bb60 con 0x7fc8981002f0 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819bcd0 con 0x7fc8981053c0 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 0x7fc89819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 0x7fc89819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44354/0 (socket says 172.21.15.100:44354) 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 -- 172.21.15.100:0/1523063106 learned_addr learned my addr 172.21.15.100:0/1523063106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc8977fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981002f0 0x7fc898196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc896ffd640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8981053c0 0x7fc898196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 -- 172.21.15.100:0/1523063106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8981053c0 msgr2=0x7fc898196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8981053c0 0x7fc898196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 -- 172.21.15.100:0/1523063106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981002f0 msgr2=0x7fc898196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981002f0 0x7fc898196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 -- 172.21.15.100:0/1523063106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888009560 con 0x7fc89819b000 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc8977fe640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981002f0 0x7fc898196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc897fff640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 0x7fc89819b4b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fc89000eca0 tx=0x7fc89000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890013070 con 0x7fc89819b000 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc890005020 con 0x7fc89819b000 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8980682e0 con 0x7fc89819b000 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900099b0 con 0x7fc89819b000 2024-03-12T05:32:26.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.076+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc898068740 con 0x7fc89819b000 2024-03-12T05:32:26.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.077+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc898067f70 con 0x7fc89819b000 2024-03-12T05:32:26.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.078+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc890005520 con 0x7fc89819b000 2024-03-12T05:32:26.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.079+0000 7fc894ff9640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.079+0000 7fc8977fe640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.079+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc8900cf090 con 0x7fc89819b000 2024-03-12T05:32:26.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.080+0000 7fc8977fe640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc888005ab0 tx=0x7fc888005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.084+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc890097fd0 con 0x7fc89819b000 2024-03-12T05:32:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 226414 26.57 MB/sec execute 309 sec latency 168.957 ms 2024-03-12T05:32:26.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.315+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc898069180 con 0x7fc89819b000 2024-03-12T05:32:26.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.316+0000 7fc894ff9640 1 -- 172.21.15.100:0/1523063106 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fc89009b690 con 0x7fc89819b000 2024-03-12T05:32:26.310 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:32:26.310 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:32:26.310 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:32:26.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 msgr2=0x7fc8800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc888005ab0 tx=0x7fc888005fd0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 msgr2=0x7fc89819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 0x7fc89819b4b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fc89000eca0 tx=0x7fc89000b690 comp rx=0 tx=0).stop 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 shutdown_connections 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc8800a1ba0 0x7fc8800a4060 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89819b000 0x7fc89819b4b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8981053c0 0x7fc898196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 --2- 172.21.15.100:0/1523063106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8981002f0 0x7fc898196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 >> 172.21.15.100:0/1523063106 conn(0x7fc898077180 msgr2=0x7fc8980fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 shutdown_connections 2024-03-12T05:32:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.318+0000 7fc89e18a640 1 -- 172.21.15.100:0/1523063106 wait complete. 2024-03-12T05:32:26.323 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:32:26.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:32:26.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:26 smithi100 ceph-mon[32210]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 424 op/s 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 -- 172.21.15.100:0/1464917637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 msgr2=0x7f7034107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 --2- 172.21.15.100:0/1464917637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034107630 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7024009900 tx=0x7f702402bcd0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 -- 172.21.15.100:0/1464917637 shutdown_connections 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 --2- 172.21.15.100:0/1464917637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7034069ec0 0x7f703410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 --2- 172.21.15.100:0/1464917637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 0x7f7034069980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 --2- 172.21.15.100:0/1464917637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034107630 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 -- 172.21.15.100:0/1464917637 >> 172.21.15.100:0/1464917637 conn(0x7f7034077110 msgr2=0x7f7034077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 -- 172.21.15.100:0/1464917637 shutdown_connections 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.440+0000 7f7038dca640 1 -- 172.21.15.100:0/1464917637 wait complete. 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 Processor -- start 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 -- start start 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 0x7f703419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7034069ec0 0x7f703419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034197c60 con 0x7f7034107250 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034197dd0 con 0x7f7034069500 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7038dca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034197f90 con 0x7f7034069ec0 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7031d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7034069ec0 0x7f703419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f703259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 0x7f703419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44374/0 (socket says 172.21.15.100:44374) 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f7032d9d640 1 -- 172.21.15.100:0/727924824 learned_addr learned my addr 172.21.15.100:0/727924824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f703259c640 1 -- 172.21.15.100:0/727924824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7034069ec0 msgr2=0x7f703419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f703259c640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7034069ec0 0x7f703419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f703259c640 1 -- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 msgr2=0x7f7034197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f703259c640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.441+0000 7f703259c640 1 -- 172.21.15.100:0/727924824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024009560 con 0x7f7034069500 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f7032d9d640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f703259c640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 0x7f703419d460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f702402c210 tx=0x7f702402c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f70237fe640 1 -- 172.21.15.100:0/727924824 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702402cac0 con 0x7f7034069500 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7034198210 con 0x7f7034069500 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f70237fe640 1 -- 172.21.15.100:0/727924824 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702402cc20 con 0x7f7034069500 2024-03-12T05:32:26.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f70237fe640 1 -- 172.21.15.100:0/727924824 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024003a10 con 0x7f7034069500 2024-03-12T05:32:26.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.442+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70341a4500 con 0x7f7034069500 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.445+0000 7f70237fe640 1 -- 172.21.15.100:0/727924824 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7024003b70 con 0x7f7034069500 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.445+0000 7f70237fe640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f701c0a1ba0 0x7f701c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.445+0000 7f703b99d740 1 Processor -- start 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.445+0000 7f70237fe640 1 -- 172.21.15.100:0/727924824 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70240ef7e0 con 0x7f7034069500 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.445+0000 7f7031d9b640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f701c0a1ba0 0x7f701c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.445+0000 7f703b99d740 1 -- start start 2024-03-12T05:32:26.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703b99d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703b99d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703b99d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6df0 0x55b73a0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703b99d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a077960 con 0x55b73a0c88e0 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703b99d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a02b310 con 0x55b73a0b6430 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703b99d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a028840 con 0x55b73a0b6df0 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44380/0 (socket says 172.21.15.100:44380) 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 -- 172.21.15.100:0/1203792089 learned_addr learned my addr 172.21.15.100:0/1203792089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f7032d9d640 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6df0 0x55b73a0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f7031d9b640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f701c0a1ba0 0x7f701c0a4060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7034077710 tx=0x7f7028005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 -- 172.21.15.100:0/1203792089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6df0 msgr2=0x55b73a0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f7031d9b640 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6df0 0x55b73a0d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 -- 172.21.15.100:0/1203792089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 msgr2=0x55b73a0b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 -- 172.21.15.100:0/1203792089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b739fe6060 con 0x55b73a0c88e0 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f7031d9b640 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.446+0000 7f703259c640 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0c8cc0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f70240c4750 tx=0x7f70240bc3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481ee464a7cec320 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f7020ff9640 1 -- 172.21.15.100:0/1203792089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70240bc4e0 con 0x55b73a0c88e0 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f7020ff9640 1 -- 172.21.15.100:0/1203792089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70240c0630 con 0x55b73a0c88e0 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f7020ff9640 1 -- 172.21.15.100:0/1203792089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70240b8b50 con 0x55b73a0c88e0 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 -- 172.21.15.100:0/1203792089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 msgr2=0x55b73a0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0c8cc0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f70240c4750 tx=0x7f70240bc3e0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 -- 172.21.15.100:0/1203792089 shutdown_connections 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6df0 0x55b73a0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 --2- 172.21.15.100:0/1203792089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0c8cc0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.447+0000 7f703b99d740 1 -- 172.21.15.100:0/1203792089 >> 172.21.15.100:0/1203792089 conn(0x55b73a0b8ef0 msgr2=0x55b73a0b92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 -- 172.21.15.100:0/1203792089 shutdown_connections 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 -- 172.21.15.100:0/1203792089 wait complete. 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 -- start start 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 Processor -- start 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6df0 0x55b73a0d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a024020 con 0x55b73a0b6df0 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739cf7430 con 0x55b73a0b6430 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739cb3e00 con 0x55b73a0c88e0 2024-03-12T05:32:26.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703b99d740 10 client.0 _ll_register_callbacks cb 0x55b739fcbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49282/0 (socket says 172.21.15.100:49282) 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f703259c640 1 -- 172.21.15.100:0/746211416 learned_addr learned my addr 172.21.15.100:0/746211416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f7031d9b640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6df0 0x55b73a0d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.448+0000 7f7032d9d640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7031d9b640 1 -- 172.21.15.100:0/746211416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 msgr2=0x55b73a0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7031d9b640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7031d9b640 1 -- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 msgr2=0x55b73a0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7031d9b640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7031d9b640 1 -- 172.21.15.100:0/746211416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70341a4500 con 0x55b73a0b6df0 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f703259c640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7031d9b640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6df0 0x55b73a0d3e20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f702801dc10 tx=0x7f7028004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed1e23b4f72c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7019ffb640 1 -- 172.21.15.100:0/746211416 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702801d220 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b739d237e0 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7019ffb640 1 -- 172.21.15.100:0/746211416 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7028004020 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f7019ffb640 1 -- 172.21.15.100:0/746211416 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70280041e0 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f703b99d740 10 client.19749 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.449+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b73a0226a0 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f7019ffb640 1 -- 172.21.15.100:0/746211416 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b739d237e0 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f7019ffb640 1 -- 172.21.15.100:0/746211416 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f702801b6c0 con 0x55b73a0b6df0 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703b99d740 10 client.19749 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703b99d740 10 client.19749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703b99d740 20 client.19749 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703b99d740 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a0de1c0 0x55b73a0e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703b99d740 4 client.19749 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:26.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b739d60720 con 0x55b73a0de1c0 2024-03-12T05:32:26.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f703259c640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a0de1c0 0x55b73a0e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.450+0000 7f7019ffb640 10 client.19749 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:26.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.451+0000 7f703259c640 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a0de1c0 0x55b73a0e05e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46d357c01effecbf server_cookie=4abd2bdb68215336 in_seq=0 out_seq=0 2024-03-12T05:32:26.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.452+0000 7f7019ffb640 1 -- 172.21.15.100:0/746211416 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b739d60720 con 0x55b73a0de1c0 2024-03-12T05:32:26.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.452+0000 7f7019ffb640 10 client.19749 handle_command_reply: tid=0 2024-03-12T05:32:26.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.452+0000 7f703b99d740 1 client.19749 shutdown 2024-03-12T05:32:26.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6df0 msgr2=0x55b73a0d3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6df0 0x55b73a0d3e20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f702801dc10 tx=0x7f7028004730 comp rx=0 tx=0).stop 2024-03-12T05:32:26.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 shutdown_connections 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a0de1c0 0x55b73a0e05e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6df0 0x55b73a0d3e20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f7019ffb640 0 client.19749 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 --2- 172.21.15.100:0/746211416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6430 0x55b73a0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 >> 172.21.15.100:0/746211416 conn(0x55b73a0b8ef0 msgr2=0x55b73a0b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 shutdown_connections 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 1 -- 172.21.15.100:0/746211416 wait complete. 2024-03-12T05:32:26.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.453+0000 7f703b99d740 20 client.19749 trim_cache size 0 max 16384 2024-03-12T05:32:26.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:26 smithi156 ceph-mon[36270]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 424 op/s 2024-03-12T05:32:26.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 Processor -- start 2024-03-12T05:32:26.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 -- start start 2024-03-12T05:32:26.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6430 0x55b73a0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739fa9130 con 0x55b73a0c88e0 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a0e55d0 con 0x55b73a0b6df0 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703b99d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739d237e0 con 0x55b73a0b6430 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f7032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f7032d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44402/0 (socket says 172.21.15.100:44402) 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f7032d9d640 1 -- 172.21.15.100:0/3073348773 learned_addr learned my addr 172.21.15.100:0/3073348773 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f7031d9b640 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.466+0000 7f703259c640 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6430 0x55b73a0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f7032d9d640 1 -- 172.21.15.100:0/3073348773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6430 msgr2=0x55b73a0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f7032d9d640 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6430 0x55b73a0c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f7032d9d640 1 -- 172.21.15.100:0/3073348773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 msgr2=0x55b73a0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f7032d9d640 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f7032d9d640 1 -- 172.21.15.100:0/3073348773 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b739fbec20 con 0x55b73a0c88e0 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f7032d9d640 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdac0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f702c00fa90 tx=0x7f702c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8382a99de0c9d34 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f701bfff640 1 -- 172.21.15.100:0/3073348773 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c017050 con 0x55b73a0c88e0 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f701bfff640 1 -- 172.21.15.100:0/3073348773 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702c018040 con 0x55b73a0c88e0 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.467+0000 7f701bfff640 1 -- 172.21.15.100:0/3073348773 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c010e60 con 0x55b73a0c88e0 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 -- 172.21.15.100:0/3073348773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 msgr2=0x55b73a0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdac0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f702c00fa90 tx=0x7f702c009500 comp rx=0 tx=0).stop 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 -- 172.21.15.100:0/3073348773 shutdown_connections 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0c88e0 0x55b73a0cdac0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 --2- 172.21.15.100:0/3073348773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0b6430 0x55b73a0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 -- 172.21.15.100:0/3073348773 >> 172.21.15.100:0/3073348773 conn(0x55b73a0b8ef0 msgr2=0x55b73a0cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 -- 172.21.15.100:0/3073348773 shutdown_connections 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 -- 172.21.15.100:0/3073348773 wait complete. 2024-03-12T05:32:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 -- start start 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.468+0000 7f703b99d740 1 Processor -- start 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 0x55b73a12e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a12ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a128b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a1290d0 con 0x55b73a0b6430 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a129240 con 0x55b73a0b6df0 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73a129400 con 0x55b73a0c88e0 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 10 client.0 _ll_register_callbacks cb 0x7f70240b8060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 0x55b73a12e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 0x55b73a12e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44412/0 (socket says 172.21.15.100:44412) 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 -- 172.21.15.100:0/3379192371 learned_addr learned my addr 172.21.15.100:0/3379192371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f7031d9b640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a12ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f7032d9d640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a128b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 -- 172.21.15.100:0/3379192371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 msgr2=0x55b73a128b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a128b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 -- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 msgr2=0x55b73a12ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a12ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 -- 172.21.15.100:0/3379192371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c00f740 con 0x55b73a0b6430 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703259c640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 0x55b73a12e700 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f702402bd10 tx=0x7f70240b8780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faf2418081ac3cd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f7019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f7019ffb640 1 -- 172.21.15.100:0/3379192371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70240ba8e0 con 0x55b73a0b6430 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f7019ffb640 1 -- 172.21.15.100:0/3379192371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70240bcbe0 con 0x55b73a0b6430 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70240c0630 con 0x55b73a0b6430 2024-03-12T05:32:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f7019ffb640 1 -- 172.21.15.100:0/3379192371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70240bc4e0 con 0x55b73a0b6430 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 10 client.19755 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.469+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b739fe48c0 con 0x55b73a0b6430 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f7019ffb640 1 -- 172.21.15.100:0/3379192371 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70240a1020 con 0x55b73a0b6430 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f7019ffb640 1 -- 172.21.15.100:0/3379192371 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70240d7030 con 0x55b73a0b6430 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f703b99d740 10 client.19755 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f703b99d740 10 client.19755 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f703b99d740 20 client.19755 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f703b99d740 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a136210 0x55b73a138600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f703b99d740 4 client.19755 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b739daf890 con 0x55b73a136210 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f7031d9b640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a136210 0x55b73a138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f7019ffb640 10 client.19755 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.470+0000 7f7031d9b640 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a136210 0x55b73a138600 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c81ab7c983ce807 server_cookie=17a2a1090d6a817f in_seq=0 out_seq=0 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f7019ffb640 1 -- 172.21.15.100:0/3379192371 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b739daf890 con 0x55b73a136210 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f7019ffb640 10 client.19755 handle_command_reply: tid=0 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 client.19755 shutdown 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 msgr2=0x55b73a12e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 0x55b73a12e700 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f702402bd10 tx=0x7f70240b8780 comp rx=0 tx=0).stop 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 shutdown_connections 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55b73a136210 0x55b73a138600 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73a0c88e0 0x55b73a128b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b73a0b6df0 0x55b73a12ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 --2- 172.21.15.100:0/3379192371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b73a0b6430 0x55b73a12e700 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f7019ffb640 0 client.19755 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.471+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 >> 172.21.15.100:0/3379192371 conn(0x55b73a0b8ef0 msgr2=0x55b73a0cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 shutdown_connections 2024-03-12T05:32:26.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f703b99d740 1 -- 172.21.15.100:0/3379192371 wait complete. 2024-03-12T05:32:26.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f703b99d740 20 client.19755 trim_cache size 0 max 16384 2024-03-12T05:32:26.473 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f701c0a1ba0 msgr2=0x7f701c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f701c0a1ba0 0x7f701c0a4060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7034077710 tx=0x7f7028005f90 comp rx=0 tx=0).stop 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 msgr2=0x7f703419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 0x7f703419d460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f702402c210 tx=0x7f702402c690 comp rx=0 tx=0).stop 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 shutdown_connections 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f701c0a1ba0 0x7f701c0a4060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7034107250 0x7f7034197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7034069ec0 0x7f703419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 --2- 172.21.15.100:0/727924824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7034069500 0x7f703419d460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 >> 172.21.15.100:0/727924824 conn(0x7f7034077110 msgr2=0x7f70340fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 shutdown_connections 2024-03-12T05:32:26.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.472+0000 7f7038dca640 1 -- 172.21.15.100:0/727924824 wait complete. 2024-03-12T05:32:26.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:26 smithi161 ceph-mon[36043]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 424 op/s 2024-03-12T05:32:26.477 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:32:26.477 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1977455156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8100260 msgr2=0x7fcdb8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1977455156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8100260 0x7fcdb8100640 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009930 tx=0x7fcda802bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1977455156 shutdown_connections 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1977455156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1977455156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100c10 0x7fcdb8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1977455156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8100260 0x7fcdb8100640 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1977455156 >> 172.21.15.100:0/1977455156 conn(0x7fcdb8077180 msgr2=0x7fcdb8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1977455156 shutdown_connections 2024-03-12T05:32:26.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.637+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1977455156 wait complete. 2024-03-12T05:32:26.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 Processor -- start 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 -- start start 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100260 0x7fcdb819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8197640 0x7fcdb8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb8198110 con 0x7fcdb8197640 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb8198280 con 0x7fcdb8100260 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdbcb99640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb8198440 con 0x7fcdb8105330 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100260 0x7fcdb819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8197640 0x7fcdb8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40184/0 (socket says 172.21.15.100:40184) 2024-03-12T05:32:26.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.638+0000 7fcdaffff640 1 -- 172.21.15.100:0/1593385119 learned_addr learned my addr 172.21.15.100:0/1593385119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb6d9d640 1 -- 172.21.15.100:0/1593385119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 msgr2=0x7fcdb819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb6d9d640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb6d9d640 1 -- 172.21.15.100:0/1593385119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100260 msgr2=0x7fcdb819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb6d9d640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100260 0x7fcdb819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb6d9d640 1 -- 172.21.15.100:0/1593385119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8009560 con 0x7fcdb8197640 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdaffff640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb659c640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100260 0x7fcdb819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdb6d9d640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8197640 0x7fcdb8197ac0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009fd0 tx=0x7fcdb000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdaf7fe640 1 -- 172.21.15.100:0/1593385119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000ce20 con 0x7fcdb8197640 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb81a43e0 con 0x7fcdb8197640 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdaf7fe640 1 -- 172.21.15.100:0/1593385119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0005020 con 0x7fcdb8197640 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdaf7fe640 1 -- 172.21.15.100:0/1593385119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0010690 con 0x7fcdb8197640 2024-03-12T05:32:26.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.639+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb81a4870 con 0x7fcdb8197640 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.641+0000 7fcdaf7fe640 1 -- 172.21.15.100:0/1593385119 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fcdb0005500 con 0x7fcdb8197640 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 Processor -- start 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 -- start start 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab474960 con 0x55d5ab4b9b70 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab428310 con 0x55d5ab4b9250 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdbf768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab425840 con 0x55d5ab4c2750 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdaf7fe640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcda40a1ba0 0x7fcda40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdb6d9d640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcda40a1ba0 0x7fcda40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44426/0 (socket says 172.21.15.100:44426) 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdb659c640 1 -- 172.21.15.100:0/2223766036 learned_addr learned my addr 172.21.15.100:0/2223766036 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.642+0000 7fcdaf7fe640 1 -- 172.21.15.100:0/1593385119 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcdb0014070 con 0x7fcdb8197640 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdaffff640 1 -- 172.21.15.100:0/2223766036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4c2750 msgr2=0x55d5ab4c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdaffff640 1 -- 172.21.15.100:0/2223766036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 msgr2=0x55d5ab4c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdaffff640 1 -- 172.21.15.100:0/2223766036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5ab3e3060 con 0x55d5ab4b9250 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdb659c640 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdb6d9d640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcda40a1ba0 0x7fcda40a4060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c760 tx=0x7fcdb0069ee0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4b9630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8198e90 tx=0x7fcda0009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149cf377c5f51f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdacff9640 1 -- 172.21.15.100:0/2223766036 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000fc20 con 0x55d5ab4b9250 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdacff9640 1 -- 172.21.15.100:0/2223766036 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda0004510 con 0x55d5ab4b9250 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.643+0000 7fcdacff9640 1 -- 172.21.15.100:0/2223766036 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00049e0 con 0x55d5ab4b9250 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 -- 172.21.15.100:0/2223766036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 msgr2=0x55d5ab4b9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4b9630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8198e90 tx=0x7fcda0009530 comp rx=0 tx=0).stop 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 -- 172.21.15.100:0/2223766036 shutdown_connections 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2223766036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4b9630 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 -- 172.21.15.100:0/2223766036 >> 172.21.15.100:0/2223766036 conn(0x55d5ab4b3980 msgr2=0x55d5ab4b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 -- 172.21.15.100:0/2223766036 shutdown_connections 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 -- 172.21.15.100:0/2223766036 wait complete. 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 -- start start 2024-03-12T05:32:26.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 Processor -- start 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.644+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab421020 con 0x55d5ab4b9b70 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab0f4430 con 0x55d5ab4c2750 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab0b0e00 con 0x55d5ab4b9250 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 10 client.0 _ll_register_callbacks cb 0x55d5ab3c8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44442/0 (socket says 172.21.15.100:44442) 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 -- 172.21.15.100:0/3831265507 learned_addr learned my addr 172.21.15.100:0/3831265507 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdaffff640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdaffff640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 -- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 msgr2=0x55d5ab4bf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4bf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 -- 172.21.15.100:0/3831265507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4c2750 msgr2=0x55d5ab4c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 -- 172.21.15.100:0/3831265507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda0019040 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdb659c640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bfc80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009230 tx=0x7fcda802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f8ba2c832e61ba server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcd9dffb640 1 -- 172.21.15.100:0/3831265507 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004310 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcd9dffb640 1 -- 172.21.15.100:0/3831265507 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcda8004470 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5ab1207e0 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcd9dffb640 1 -- 172.21.15.100:0/3831265507 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcda8004df0 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 10 client.19767 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.645+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d5ab41f6a0 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcd9dffb640 1 -- 172.21.15.100:0/3831265507 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5ab1207e0 con 0x55d5ab4b9b70 2024-03-12T05:32:26.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcd9dffb640 1 -- 172.21.15.100:0/3831265507 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcda803c040 con 0x55d5ab4b9b70 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdbf768740 10 client.19767 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdbf768740 10 client.19767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdbf768740 20 client.19767 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdbf768740 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab4d2b80 0x55d5ab4d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdbf768740 4 client.19767 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5ab15d720 con 0x55d5ab4d2b80 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdaffff640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab4d2b80 0x55d5ab4d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcd9dffb640 10 client.19767 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.646+0000 7fcdaffff640 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab4d2b80 0x55d5ab4d4fa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a28f530854f0e01 server_cookie=2068e3d70ae985b1 in_seq=0 out_seq=0 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcd9dffb640 1 -- 172.21.15.100:0/3831265507 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5ab15d720 con 0x55d5ab4d2b80 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcd9dffb640 10 client.19767 handle_command_reply: tid=0 2024-03-12T05:32:26.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 client.19767 shutdown 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 msgr2=0x55d5ab4bfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bfc80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009230 tx=0x7fcda802c660 comp rx=0 tx=0).stop 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 shutdown_connections 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab4d2b80 0x55d5ab4d4fa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bfc80 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 --2- 172.21.15.100:0/3831265507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4bf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 >> 172.21.15.100:0/3831265507 conn(0x55d5ab4b3980 msgr2=0x55d5ab4b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcd9dffb640 0 client.19767 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 shutdown_connections 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 1 -- 172.21.15.100:0/3831265507 wait complete. 2024-03-12T05:32:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.647+0000 7fcdbf768740 20 client.19767 trim_cache size 0 max 16384 2024-03-12T05:32:26.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 Processor -- start 2024-03-12T05:32:26.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 -- start start 2024-03-12T05:32:26.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab3a6130 con 0x55d5ab4c2750 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab4b3820 con 0x55d5ab4b9250 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdbf768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab4b8eb0 con 0x55d5ab4b9b70 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4baa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44454/0 (socket says 172.21.15.100:44454) 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 -- 172.21.15.100:0/2283422157 learned_addr learned my addr 172.21.15.100:0/2283422157 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdb659c640 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 -- 172.21.15.100:0/2283422157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9b70 msgr2=0x55d5ab4bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 -- 172.21.15.100:0/2283422157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 msgr2=0x55d5ab4c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 -- 172.21.15.100:0/2283422157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5ab3bbc20 con 0x55d5ab4c2750 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.661+0000 7fcdaffff640 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4baa30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097f0 tx=0x7fcda0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230462f95deaae31 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcd9ffff640 1 -- 172.21.15.100:0/2283422157 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000fdb0 con 0x55d5ab4c2750 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcd9ffff640 1 -- 172.21.15.100:0/2283422157 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda00328d0 con 0x55d5ab4c2750 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcd9ffff640 1 -- 172.21.15.100:0/2283422157 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0004b40 con 0x55d5ab4c2750 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcdbf768740 1 -- 172.21.15.100:0/2283422157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 msgr2=0x55d5ab4baa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4baa30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097f0 tx=0x7fcda0009530 comp rx=0 tx=0).stop 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcdbf768740 1 -- 172.21.15.100:0/2283422157 shutdown_connections 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.662+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4baa30 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4bffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 --2- 172.21.15.100:0/2283422157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- 172.21.15.100:0/2283422157 >> 172.21.15.100:0/2283422157 conn(0x55d5ab4b3980 msgr2=0x55d5ab4d98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- 172.21.15.100:0/2283422157 shutdown_connections 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- 172.21.15.100:0/2283422157 wait complete. 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- start start 2024-03-12T05:32:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 Processor -- start 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab4c8d10 con 0x55d5ab4c2750 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab529050 con 0x55d5ab4b9b70 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ab529190 con 0x55d5ab4b9250 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdbf768740 10 client.0 _ll_register_callbacks cb 0x7fcda0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40210/0 (socket says 172.21.15.100:40210) 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdaffff640 1 -- 172.21.15.100:0/672555188 learned_addr learned my addr 172.21.15.100:0/672555188 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdb659c640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49342/0 (socket says 172.21.15.100:49342) 2024-03-12T05:32:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.663+0000 7fcdaffff640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdaffff640 1 -- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 msgr2=0x55d5ab4c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdaffff640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdaffff640 1 -- 172.21.15.100:0/672555188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 msgr2=0x55d5ab4c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdaffff640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdaffff640 1 -- 172.21.15.100:0/672555188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5ab3e3060 con 0x55d5ab4c2750 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdb659c640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdaffff640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c8610 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009600 tx=0x7fcda0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f69472e33dc2349 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcd9dffb640 1 -- 172.21.15.100:0/672555188 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0011020 con 0x55d5ab4c2750 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcd9dffb640 1 -- 172.21.15.100:0/672555188 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcda0011180 con 0x55d5ab4c2750 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcda0004510 con 0x55d5ab4c2750 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcd9dffb640 1 -- 172.21.15.100:0/672555188 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcda0011340 con 0x55d5ab4c2750 2024-03-12T05:32:26.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdbf768740 10 client.19773 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d5ab3e18c0 con 0x55d5ab4c2750 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.664+0000 7fcd9dffb640 1 -- 172.21.15.100:0/672555188 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda0004510 con 0x55d5ab4c2750 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcd9dffb640 1 -- 172.21.15.100:0/672555188 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcda001d070 con 0x55d5ab4c2750 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdbf768740 10 client.19773 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdbf768740 10 client.19773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdbf768740 20 client.19773 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdbf768740 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab52a490 0x55d5ab52c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdbf768740 4 client.19773 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5ab1ac890 con 0x55d5ab52a490 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdb659c640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab52a490 0x55d5ab52c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcd9dffb640 10 client.19773 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.665+0000 7fcdb659c640 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab52a490 0x55d5ab52c880 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1b6595a8e71d6ba server_cookie=29e36eb27d64055d in_seq=0 out_seq=0 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcd9dffb640 1 -- 172.21.15.100:0/672555188 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5ab1ac890 con 0x55d5ab52a490 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcd9dffb640 10 client.19773 handle_command_reply: tid=0 2024-03-12T05:32:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 client.19773 shutdown 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 msgr2=0x55d5ab4c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c8610 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009600 tx=0x7fcda0000bc0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 shutdown_connections 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5ab4c2750 0x55d5ab4c8610 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55d5ab52a490 0x55d5ab52c880 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5ab4b9b70 0x55d5ab4c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 --2- 172.21.15.100:0/672555188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5ab4b9250 0x55d5ab4c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcd9dffb640 0 client.19773 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 >> 172.21.15.100:0/672555188 conn(0x55d5ab4b3980 msgr2=0x55d5ab4baa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 shutdown_connections 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 1 -- 172.21.15.100:0/672555188 wait complete. 2024-03-12T05:32:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbf768740 20 client.19773 trim_cache size 0 max 16384 2024-03-12T05:32:26.667 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcda40a1ba0 msgr2=0x7fcda40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcda40a1ba0 0x7fcda40a4060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c760 tx=0x7fcdb0069ee0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8197640 msgr2=0x7fcdb8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8197640 0x7fcdb8197ac0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009fd0 tx=0x7fcdb000ef10 comp rx=0 tx=0).stop 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 shutdown_connections 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcda40a1ba0 0x7fcda40a4060 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb8197640 0x7fcdb8197ac0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb8105330 0x7fcdb819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 --2- 172.21.15.100:0/1593385119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdb8100260 0x7fcdb819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.666+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 >> 172.21.15.100:0/1593385119 conn(0x7fcdb8077180 msgr2=0x7fcdb8068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.667+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 shutdown_connections 2024-03-12T05:32:26.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.667+0000 7fcdbcb99640 1 -- 172.21.15.100:0/1593385119 wait complete. 2024-03-12T05:32:26.675 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:32:26.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:32:26.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3564530070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b00ff8f0 msgr2=0x7ff5b00ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3564530070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b00ffcd0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009930 tx=0x7ff5a002bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:26.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3564530070 shutdown_connections 2024-03-12T05:32:26.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3564530070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b0106500 0x7ff5b010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3564530070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b01002a0 0x7ff5b0105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3564530070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b00ffcd0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3564530070 >> 172.21.15.100:0/3564530070 conn(0x7ff5b0076640 msgr2=0x7ff5b00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3564530070 shutdown_connections 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3564530070 wait complete. 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.804+0000 7ff5b55c7640 1 Processor -- start 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 -- start start 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b0106500 0x7ff5b019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 0x7ff5b0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0198150 con 0x7ff5b0197680 2024-03-12T05:32:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01982c0 con 0x7ff5b0106500 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5b55c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0198480 con 0x7ff5b00ff8f0 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 0x7ff5b0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b0106500 0x7ff5b019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40232/0 (socket says 172.21.15.100:40232) 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 0x7ff5b0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44482/0 (socket says 172.21.15.100:44482) 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5aeffd640 1 -- 172.21.15.100:0/3087565731 learned_addr learned my addr 172.21.15.100:0/3087565731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 -- 172.21.15.100:0/3087565731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 msgr2=0x7ff5b019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 -- 172.21.15.100:0/3087565731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b0106500 msgr2=0x7ff5b019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b0106500 0x7ff5b019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 -- 172.21.15.100:0/3087565731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009560 con 0x7ff5b0197680 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.805+0000 7ff5af7fe640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 0x7ff5b0197b00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009fd0 tx=0x7ff5a800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.806+0000 7ff597fff640 1 -- 172.21.15.100:0/3087565731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a800ce20 con 0x7ff5b0197680 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.806+0000 7ff597fff640 1 -- 172.21.15.100:0/3087565731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5a8005020 con 0x7ff5b0197680 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.806+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b01a4420 con 0x7ff5b0197680 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.806+0000 7ff597fff640 1 -- 172.21.15.100:0/3087565731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8010690 con 0x7ff5b0197680 2024-03-12T05:32:26.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.806+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01a4900 con 0x7ff5b0197680 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff597fff640 1 -- 172.21.15.100:0/3087565731 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff5a8005500 con 0x7ff5b0197680 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 Processor -- start 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 -- start start 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d73681f80 0x556d73682360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 0x556d73691f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7363d960 con 0x556d7368fb60 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d735f1310 con 0x556d73681f80 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5b8198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d735ee840 con 0x556d736828a0 2024-03-12T05:32:26.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 0x556d73691f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 0x556d73691f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44490/0 (socket says 172.21.15.100:44490) 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/325376691 learned_addr learned my addr 172.21.15.100:0/325376691 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d73681f80 0x556d73682360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff597fff640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5980a1ba0 0x7ff5980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/325376691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 msgr2=0x556d7368f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/325376691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d73681f80 msgr2=0x556d73682360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d73681f80 0x556d73682360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.809+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/325376691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d735ac060 con 0x556d7368fb60 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff5af7fe640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5980a1ba0 0x7ff5980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff597fff640 1 -- 172.21.15.100:0/3087565731 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5a8014070 con 0x7ff5b0197680 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 0x556d73691f50 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4011790 tx=0x7ff5a4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c66a125da5f01d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff5957fa640 1 -- 172.21.15.100:0/325376691 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400f7c0 con 0x556d7368fb60 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff5957fa640 1 -- 172.21.15.100:0/325376691 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5a400fe00 con 0x556d7368fb60 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff5957fa640 1 -- 172.21.15.100:0/325376691 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a401fe30 con 0x556d7368fb60 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.810+0000 7ff5af7fe640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5980a1ba0 0x7ff5980a4060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80cb650 tx=0x7ff5a8069e60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 -- 172.21.15.100:0/325376691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 msgr2=0x556d73691f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 0x556d73691f50 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4011790 tx=0x7ff5a4005850 comp rx=0 tx=0).stop 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 -- 172.21.15.100:0/325376691 shutdown_connections 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d7368fb60 0x556d73691f50 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 --2- 172.21.15.100:0/325376691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d73681f80 0x556d73682360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 -- 172.21.15.100:0/325376691 >> 172.21.15.100:0/325376691 conn(0x556d7367c980 msgr2=0x556d7367cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 -- 172.21.15.100:0/325376691 shutdown_connections 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.811+0000 7ff5b8198740 1 -- 172.21.15.100:0/325376691 wait complete. 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 -- start start 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 Processor -- start 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d7368d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d7368d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d735ea020 con 0x556d73681f80 2024-03-12T05:32:26.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d732bd430 con 0x556d736828a0 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73279e00 con 0x556d7368fb60 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5b8198740 10 client.0 _ll_register_callbacks cb 0x556d73591cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d7368d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44502/0 (socket says 172.21.15.100:44502) 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d7368d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49384/0 (socket says 172.21.15.100:49384) 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/3612504399 learned_addr learned my addr 172.21.15.100:0/3612504399 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.812+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d7368d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/3612504399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 msgr2=0x556d7368d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d7368d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 msgr2=0x556d7368d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d7368d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/3612504399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a4019040 con 0x556d73681f80 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368cbe0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006010 tx=0x7ff5a4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f3f10b98ddec89 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff592ff5640 1 -- 172.21.15.100:0/3612504399 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a401f7d0 con 0x556d73681f80 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff592ff5640 1 -- 172.21.15.100:0/3612504399 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5a4004040 con 0x556d73681f80 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d732e97e0 con 0x556d73681f80 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff592ff5640 1 -- 172.21.15.100:0/3612504399 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5a4004200 con 0x556d73681f80 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5b8198740 10 client.19785 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556d735e86a0 con 0x556d73681f80 2024-03-12T05:32:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff592ff5640 1 -- 172.21.15.100:0/3612504399 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d732e97e0 con 0x556d73681f80 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.813+0000 7ff592ff5640 1 -- 172.21.15.100:0/3612504399 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5a4005270 con 0x556d73681f80 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5b8198740 10 client.19785 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5b8198740 10 client.19785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5b8198740 20 client.19785 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5b8198740 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d7369fe70 0x556d736a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5b8198740 4 client.19785 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d73326720 con 0x556d7369fe70 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d7369fe70 0x556d736a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff592ff5640 10 client.19785 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.814+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d7369fe70 0x556d736a2260 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d813320048276c1 server_cookie=54168f7cd9f3cdfc in_seq=0 out_seq=0 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff592ff5640 1 -- 172.21.15.100:0/3612504399 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d73326720 con 0x556d7369fe70 2024-03-12T05:32:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff592ff5640 10 client.19785 handle_command_reply: tid=0 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 client.19785 shutdown 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 msgr2=0x556d7368cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368cbe0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006010 tx=0x7ff5a4001e50 comp rx=0 tx=0).stop 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 shutdown_connections 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d7369fe70 0x556d736a2260 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d7368d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d7368d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 --2- 172.21.15.100:0/3612504399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368cbe0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 >> 172.21.15.100:0/3612504399 conn(0x556d7367c980 msgr2=0x556d7367e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff592ff5640 0 client.19785 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:26.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.815+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 shutdown_connections 2024-03-12T05:32:26.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.816+0000 7ff5b8198740 1 -- 172.21.15.100:0/3612504399 wait complete. 2024-03-12T05:32:26.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.816+0000 7ff5b8198740 20 client.19785 trim_cache size 0 max 16384 2024-03-12T05:32:26.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 Processor -- start 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 -- start start 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d7368fb60 0x556d7368b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7356f130 con 0x556d73681f80 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7368ca00 con 0x556d7368fb60 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5b8198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d736815c0 con 0x556d736828a0 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44516/0 (socket says 172.21.15.100:44516) 2024-03-12T05:32:26.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/2775681237 learned_addr learned my addr 172.21.15.100:0/2775681237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/2775681237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 msgr2=0x556d7368b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/2775681237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d7368fb60 msgr2=0x556d7368b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d7368fb60 0x556d7368b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.829+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/2775681237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d73584c20 con 0x556d73681f80 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.830+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368f310 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48cad2ecd4c350a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.830+0000 7ff594ff9640 1 -- 172.21.15.100:0/2775681237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40047b0 con 0x556d73681f80 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.830+0000 7ff594ff9640 1 -- 172.21.15.100:0/2775681237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5a400fc70 con 0x556d73681f80 2024-03-12T05:32:26.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.830+0000 7ff594ff9640 1 -- 172.21.15.100:0/2775681237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004040 con 0x556d73681f80 2024-03-12T05:32:26.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- 172.21.15.100:0/2775681237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 msgr2=0x556d7368f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368f310 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4005d50 comp rx=0 tx=0).stop 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- 172.21.15.100:0/2775681237 shutdown_connections 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d7368fb60 0x556d7368b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d736828a0 0x556d7368b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- 172.21.15.100:0/2775681237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d7368f310 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- 172.21.15.100:0/2775681237 >> 172.21.15.100:0/2775681237 conn(0x556d7367c980 msgr2=0x556d73687ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- 172.21.15.100:0/2775681237 shutdown_connections 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- 172.21.15.100:0/2775681237 wait complete. 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- start start 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 Processor -- start 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d73696b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d73695000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d73695540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73695a80 con 0x556d73681f80 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73695bf0 con 0x556d736828a0 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73695db0 con 0x556d7368fb60 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5b8198740 10 client.0 _ll_register_callbacks cb 0x7ff5a4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d73696b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d73696b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44520/0 (socket says 172.21.15.100:44520) 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.831+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/1766833298 learned_addr learned my addr 172.21.15.100:0/1766833298 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d73695000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d73695540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/1766833298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 msgr2=0x556d73695540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d73695540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 msgr2=0x556d73695000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d73695000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 -- 172.21.15.100:0/1766833298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d735e86a0 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5ae7fc640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d73696b50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40059f0 tx=0x7ff5a401f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a377a5277ccc33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff592ff5640 1 -- 172.21.15.100:0/1766833298 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40047b0 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff592ff5640 1 -- 172.21.15.100:0/1766833298 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5a4000e40 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5a400fe00 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff592ff5640 1 -- 172.21.15.100:0/1766833298 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5a4011030 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5b8198740 10 client.19791 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556d735aa8c0 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff592ff5640 1 -- 172.21.15.100:0/1766833298 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5a400fe00 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.832+0000 7ff592ff5640 1 -- 172.21.15.100:0/1766833298 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5a4003890 con 0x556d73681f80 2024-03-12T05:32:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5b8198740 10 client.19791 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5b8198740 10 client.19791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5b8198740 20 client.19791 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5b8198740 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d736f72b0 0x556d736f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5b8198740 4 client.19791 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d73375890 con 0x556d736f72b0 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d736f72b0 0x556d736f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff592ff5640 10 client.19791 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.833+0000 7ff5aeffd640 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d736f72b0 0x556d736f96a0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd394f99864f5e28 server_cookie=d6c2166f6598f782 in_seq=0 out_seq=0 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff592ff5640 1 -- 172.21.15.100:0/1766833298 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d73375890 con 0x556d736f72b0 2024-03-12T05:32:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff592ff5640 10 client.19791 handle_command_reply: tid=0 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 client.19791 shutdown 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 msgr2=0x556d73696b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d73696b50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40059f0 tx=0x7ff5a401f950 comp rx=0 tx=0).stop 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 shutdown_connections 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556d736f72b0 0x556d736f96a0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7368fb60 0x556d73695540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d736828a0 0x556d73695000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff592ff5640 0 client.19791 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 --2- 172.21.15.100:0/1766833298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d73681f80 0x556d73696b50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 >> 172.21.15.100:0/1766833298 conn(0x556d7367c980 msgr2=0x556d7367b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 shutdown_connections 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 1 -- 172.21.15.100:0/1766833298 wait complete. 2024-03-12T05:32:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.834+0000 7ff5b8198740 20 client.19791 trim_cache size 0 max 16384 2024-03-12T05:32:26.836 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:32:26.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5980a1ba0 msgr2=0x7ff5980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5980a1ba0 0x7ff5980a4060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80cb650 tx=0x7ff5a8069e60 comp rx=0 tx=0).stop 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 msgr2=0x7ff5b0197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 0x7ff5b0197b00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009fd0 tx=0x7ff5a800ef10 comp rx=0 tx=0).stop 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 shutdown_connections 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff5980a1ba0 0x7ff5980a4060 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b0197680 0x7ff5b0197b00 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b0106500 0x7ff5b019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 --2- 172.21.15.100:0/3087565731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b00ff8f0 0x7ff5b019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 >> 172.21.15.100:0/3087565731 conn(0x7ff5b0076640 msgr2=0x7ff5b01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 shutdown_connections 2024-03-12T05:32:26.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:26.835+0000 7ff5b55c7640 1 -- 172.21.15.100:0/3087565731 wait complete. 2024-03-12T05:32:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 226915 26.59 MB/sec execute 310 sec latency 192.766 ms 2024-03-12T05:32:27.352 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:27 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1523063106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:32:27.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:27 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1523063106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:32:27.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:27 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1523063106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:32:27.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:32:27.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:32:27.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 -- 172.21.15.100:0/2514557548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c0685f0 msgr2=0x7fc58c0689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 --2- 172.21.15.100:0/2514557548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c0685f0 0x7fc58c0689d0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc588009930 tx=0x7fc58802bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 -- 172.21.15.100:0/2514557548 shutdown_connections 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 --2- 172.21.15.100:0/2514557548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c1064c0 0x7fc58c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 --2- 172.21.15.100:0/2514557548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c068fa0 0x7fc58c105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 --2- 172.21.15.100:0/2514557548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c0685f0 0x7fc58c0689d0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 -- 172.21.15.100:0/2514557548 >> 172.21.15.100:0/2514557548 conn(0x7fc58c077180 msgr2=0x7fc58c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 -- 172.21.15.100:0/2514557548 shutdown_connections 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.973+0000 7fc593423640 1 -- 172.21.15.100:0/2514557548 wait complete. 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 Processor -- start 2024-03-12T05:32:27.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 -- start start 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c0685f0 0x7fc58c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c1064c0 0x7fc58c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 0x7fc58c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c197ff0 con 0x7fc58c197600 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c198160 con 0x7fc58c1064c0 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc593423640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c198320 con 0x7fc58c0685f0 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc5911bf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c0685f0 0x7fc58c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc5919c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 0x7fc58c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc5909be640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c1064c0 0x7fc58c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc5919c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 0x7fc58c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44534/0 (socket says 172.21.15.100:44534) 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc5911bf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c0685f0 0x7fc58c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40292/0 (socket says 172.21.15.100:40292) 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.974+0000 7fc5919c0640 1 -- 172.21.15.100:0/3551434224 learned_addr learned my addr 172.21.15.100:0/3551434224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:27.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5919c0640 1 -- 172.21.15.100:0/3551434224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c0685f0 msgr2=0x7fc58c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5919c0640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c0685f0 0x7fc58c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5919c0640 1 -- 172.21.15.100:0/3551434224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c1064c0 msgr2=0x7fc58c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5919c0640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c1064c0 0x7fc58c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5919c0640 1 -- 172.21.15.100:0/3551434224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc588009560 con 0x7fc58c197600 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5919c0640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 0x7fc58c197ab0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc58400eb10 tx=0x7fc58400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5827fc640 1 -- 172.21.15.100:0/3551434224 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584004b20 con 0x7fc58c197600 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5827fc640 1 -- 172.21.15.100:0/3551434224 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc584004c80 con 0x7fc58c197600 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c1a4310 con 0x7fc58c197600 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc5827fc640 1 -- 172.21.15.100:0/3551434224 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584010650 con 0x7fc58c197600 2024-03-12T05:32:27.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.975+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c1a4820 con 0x7fc58c197600 2024-03-12T05:32:27.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.978+0000 7fc5827fc640 1 -- 172.21.15.100:0/3551434224 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc584005020 con 0x7fc58c197600 2024-03-12T05:32:27.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.978+0000 7fc595ffc740 1 Processor -- start 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.978+0000 7fc595ffc740 1 -- start start 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b934760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b93d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b93fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc595ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b8a2f90 con 0x558d3b934d30 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc595ffc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b8a04c0 con 0x558d3b934380 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc595ffc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b89bca0 con 0x558d3b93d8b0 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b93d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5909be640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b934760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b93d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44536/0 (socket says 172.21.15.100:44536) 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 -- 172.21.15.100:0/4088446136 learned_addr learned my addr 172.21.15.100:0/4088446136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:27.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5909be640 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b93fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5827fc640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5780a1ba0 0x7fc5780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 -- 172.21.15.100:0/4088446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 msgr2=0x558d3b93fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b93fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 -- 172.21.15.100:0/4088446136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 msgr2=0x558d3b934760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b934760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5911bf640 1 -- 172.21.15.100:0/4088446136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3b85e0b0 con 0x558d3b934d30 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5909be640 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b934760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5827fc640 1 -- 172.21.15.100:0/3551434224 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc584014070 con 0x7fc58c197600 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.979+0000 7fc5919c0640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5780a1ba0 0x7fc5780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.980+0000 7fc5911bf640 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b93d2e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc588005ab0 tx=0x7fc58802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645f43d1c3418b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.980+0000 7fc5777fe640 1 -- 172.21.15.100:0/4088446136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588004840 con 0x558d3b934d30 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.980+0000 7fc5919c0640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5780a1ba0 0x7fc5780a4060 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5840a2c00 tx=0x7fc584000f40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.980+0000 7fc5777fe640 1 -- 172.21.15.100:0/4088446136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5880049a0 con 0x558d3b934d30 2024-03-12T05:32:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.980+0000 7fc5777fe640 1 -- 172.21.15.100:0/4088446136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588005050 con 0x558d3b934d30 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 -- 172.21.15.100:0/4088446136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 msgr2=0x558d3b93d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b93d2e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc588005ab0 tx=0x7fc58802c6e0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 -- 172.21.15.100:0/4088446136 shutdown_connections 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b93fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b93d2e0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 --2- 172.21.15.100:0/4088446136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b934760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 -- 172.21.15.100:0/4088446136 >> 172.21.15.100:0/4088446136 conn(0x558d3b92e840 msgr2=0x558d3b92ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 -- 172.21.15.100:0/4088446136 shutdown_connections 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 -- 172.21.15.100:0/4088446136 wait complete. 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 -- start start 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 Processor -- start 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934380 0x558d3b93a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.981+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934d30 0x558d3b93ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 0x558d3b93b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b840f40 con 0x558d3b93d8b0 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b56f4b0 con 0x558d3b934d30 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b821670 con 0x558d3b934380 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 10 client.0 _ll_register_callbacks cb 0x558d3b88ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 0x558d3b93b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5911bf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934d30 0x558d3b93ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5911bf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934d30 0x558d3b93ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49444/0 (socket says 172.21.15.100:49444) 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5911bf640 1 -- 172.21.15.100:0/1245715285 learned_addr learned my addr 172.21.15.100:0/1245715285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 0x558d3b93b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44540/0 (socket says 172.21.15.100:44540) 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934380 0x558d3b93a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 -- 172.21.15.100:0/1245715285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934380 msgr2=0x558d3b93a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934380 0x558d3b93a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 -- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934d30 msgr2=0x558d3b93ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934d30 0x558d3b93ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 -- 172.21.15.100:0/1245715285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c10e880 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc5909be640 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 0x558d3b93b290 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009730 tx=0x7fc57c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105ed3f041dda777 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc574ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc574ff9640 1 -- 172.21.15.100:0/1245715285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c011050 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc574ff9640 1 -- 172.21.15.100:0/1245715285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc57c012e40 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d3b875f40 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc574ff9640 1 -- 172.21.15.100:0/1245715285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc57c0111b0 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.982+0000 7fc595ffc740 10 client.19803 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d3b89a320 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc574ff9640 1 -- 172.21.15.100:0/1245715285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558d3b875f40 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc574ff9640 1 -- 172.21.15.100:0/1245715285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc57c027070 con 0x558d3b93d8b0 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 10 client.19803 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:27.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 10 client.19803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 20 client.19803 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b94dd20 0x558d3b950140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 4 client.19803 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d3b892310 con 0x558d3b94dd20 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.983+0000 7fc5911bf640 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b94dd20 0x558d3b950140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc5911bf640 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b94dd20 0x558d3b950140 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d73d278682f3241 server_cookie=51288b13b4e487fb in_seq=0 out_seq=0 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc574ff9640 10 client.19803 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc574ff9640 1 -- 172.21.15.100:0/1245715285 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d3b892310 con 0x558d3b94dd20 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc574ff9640 10 client.19803 handle_command_reply: tid=0 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 client.19803 shutdown 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 msgr2=0x558d3b93b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 0x558d3b93b290 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009730 tx=0x7fc57c004730 comp rx=0 tx=0).stop 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 shutdown_connections 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b94dd20 0x558d3b950140 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b93d8b0 0x558d3b93b290 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934d30 0x558d3b93ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1245715285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934380 0x558d3b93a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 >> 172.21.15.100:0/1245715285 conn(0x558d3b92e840 msgr2=0x558d3b9307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.984+0000 7fc574ff9640 0 client.19803 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.985+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 shutdown_connections 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.985+0000 7fc595ffc740 1 -- 172.21.15.100:0/1245715285 wait complete. 2024-03-12T05:32:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:27.985+0000 7fc595ffc740 20 client.19803 trim_cache size 0 max 16384 2024-03-12T05:32:27.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.002+0000 7fc595ffc740 1 Processor -- start 2024-03-12T05:32:27.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.002+0000 7fc595ffc740 1 -- start start 2024-03-12T05:32:27.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 0x558d3b93cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934d30 0x558d3b937ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b93d8b0 0x558d3b937ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc595ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b940100 con 0x558d3b934380 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc595ffc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b935670 con 0x558d3b93d8b0 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc595ffc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b92a110 con 0x558d3b934d30 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 0x558d3b93cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 0x558d3b93cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44554/0 (socket says 172.21.15.100:44554) 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 -- 172.21.15.100:0/3762543212 learned_addr learned my addr 172.21.15.100:0/3762543212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5911bf640 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934d30 0x558d3b937ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b93d8b0 0x558d3b937ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 -- 172.21.15.100:0/3762543212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934d30 msgr2=0x558d3b937ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934d30 0x558d3b937ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 -- 172.21.15.100:0/3762543212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b93d8b0 msgr2=0x558d3b937ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b93d8b0 0x558d3b937ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:27.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 -- 172.21.15.100:0/3762543212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3b836c00 con 0x558d3b934380 2024-03-12T05:32:27.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.003+0000 7fc5909be640 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 0x558d3b93cf40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc57c012a50 tx=0x7fc57c012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8486a68f3629a5d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:27.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc576ffd640 1 -- 172.21.15.100:0/3762543212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c009e90 con 0x558d3b934380 2024-03-12T05:32:27.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc576ffd640 1 -- 172.21.15.100:0/3762543212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c0047a0 con 0x558d3b934380 2024-03-12T05:32:27.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc576ffd640 1 -- 172.21.15.100:0/3762543212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c004040 con 0x558d3b934380 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 -- 172.21.15.100:0/3762543212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 msgr2=0x558d3b93cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 0x558d3b93cf40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc57c012a50 tx=0x7fc57c012540 comp rx=0 tx=0).stop 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 -- 172.21.15.100:0/3762543212 shutdown_connections 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b93d8b0 0x558d3b937ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b934d30 0x558d3b937ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 --2- 172.21.15.100:0/3762543212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934380 0x558d3b93cf40 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.004+0000 7fc595ffc740 1 -- 172.21.15.100:0/3762543212 >> 172.21.15.100:0/3762543212 conn(0x558d3b92e840 msgr2=0x558d3b939100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 -- 172.21.15.100:0/3762543212 shutdown_connections 2024-03-12T05:32:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 -- 172.21.15.100:0/3762543212 wait complete. 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 -- start start 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 Processor -- start 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b943c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b944140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b9a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b9a2cc0 con 0x558d3b934d30 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b9a2e30 con 0x558d3b934380 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.005+0000 7fc595ffc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b9a2ff0 con 0x558d3b93d8b0 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc595ffc740 10 client.0 _ll_register_callbacks cb 0x7fc5840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5909be640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b943c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5909be640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b943c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49462/0 (socket says 172.21.15.100:49462) 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5909be640 1 -- 172.21.15.100:0/1903336353 learned_addr learned my addr 172.21.15.100:0/1903336353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b944140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5909be640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b9a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 -- 172.21.15.100:0/1903336353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 msgr2=0x558d3b9a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b9a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 -- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 msgr2=0x558d3b943c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b943c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 -- 172.21.15.100:0/1903336353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d3b89a320 con 0x558d3b934d30 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5909be640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b943c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc5911bf640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b944140 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc58802c170 tx=0x7fc588004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a229105b633ffed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc574ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc574ff9640 1 -- 172.21.15.100:0/1903336353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58804f070 con 0x558d3b934d30 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d3b8c1900 con 0x558d3b934d30 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.006+0000 7fc574ff9640 1 -- 172.21.15.100:0/1903336353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc588051d80 con 0x558d3b934d30 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc574ff9640 1 -- 172.21.15.100:0/1903336353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc588004520 con 0x558d3b934d30 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 10 client.19809 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d3b85c910 con 0x558d3b934d30 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc574ff9640 1 -- 172.21.15.100:0/1903336353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558d3b8c1900 con 0x558d3b934d30 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc574ff9640 1 -- 172.21.15.100:0/1903336353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc58803c050 con 0x558d3b934d30 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 10 client.19809 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 10 client.19809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 20 client.19809 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b9aa570 0x558d3b9ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 4 client.19809 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558d3b59b860 con 0x558d3b9aa570 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.007+0000 7fc5909be640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b9aa570 0x558d3b9ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.008+0000 7fc574ff9640 10 client.19809 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.008+0000 7fc5909be640 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b9aa570 0x558d3b9ac990 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1c41858bb863b server_cookie=725d8f2aef5020fc in_seq=0 out_seq=0 2024-03-12T05:32:28.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc574ff9640 1 -- 172.21.15.100:0/1903336353 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558d3b59b860 con 0x558d3b9aa570 2024-03-12T05:32:28.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc574ff9640 10 client.19809 handle_command_reply: tid=0 2024-03-12T05:32:28.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 client.19809 shutdown 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 msgr2=0x558d3b944140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b944140 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc58802c170 tx=0x7fc588004410 comp rx=0 tx=0).stop 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 shutdown_connections 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x558d3b9aa570 0x558d3b9ac990 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d3b93d8b0 0x558d3b9a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3b934d30 0x558d3b944140 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 --2- 172.21.15.100:0/1903336353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d3b934380 0x558d3b943c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 >> 172.21.15.100:0/1903336353 conn(0x558d3b92e840 msgr2=0x558d3b93a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc574ff9640 0 client.19809 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 shutdown_connections 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 1 -- 172.21.15.100:0/1903336353 wait complete. 2024-03-12T05:32:28.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.009+0000 7fc595ffc740 20 client.19809 trim_cache size 0 max 16384 2024-03-12T05:32:28.008 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:32:28.008 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:32:28.008 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "6174f05e-9352-4658-bda7-188badb777df", 2024-03-12T05:32:28.008 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:32:28.008 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5780a1ba0 msgr2=0x7fc5780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5780a1ba0 0x7fc5780a4060 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5840a2c00 tx=0x7fc584000f40 comp rx=0 tx=0).stop 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 msgr2=0x7fc58c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 0x7fc58c197ab0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc58400eb10 tx=0x7fc58400c2d0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 shutdown_connections 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fc5780a1ba0 0x7fc5780a4060 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58c197600 0x7fc58c197ab0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c1064c0 0x7fc58c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 --2- 172.21.15.100:0/3551434224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c0685f0 0x7fc58c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 >> 172.21.15.100:0/3551434224 conn(0x7fc58c077180 msgr2=0x7fc58c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 shutdown_connections 2024-03-12T05:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.010+0000 7fc593423640 1 -- 172.21.15.100:0/3551434224 wait complete. 2024-03-12T05:32:28.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:32:28.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:32:28.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/3927609990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 msgr2=0x7ff1f010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/3927609990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f010e000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009c10 tx=0x7ff1e802bee0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/3927609990 shutdown_connections 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/3927609990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f010e000 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/3927609990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101d10 0x7ff1f0105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/3927609990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f0101360 0x7ff1f0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/3927609990 >> 172.21.15.100:0/3927609990 conn(0x7ff1f00fcc20 msgr2=0x7ff1f00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/3927609990 shutdown_connections 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.144+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/3927609990 wait complete. 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.145+0000 7ff1f7d4f640 1 Processor -- start 2024-03-12T05:32:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.145+0000 7ff1f7d4f640 1 -- start start 2024-03-12T05:32:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.145+0000 7ff1f7d4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101360 0x7ff1f019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.145+0000 7ff1f7d4f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.145+0000 7ff1f7d4f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 0x7ff1f0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f7d4f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019a250 con 0x7ff1f01997a0 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f7d4f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019a3c0 con 0x7ff1f0106430 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f7d4f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019a530 con 0x7ff1f0101360 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 0x7ff1f0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 0x7ff1f0199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44590/0 (socket says 172.21.15.100:44590) 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 -- 172.21.15.100:0/1848330763 learned_addr learned my addr 172.21.15.100:0/1848330763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f019fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49476/0 (socket says 172.21.15.100:49476) 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101360 0x7ff1f019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 -- 172.21.15.100:0/1848330763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101360 msgr2=0x7ff1f019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101360 0x7ff1f019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 -- 172.21.15.100:0/1848330763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 msgr2=0x7ff1f019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 -- 172.21.15.100:0/1848330763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec009560 con 0x7ff1f01997a0 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101360 0x7ff1f019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1f62ec640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 0x7ff1f0199c50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8036040 tx=0x7ff1e8033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.146+0000 7ff1e6ffd640 1 -- 172.21.15.100:0/1848330763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8003b80 con 0x7ff1f01997a0 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.147+0000 7ff1e6ffd640 1 -- 172.21.15.100:0/1848330763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e8003ce0 con 0x7ff1f01997a0 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.147+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e8009840 con 0x7ff1f01997a0 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.147+0000 7ff1e6ffd640 1 -- 172.21.15.100:0/1848330763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e803e910 con 0x7ff1f01997a0 2024-03-12T05:32:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.147+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f01a6800 con 0x7ff1f01997a0 2024-03-12T05:32:28.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1e6ffd640 1 -- 172.21.15.100:0/1848330763 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff1e80045b0 con 0x7ff1f01997a0 2024-03-12T05:32:28.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 Processor -- start 2024-03-12T05:32:28.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 -- start start 2024-03-12T05:32:28.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9b2b0 0x55e79aa9b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aaa47b0 0x55e79aaa6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79aa0a280 con 0x55e79aa9bbd0 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79aa077b0 con 0x55e79aaa47b0 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.150+0000 7ff1faa01740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79aa02f90 con 0x55e79aa9b2b0 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9b2b0 0x55e79aa9b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9b2b0 0x55e79aa9b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:40354/0 (socket says 172.21.15.100:40354) 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f52ea640 1 -- 172.21.15.100:0/2561507428 learned_addr learned my addr 172.21.15.100:0/2561507428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aaa47b0 0x55e79aaa6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1e6ffd640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1dc0a1ca0 0x7ff1dc0a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2561507428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9b2b0 msgr2=0x55e79aa9b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9b2b0 0x55e79aa9b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2561507428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aaa47b0 msgr2=0x55e79aaa6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aaa47b0 0x55e79aaa6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2561507428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e79a9c1570 con 0x55e79aa9bbd0 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f62ec640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1dc0a1ca0 0x7ff1dc0a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa4270 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009dc0 tx=0x7ff1ec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4944ea9f71fcc25c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1e6ffd640 1 -- 172.21.15.100:0/1848330763 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1e80f05f0 con 0x7ff1f01997a0 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.151+0000 7ff1dbfff640 1 -- 172.21.15.100:0/2561507428 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec012450 con 0x55e79aa9bbd0 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1dbfff640 1 -- 172.21.15.100:0/2561507428 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec012a90 con 0x55e79aa9bbd0 2024-03-12T05:32:28.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1dbfff640 1 -- 172.21.15.100:0/2561507428 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00fbf0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 -- 172.21.15.100:0/2561507428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aa9bbd0 msgr2=0x55e79aaa4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1f62ec640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1dc0a1ca0 0x7ff1dc0a4160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80bcf90 tx=0x7ff1e80b95b0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa4270 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009dc0 tx=0x7ff1ec0117b0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 -- 172.21.15.100:0/2561507428 shutdown_connections 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aaa47b0 0x55e79aaa6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa4270 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2561507428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9b2b0 0x55e79aa9b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.152+0000 7ff1faa01740 1 -- 172.21.15.100:0/2561507428 >> 172.21.15.100:0/2561507428 conn(0x55e79aa95990 msgr2=0x55e79aa95d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 -- 172.21.15.100:0/2561507428 shutdown_connections 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 -- 172.21.15.100:0/2561507428 wait complete. 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 -- start start 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 Processor -- start 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.153+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1faa01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79a6d6430 con 0x55e79aaa47b0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1faa01740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79a692e00 con 0x55e79aa9b2b0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1faa01740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79a987fd0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44608/0 (socket says 172.21.15.100:44608) 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f52ea640 1 -- 172.21.15.100:0/2468905303 learned_addr learned my addr 172.21.15.100:0/2468905303 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1faa01740 10 client.0 _ll_register_callbacks cb 0x55e79a9ddfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 msgr2=0x55e79aaa4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2468905303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 msgr2=0x55e79aaa1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2468905303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec009a20 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa14f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec01c040 tx=0x7ff1ec012700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e778d4ef3de799a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1d97fa640 1 -- 172.21.15.100:0/2468905303 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00f550 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1d97fa640 1 -- 172.21.15.100:0/2468905303 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1ec00fb40 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.154+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e79a7027e0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1d97fa640 1 -- 172.21.15.100:0/2468905303 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1ec0128f0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 10 client.29092 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e79aa01610 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1d97fa640 1 -- 172.21.15.100:0/2468905303 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e79a7027e0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1d97fa640 1 -- 172.21.15.100:0/2468905303 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1ec022040 con 0x55e79aa9bbd0 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 10 client.29092 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 10 client.29092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 20 client.29092 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79aab6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 4 client.29092 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e79a73f720 con 0x55e79aab48c0 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.155+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79aab6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.156+0000 7ff1d97fa640 10 client.29092 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.156+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79aab6ce0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b22487e634645 server_cookie=af7fdb6d2840f5b7 in_seq=0 out_seq=0 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.156+0000 7ff1d97fa640 1 -- 172.21.15.100:0/2468905303 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e79a73f720 con 0x55e79aab48c0 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.156+0000 7ff1d97fa640 10 client.29092 handle_command_reply: tid=0 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.156+0000 7ff1faa01740 1 client.29092 shutdown 2024-03-12T05:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 msgr2=0x55e79aaa14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa14f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec01c040 tx=0x7ff1ec012700 comp rx=0 tx=0).stop 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 shutdown_connections 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79aab6ce0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1d97fa640 0 client.29092 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa14f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2468905303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 >> 172.21.15.100:0/2468905303 conn(0x55e79aa95990 msgr2=0x55e79aa97730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 shutdown_connections 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 1 -- 172.21.15.100:0/2468905303 wait complete. 2024-03-12T05:32:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.157+0000 7ff1faa01740 20 client.29092 trim_cache size 0 max 16384 2024-03-12T05:32:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.175+0000 7ff1faa01740 1 Processor -- start 2024-03-12T05:32:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.175+0000 7ff1faa01740 1 -- start start 2024-03-12T05:32:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aa95d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1faa01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79aa9af10 con 0x55e79aaa47b0 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1faa01740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79aa969e0 con 0x55e79aa9b2b0 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1faa01740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79aabbef0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aa95d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f5aeb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aa95d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:49510/0 (socket says 172.21.15.100:49510) 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f52ea640 1 -- 172.21.15.100:0/2390713818 learned_addr learned my addr 172.21.15.100:0/2390713818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2390713818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 msgr2=0x55e79aa95d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aa95d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2390713818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 msgr2=0x55e79aaa4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.176+0000 7ff1f5aeb640 1 -- 172.21.15.100:0/2390713818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e79a99dac0 con 0x55e79aa9bbd0 2024-03-12T05:32:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.177+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa3c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009dc0 tx=0x7ff1ec0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2839c26a2f2f1d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.177+0000 7ff1db7fe640 1 -- 172.21.15.100:0/2390713818 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec009840 con 0x55e79aa9bbd0 2024-03-12T05:32:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.177+0000 7ff1db7fe640 1 -- 172.21.15.100:0/2390713818 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec004c00 con 0x55e79aa9bbd0 2024-03-12T05:32:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.177+0000 7ff1db7fe640 1 -- 172.21.15.100:0/2390713818 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec004040 con 0x55e79aa9bbd0 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.177+0000 7ff1faa01740 1 -- 172.21.15.100:0/2390713818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 msgr2=0x55e79aaa3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.177+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa3c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009dc0 tx=0x7ff1ec0117b0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- 172.21.15.100:0/2390713818 shutdown_connections 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79aaa4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa3c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 --2- 172.21.15.100:0/2390713818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aa95d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- 172.21.15.100:0/2390713818 >> 172.21.15.100:0/2390713818 conn(0x55e79aa95990 msgr2=0x55e79aaa6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- 172.21.15.100:0/2390713818 shutdown_connections 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- 172.21.15.100:0/2390713818 wait complete. 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- start start 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 Processor -- start 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79ab03df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79ab04330 con 0x55e79aaa47b0 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79ab044a0 con 0x55e79aa9b2b0 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.178+0000 7ff1faa01740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79ab04660 con 0x55e79aa9bbd0 2024-03-12T05:32:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1faa01740 10 client.0 _ll_register_callbacks cb 0x7ff1ec009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79ab03df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79ab03df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44636/0 (socket says 172.21.15.100:44636) 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 -- 172.21.15.100:0/3133591099 learned_addr learned my addr 172.21.15.100:0/3133591099 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 -- 172.21.15.100:0/3133591099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 msgr2=0x55e79aaa7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 -- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 msgr2=0x55e79aaa78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 -- 172.21.15.100:0/3133591099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e79aa01610 con 0x55e79aaa47b0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1f52ea640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79ab03df0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005b90 tx=0x7ff1ec012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b54565dae9485dd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1d97fa640 1 -- 172.21.15.100:0/3133591099 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0016040 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1d97fa640 1 -- 172.21.15.100:0/3133591099 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff1e00035f0 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1ec012a90 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1d97fa640 1 -- 172.21.15.100:0/3133591099 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1e0004500 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1faa01740 10 client.19827 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.179+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e79a9c36a0 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1d97fa640 1 -- 172.21.15.100:0/3133591099 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec012a90 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1d97fa640 1 -- 172.21.15.100:0/3133591099 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1e00237e0 con 0x55e79aaa47b0 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1faa01740 10 client.19827 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1faa01740 10 client.19827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1faa01740 20 client.19827 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1faa01740 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79ab0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1faa01740 4 client.19827 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e79a78e890 con 0x55e79aab48c0 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79ab0df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1f5aeb640 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79ab0df90 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ffcb8e751e82cd server_cookie=2d33a926807e4a4b in_seq=0 out_seq=0 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.180+0000 7ff1d97fa640 10 client.19827 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1d97fa640 1 -- 172.21.15.100:0/3133591099 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e79a78e890 con 0x55e79aab48c0 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1d97fa640 10 client.19827 handle_command_reply: tid=0 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 client.19827 shutdown 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 msgr2=0x55e79ab03df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79ab03df0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005b90 tx=0x7ff1ec012f50 comp rx=0 tx=0).stop 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 shutdown_connections 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e79aab48c0 0x55e79ab0df90 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e79aaa47b0 0x55e79ab03df0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1d97fa640 0 client.19827 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e79aa9bbd0 0x55e79aaa7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 --2- 172.21.15.100:0/3133591099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e79aa9b2b0 0x55e79aaa78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 >> 172.21.15.100:0/3133591099 conn(0x55e79aa95990 msgr2=0x55e79aa9c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 shutdown_connections 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.182+0000 7ff1faa01740 1 -- 172.21.15.100:0/3133591099 wait complete. 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1faa01740 20 client.19827 trim_cache size 0 max 16384 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stdout: "6174f05e-9352-4658-bda7-188badb777df": { 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:32:28.182 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "6174f05e-9352-4658-bda7-188badb777df", 2024-03-12T05:32:28.183 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:32:28.183 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:32:28.183 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:32:28.183 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1dc0a1ca0 msgr2=0x7ff1dc0a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1dc0a1ca0 0x7ff1dc0a4160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80bcf90 tx=0x7ff1e80b95b0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 msgr2=0x7ff1f0199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 0x7ff1f0199c50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8036040 tx=0x7ff1e8033680 comp rx=0 tx=0).stop 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 shutdown_connections 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff1dc0a1ca0 0x7ff1dc0a4160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f01997a0 0x7ff1f0199c50 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1f0106430 0x7ff1f019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 --2- 172.21.15.100:0/1848330763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f0101360 0x7ff1f019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 >> 172.21.15.100:0/1848330763 conn(0x7ff1f00fcc20 msgr2=0x7ff1f00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 shutdown_connections 2024-03-12T05:32:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:28.183+0000 7ff1f7d4f640 1 -- 172.21.15.100:0/1848330763 wait complete. 2024-03-12T05:32:28.190 INFO:tasks.cephfs.filesystem:scrub status for tag:6174f05e-9352-4658-bda7-188badb777df - {'path': '/', 'tag': '6174f05e-9352-4658-bda7-188badb777df', 'options': 'recursive,force'} 2024-03-12T05:32:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 227378 26.56 MB/sec execute 311 sec latency 161.769 ms 2024-03-12T05:32:28.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:28 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:32:28.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:28 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:28.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:28 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:32:28.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:28 smithi100 ceph-mon[32210]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 628 op/s 2024-03-12T05:32:28.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:28 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:32:28.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:28 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:28.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:28 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:32:28.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:28 smithi156 ceph-mon[36270]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 628 op/s 2024-03-12T05:32:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:28 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:32:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:28 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:28 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:32:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:28 smithi161 ceph-mon[36043]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 628 op/s 2024-03-12T05:32:29.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 227806 26.56 MB/sec execute 312 sec latency 308.542 ms 2024-03-12T05:32:29.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:29 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:32:29.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:29 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:32:29.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:29 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:29 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:32:29.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:29 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:29 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:29 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:32:29.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:29 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:32:29.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:29 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:29 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:32:29.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:29 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:29 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:29 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:32:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:29 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:32:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:29 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:29 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:32:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:29 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:29 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 228201 26.56 MB/sec execute 313 sec latency 356.130 ms 2024-03-12T05:32:30.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:30 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:30.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:30 smithi100 ceph-mon[32210]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.9 MiB/s wr, 421 op/s 2024-03-12T05:32:30.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:30 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:30.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:30 smithi156 ceph-mon[36270]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.9 MiB/s wr, 421 op/s 2024-03-12T05:32:30.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:30 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:32:30.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:30 smithi161 ceph-mon[36043]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.9 MiB/s wr, 421 op/s 2024-03-12T05:32:31.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 228679 26.55 MB/sec execute 314 sec latency 171.323 ms 2024-03-12T05:32:32.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 229098 26.53 MB/sec execute 315 sec latency 165.100 ms 2024-03-12T05:32:32.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:32 smithi156 ceph-mon[36270]: pgmap v644: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:32:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:32 smithi161 ceph-mon[36043]: pgmap v644: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:32:32.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:32 smithi100 ceph-mon[32210]: pgmap v644: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-03-12T05:32:33.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 229556 26.52 MB/sec execute 316 sec latency 156.092 ms 2024-03-12T05:32:34.244 INFO:tasks.workunit.client.0.smithi100.stdout: 10 229984 26.51 MB/sec execute 317 sec latency 228.771 ms 2024-03-12T05:32:34.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:34 smithi156 ceph-mon[36270]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 528 op/s 2024-03-12T05:32:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:34 smithi161 ceph-mon[36043]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 528 op/s 2024-03-12T05:32:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:34 smithi100 ceph-mon[32210]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 528 op/s 2024-03-12T05:32:35.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 230446 26.50 MB/sec execute 318 sec latency 299.383 ms 2024-03-12T05:32:36.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 230995 26.52 MB/sec execute 319 sec latency 142.687 ms 2024-03-12T05:32:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:36 smithi156 ceph-mon[36270]: pgmap v646: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 450 op/s 2024-03-12T05:32:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:36 smithi161 ceph-mon[36043]: pgmap v646: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 450 op/s 2024-03-12T05:32:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:36 smithi100 ceph-mon[32210]: pgmap v646: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 450 op/s 2024-03-12T05:32:37.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 231444 26.50 MB/sec execute 320 sec latency 169.072 ms 2024-03-12T05:32:37.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:37 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:32:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:37 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:32:37.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:37 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:32:38.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 231786 26.47 MB/sec execute 321 sec latency 194.518 ms 2024-03-12T05:32:38.541 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:38 smithi100 ceph-mon[32210]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 630 op/s 2024-03-12T05:32:38.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:38 smithi156 ceph-mon[36270]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 630 op/s 2024-03-12T05:32:38.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:38 smithi161 ceph-mon[36043]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 630 op/s 2024-03-12T05:32:39.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 232289 26.45 MB/sec execute 322 sec latency 181.279 ms 2024-03-12T05:32:40.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 232597 26.46 MB/sec execute 323 sec latency 388.381 ms 2024-03-12T05:32:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:40 smithi156 ceph-mon[36270]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.8 MiB/s wr, 425 op/s 2024-03-12T05:32:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:40 smithi161 ceph-mon[36043]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.8 MiB/s wr, 425 op/s 2024-03-12T05:32:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:32:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi100 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:40.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:40 smithi100 ceph-mon[32210]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.8 MiB/s wr, 425 op/s 2024-03-12T05:32:41.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 232995 26.45 MB/sec execute 324 sec latency 168.385 ms 2024-03-12T05:32:41.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:32:41.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:32:41.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:32:41.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:41 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:32:41.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:32:41.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:32:41.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:32:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:41 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:32:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:32:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:32:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:41.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-12T05:32:41.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:41 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-12T05:32:42.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 233405 26.43 MB/sec execute 325 sec latency 157.752 ms 2024-03-12T05:32:42.396 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:42 smithi161 ceph-mon[36043]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.9 MiB/s wr, 536 op/s 2024-03-12T05:32:42.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:42 smithi156 ceph-mon[36270]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.9 MiB/s wr, 536 op/s 2024-03-12T05:32:42.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:42 smithi100 ceph-mon[32210]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.9 MiB/s wr, 536 op/s 2024-03-12T05:32:43.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 233828 26.43 MB/sec execute 326 sec latency 194.209 ms 2024-03-12T05:32:44.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 234205 26.41 MB/sec execute 327 sec latency 194.748 ms 2024-03-12T05:32:44.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:44 smithi156 ceph-mon[36270]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 493 op/s 2024-03-12T05:32:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:44 smithi161 ceph-mon[36043]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 493 op/s 2024-03-12T05:32:44.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:44 smithi100 ceph-mon[32210]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 493 op/s 2024-03-12T05:32:45.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 234461 26.35 MB/sec execute 328 sec latency 320.241 ms 2024-03-12T05:32:46.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 234928 26.35 MB/sec execute 329 sec latency 215.162 ms 2024-03-12T05:32:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:46 smithi156 ceph-mon[36270]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.9 MiB/s wr, 376 op/s 2024-03-12T05:32:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:46 smithi161 ceph-mon[36043]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.9 MiB/s wr, 376 op/s 2024-03-12T05:32:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:46 smithi100 ceph-mon[32210]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.9 MiB/s wr, 376 op/s 2024-03-12T05:32:47.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 235484 26.37 MB/sec execute 330 sec latency 183.731 ms 2024-03-12T05:32:47.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:32:47.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:32:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:32:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:32:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:32:47.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:47.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-12T05:32:47.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: Adjusting osd_memory_target on smithi161 to 174.1M 2024-03-12T05:32:47.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: Unable to set osd_memory_target on smithi161 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-12T05:32:47.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:32:47.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:32:47.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:32:48.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 235851 26.35 MB/sec execute 331 sec latency 231.675 ms 2024-03-12T05:32:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:48 smithi161 ceph-mon[36043]: pgmap v652: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 562 op/s 2024-03-12T05:32:48.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:48 smithi100 ceph-mon[32210]: pgmap v652: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 562 op/s 2024-03-12T05:32:48.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:48 smithi156 ceph-mon[36270]: pgmap v652: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 562 op/s 2024-03-12T05:32:49.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 236327 26.35 MB/sec execute 332 sec latency 126.344 ms 2024-03-12T05:32:50.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 236664 26.30 MB/sec execute 333 sec latency 267.780 ms 2024-03-12T05:32:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:50 smithi156 ceph-mon[36270]: pgmap v653: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 382 op/s 2024-03-12T05:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:50 smithi161 ceph-mon[36043]: pgmap v653: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 382 op/s 2024-03-12T05:32:50.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:50 smithi100 ceph-mon[32210]: pgmap v653: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 382 op/s 2024-03-12T05:32:51.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 237171 26.29 MB/sec execute 334 sec latency 149.914 ms 2024-03-12T05:32:52.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 237816 26.36 MB/sec execute 335 sec latency 164.701 ms 2024-03-12T05:32:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:52 smithi156 ceph-mon[36270]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 495 op/s 2024-03-12T05:32:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:52 smithi161 ceph-mon[36043]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 495 op/s 2024-03-12T05:32:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:52 smithi100 ceph-mon[32210]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 495 op/s 2024-03-12T05:32:53.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 238244 26.34 MB/sec execute 336 sec latency 191.901 ms 2024-03-12T05:32:54.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 238620 26.32 MB/sec execute 337 sec latency 169.050 ms 2024-03-12T05:32:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:54 smithi156 ceph-mon[36270]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 485 op/s 2024-03-12T05:32:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:54 smithi161 ceph-mon[36043]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 485 op/s 2024-03-12T05:32:54.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:54 smithi100 ceph-mon[32210]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 485 op/s 2024-03-12T05:32:55.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 238915 26.28 MB/sec execute 338 sec latency 362.938 ms 2024-03-12T05:32:56.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 239448 26.29 MB/sec execute 339 sec latency 188.087 ms 2024-03-12T05:32:56.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:56 smithi156 ceph-mon[36270]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.2 MiB/s wr, 400 op/s 2024-03-12T05:32:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:56 smithi161 ceph-mon[36043]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.2 MiB/s wr, 400 op/s 2024-03-12T05:32:56.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:56 smithi100 ceph-mon[32210]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.2 MiB/s wr, 400 op/s 2024-03-12T05:32:57.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 239936 26.31 MB/sec execute 340 sec latency 165.274 ms 2024-03-12T05:32:58.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:32:58.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:32:58.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 240435 26.28 MB/sec execute 341 sec latency 140.396 ms 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 -- 172.21.15.100:0/1775304340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398100c70 msgr2=0x7fb39810ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 --2- 172.21.15.100:0/1775304340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398100c70 0x7fb39810ce80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb390009cd0 tx=0x7fb39002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 -- 172.21.15.100:0/1775304340 shutdown_connections 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 --2- 172.21.15.100:0/1775304340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398100c70 0x7fb39810ce80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 --2- 172.21.15.100:0/1775304340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb398100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 --2- 172.21.15.100:0/1775304340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398105010 0x7fb3981053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 -- 172.21.15.100:0/1775304340 >> 172.21.15.100:0/1775304340 conn(0x7fb398076ee0 msgr2=0x7fb3980772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 -- 172.21.15.100:0/1775304340 shutdown_connections 2024-03-12T05:32:58.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.318+0000 7fb39cb48640 1 -- 172.21.15.100:0/1775304340 wait complete. 2024-03-12T05:32:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 Processor -- start 2024-03-12T05:32:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 -- start start 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb39819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398100c70 0x7fb39819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398105010 0x7fb398197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398197c70 con 0x7fb3981002d0 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398197de0 con 0x7fb398105010 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39cb48640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398197fa0 con 0x7fb398100c70 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb39819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb39819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47732/0 (socket says 172.21.15.100:47732) 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 -- 172.21.15.100:0/662992995 learned_addr learned my addr 172.21.15.100:0/662992995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb395d9b640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398100c70 0x7fb39819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb396d9d640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398105010 0x7fb398197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 -- 172.21.15.100:0/662992995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398100c70 msgr2=0x7fb39819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398100c70 0x7fb39819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 -- 172.21.15.100:0/662992995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398105010 msgr2=0x7fb398197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398105010 0x7fb398197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb39659c640 1 -- 172.21.15.100:0/662992995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb380009490 con 0x7fb3981002d0 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.319+0000 7fb395d9b640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398100c70 0x7fb39819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.320+0000 7fb39659c640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb39819d420 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb38800ea60 tx=0x7fb38800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.320+0000 7fb38f7fe640 1 -- 172.21.15.100:0/662992995 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800cde0 con 0x7fb3981002d0 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.320+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb388009490 con 0x7fb3981002d0 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.320+0000 7fb38f7fe640 1 -- 172.21.15.100:0/662992995 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3880035f0 con 0x7fb3981002d0 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.320+0000 7fb38f7fe640 1 -- 172.21.15.100:0/662992995 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388010690 con 0x7fb3981002d0 2024-03-12T05:32:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.320+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb390009900 con 0x7fb3981002d0 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.322+0000 7fb38f7fe640 1 -- 172.21.15.100:0/662992995 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb388003af0 con 0x7fb3981002d0 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.322+0000 7fb39f7f8740 1 Processor -- start 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.322+0000 7fb39f7f8740 1 -- start start 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1c87b0 0x555a4a1caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a12e280 con 0x555a4a1bfbd0 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a12b7b0 con 0x555a4a1c87b0 2024-03-12T05:32:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a126f90 con 0x555a4a1bf2b0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb38f7fe640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb3840a1ba0 0x7fb3840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1bf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42240/0 (socket says 172.21.15.100:42240) 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 -- 172.21.15.100:0/1503188261 learned_addr learned my addr 172.21.15.100:0/1503188261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb3840a1ba0 0x7fb3840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb395d9b640 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1c87b0 0x555a4a1caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb38f7fe640 1 -- 172.21.15.100:0/662992995 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb388014070 con 0x7fb3981002d0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 -- 172.21.15.100:0/1503188261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1c87b0 msgr2=0x555a4a1caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1c87b0 0x555a4a1caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 -- 172.21.15.100:0/1503188261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 msgr2=0x555a4a1c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 -- 172.21.15.100:0/1503188261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4a0e5570 con 0x555a4a1bf2b0 2024-03-12T05:32:58.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb39659c640 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.323+0000 7fb396d9d640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb3840a1ba0 0x7fb3840a4060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb390033000 tx=0x7fb39003c000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb396d9d640 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1bf690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb390037040 tx=0x7fb390034710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb272d0f8565b853 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb38cff9640 1 -- 172.21.15.100:0/1503188261 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390048070 con 0x555a4a1bf2b0 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb38cff9640 1 -- 172.21.15.100:0/1503188261 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb390032dc0 con 0x555a4a1bf2b0 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb38cff9640 1 -- 172.21.15.100:0/1503188261 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390004330 con 0x555a4a1bf2b0 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb39f7f8740 1 -- 172.21.15.100:0/1503188261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 msgr2=0x555a4a1bf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1bf690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb390037040 tx=0x7fb390034710 comp rx=0 tx=0).stop 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb39f7f8740 1 -- 172.21.15.100:0/1503188261 shutdown_connections 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1c87b0 0x555a4a1caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.324+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/1503188261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1bf690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- 172.21.15.100:0/1503188261 >> 172.21.15.100:0/1503188261 conn(0x555a4a1b9990 msgr2=0x555a4a1b9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- 172.21.15.100:0/1503188261 shutdown_connections 2024-03-12T05:32:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- 172.21.15.100:0/1503188261 wait complete. 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- start start 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 Processor -- start 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49dfa430 con 0x555a4a1c87b0 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49db6e00 con 0x555a4a1bfbd0 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a0abfd0 con 0x555a4a1bf2b0 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39f7f8740 10 client.0 _ll_register_callbacks cb 0x555a4a101fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47750/0 (socket says 172.21.15.100:47750) 2024-03-12T05:32:58.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb39659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42256/0 (socket says 172.21.15.100:42256) 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb396d9d640 1 -- 172.21.15.100:0/4137705722 learned_addr learned my addr 172.21.15.100:0/4137705722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.325+0000 7fb395d9b640 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb396d9d640 1 -- 172.21.15.100:0/4137705722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 msgr2=0x555a4a1c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb396d9d640 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb396d9d640 1 -- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 msgr2=0x555a4a1c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb396d9d640 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb396d9d640 1 -- 172.21.15.100:0/4137705722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3980694e0 con 0x555a4a1c87b0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb396d9d640 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1c5cd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb390009a30 tx=0x7fb390005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fa440419286778 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb37dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb37dffb640 1 -- 172.21.15.100:0/4137705722 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390048040 con 0x555a4a1c87b0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb37dffb640 1 -- 172.21.15.100:0/4137705722 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb390032770 con 0x555a4a1c87b0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a49e63720 con 0x555a4a1c87b0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb37dffb640 1 -- 172.21.15.100:0/4137705722 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb390003c40 con 0x555a4a1c87b0 2024-03-12T05:32:58.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb39f7f8740 10 client.19839 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555a4a125610 con 0x555a4a1c87b0 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.326+0000 7fb37dffb640 1 -- 172.21.15.100:0/4137705722 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555a49e63720 con 0x555a4a1c87b0 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb37dffb640 1 -- 172.21.15.100:0/4137705722 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb390042490 con 0x555a4a1c87b0 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39f7f8740 10 client.19839 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39f7f8740 10 client.19839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39f7f8740 20 client.19839 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a1d85f0 0x555a4a1daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39f7f8740 4 client.19839 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a49eb2890 con 0x555a4a1d85f0 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39659c640 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a1d85f0 0x555a4a1daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb37dffb640 10 client.19839 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:58.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.327+0000 7fb39659c640 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a1d85f0 0x555a4a1daa10 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cb40b8d8e01bfb server_cookie=e3d508385eccc405 in_seq=0 out_seq=0 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb37dffb640 1 -- 172.21.15.100:0/4137705722 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a49eb2890 con 0x555a4a1d85f0 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb37dffb640 10 client.19839 handle_command_reply: tid=0 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 client.19839 shutdown 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 msgr2=0x555a4a1c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1c5cd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb390009a30 tx=0x7fb390005fb0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 shutdown_connections 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a1d85f0 0x555a4a1daa10 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1c5cd0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/4137705722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 >> 172.21.15.100:0/4137705722 conn(0x555a4a1b9990 msgr2=0x555a4a1bb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb37dffb640 0 client.19839 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:58.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 shutdown_connections 2024-03-12T05:32:58.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 1 -- 172.21.15.100:0/4137705722 wait complete. 2024-03-12T05:32:58.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.328+0000 7fb39f7f8740 20 client.19839 trim_cache size 0 max 16384 2024-03-12T05:32:58.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 Processor -- start 2024-03-12T05:32:58.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 -- start start 2024-03-12T05:32:58.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1c87b0 0x555a4a1cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49eb2890 con 0x555a4a1bfbd0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a1c65a0 con 0x555a4a1bf2b0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a1ba9e0 con 0x555a4a1c87b0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47762/0 (socket says 172.21.15.100:47762) 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 -- 172.21.15.100:0/279940449 learned_addr learned my addr 172.21.15.100:0/279940449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb396d9d640 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1c87b0 0x555a4a1cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb39659c640 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 -- 172.21.15.100:0/279940449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1c87b0 msgr2=0x555a4a1cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1c87b0 0x555a4a1cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 -- 172.21.15.100:0/279940449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bf2b0 msgr2=0x555a4a1c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.344+0000 7fb395d9b640 1 -- 172.21.15.100:0/279940449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4a0c1ac0 con 0x555a4a1bfbd0 2024-03-12T05:32:58.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.345+0000 7fb395d9b640 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8250 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb3800127a0 tx=0x7fb380005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152f7087664c350d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.345+0000 7fb37ffff640 1 -- 172.21.15.100:0/279940449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380020070 con 0x555a4a1bfbd0 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.345+0000 7fb37ffff640 1 -- 172.21.15.100:0/279940449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb380018040 con 0x555a4a1bfbd0 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.345+0000 7fb37ffff640 1 -- 172.21.15.100:0/279940449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380017050 con 0x555a4a1bfbd0 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- 172.21.15.100:0/279940449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 msgr2=0x555a4a1c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8250 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb3800127a0 tx=0x7fb380005f90 comp rx=0 tx=0).stop 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- 172.21.15.100:0/279940449 shutdown_connections 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1c87b0 0x555a4a1cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1bfbd0 0x555a4a1c8250 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/279940449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bf2b0 0x555a4a1c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- 172.21.15.100:0/279940449 >> 172.21.15.100:0/279940449 conn(0x555a4a1b9990 msgr2=0x555a4a1c0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- 172.21.15.100:0/279940449 shutdown_connections 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- 172.21.15.100:0/279940449 wait complete. 2024-03-12T05:32:58.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- start start 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 Processor -- start 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a1ce600 con 0x555a4a1c87b0 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a1ce770 con 0x555a4a1bfbd0 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4a1ce8e0 con 0x555a4a1bf2b0 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39f7f8740 10 client.0 _ll_register_callbacks cb 0x7fb380005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47772/0 (socket says 172.21.15.100:47772) 2024-03-12T05:32:58.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb396d9d640 1 -- 172.21.15.100:0/3545187612 learned_addr learned my addr 172.21.15.100:0/3545187612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb395d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.346+0000 7fb39659c640 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb396d9d640 1 -- 172.21.15.100:0/3545187612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 msgr2=0x555a4a1cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb396d9d640 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb396d9d640 1 -- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 msgr2=0x555a4a1cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb396d9d640 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb396d9d640 1 -- 172.21.15.100:0/3545187612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb380012450 con 0x555a4a1c87b0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb396d9d640 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1ce0c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fb390042940 tx=0x7fb390009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5211d8e2defa52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb37dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb37dffb640 1 -- 172.21.15.100:0/3545187612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390048040 con 0x555a4a1c87b0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb37dffb640 1 -- 172.21.15.100:0/3545187612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb390042cc0 con 0x555a4a1c87b0 2024-03-12T05:32:58.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a49e267e0 con 0x555a4a1c87b0 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb37dffb640 1 -- 172.21.15.100:0/3545187612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb390042e80 con 0x555a4a1c87b0 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb39f7f8740 10 client.19845 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555a4a0e76a0 con 0x555a4a1c87b0 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb37dffb640 1 -- 172.21.15.100:0/3545187612 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb390074080 con 0x555a4a1c87b0 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb37dffb640 1 -- 172.21.15.100:0/3545187612 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb3900045b0 con 0x555a4a1c87b0 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb39f7f8740 10 client.19845 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb39f7f8740 10 client.19845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.347+0000 7fb39f7f8740 20 client.19845 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a22fc10 0x555a4a232000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39f7f8740 4 client.19845 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a49e154e0 con 0x555a4a22fc10 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39659c640 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a22fc10 0x555a4a232000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb37dffb640 10 client.19845 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39659c640 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a22fc10 0x555a4a232000 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39dac041a99e17b9 server_cookie=613a82880bb6ff4a in_seq=0 out_seq=0 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb37dffb640 1 -- 172.21.15.100:0/3545187612 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555a49e154e0 con 0x555a4a22fc10 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb37dffb640 10 client.19845 handle_command_reply: tid=0 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39f7f8740 1 client.19845 shutdown 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 msgr2=0x555a4a1ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.348+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1ce0c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fb390042940 tx=0x7fb390009360 comp rx=0 tx=0).stop 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 shutdown_connections 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x555a4a22fc10 0x555a4a232000 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a4a1c87b0 0x555a4a1ce0c0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4a1bfbd0 0x555a4a1cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 --2- 172.21.15.100:0/3545187612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a4a1bf2b0 0x555a4a1cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 >> 172.21.15.100:0/3545187612 conn(0x555a4a1b9990 msgr2=0x555a4a1c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb37dffb640 0 client.19845 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 shutdown_connections 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 1 -- 172.21.15.100:0/3545187612 wait complete. 2024-03-12T05:32:58.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39f7f8740 20 client.19845 trim_cache size 0 max 16384 2024-03-12T05:32:58.350 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:32:58.350 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:32:58.350 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:32:58.350 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb3840a1ba0 msgr2=0x7fb3840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb3840a1ba0 0x7fb3840a4060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb390033000 tx=0x7fb39003c000 comp rx=0 tx=0).stop 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 msgr2=0x7fb39819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb39819d420 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb38800ea60 tx=0x7fb38800ef20 comp rx=0 tx=0).stop 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 shutdown_connections 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb3840a1ba0 0x7fb3840a4060 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb398105010 0x7fb398197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb398100c70 0x7fb39819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 --2- 172.21.15.100:0/662992995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3981002d0 0x7fb39819d420 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 >> 172.21.15.100:0/662992995 conn(0x7fb398076ee0 msgr2=0x7fb3980fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 shutdown_connections 2024-03-12T05:32:58.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.349+0000 7fb39cb48640 1 -- 172.21.15.100:0/662992995 wait complete. 2024-03-12T05:32:58.353 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:58 smithi100 ceph-mon[32210]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 594 op/s 2024-03-12T05:32:58.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:32:58.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:32:58.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:58 smithi156 ceph-mon[36270]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 594 op/s 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 -- 172.21.15.100:0/2180483300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc107010 msgr2=0x7f12fc1073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 --2- 172.21.15.100:0/2180483300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc107010 0x7f12fc1073f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009930 tx=0x7f12f002bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 -- 172.21.15.100:0/2180483300 shutdown_connections 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 --2- 172.21.15.100:0/2180483300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc068c80 0x7f12fc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 --2- 172.21.15.100:0/2180483300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc0682e0 0x7f12fc068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 --2- 172.21.15.100:0/2180483300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc107010 0x7f12fc1073f0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 -- 172.21.15.100:0/2180483300 >> 172.21.15.100:0/2180483300 conn(0x7f12fc075a90 msgr2=0x7f12fc075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 -- 172.21.15.100:0/2180483300 shutdown_connections 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.474+0000 7f1301da0640 1 -- 172.21.15.100:0/2180483300 wait complete. 2024-03-12T05:32:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 Processor -- start 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 -- start start 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 0x7f12fc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc068c80 0x7f12fc197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc107010 0x7f12fc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc19c600 con 0x7f12fc0682e0 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc19c770 con 0x7f12fc107010 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1301da0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc19c930 con 0x7f12fc068c80 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 0x7f12fc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 0x7f12fc197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47796/0 (socket says 172.21.15.100:47796) 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 -- 172.21.15.100:0/1988953508 learned_addr learned my addr 172.21.15.100:0/1988953508 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f12fbfff640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc068c80 0x7f12fc197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f130159f640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc107010 0x7f12fc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 -- 172.21.15.100:0/1988953508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc068c80 msgr2=0x7f12fc197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc068c80 0x7f12fc197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 -- 172.21.15.100:0/1988953508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc107010 msgr2=0x7f12fc19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc107010 0x7f12fc19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.475+0000 7f1300d9e640 1 -- 172.21.15.100:0/1988953508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f0009560 con 0x7f12fc0682e0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.476+0000 7f1300d9e640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 0x7f12fc197740 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009230 tx=0x7f12f002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.476+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0004350 con 0x7f12fc0682e0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.476+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12f00044b0 con 0x7f12fc0682e0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.476+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc19cbb0 con 0x7f12fc0682e0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.476+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0003d00 con 0x7f12fc0682e0 2024-03-12T05:32:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.476+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc100310 con 0x7f12fc0682e0 2024-03-12T05:32:58.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:58 smithi161 ceph-mon[36043]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 594 op/s 2024-03-12T05:32:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.477+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc102690 con 0x7f12fc0682e0 2024-03-12T05:32:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.478+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f12f0004df0 con 0x7f12fc0682e0 2024-03-12T05:32:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.479+0000 7f12f9ffb640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12ec0a1ba0 0x7f12ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.479+0000 7f12fbfff640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12ec0a1ba0 0x7f12ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.479+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f12f00efe70 con 0x7f12fc0682e0 2024-03-12T05:32:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.480+0000 7f12fbfff640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12ec0a1ba0 0x7f12ec0a4060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12fc076060 tx=0x7f12e8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.483+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12f00b8e10 con 0x7f12fc0682e0 2024-03-12T05:32:58.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.727+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12fc198a90 con 0x7f12fc0682e0 2024-03-12T05:32:58.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.728+0000 7f12f9ffb640 1 -- 172.21.15.100:0/1988953508 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f12f0003e60 con 0x7f12fc0682e0 2024-03-12T05:32:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:32:58.722 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:32:58.722 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:32:58.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12ec0a1ba0 msgr2=0x7f12ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12ec0a1ba0 0x7f12ec0a4060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12fc076060 tx=0x7f12e8005f90 comp rx=0 tx=0).stop 2024-03-12T05:32:58.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 msgr2=0x7f12fc197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 0x7f12fc197740 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009230 tx=0x7f12f002c650 comp rx=0 tx=0).stop 2024-03-12T05:32:58.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 shutdown_connections 2024-03-12T05:32:58.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f12ec0a1ba0 0x7f12ec0a4060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc107010 0x7f12fc19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc068c80 0x7f12fc197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 --2- 172.21.15.100:0/1988953508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0682e0 0x7f12fc197740 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 >> 172.21.15.100:0/1988953508 conn(0x7f12fc075a90 msgr2=0x7f12fc0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 shutdown_connections 2024-03-12T05:32:58.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.730+0000 7f1301da0640 1 -- 172.21.15.100:0/1988953508 wait complete. 2024-03-12T05:32:58.736 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:32:58.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 -- 172.21.15.100:0/2511503160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2981083f0 msgr2=0x7fd2981087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/2511503160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2981083f0 0x7fd2981087d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28802bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 -- 172.21.15.100:0/2511503160 shutdown_connections 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/2511503160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29810cd10 0x7fd29810d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/2511503160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd298108d10 0x7fd29810c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/2511503160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2981083f0 0x7fd2981087d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 -- 172.21.15.100:0/2511503160 >> 172.21.15.100:0/2511503160 conn(0x7fd29806c6c0 msgr2=0x7fd29806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 -- 172.21.15.100:0/2511503160 shutdown_connections 2024-03-12T05:32:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.875+0000 7fd29ea5c640 1 -- 172.21.15.100:0/2511503160 wait complete. 2024-03-12T05:32:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.876+0000 7fd29ea5c640 1 Processor -- start 2024-03-12T05:32:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.876+0000 7fd29ea5c640 1 -- start start 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.876+0000 7fd29ea5c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 0x7fd2981a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.876+0000 7fd29ea5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd298108d10 0x7fd2981a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.876+0000 7fd29ea5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29810cd10 0x7fd2981a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd29ea5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981a7150 con 0x7fd29810cd10 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd29ea5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981a72c0 con 0x7fd298108d10 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd29ea5c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981a7480 con 0x7fd2981083f0 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd297fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 0x7fd2981a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd297fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 0x7fd2981a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42308/0 (socket says 172.21.15.100:42308) 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd297fff640 1 -- 172.21.15.100:0/4105830624 learned_addr learned my addr 172.21.15.100:0/4105830624 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd29cff9640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29810cd10 0x7fd2981a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd2977fe640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd298108d10 0x7fd2981a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd297fff640 1 -- 172.21.15.100:0/4105830624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd298108d10 msgr2=0x7fd2981a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd297fff640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd298108d10 0x7fd2981a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.877+0000 7fd297fff640 1 -- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29810cd10 msgr2=0x7fd2981a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd297fff640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29810cd10 0x7fd2981a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd297fff640 1 -- 172.21.15.100:0/4105830624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009560 con 0x7fd2981083f0 2024-03-12T05:32:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd297fff640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 0x7fd2981a2290 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd288005940 tx=0x7fd28802cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd2957fa640 1 -- 172.21.15.100:0/4105830624 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288004250 con 0x7fd2981083f0 2024-03-12T05:32:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd2957fa640 1 -- 172.21.15.100:0/4105830624 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2880043b0 con 0x7fd2981083f0 2024-03-12T05:32:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2981a7700 con 0x7fd2981083f0 2024-03-12T05:32:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd2957fa640 1 -- 172.21.15.100:0/4105830624 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880039e0 con 0x7fd2981083f0 2024-03-12T05:32:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.878+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981188a0 con 0x7fd2981083f0 2024-03-12T05:32:58.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.881+0000 7fd2957fa640 1 -- 172.21.15.100:0/4105830624 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd288005360 con 0x7fd2981083f0 2024-03-12T05:32:58.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.881+0000 7fd2a1632740 1 Processor -- start 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.881+0000 7fd2a1632740 1 -- start start 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0b70 0x55fb1abc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc9750 0x55fb1abcbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2a1632740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1ab7b960 con 0x55fb1abc0250 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2a1632740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1ab2f310 con 0x55fb1abc9750 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2a1632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1ab2c840 con 0x55fb1abc0b70 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd29cff9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd29cff9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47818/0 (socket says 172.21.15.100:47818) 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd29cff9640 1 -- 172.21.15.100:0/4071310248 learned_addr learned my addr 172.21.15.100:0/4071310248 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd297fff640 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0b70 0x55fb1abc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2957fa640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2800a1ba0 0x7fd2800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2977fe640 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc9750 0x55fb1abcbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd29cff9640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2800a1ba0 0x7fd2800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd2957fa640 1 -- 172.21.15.100:0/4105830624 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd288100880 con 0x7fd2981083f0 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd297fff640 1 -- 172.21.15.100:0/4071310248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc9750 msgr2=0x55fb1abcbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd297fff640 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc9750 0x55fb1abcbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd297fff640 1 -- 172.21.15.100:0/4071310248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 msgr2=0x55fb1abc0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd297fff640 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.882+0000 7fd297fff640 1 -- 172.21.15.100:0/4071310248 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb1aaea060 con 0x55fb1abc0b70 2024-03-12T05:32:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.883+0000 7fd297fff640 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0b70 0x55fb1abc9210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2880fe0a0 tx=0x7fd28802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5eb976d649a3a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.883+0000 7fd29cff9640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2800a1ba0 0x7fd2800a4060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2900049d0 tx=0x7fd290004ab0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.883+0000 7fd27e7fc640 1 -- 172.21.15.100:0/4071310248 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288103ea0 con 0x55fb1abc0b70 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.883+0000 7fd27e7fc640 1 -- 172.21.15.100:0/4071310248 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2880d5d20 con 0x55fb1abc0b70 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.883+0000 7fd27e7fc640 1 -- 172.21.15.100:0/4071310248 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880fe210 con 0x55fb1abc0b70 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- 172.21.15.100:0/4071310248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0b70 msgr2=0x55fb1abc9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0b70 0x55fb1abc9210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2880fe0a0 tx=0x7fd28802bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- 172.21.15.100:0/4071310248 shutdown_connections 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc9750 0x55fb1abcbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0b70 0x55fb1abc9210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- 172.21.15.100:0/4071310248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- 172.21.15.100:0/4071310248 >> 172.21.15.100:0/4071310248 conn(0x55fb1abba980 msgr2=0x55fb1abbad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- 172.21.15.100:0/4071310248 shutdown_connections 2024-03-12T05:32:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- 172.21.15.100:0/4071310248 wait complete. 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- start start 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 Processor -- start 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1ab28020 con 0x55fb1abc0250 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1a7fb430 con 0x55fb1abc0b70 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.884+0000 7fd2a1632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1a7b7e00 con 0x55fb1abc9750 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd2a1632740 10 client.0 _ll_register_callbacks cb 0x55fb1aacfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47832/0 (socket says 172.21.15.100:47832) 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 -- 172.21.15.100:0/873474752 learned_addr learned my addr 172.21.15.100:0/873474752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd29cff9640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd2977fe640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 -- 172.21.15.100:0/873474752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 msgr2=0x55fb1abc71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 -- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 msgr2=0x55fb1abc6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 -- 172.21.15.100:0/873474752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981188a0 con 0x55fb1abc0250 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd297fff640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc6730 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fd2880fe590 tx=0x7fd2880b82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808bd03fb3fd0041 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd2777fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd2777fe640 1 -- 172.21.15.100:0/873474752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288103ea0 con 0x55fb1abc0250 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd2777fe640 1 -- 172.21.15.100:0/873474752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd2880cdc80 con 0x55fb1abc0250 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.885+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb1a8277e0 con 0x55fb1abc0250 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2777fe640 1 -- 172.21.15.100:0/873474752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2880cd580 con 0x55fb1abc0250 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 10 client.19863 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fb1ab266a0 con 0x55fb1abc0250 2024-03-12T05:32:58.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2777fe640 1 -- 172.21.15.100:0/873474752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fb1a8277e0 con 0x55fb1abc0250 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2777fe640 1 -- 172.21.15.100:0/873474752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2880e7030 con 0x55fb1abc0250 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 10 client.19863 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 10 client.19863 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 20 client.19863 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1abd94a0 0x55fb1abdb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 4 client.19863 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb1a864720 con 0x55fb1abd94a0 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.886+0000 7fd2977fe640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1abd94a0 0x55fb1abdb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.887+0000 7fd2977fe640 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1abd94a0 0x55fb1abdb8d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f76a312d5be04f93 server_cookie=4cb3463ee8c5e07c in_seq=0 out_seq=0 2024-03-12T05:32:58.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.887+0000 7fd2777fe640 10 client.19863 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:58.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.888+0000 7fd2777fe640 1 -- 172.21.15.100:0/873474752 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb1a864720 con 0x55fb1abd94a0 2024-03-12T05:32:58.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.888+0000 7fd2777fe640 10 client.19863 handle_command_reply: tid=0 2024-03-12T05:32:58.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.888+0000 7fd2a1632740 1 client.19863 shutdown 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.888+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 msgr2=0x55fb1abc6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.888+0000 7fd2a1632740 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc6730 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fd2880fe590 tx=0x7fd2880b82f0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 shutdown_connections 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1abd94a0 0x55fb1abdb8d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 --2- 172.21.15.100:0/873474752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc6730 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2777fe640 0 client.19863 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 >> 172.21.15.100:0/873474752 conn(0x55fb1abba980 msgr2=0x55fb1abbc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 shutdown_connections 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 1 -- 172.21.15.100:0/873474752 wait complete. 2024-03-12T05:32:58.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.889+0000 7fd2a1632740 20 client.19863 trim_cache size 0 max 16384 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 Processor -- start 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 -- start start 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1aaad130 con 0x55fb1abc0250 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1abb62b0 con 0x55fb1abc0b70 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd2a1632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1abcc110 con 0x55fb1abc9750 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.903+0000 7fd297fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47840/0 (socket says 172.21.15.100:47840) 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 -- 172.21.15.100:0/1414292846 learned_addr learned my addr 172.21.15.100:0/1414292846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd2977fe640 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd29cff9640 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 -- 172.21.15.100:0/1414292846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 msgr2=0x55fb1abc39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 -- 172.21.15.100:0/1414292846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 msgr2=0x55fb1abc6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 -- 172.21.15.100:0/1414292846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb1aac2c20 con 0x55fb1abc0250 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd297fff640 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc8e60 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd2880cd510 tx=0x7fd28802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4b3a27208321c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd27dffb640 1 -- 172.21.15.100:0/1414292846 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880cd580 con 0x55fb1abc0250 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd27dffb640 1 -- 172.21.15.100:0/1414292846 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd288100b40 con 0x55fb1abc0250 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.904+0000 7fd27dffb640 1 -- 172.21.15.100:0/1414292846 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880d16d0 con 0x55fb1abc0250 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 -- 172.21.15.100:0/1414292846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 msgr2=0x55fb1abc8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc8e60 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd2880cd510 tx=0x7fd28802bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 -- 172.21.15.100:0/1414292846 shutdown_connections 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc9750 0x55fb1abc39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abc6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 --2- 172.21.15.100:0/1414292846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc0250 0x55fb1abc8e60 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 -- 172.21.15.100:0/1414292846 >> 172.21.15.100:0/1414292846 conn(0x55fb1abba980 msgr2=0x55fb1abe01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 -- 172.21.15.100:0/1414292846 shutdown_connections 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 -- 172.21.15.100:0/1414292846 wait complete. 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.905+0000 7fd2a1632740 1 -- start start 2024-03-12T05:32:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 Processor -- start 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0250 0x55fb1abd08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abcec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 0x55fb1abcf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1abcfaa0 con 0x55fb1abc9750 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1ac2f970 con 0x55fb1abc0b70 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1ac2fab0 con 0x55fb1abc0250 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2a1632740 10 client.0 _ll_register_callbacks cb 0x7fd2880cd350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 0x55fb1abcf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 0x55fb1abcf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47844/0 (socket says 172.21.15.100:47844) 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 -- 172.21.15.100:0/3773324891 learned_addr learned my addr 172.21.15.100:0/3773324891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd297fff640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0250 0x55fb1abd08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2977fe640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abcec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 -- 172.21.15.100:0/3773324891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0250 msgr2=0x55fb1abd08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0250 0x55fb1abd08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 -- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 msgr2=0x55fb1abcec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abcec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 -- 172.21.15.100:0/3773324891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb1ab266a0 con 0x55fb1abc9750 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd29cff9640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 0x55fb1abcf3d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd290013ba0 tx=0x7fd290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604b06e7ba3a206f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.906+0000 7fd2777fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2777fe640 1 -- 172.21.15.100:0/3773324891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290024c10 con 0x55fb1abc9750 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2777fe640 1 -- 172.21.15.100:0/3773324891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd290024d70 con 0x55fb1abc9750 2024-03-12T05:32:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb1a8b3890 con 0x55fb1abc9750 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2777fe640 1 -- 172.21.15.100:0/3773324891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd290004b20 con 0x55fb1abc9750 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 10 client.19869 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fb1aae88c0 con 0x55fb1abc9750 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2777fe640 1 -- 172.21.15.100:0/3773324891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd290038020 con 0x55fb1abc9750 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2777fe640 1 -- 172.21.15.100:0/3773324891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd290010430 con 0x55fb1abc9750 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 10 client.19869 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 10 client.19869 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 20 client.19869 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1ac31300 0x55fb1ac33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 4 client.19869 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fb1a7d3700 con 0x55fb1ac31300 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.907+0000 7fd297fff640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1ac31300 0x55fb1ac33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.908+0000 7fd2777fe640 10 client.19869 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.908+0000 7fd297fff640 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1ac31300 0x55fb1ac33720 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c74b5a9f2b2dffb server_cookie=c995bda4e5579ff6 in_seq=0 out_seq=0 2024-03-12T05:32:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2777fe640 1 -- 172.21.15.100:0/3773324891 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fb1a7d3700 con 0x55fb1ac31300 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2777fe640 10 client.19869 handle_command_reply: tid=0 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 client.19869 shutdown 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 msgr2=0x55fb1abcf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 0x55fb1abcf3d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd290013ba0 tx=0x7fd290004a40 comp rx=0 tx=0).stop 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 shutdown_connections 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x55fb1ac31300 0x55fb1ac33720 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2777fe640 0 client.19869 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb1abc9750 0x55fb1abcf3d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb1abc0b70 0x55fb1abcec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 --2- 172.21.15.100:0/3773324891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1abc0250 0x55fb1abd08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 >> 172.21.15.100:0/3773324891 conn(0x55fb1abba980 msgr2=0x55fb1abb92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 shutdown_connections 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 1 -- 172.21.15.100:0/3773324891 wait complete. 2024-03-12T05:32:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.910+0000 7fd2a1632740 20 client.19869 trim_cache size 0 max 16384 2024-03-12T05:32:58.909 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2800a1ba0 msgr2=0x7fd2800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2800a1ba0 0x7fd2800a4060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2900049d0 tx=0x7fd290004ab0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 msgr2=0x7fd2981a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 0x7fd2981a2290 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd288005940 tx=0x7fd28802cb00 comp rx=0 tx=0).stop 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 shutdown_connections 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd2800a1ba0 0x7fd2800a4060 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd29810cd10 0x7fd2981a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd298108d10 0x7fd2981a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 --2- 172.21.15.100:0/4105830624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2981083f0 0x7fd2981a2290 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:58.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 >> 172.21.15.100:0/4105830624 conn(0x7fd29806c6c0 msgr2=0x7fd298116680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:58.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 shutdown_connections 2024-03-12T05:32:58.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:58.911+0000 7fd29ea5c640 1 -- 172.21.15.100:0/4105830624 wait complete. 2024-03-12T05:32:58.915 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:32:58.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 -- 172.21.15.100:0/3561956995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0105330 msgr2=0x7f49b010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 --2- 172.21.15.100:0/3561956995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0105330 0x7f49b010cf00 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009cd0 tx=0x7f49a802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 -- 172.21.15.100:0/3561956995 shutdown_connections 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 --2- 172.21.15.100:0/3561956995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0105330 0x7f49b010cf00 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 --2- 172.21.15.100:0/3561956995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49b0100c10 0x7f49b0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 --2- 172.21.15.100:0/3561956995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b0100260 0x7f49b0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 -- 172.21.15.100:0/3561956995 >> 172.21.15.100:0/3561956995 conn(0x7f49b0077180 msgr2=0x7f49b0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 -- 172.21.15.100:0/3561956995 shutdown_connections 2024-03-12T05:32:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 -- 172.21.15.100:0/3561956995 wait complete. 2024-03-12T05:32:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.048+0000 7f49b590d640 1 Processor -- start 2024-03-12T05:32:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 -- start start 2024-03-12T05:32:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 0x7f49b01966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b0105330 0x7f49b0196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49b019afb0 0x7f49b019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b019b9a0 con 0x7f49b0100260 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b019bb10 con 0x7f49b019afb0 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49b590d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b019bc80 con 0x7f49b0105330 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 0x7f49b01966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 0x7f49b01966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47860/0 (socket says 172.21.15.100:47860) 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 -- 172.21.15.100:0/2830803122 learned_addr learned my addr 172.21.15.100:0/2830803122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49b019afb0 0x7f49b019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 -- 172.21.15.100:0/2830803122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b0105330 msgr2=0x7f49b0196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b0105330 0x7f49b0196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 -- 172.21.15.100:0/2830803122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49b019afb0 msgr2=0x7f49b019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49b019afb0 0x7f49b019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.049+0000 7f49aeffd640 1 -- 172.21.15.100:0/2830803122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8009900 con 0x7f49b0100260 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.050+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 0x7f49b01966f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f49a000b4a0 tx=0x7f49a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.050+0000 7f49b490b640 1 -- 172.21.15.100:0/2830803122 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0010610 con 0x7f49b0100260 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.050+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b01a3260 con 0x7f49b0100260 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.050+0000 7f49b490b640 1 -- 172.21.15.100:0/2830803122 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a0010c50 con 0x7f49b0100260 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.050+0000 7f49b490b640 1 -- 172.21.15.100:0/2830803122 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0017e20 con 0x7f49b0100260 2024-03-12T05:32:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.050+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b01a3710 con 0x7f49b0100260 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.052+0000 7f49b490b640 1 -- 172.21.15.100:0/2830803122 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f49a000f360 con 0x7f49b0100260 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.052+0000 7f49b85bc740 1 Processor -- start 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.052+0000 7f49b85bc740 1 -- start start 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae87b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b85bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dae37960 con 0x5611dae85750 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b85bc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dadeb310 con 0x5611dae7c250 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b85bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dade8840 con 0x5611dae7cb70 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b490b640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f49980a1ba0 0x7f49980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae87b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae7c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59284/0 (socket says 172.21.15.100:59284) 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae87b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47862/0 (socket says 172.21.15.100:47862) 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 -- 172.21.15.100:0/3609522233 learned_addr learned my addr 172.21.15.100:0/3609522233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f49980a1ba0 0x7f49980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49b490b640 1 -- 172.21.15.100:0/2830803122 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f49a00ccde0 con 0x7f49b0100260 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 -- 172.21.15.100:0/3609522233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 msgr2=0x5611dae85210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae85210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 -- 172.21.15.100:0/3609522233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 msgr2=0x5611dae87b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae87b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.053+0000 7f49af7fe640 1 -- 172.21.15.100:0/3609522233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611dada6060 con 0x5611dae7c250 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.054+0000 7f49af7fe640 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae7c630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49a802c700 tx=0x7f49a8034540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9405e8cdcfd918ed server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.054+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f49980a1ba0 0x7f49980a4060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f49a8048000 tx=0x7f49a804b000 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.054+0000 7f4995ffb640 1 -- 172.21.15.100:0/3609522233 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80328b0 con 0x5611dae7c250 2024-03-12T05:32:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.054+0000 7f4995ffb640 1 -- 172.21.15.100:0/3609522233 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a8003e50 con 0x5611dae7c250 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.054+0000 7f4995ffb640 1 -- 172.21.15.100:0/3609522233 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80045f0 con 0x5611dae7c250 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 -- 172.21.15.100:0/3609522233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 msgr2=0x5611dae7c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae7c630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49a802c700 tx=0x7f49a8034540 comp rx=0 tx=0).stop 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 -- 172.21.15.100:0/3609522233 shutdown_connections 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae87b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae85210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 --2- 172.21.15.100:0/3609522233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae7c630 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 -- 172.21.15.100:0/3609522233 >> 172.21.15.100:0/3609522233 conn(0x5611dae76980 msgr2=0x5611dae76d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 -- 172.21.15.100:0/3609522233 shutdown_connections 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 -- 172.21.15.100:0/3609522233 wait complete. 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.055+0000 7f49b85bc740 1 -- start start 2024-03-12T05:32:59.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 Processor -- start 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae88590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae93610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dade4020 con 0x5611dae85750 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611daab7430 con 0x5611dae7c250 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611daa73e00 con 0x5611dae7cb70 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49b85bc740 10 client.0 _ll_register_callbacks cb 0x5611dad8bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae93610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae93610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47872/0 (socket says 172.21.15.100:47872) 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 -- 172.21.15.100:0/2417082648 learned_addr learned my addr 172.21.15.100:0/2417082648 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae88590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 -- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 msgr2=0x5611dae88ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae88ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 -- 172.21.15.100:0/2417082648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 msgr2=0x5611dae88590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae88590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49af7fe640 1 -- 172.21.15.100:0/2417082648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b010d710 con 0x5611dae85750 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae88590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.056+0000 7f49ae7fc640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae88ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae93610 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f49a8032560 tx=0x7f49a8003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf0b3e43fa55fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f498effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f498effd640 1 -- 172.21.15.100:0/2417082648 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8004780 con 0x5611dae85750 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f498effd640 1 -- 172.21.15.100:0/2417082648 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f49a804a070 con 0x5611dae85750 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611daae37e0 con 0x5611dae85750 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f498effd640 1 -- 172.21.15.100:0/2417082648 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49a8032ae0 con 0x5611dae85750 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 10 client.19881 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5611dade26a0 con 0x5611dae85750 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f498effd640 1 -- 172.21.15.100:0/2417082648 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611daae37e0 con 0x5611dae85750 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f498effd640 1 -- 172.21.15.100:0/2417082648 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f49a8067070 con 0x5611dae85750 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 10 client.19881 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 10 client.19881 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 20 client.19881 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611dae83a20 0x5611dae83e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 4 client.19881 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.057+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611dab20720 con 0x5611dae83a20 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.058+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611dae83a20 0x5611dae83e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.058+0000 7f498effd640 10 client.19881 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.058+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611dae83a20 0x5611dae83e30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87e8e1dff4e941b4 server_cookie=e4d1f9419742d1a in_seq=0 out_seq=0 2024-03-12T05:32:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f498effd640 1 -- 172.21.15.100:0/2417082648 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611dab20720 con 0x5611dae83a20 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f498effd640 10 client.19881 handle_command_reply: tid=0 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 client.19881 shutdown 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 msgr2=0x5611dae93610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae93610 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f49a8032560 tx=0x7f49a8003de0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 shutdown_connections 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae93610 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae88ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611dae83a20 0x5611dae83e30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2417082648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae88590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f498effd640 0 client.19881 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 >> 172.21.15.100:0/2417082648 conn(0x5611dae76980 msgr2=0x5611dae786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 shutdown_connections 2024-03-12T05:32:59.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.059+0000 7f49b85bc740 1 -- 172.21.15.100:0/2417082648 wait complete. 2024-03-12T05:32:59.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.060+0000 7f49b85bc740 20 client.19881 trim_cache size 0 max 16384 2024-03-12T05:32:59.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.073+0000 7f49b85bc740 1 Processor -- start 2024-03-12T05:32:59.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.073+0000 7f49b85bc740 1 -- start start 2024-03-12T05:32:59.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.073+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae9a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.073+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae9ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.073+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae81080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49b85bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dad69130 con 0x5611dae85750 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49b85bc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dae77980 con 0x5611dae7c250 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49b85bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611dae94840 con 0x5611dae7cb70 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae81080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae81080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47878/0 (socket says 172.21.15.100:47878) 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 -- 172.21.15.100:0/2033237990 learned_addr learned my addr 172.21.15.100:0/2033237990 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49ae7fc640 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae9ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae9a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 -- 172.21.15.100:0/2033237990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 msgr2=0x5611dae9ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae9ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 -- 172.21.15.100:0/2033237990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 msgr2=0x5611dae9a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae9a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 -- 172.21.15.100:0/2033237990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611dad7ec20 con 0x5611dae85750 2024-03-12T05:32:59.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae81080 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f49a8036010 tx=0x7f49a8034540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee2d3f5c00bfeb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49957fa640 1 -- 172.21.15.100:0/2033237990 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80328c0 con 0x5611dae85750 2024-03-12T05:32:59.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49957fa640 1 -- 172.21.15.100:0/2033237990 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a8032a20 con 0x5611dae85750 2024-03-12T05:32:59.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.074+0000 7f49957fa640 1 -- 172.21.15.100:0/2033237990 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80047e0 con 0x5611dae85750 2024-03-12T05:32:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 -- 172.21.15.100:0/2033237990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 msgr2=0x5611dae81080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae81080 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f49a8036010 tx=0x7f49a8034540 comp rx=0 tx=0).stop 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 -- 172.21.15.100:0/2033237990 shutdown_connections 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae85750 0x5611dae81080 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae7cb70 0x5611dae9ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2033237990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7c250 0x5611dae9a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 -- 172.21.15.100:0/2033237990 >> 172.21.15.100:0/2033237990 conn(0x5611dae76980 msgr2=0x5611dae71a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 -- 172.21.15.100:0/2033237990 shutdown_connections 2024-03-12T05:32:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.075+0000 7f49b85bc740 1 -- 172.21.15.100:0/2033237990 wait complete. 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 -- start start 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 Processor -- start 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 0x5611daee5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7cb70 0x5611daee64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae85750 0x5611daee03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611daee0960 con 0x5611dae7c250 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611daee0ad0 con 0x5611dae7cb70 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611daee0c90 con 0x5611dae85750 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.076+0000 7f49b85bc740 10 client.0 _ll_register_callbacks cb 0x7f49a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 0x5611daee5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7cb70 0x5611daee64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae85750 0x5611daee03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 0x5611daee5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47890/0 (socket says 172.21.15.100:47890) 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49aeffd640 1 -- 172.21.15.100:0/2719392445 learned_addr learned my addr 172.21.15.100:0/2719392445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 -- 172.21.15.100:0/2719392445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7cb70 msgr2=0x5611daee64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7cb70 0x5611daee64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 -- 172.21.15.100:0/2719392445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 msgr2=0x5611daee5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 0x5611daee5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 -- 172.21.15.100:0/2719392445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611dade26a0 con 0x5611dae85750 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49ae7fc640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7cb70 0x5611daee64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 0x5611daee5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49af7fe640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae85750 0x5611daee03a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49a802c630 tx=0x7f49a80048b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1435adb04f65dce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f498effd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f498effd640 1 -- 172.21.15.100:0/2719392445 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80328c0 con 0x5611dae85750 2024-03-12T05:32:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49a8003e50 con 0x5611dae85750 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.077+0000 7f498effd640 1 -- 172.21.15.100:0/2719392445 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f49a804a040 con 0x5611dae85750 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f498effd640 1 -- 172.21.15.100:0/2719392445 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49a8003cc0 con 0x5611dae85750 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 10 client.29152 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5611dada48c0 con 0x5611dae85750 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f498effd640 1 -- 172.21.15.100:0/2719392445 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a8003e50 con 0x5611dae85750 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f498effd640 1 -- 172.21.15.100:0/2719392445 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f49a8067040 con 0x5611dae85750 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 10 client.29152 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 10 client.29152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 20 client.29152 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611daeed970 0x5611daeefd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 4 client.29152 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5611dab6f890 con 0x5611daeed970 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.078+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611daeed970 0x5611daeefd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.079+0000 7f498effd640 10 client.29152 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:59.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.079+0000 7f49aeffd640 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611daeed970 0x5611daeefd60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87b1111754f00c60 server_cookie=be8d681d712a01ad in_seq=0 out_seq=0 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f498effd640 1 -- 172.21.15.100:0/2719392445 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5611dab6f890 con 0x5611daeed970 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f498effd640 10 client.29152 handle_command_reply: tid=0 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 client.29152 shutdown 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae85750 msgr2=0x5611daee03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae85750 0x5611daee03a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49a802c630 tx=0x7f49a80048b0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 shutdown_connections 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611dae85750 0x5611daee03a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x5611daeed970 0x5611daeefd60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611dae7cb70 0x5611daee64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 --2- 172.21.15.100:0/2719392445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5611dae7c250 0x5611daee5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f498effd640 0 client.29152 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 >> 172.21.15.100:0/2719392445 conn(0x5611dae76980 msgr2=0x5611dae77db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 shutdown_connections 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 1 -- 172.21.15.100:0/2719392445 wait complete. 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.081+0000 7f49b85bc740 20 client.29152 trim_cache size 0 max 16384 2024-03-12T05:32:59.080 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:32:59.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f49980a1ba0 msgr2=0x7f49980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f49980a1ba0 0x7f49980a4060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f49a8048000 tx=0x7f49a804b000 comp rx=0 tx=0).stop 2024-03-12T05:32:59.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 msgr2=0x7f49b01966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 0x7f49b01966f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f49a000b4a0 tx=0x7f49a000b960 comp rx=0 tx=0).stop 2024-03-12T05:32:59.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 shutdown_connections 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f49980a1ba0 0x7f49980a4060 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49b019afb0 0x7f49b019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b0105330 0x7f49b0196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 --2- 172.21.15.100:0/2830803122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49b0100260 0x7f49b01966f0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 >> 172.21.15.100:0/2830803122 conn(0x7f49b0077180 msgr2=0x7f49b00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 shutdown_connections 2024-03-12T05:32:59.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.082+0000 7f49b590d640 1 -- 172.21.15.100:0/2830803122 wait complete. 2024-03-12T05:32:59.086 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:32:59.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.204+0000 7f4ef27d7640 1 -- 172.21.15.100:0/159049409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec100c10 msgr2=0x7f4eec104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.204+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/159049409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec100c10 0x7f4eec104c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009900 tx=0x7f4ee002bc30 comp rx=0 tx=0).stop 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 -- 172.21.15.100:0/159049409 shutdown_connections 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/159049409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec105330 0x7f4eec10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/159049409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec100c10 0x7f4eec104c50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/159049409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec100260 0x7f4eec100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 -- 172.21.15.100:0/159049409 >> 172.21.15.100:0/159049409 conn(0x7f4eec077180 msgr2=0x7f4eec077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 -- 172.21.15.100:0/159049409 shutdown_connections 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 -- 172.21.15.100:0/159049409 wait complete. 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 Processor -- start 2024-03-12T05:32:59.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.205+0000 7f4ef27d7640 1 -- start start 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef27d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec100260 0x7f4eec19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef27d7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec105330 0x7f4eec19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef27d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec199ef0 0x7f4eec19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef27d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19a9f0 con 0x7f4eec199ef0 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef27d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19ab60 con 0x7f4eec100260 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef27d7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19ad20 con 0x7f4eec105330 2024-03-12T05:32:59.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef0d74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec199ef0 0x7f4eec19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec105330 0x7f4eec19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec100260 0x7f4eec19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef0d74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec199ef0 0x7f4eec19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47898/0 (socket says 172.21.15.100:47898) 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4ef0d74640 1 -- 172.21.15.100:0/808144054 learned_addr learned my addr 172.21.15.100:0/808144054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/808144054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec100260 msgr2=0x7f4eec19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec100260 0x7f4eec19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec199ef0 msgr2=0x7f4eec19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec199ef0 0x7f4eec19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.206+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/808144054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009560 con 0x7f4eec105330 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.207+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec105330 0x7f4eec19fbb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0036040 tx=0x7f4ee002c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.207+0000 7f4ee97fa640 1 -- 172.21.15.100:0/808144054 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00043d0 con 0x7f4eec105330 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.207+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec1a6490 con 0x7f4eec105330 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.207+0000 7f4ee97fa640 1 -- 172.21.15.100:0/808144054 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee00052d0 con 0x7f4eec105330 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.207+0000 7f4ee97fa640 1 -- 172.21.15.100:0/808144054 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0003c90 con 0x7f4eec105330 2024-03-12T05:32:59.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.207+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec1a69a0 con 0x7f4eec105330 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ee97fa640 1 -- 172.21.15.100:0/808144054 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4ee003e450 con 0x7f4eec105330 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 Processor -- start 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 -- start start 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620250 0x556989620630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989629210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x55698962bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569895db960 con 0x556989620250 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698958f310 con 0x556989629750 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698958c840 con 0x556989620b70 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.209+0000 7f4ee97fa640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4ed40a1ba0 0x7f4ed40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989629210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ef0d74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620250 0x556989620630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x55698962bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ef0d74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620250 0x556989620630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47900/0 (socket says 172.21.15.100:47900) 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ef0d74640 1 -- 172.21.15.100:0/41299289 learned_addr learned my addr 172.21.15.100:0/41299289 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989629210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42392/0 (socket says 172.21.15.100:42392) 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4ed40a1ba0 0x7f4ed40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ee97fa640 1 -- 172.21.15.100:0/808144054 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ee00ef800 con 0x7f4eec105330 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 -- 172.21.15.100:0/41299289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 msgr2=0x55698962bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x55698962bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 -- 172.21.15.100:0/41299289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620250 msgr2=0x556989620630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620250 0x556989620630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 -- 172.21.15.100:0/41299289 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55698954a060 con 0x556989620b70 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4eebfff640 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989629210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009c20 tx=0x7f4edc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be17a50d0c4e6e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4ed40a1ba0 0x7f4ed40a4060 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40078b0 tx=0x7f4ee40075e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ed27fc640 1 -- 172.21.15.100:0/41299289 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc004050 con 0x556989620b70 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ed27fc640 1 -- 172.21.15.100:0/41299289 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4edc005500 con 0x556989620b70 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.210+0000 7f4ed27fc640 1 -- 172.21.15.100:0/41299289 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc004c30 con 0x556989620b70 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 -- 172.21.15.100:0/41299289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 msgr2=0x556989629210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989629210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009c20 tx=0x7f4edc011810 comp rx=0 tx=0).stop 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 -- 172.21.15.100:0/41299289 shutdown_connections 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x55698962bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989629210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/41299289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620250 0x556989620630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 -- 172.21.15.100:0/41299289 >> 172.21.15.100:0/41299289 conn(0x55698961a980 msgr2=0x55698961ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 -- 172.21.15.100:0/41299289 shutdown_connections 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 -- 172.21.15.100:0/41299289 wait complete. 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.211+0000 7f4ef53a9740 1 -- start start 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 Processor -- start 2024-03-12T05:32:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 0x5569896267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989626ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x556989627220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556989588020 con 0x556989620b70 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698925b430 con 0x556989629750 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556989217e00 con 0x556989620250 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef53a9740 10 client.0 _ll_register_callbacks cb 0x55698952fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 0x5569896267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989626ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 0x5569896267a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42398/0 (socket says 172.21.15.100:42398) 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 -- 172.21.15.100:0/1655329561 learned_addr learned my addr 172.21.15.100:0/1655329561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x556989627220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 -- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 msgr2=0x556989627220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x556989627220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 -- 172.21.15.100:0/1655329561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 msgr2=0x556989626ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989626ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 -- 172.21.15.100:0/1655329561 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc009850 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x556989627220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989626ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4eebfff640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 0x5569896267a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4edc004910 tx=0x7f4edc004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faf6ece2669c3489 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.212+0000 7f4ecb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/1655329561 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc003a00 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569892877e0 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/1655329561 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4edc0046c0 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/1655329561 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4edc003b60 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 10 client.29164 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5569895866a0 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/1655329561 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5569892877e0 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/1655329561 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4edc021040 con 0x556989620250 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 10 client.29164 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:59.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 10 client.29164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 20 client.29164 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x55698963beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 4 client.29164 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.213+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569892c4720 con 0x556989639a70 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.214+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x55698963beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.214+0000 7f4ecb7fe640 10 client.29164 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.214+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x55698963beb0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1060ec84d50cbfe6 server_cookie=84c69c3c835e72ba in_seq=0 out_seq=0 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.214+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/1655329561 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569892c4720 con 0x556989639a70 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.214+0000 7f4ecb7fe640 10 client.29164 handle_command_reply: tid=0 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.214+0000 7f4ef53a9740 1 client.29164 shutdown 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 msgr2=0x5569896267a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 0x5569896267a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4edc004910 tx=0x7f4edc004430 comp rx=0 tx=0).stop 2024-03-12T05:32:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 shutdown_connections 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x55698963beb0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989629750 0x556989627220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ecb7fe640 0 client.29164 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989626ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/1655329561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620250 0x5569896267a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 >> 172.21.15.100:0/1655329561 conn(0x55698961a980 msgr2=0x55698961c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 shutdown_connections 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 1 -- 172.21.15.100:0/1655329561 wait complete. 2024-03-12T05:32:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.215+0000 7f4ef53a9740 20 client.29164 trim_cache size 0 max 16384 2024-03-12T05:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 Processor -- start 2024-03-12T05:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 -- start start 2024-03-12T05:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x556989628ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989627040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989629750 0x556989640850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698950d130 con 0x556989629750 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698961b980 con 0x556989620250 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569896209a0 con 0x556989620b70 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x556989628ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x556989628ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:59352/0 (socket says 172.21.15.100:59352) 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4eebfff640 1 -- 172.21.15.100:0/2199163773 learned_addr learned my addr 172.21.15.100:0/2199163773 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989627040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989629750 0x556989640850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef0d74640 1 -- 172.21.15.100:0/2199163773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 msgr2=0x556989627040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989627040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef0d74640 1 -- 172.21.15.100:0/2199163773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 msgr2=0x556989628ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x556989628ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.228+0000 7f4ef0d74640 1 -- 172.21.15.100:0/2199163773 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556989522c20 con 0x556989629750 2024-03-12T05:32:59.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.229+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989629750 0x556989640850 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4007e90 tx=0x7f4ee4007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2fb8e2f169f5f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.229+0000 7f4ed1ffb640 1 -- 172.21.15.100:0/2199163773 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004d20 con 0x556989629750 2024-03-12T05:32:59.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.229+0000 7f4ed1ffb640 1 -- 172.21.15.100:0/2199163773 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee4025070 con 0x556989629750 2024-03-12T05:32:59.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.229+0000 7f4ed1ffb640 1 -- 172.21.15.100:0/2199163773 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee402aa70 con 0x556989629750 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 -- 172.21.15.100:0/2199163773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989629750 msgr2=0x556989640850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989629750 0x556989640850 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4007e90 tx=0x7f4ee4007a60 comp rx=0 tx=0).stop 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 -- 172.21.15.100:0/2199163773 shutdown_connections 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989629750 0x556989640850 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989620b70 0x556989627040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/2199163773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x556989628ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 -- 172.21.15.100:0/2199163773 >> 172.21.15.100:0/2199163773 conn(0x55698961a980 msgr2=0x556989615a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 -- 172.21.15.100:0/2199163773 shutdown_connections 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 -- 172.21.15.100:0/2199163773 wait complete. 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 -- start start 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.230+0000 7f4ef53a9740 1 Processor -- start 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x5569896302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989630810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989629750 0x55698962e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698962ee00 con 0x556989620b70 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698962ef70 con 0x556989620250 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698962f130 con 0x556989629750 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef53a9740 10 client.0 _ll_register_callbacks cb 0x7f4edc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989630810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x5569896302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989630810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47940/0 (socket says 172.21.15.100:47940) 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/3605271590 learned_addr learned my addr 172.21.15.100:0/3605271590 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ef0d74640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989629750 0x55698962e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/3605271590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989629750 msgr2=0x55698962e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989629750 0x55698962e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 msgr2=0x5569896302d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x5569896302d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 -- 172.21.15.100:0/3605271590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee4014940 con 0x556989620b70 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eebfff640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x5569896302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4eeb7fe640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989630810 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006010 tx=0x7f4ee0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba32661b9d00bfd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ecb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.231+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/3605271590 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00bc500 con 0x556989620b70 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/3605271590 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ee00c0650 con 0x556989620b70 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4edc005500 con 0x556989620b70 2024-03-12T05:32:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/3605271590 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee00c4ca0 con 0x556989620b70 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 10 client.19902 fetch_fsmap learned FSMap version 30 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5569895488c0 con 0x556989620b70 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/3605271590 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4edc005500 con 0x556989620b70 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/3605271590 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ee003c070 con 0x556989620b70 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 10 client.19902 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 10 client.19902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 20 client.19902 populate_metadata read hostname 'smithi100' 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x556989692fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 4 client.19902 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556989313890 con 0x556989639a70 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.232+0000 7f4eebfff640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x556989692fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ecb7fe640 10 client.19902 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4eebfff640 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x556989692fd0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83753e0cfea6e925 server_cookie=d368eea94e7922d7 in_seq=0 out_seq=0 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ecb7fe640 1 -- 172.21.15.100:0/3605271590 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556989313890 con 0x556989639a70 2024-03-12T05:32:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ecb7fe640 10 client.19902 handle_command_reply: tid=0 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 client.19902 shutdown 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 msgr2=0x556989630810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989630810 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006010 tx=0x7f4ee0000bc0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 shutdown_connections 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556989639a70 0x556989692fd0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556989629750 0x55698962e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556989620b70 0x556989630810 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 --2- 172.21.15.100:0/3605271590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556989620250 0x5569896302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 >> 172.21.15.100:0/3605271590 conn(0x55698961a980 msgr2=0x556989640850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ecb7fe640 0 client.19902 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 shutdown_connections 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 1 -- 172.21.15.100:0/3605271590 wait complete. 2024-03-12T05:32:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.233+0000 7f4ef53a9740 20 client.19902 trim_cache size 0 max 16384 2024-03-12T05:32:59.233 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:32:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4ed40a1ba0 msgr2=0x7f4ed40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4ed40a1ba0 0x7f4ed40a4060 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40078b0 tx=0x7f4ee40075e0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec105330 msgr2=0x7f4eec19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:32:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec105330 0x7f4eec19fbb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0036040 tx=0x7f4ee002c750 comp rx=0 tx=0).stop 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 shutdown_connections 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f4ed40a1ba0 0x7f4ed40a4060 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec199ef0 0x7f4eec19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eec105330 0x7f4eec19fbb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 --2- 172.21.15.100:0/808144054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec100260 0x7f4eec19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 >> 172.21.15.100:0/808144054 conn(0x7f4eec077180 msgr2=0x7f4eec0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 shutdown_connections 2024-03-12T05:32:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:32:59.234+0000 7f4ef27d7640 1 -- 172.21.15.100:0/808144054 wait complete. 2024-03-12T05:32:59.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 240953 26.30 MB/sec execute 342 sec latency 162.866 ms 2024-03-12T05:32:59.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:32:59 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1988953508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:32:59.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:32:59 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1988953508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:32:59.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:32:59 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1988953508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:33:00.239 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:33:00.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:33:00.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:33:00.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 241333 26.30 MB/sec execute 343 sec latency 348.330 ms 2024-03-12T05:33:00.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 -- 172.21.15.100:0/1338287412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 msgr2=0x7f89b81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 --2- 172.21.15.100:0/1338287412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b81004a0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009900 tx=0x7f89ac02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 -- 172.21.15.100:0/1338287412 shutdown_connections 2024-03-12T05:33:00.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 --2- 172.21.15.100:0/1338287412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89b81009e0 0x7f89b810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 --2- 172.21.15.100:0/1338287412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b81004a0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 --2- 172.21.15.100:0/1338287412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89b8106100 0x7f89b81064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 -- 172.21.15.100:0/1338287412 >> 172.21.15.100:0/1338287412 conn(0x7f89b8077110 msgr2=0x7f89b8077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 -- 172.21.15.100:0/1338287412 shutdown_connections 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.363+0000 7f89be864640 1 -- 172.21.15.100:0/1338287412 wait complete. 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 Processor -- start 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 -- start start 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89b81009e0 0x7f89b8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89b8106100 0x7f89b81a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81993a0 con 0x7f89b8100040 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81a3bc0 con 0x7f89b8106100 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89be864640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81a3d00 con 0x7f89b81009e0 2024-03-12T05:33:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47952/0 (socket says 172.21.15.100:47952) 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89b7fff640 1 -- 172.21.15.100:0/1717422875 learned_addr learned my addr 172.21.15.100:0/1717422875 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89bce01640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89b8106100 0x7f89b81a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.364+0000 7f89b77fe640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89b81009e0 0x7f89b8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b7fff640 1 -- 172.21.15.100:0/1717422875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89b81009e0 msgr2=0x7f89b8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b7fff640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89b81009e0 0x7f89b8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b7fff640 1 -- 172.21.15.100:0/1717422875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89b8106100 msgr2=0x7f89b81a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b7fff640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89b8106100 0x7f89b81a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b7fff640 1 -- 172.21.15.100:0/1717422875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac009560 con 0x7f89b8100040 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b7fff640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b8198600 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f89b000ebc0 tx=0x7f89b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b57fa640 1 -- 172.21.15.100:0/1717422875 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00099b0 con 0x7f89b8100040 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b57fa640 1 -- 172.21.15.100:0/1717422875 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89b000eec0 con 0x7f89b8100040 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b81a3f60 con 0x7f89b8100040 2024-03-12T05:33:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89b57fa640 1 -- 172.21.15.100:0/1717422875 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0018b60 con 0x7f89b8100040 2024-03-12T05:33:00.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.365+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b81a4390 con 0x7f89b8100040 2024-03-12T05:33:00.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.367+0000 7f89b57fa640 1 -- 172.21.15.100:0/1717422875 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f89b0018cc0 con 0x7f89b8100040 2024-03-12T05:33:00.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.367+0000 7f89c1433740 1 Processor -- start 2024-03-12T05:33:00.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 -- start start 2024-03-12T05:33:00.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b57fa640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89a00a1ba0 0x7f89a00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb834640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb83e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb7a2f90 con 0x55e3bb834c10 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb7a04c0 con 0x55e3bb834260 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89c1433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb79bca0 con 0x55e3bb83dd10 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89bce01640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89a00a1ba0 0x7f89a00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47968/0 (socket says 172.21.15.100:47968) 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 -- 172.21.15.100:0/652333403 learned_addr learned my addr 172.21.15.100:0/652333403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b7fff640 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb834640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89bce01640 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb83e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b57fa640 1 -- 172.21.15.100:0/1717422875 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89b0014070 con 0x7f89b8100040 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 -- 172.21.15.100:0/652333403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 msgr2=0x55e3bb83e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb83e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 -- 172.21.15.100:0/652333403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 msgr2=0x55e3bb834640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb834640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 -- 172.21.15.100:0/652333403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3bb75e0b0 con 0x55e3bb834c10 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89bce01640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89a00a1ba0 0x7f89a00a4060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89a800baa0 tx=0x7f89a8004230 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.368+0000 7f89b77fe640 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83d740 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009900 tx=0x7f89ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48d2f2a2ff5cc4d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f899e7fc640 1 -- 172.21.15.100:0/652333403 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0048b0 con 0x55e3bb834c10 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f899e7fc640 1 -- 172.21.15.100:0/652333403 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89ac004a10 con 0x55e3bb834c10 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f899e7fc640 1 -- 172.21.15.100:0/652333403 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0050a0 con 0x55e3bb834c10 2024-03-12T05:33:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 -- 172.21.15.100:0/652333403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 msgr2=0x55e3bb83d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83d740 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009900 tx=0x7f89ac0046c0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 -- 172.21.15.100:0/652333403 shutdown_connections 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb83e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83d740 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 --2- 172.21.15.100:0/652333403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb834640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 -- 172.21.15.100:0/652333403 >> 172.21.15.100:0/652333403 conn(0x55e3bb82e840 msgr2=0x55e3bb82ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 -- 172.21.15.100:0/652333403 shutdown_connections 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.369+0000 7f89c1433740 1 -- 172.21.15.100:0/652333403 wait complete. 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 -- start start 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 Processor -- start 2024-03-12T05:33:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb841710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb841c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb842190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb740f40 con 0x55e3bb834c10 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb46f4b0 con 0x55e3bb83dd10 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb721670 con 0x55e3bb834260 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89c1433740 10 client.0 _ll_register_callbacks cb 0x55e3bb78ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb841c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb841c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47982/0 (socket says 172.21.15.100:47982) 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb841710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89b77fe640 1 -- 172.21.15.100:0/755805917 learned_addr learned my addr 172.21.15.100:0/755805917 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.370+0000 7f89bce01640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb842190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b77fe640 1 -- 172.21.15.100:0/755805917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 msgr2=0x55e3bb841710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b77fe640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb841710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b77fe640 1 -- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 msgr2=0x55e3bb842190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b77fe640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb842190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b77fe640 1 -- 172.21.15.100:0/755805917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b810d7a0 con 0x55e3bb834c10 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b7fff640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb841710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89b77fe640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb841c50 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f89ac02cab0 tx=0x7f89ac02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cd8d2c9aab5f1d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89977fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89977fe640 1 -- 172.21.15.100:0/755805917 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac005270 con 0x55e3bb834c10 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89977fe640 1 -- 172.21.15.100:0/755805917 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89ac02cc90 con 0x55e3bb834c10 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3bb775f40 con 0x55e3bb834c10 2024-03-12T05:33:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89977fe640 1 -- 172.21.15.100:0/755805917 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac0053d0 con 0x55e3bb834c10 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89c1433740 10 client.19914 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e3bb79a320 con 0x55e3bb834c10 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89977fe640 1 -- 172.21.15.100:0/755805917 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3bb775f40 con 0x55e3bb834c10 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.371+0000 7f89977fe640 1 -- 172.21.15.100:0/755805917 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f89ac058070 con 0x55e3bb834c10 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89c1433740 10 client.19914 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89c1433740 10 client.19914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89c1433740 20 client.19914 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89c1433740 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb851be0 0x55e3bb854000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89c1433740 4 client.19914 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3bb792310 con 0x55e3bb851be0 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89b7fff640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb851be0 0x55e3bb854000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89b7fff640 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb851be0 0x55e3bb854000 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2551d7392acb0b0e server_cookie=62cb206c005050cd in_seq=0 out_seq=0 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.372+0000 7f89977fe640 10 client.19914 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89977fe640 1 -- 172.21.15.100:0/755805917 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3bb792310 con 0x55e3bb851be0 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89977fe640 10 client.19914 handle_command_reply: tid=0 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 client.19914 shutdown 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 msgr2=0x55e3bb841c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb841c50 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f89ac02cab0 tx=0x7f89ac02bcd0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 shutdown_connections 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb851be0 0x55e3bb854000 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb842190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb841c50 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89977fe640 0 client.19914 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 --2- 172.21.15.100:0/755805917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb841710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 >> 172.21.15.100:0/755805917 conn(0x55e3bb82e840 msgr2=0x55e3bb8307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 shutdown_connections 2024-03-12T05:33:00.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 1 -- 172.21.15.100:0/755805917 wait complete. 2024-03-12T05:33:00.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.373+0000 7f89c1433740 20 client.19914 trim_cache size 0 max 16384 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 Processor -- start 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 -- start start 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb843e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb841fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb82f640 con 0x55e3bb834c10 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb828dc0 con 0x55e3bb834260 2024-03-12T05:33:00.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89c1433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb859230 con 0x55e3bb83dd10 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb843e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47984/0 (socket says 172.21.15.100:47984) 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89b77fe640 1 -- 172.21.15.100:0/354080491 learned_addr learned my addr 172.21.15.100:0/354080491 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.394+0000 7f89bce01640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb841fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f89b77fe640 1 -- 172.21.15.100:0/354080491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 msgr2=0x55e3bb841fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f89b77fe640 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb841fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f89b77fe640 1 -- 172.21.15.100:0/354080491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 msgr2=0x55e3bb843e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f89b77fe640 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb843e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f89b77fe640 1 -- 172.21.15.100:0/354080491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3bb736c00 con 0x55e3bb834c10 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f89b77fe640 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83f100 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004e30 tx=0x7f89ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706b7744fd1a8481 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f899dffb640 1 -- 172.21.15.100:0/354080491 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac03f6e0 con 0x55e3bb834c10 2024-03-12T05:33:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f899dffb640 1 -- 172.21.15.100:0/354080491 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89ac005d50 con 0x55e3bb834c10 2024-03-12T05:33:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.395+0000 7f899dffb640 1 -- 172.21.15.100:0/354080491 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0054b0 con 0x55e3bb834c10 2024-03-12T05:33:00.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 -- 172.21.15.100:0/354080491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 msgr2=0x55e3bb83f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83f100 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004e30 tx=0x7f89ac000bc0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 -- 172.21.15.100:0/354080491 shutdown_connections 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb83dd10 0x55e3bb841fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb83f100 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 --2- 172.21.15.100:0/354080491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb834260 0x55e3bb843e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 -- 172.21.15.100:0/354080491 >> 172.21.15.100:0/354080491 conn(0x55e3bb82e840 msgr2=0x55e3bb83c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 -- 172.21.15.100:0/354080491 shutdown_connections 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 -- 172.21.15.100:0/354080491 wait complete. 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.396+0000 7f89c1433740 1 -- start start 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 Processor -- start 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb8487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb848d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb846cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb8472e0 con 0x55e3bb834c10 2024-03-12T05:33:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb847450 con 0x55e3bb83dd10 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bb847610 con 0x55e3bb834260 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb848d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89c1433740 10 client.0 _ll_register_callbacks cb 0x7f89ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb8487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb848d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47986/0 (socket says 172.21.15.100:47986) 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb8487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:42480/0 (socket says 172.21.15.100:42480) 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 -- 172.21.15.100:0/2043113010 learned_addr learned my addr 172.21.15.100:0/2043113010 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89bce01640 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb846cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 -- 172.21.15.100:0/2043113010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 msgr2=0x55e3bb8487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb8487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 -- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 msgr2=0x55e3bb846cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb846cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.397+0000 7f89b77fe640 1 -- 172.21.15.100:0/2043113010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3bb79a320 con 0x55e3bb834c10 2024-03-12T05:33:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89b77fe640 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb848d00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004650 tx=0x7f89ac004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91f4d81c7eb1ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89977fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89977fe640 1 -- 172.21.15.100:0/2043113010 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac068070 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89977fe640 1 -- 172.21.15.100:0/2043113010 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89ac000e40 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3bb7c1900 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89977fe640 1 -- 172.21.15.100:0/2043113010 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac04a450 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89c1433740 10 client.19920 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.398+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e3bb75c910 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89977fe640 1 -- 172.21.15.100:0/2043113010 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3bb7c1900 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89977fe640 1 -- 172.21.15.100:0/2043113010 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f89ac06b030 con 0x55e3bb834c10 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89c1433740 10 client.19920 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89c1433740 10 client.19920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89c1433740 20 client.19920 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:00.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89c1433740 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb8a8da0 0x55e3bb8ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89c1433740 4 client.19920 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e3bb49b860 con 0x55e3bb8a8da0 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.399+0000 7f89b7fff640 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb8a8da0 0x55e3bb8ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.400+0000 7f89977fe640 10 client.19920 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.400+0000 7f89b7fff640 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb8a8da0 0x55e3bb8ab190 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf8b0cd625f12ee server_cookie=619c57cbb8202fb1 in_seq=0 out_seq=0 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89977fe640 1 -- 172.21.15.100:0/2043113010 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e3bb49b860 con 0x55e3bb8a8da0 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89977fe640 10 client.19920 handle_command_reply: tid=0 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 client.19920 shutdown 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 msgr2=0x55e3bb848d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb848d00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004650 tx=0x7f89ac004560 comp rx=0 tx=0).stop 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 shutdown_connections 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55e3bb8a8da0 0x55e3bb8ab190 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e3bb83dd10 0x55e3bb846cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3bb834c10 0x55e3bb848d00 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 --2- 172.21.15.100:0/2043113010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e3bb834260 0x55e3bb8487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89977fe640 0 client.19920 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 >> 172.21.15.100:0/2043113010 conn(0x55e3bb82e840 msgr2=0x55e3bb841ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 shutdown_connections 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.401+0000 7f89c1433740 1 -- 172.21.15.100:0/2043113010 wait complete. 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89c1433740 20 client.19920 trim_cache size 0 max 16384 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:33:00.401 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:33:00.402 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "129b0517-475b-427c-9a6f-a0f1fd11991d", 2024-03-12T05:33:00.402 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:33:00.402 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:33:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89a00a1ba0 msgr2=0x7f89a00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89a00a1ba0 0x7f89a00a4060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89a800baa0 tx=0x7f89a8004230 comp rx=0 tx=0).stop 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 msgr2=0x7f89b8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b8198600 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f89b000ebc0 tx=0x7f89b000c6a0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 shutdown_connections 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89a00a1ba0 0x7f89a00a4060 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89b8106100 0x7f89b81a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89b81009e0 0x7f89b8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 --2- 172.21.15.100:0/1717422875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b8100040 0x7f89b8198600 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.402+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 >> 172.21.15.100:0/1717422875 conn(0x7f89b8077110 msgr2=0x7f89b80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.403+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 shutdown_connections 2024-03-12T05:33:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.403+0000 7f89be864640 1 -- 172.21.15.100:0/1717422875 wait complete. 2024-03-12T05:33:00.409 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:00 smithi100 ceph-mon[32210]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 408 op/s 2024-03-12T05:33:00.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:33:00.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:33:00.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:00 smithi156 ceph-mon[36270]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 408 op/s 2024-03-12T05:33:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:00 smithi161 ceph-mon[36043]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 408 op/s 2024-03-12T05:33:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 -- 172.21.15.100:0/2151039070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7734100270 msgr2=0x7f7734100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 --2- 172.21.15.100:0/2151039070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7734100270 0x7f7734100650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7730009900 tx=0x7f773002bc30 comp rx=0 tx=0).stop 2024-03-12T05:33:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 -- 172.21.15.100:0/2151039070 shutdown_connections 2024-03-12T05:33:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 --2- 172.21.15.100:0/2151039070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734105340 0x7f773410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 --2- 172.21.15.100:0/2151039070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7734100c20 0x7f7734104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 --2- 172.21.15.100:0/2151039070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7734100270 0x7f7734100650 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 -- 172.21.15.100:0/2151039070 >> 172.21.15.100:0/2151039070 conn(0x7f7734077230 msgr2=0x7f7734077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 -- 172.21.15.100:0/2151039070 shutdown_connections 2024-03-12T05:33:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.580+0000 7f773c0c0640 1 -- 172.21.15.100:0/2151039070 wait complete. 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 Processor -- start 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 -- start start 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 0x7f773419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7734105340 0x7f773419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77341997e0 0x7f7734199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419a290 con 0x7f7734105340 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419a400 con 0x7f7734100270 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773c0c0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419a570 con 0x7f77341997e0 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f7739e5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 0x7f773419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773965b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7734105340 0x7f773419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f7739e5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 0x7f773419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:45484/0 (socket says 172.21.15.100:45484) 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f7739e5c640 1 -- 172.21.15.100:0/4215467725 learned_addr learned my addr 172.21.15.100:0/4215467725 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.581+0000 7f773a65d640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77341997e0 0x7f7734199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f7739e5c640 1 -- 172.21.15.100:0/4215467725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77341997e0 msgr2=0x7f7734199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f7739e5c640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77341997e0 0x7f7734199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f7739e5c640 1 -- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7734105340 msgr2=0x7f773419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f7739e5c640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7734105340 0x7f773419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f7739e5c640 1 -- 172.21.15.100:0/4215467725 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009490 con 0x7f7734100270 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f7739e5c640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 0x7f773419f620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7730005940 tx=0x7f773002c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f772affd640 1 -- 172.21.15.100:0/4215467725 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730042c60 con 0x7f7734100270 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f772affd640 1 -- 172.21.15.100:0/4215467725 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7730046070 con 0x7f7734100270 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730009560 con 0x7f7734100270 2024-03-12T05:33:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f772affd640 1 -- 172.21.15.100:0/4215467725 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300035f0 con 0x7f7734100270 2024-03-12T05:33:00.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.582+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7734107190 con 0x7f7734100270 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 Processor -- start 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 -- start start 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf043f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0448270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 0x556cf044aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf03ae280 con 0x556cf04487b0 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf03ab7b0 con 0x556cf043fbd0 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f772affd640 1 -- 172.21.15.100:0/4215467725 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f77300040c0 con 0x7f7734100270 2024-03-12T05:33:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.585+0000 7f773ed6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf03a6f90 con 0x556cf043f2b0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 0x556cf044aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 0x556cf044aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36158/0 (socket says 172.21.15.100:36158) 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 -- 172.21.15.100:0/2980729339 learned_addr learned my addr 172.21.15.100:0/2980729339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f7739e5c640 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0448270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773a65d640 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf043f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 -- 172.21.15.100:0/2980729339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 msgr2=0x556cf043f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf043f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 -- 172.21.15.100:0/2980729339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 msgr2=0x556cf0448270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0448270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 -- 172.21.15.100:0/2980729339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf0365570 con 0x556cf04487b0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773965b640 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 0x556cf044aba0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7724009db0 tx=0x7f7724011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f364f43b044d274 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f772affd640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f77200a1ba0 0x7f77200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f771ffff640 1 -- 172.21.15.100:0/2980729339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724004230 con 0x556cf04487b0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f773a65d640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f77200a1ba0 0x7f77200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f772affd640 1 -- 172.21.15.100:0/4215467725 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7730048070 con 0x7f7734100270 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f771ffff640 1 -- 172.21.15.100:0/2980729339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7724004390 con 0x556cf04487b0 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.586+0000 7f771ffff640 1 -- 172.21.15.100:0/2980729339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240035f0 con 0x556cf04487b0 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 -- 172.21.15.100:0/2980729339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 msgr2=0x556cf044aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773a65d640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f77200a1ba0 0x7f77200a4060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f772c005fd0 tx=0x7f772c009500 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 0x556cf044aba0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7724009db0 tx=0x7f7724011810 comp rx=0 tx=0).stop 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 -- 172.21.15.100:0/2980729339 shutdown_connections 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf04487b0 0x556cf044aba0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0448270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2980729339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf043f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 -- 172.21.15.100:0/2980729339 >> 172.21.15.100:0/2980729339 conn(0x556cf0439990 msgr2=0x556cf0439d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 -- 172.21.15.100:0/2980729339 shutdown_connections 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 -- 172.21.15.100:0/2980729339 wait complete. 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.587+0000 7f773ed6c740 1 -- start start 2024-03-12T05:33:00.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 Processor -- start 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf04480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0445470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf04459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf007a430 con 0x556cf043f2b0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf0036e00 con 0x556cf043fbd0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf032bfd0 con 0x556cf04487b0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773ed6c740 10 client.0 _ll_register_callbacks cb 0x556cf0381fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f7739e5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf04480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773a65d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf04459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f773965b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0445470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f7739e5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf04480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36174/0 (socket says 172.21.15.100:36174) 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.588+0000 7f7739e5c640 1 -- 172.21.15.100:0/1466861413 learned_addr learned my addr 172.21.15.100:0/1466861413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773965b640 1 -- 172.21.15.100:0/1466861413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 msgr2=0x556cf04459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773965b640 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf04459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773965b640 1 -- 172.21.15.100:0/1466861413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 msgr2=0x556cf04480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773965b640 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf04480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773965b640 1 -- 172.21.15.100:0/1466861413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77240099e0 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773965b640 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0445470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7724011d10 tx=0x7f7724004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0f5f21ee2004a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f771d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f771d7fa640 1 -- 172.21.15.100:0/1466861413 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724005470 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f771d7fa640 1 -- 172.21.15.100:0/1466861413 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f77240055d0 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cf00a67e0 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f771d7fa640 1 -- 172.21.15.100:0/1466861413 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7724024420 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773ed6c740 10 client.29267 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.589+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556cf03a5610 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f771d7fa640 1 -- 172.21.15.100:0/1466861413 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556cf00a67e0 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f771d7fa640 1 -- 172.21.15.100:0/1466861413 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77240196c0 con 0x556cf043fbd0 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f773ed6c740 10 client.29267 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:00.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f773ed6c740 10 client.29267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f773ed6c740 20 client.29267 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f773ed6c740 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf0458890 0x556cf045acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f773ed6c740 4 client.29267 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cf00e3720 con 0x556cf0458890 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.590+0000 7f7739e5c640 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf0458890 0x556cf045acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f771d7fa640 10 client.29267 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f7739e5c640 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf0458890 0x556cf045acd0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9966f30e09d0af server_cookie=968828c9303d4cb3 in_seq=0 out_seq=0 2024-03-12T05:33:00.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f771d7fa640 1 -- 172.21.15.100:0/1466861413 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cf00e3720 con 0x556cf0458890 2024-03-12T05:33:00.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f771d7fa640 10 client.29267 handle_command_reply: tid=0 2024-03-12T05:33:00.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f773ed6c740 1 client.29267 shutdown 2024-03-12T05:33:00.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 msgr2=0x556cf0445470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.591+0000 7f773ed6c740 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0445470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7724011d10 tx=0x7f7724004e70 comp rx=0 tx=0).stop 2024-03-12T05:33:00.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 shutdown_connections 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf0458890 0x556cf045acd0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf04459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf0445470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 --2- 172.21.15.100:0/1466861413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf04480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 >> 172.21.15.100:0/1466861413 conn(0x556cf0439990 msgr2=0x556cf043b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f771d7fa640 0 client.29267 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 shutdown_connections 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 1 -- 172.21.15.100:0/1466861413 wait complete. 2024-03-12T05:33:00.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.592+0000 7f773ed6c740 20 client.29267 trim_cache size 0 max 16384 2024-03-12T05:33:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.609+0000 7f773ed6c740 1 Processor -- start 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.609+0000 7f773ed6c740 1 -- start start 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.609+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf0447ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.609+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 0x556cf04480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.609+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf04487b0 0x556cf044f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.609+0000 7f773965b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 0x556cf04480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773ed6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf0435320 con 0x556cf043fbd0 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 0x556cf04480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36176/0 (socket says 172.21.15.100:36176) 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 -- 172.21.15.100:0/2883745490 learned_addr learned my addr 172.21.15.100:0/2883745490 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf04553a0 con 0x556cf04487b0 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf043ec80 con 0x556cf043f2b0 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f7739e5c640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf0447ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773a65d640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf04487b0 0x556cf044f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 -- 172.21.15.100:0/2883745490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 msgr2=0x556cf0447ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf0447ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 -- 172.21.15.100:0/2883745490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf04487b0 msgr2=0x556cf044f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf04487b0 0x556cf044f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 -- 172.21.15.100:0/2883745490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf0341ac0 con 0x556cf043fbd0 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773a65d640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf04487b0 0x556cf044f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f7739e5c640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf0447ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f773965b640 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 0x556cf04480e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7724019a20 tx=0x7f7724009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8663f78d00950b27 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f771f7fe640 1 -- 172.21.15.100:0/2883745490 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724005470 con 0x556cf043fbd0 2024-03-12T05:33:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f771f7fe640 1 -- 172.21.15.100:0/2883745490 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7724004ee0 con 0x556cf043fbd0 2024-03-12T05:33:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.610+0000 7f771f7fe640 1 -- 172.21.15.100:0/2883745490 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724024660 con 0x556cf043fbd0 2024-03-12T05:33:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 msgr2=0x556cf04480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 0x556cf04480e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7724019a20 tx=0x7f7724009f60 comp rx=0 tx=0).stop 2024-03-12T05:33:00.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 shutdown_connections 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf04487b0 0x556cf044f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043fbd0 0x556cf04480e0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 --2- 172.21.15.100:0/2883745490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf043f2b0 0x556cf0447ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.611+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 >> 172.21.15.100:0/2883745490 conn(0x556cf0439990 msgr2=0x556cf045f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 shutdown_connections 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 -- 172.21.15.100:0/2883745490 wait complete. 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 -- start start 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 Processor -- start 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf044f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf044fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf0441b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf0442210 con 0x556cf043f2b0 2024-03-12T05:33:00.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf0442380 con 0x556cf043fbd0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf0442540 con 0x556cf04487b0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.612+0000 7f773ed6c740 10 client.0 _ll_register_callbacks cb 0x7f7724009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf044f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf044f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36190/0 (socket says 172.21.15.100:36190) 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 -- 172.21.15.100:0/70137047 learned_addr learned my addr 172.21.15.100:0/70137047 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f773965b640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf044fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f773a65d640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf0441b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 -- 172.21.15.100:0/70137047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 msgr2=0x556cf0441b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf0441b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 -- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 msgr2=0x556cf044fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf044fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 -- 172.21.15.100:0/70137047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7734107190 con 0x556cf043f2b0 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f773a65d640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf0441b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f7739e5c640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf044f690 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f77300f5280 tx=0x7f77300af040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87726bdcb483dec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f771d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f771d7fa640 1 -- 172.21.15.100:0/70137047 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300c4010 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f771d7fa640 1 -- 172.21.15.100:0/70137047 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f77300c4db0 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cf0132890 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f771d7fa640 1 -- 172.21.15.100:0/70137047 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77300c46b0 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f773ed6c740 10 client.19938 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.613+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556cf03a5610 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f771d7fa640 1 -- 172.21.15.100:0/70137047 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556cf0132890 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f771d7fa640 1 -- 172.21.15.100:0/70137047 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77300ff040 con 0x556cf043f2b0 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773ed6c740 10 client.19938 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773ed6c740 10 client.19938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773ed6c740 20 client.19938 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773ed6c740 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf04afd60 0x556cf04b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:00.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773ed6c740 4 client.19938 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556cf00954e0 con 0x556cf04afd60 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773965b640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf04afd60 0x556cf04b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f773965b640 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf04afd60 0x556cf04b2150 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9edd311b201b19 server_cookie=511d724d84020cb4 in_seq=0 out_seq=0 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.614+0000 7f771d7fa640 10 client.19938 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f771d7fa640 1 -- 172.21.15.100:0/70137047 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556cf00954e0 con 0x556cf04afd60 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f771d7fa640 10 client.19938 handle_command_reply: tid=0 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 client.19938 shutdown 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 msgr2=0x556cf044f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf044f690 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f77300f5280 tx=0x7f77300af040 comp rx=0 tx=0).stop 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 shutdown_connections 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x556cf04afd60 0x556cf04b2150 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf04487b0 0x556cf0441b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556cf043fbd0 0x556cf044fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f771d7fa640 0 client.19938 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 --2- 172.21.15.100:0/70137047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556cf043f2b0 0x556cf044f690 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 >> 172.21.15.100:0/70137047 conn(0x556cf0439990 msgr2=0x556cf0443910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 shutdown_connections 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 1 -- 172.21.15.100:0/70137047 wait complete. 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.615+0000 7f773ed6c740 20 client.19938 trim_cache size 0 max 16384 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout: "129b0517-475b-427c-9a6f-a0f1fd11991d": { 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "129b0517-475b-427c-9a6f-a0f1fd11991d", 2024-03-12T05:33:00.614 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:33:00.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:33:00.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:33:00.615 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:33:00.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f77200a1ba0 msgr2=0x7f77200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f77200a1ba0 0x7f77200a4060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f772c005fd0 tx=0x7f772c009500 comp rx=0 tx=0).stop 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 msgr2=0x7f773419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 0x7f773419f620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7730005940 tx=0x7f773002c670 comp rx=0 tx=0).stop 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 shutdown_connections 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f77200a1ba0 0x7f77200a4060 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77341997e0 0x7f7734199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7734105340 0x7f773419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 --2- 172.21.15.100:0/4215467725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7734100270 0x7f773419f620 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 >> 172.21.15.100:0/4215467725 conn(0x7f7734077230 msgr2=0x7f77340ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 shutdown_connections 2024-03-12T05:33:00.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:00.616+0000 7f773c0c0640 1 -- 172.21.15.100:0/4215467725 wait complete. 2024-03-12T05:33:00.621 INFO:tasks.cephfs.filesystem:scrub status for tag:129b0517-475b-427c-9a6f-a0f1fd11991d - {'path': '/', 'tag': '129b0517-475b-427c-9a6f-a0f1fd11991d', 'options': 'recursive,force'} 2024-03-12T05:33:01.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 241600 26.26 MB/sec execute 344 sec latency 220.109 ms 2024-03-12T05:33:01.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:33:01.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:01 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:01 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:01.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:01.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:01.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:01 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:02.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 241827 26.21 MB/sec execute 345 sec latency 176.789 ms 2024-03-12T05:33:02.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:02 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:02.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:02 smithi156 ceph-mon[36270]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 594 op/s 2024-03-12T05:33:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:02 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:02 smithi161 ceph-mon[36043]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 594 op/s 2024-03-12T05:33:02.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:02 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:02.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:02 smithi100 ceph-mon[32210]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 594 op/s 2024-03-12T05:33:03.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 242147 26.18 MB/sec execute 346 sec latency 194.811 ms 2024-03-12T05:33:04.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 242517 26.17 MB/sec execute 347 sec latency 192.010 ms 2024-03-12T05:33:04.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:04 smithi156 ceph-mon[36270]: pgmap v660: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-03-12T05:33:04.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:04 smithi161 ceph-mon[36043]: pgmap v660: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-03-12T05:33:04.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:04 smithi100 ceph-mon[32210]: pgmap v660: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-03-12T05:33:05.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 242867 26.13 MB/sec execute 348 sec latency 294.765 ms 2024-03-12T05:33:06.245 INFO:tasks.workunit.client.0.smithi100.stdout: 10 243324 26.13 MB/sec execute 349 sec latency 376.526 ms 2024-03-12T05:33:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:06 smithi156 ceph-mon[36270]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.0 MiB/s wr, 498 op/s 2024-03-12T05:33:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:06 smithi161 ceph-mon[36043]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.0 MiB/s wr, 498 op/s 2024-03-12T05:33:06.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:06 smithi100 ceph-mon[32210]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.0 MiB/s wr, 498 op/s 2024-03-12T05:33:07.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 243775 26.14 MB/sec execute 350 sec latency 149.612 ms 2024-03-12T05:33:08.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 244280 26.14 MB/sec execute 351 sec latency 179.982 ms 2024-03-12T05:33:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:08 smithi156 ceph-mon[36270]: pgmap v662: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 836 op/s 2024-03-12T05:33:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:08 smithi161 ceph-mon[36043]: pgmap v662: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 836 op/s 2024-03-12T05:33:08.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:08 smithi100 ceph-mon[32210]: pgmap v662: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 836 op/s 2024-03-12T05:33:09.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 244809 26.15 MB/sec execute 352 sec latency 159.654 ms 2024-03-12T05:33:10.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 245124 26.13 MB/sec execute 353 sec latency 999.810 ms 2024-03-12T05:33:11.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:10 smithi100 ceph-mon[32210]: osdmap e118: 12 total, 12 up, 12 in 2024-03-12T05:33:11.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:10 smithi100 ceph-mon[32210]: pgmap v664: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 771 op/s 2024-03-12T05:33:11.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:10 smithi156 ceph-mon[36270]: osdmap e118: 12 total, 12 up, 12 in 2024-03-12T05:33:11.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:10 smithi156 ceph-mon[36270]: pgmap v664: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 771 op/s 2024-03-12T05:33:11.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:10 smithi161 ceph-mon[36043]: osdmap e118: 12 total, 12 up, 12 in 2024-03-12T05:33:11.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:10 smithi161 ceph-mon[36043]: pgmap v664: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 771 op/s 2024-03-12T05:33:11.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 245527 26.12 MB/sec execute 354 sec latency 1999.847 ms 2024-03-12T05:33:12.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 245907 26.08 MB/sec execute 355 sec latency 2999.874 ms 2024-03-12T05:33:12.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:12 smithi156 ceph-mon[36270]: pgmap v665: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 673 op/s 2024-03-12T05:33:12.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:12 smithi161 ceph-mon[36043]: pgmap v665: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 673 op/s 2024-03-12T05:33:12.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:12 smithi100 ceph-mon[32210]: pgmap v665: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 673 op/s 2024-03-12T05:33:13.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 246468 26.09 MB/sec execute 356 sec latency 3999.904 ms 2024-03-12T05:33:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 247049 26.12 MB/sec execute 357 sec latency 4940.433 ms 2024-03-12T05:33:14.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:14 smithi156 ceph-mon[36270]: pgmap v666: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 638 op/s 2024-03-12T05:33:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:14 smithi161 ceph-mon[36043]: pgmap v666: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 638 op/s 2024-03-12T05:33:14.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:14 smithi100 ceph-mon[32210]: pgmap v666: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 638 op/s 2024-03-12T05:33:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 247492 26.15 MB/sec execute 358 sec latency 1193.518 ms 2024-03-12T05:33:15.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:15 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:15 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:33:15.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:15 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:33:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 247794 26.12 MB/sec execute 359 sec latency 2193.551 ms 2024-03-12T05:33:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:16 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:33:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:16 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:33:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:16 smithi156 ceph-mon[36270]: osdmap e119: 12 total, 12 up, 12 in 2024-03-12T05:33:16.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:16 smithi156 ceph-mon[36270]: pgmap v668: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 290 op/s 2024-03-12T05:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:16 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:16 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:16 smithi161 ceph-mon[36043]: osdmap e119: 12 total, 12 up, 12 in 2024-03-12T05:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:16 smithi161 ceph-mon[36043]: pgmap v668: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 290 op/s 2024-03-12T05:33:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:16 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:33:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:16 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:33:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:16 smithi100 ceph-mon[32210]: osdmap e119: 12 total, 12 up, 12 in 2024-03-12T05:33:16.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:16 smithi100 ceph-mon[32210]: pgmap v668: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 290 op/s 2024-03-12T05:33:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 248073 26.08 MB/sec execute 360 sec latency 2445.516 ms 2024-03-12T05:33:18.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 248418 26.05 MB/sec execute 361 sec latency 2346.119 ms 2024-03-12T05:33:18.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:18 smithi156 ceph-mon[36270]: pgmap v669: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 585 op/s 2024-03-12T05:33:18.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:18 smithi161 ceph-mon[36043]: pgmap v669: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 585 op/s 2024-03-12T05:33:18.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:18 smithi100 ceph-mon[32210]: pgmap v669: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 585 op/s 2024-03-12T05:33:19.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 248767 26.01 MB/sec execute 362 sec latency 3346.156 ms 2024-03-12T05:33:20.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 249276 26.03 MB/sec execute 363 sec latency 3381.389 ms 2024-03-12T05:33:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:20 smithi156 ceph-mon[36270]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-03-12T05:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:20 smithi161 ceph-mon[36043]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-03-12T05:33:20.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:20 smithi100 ceph-mon[32210]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-03-12T05:33:21.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 249557 26.00 MB/sec execute 364 sec latency 364.761 ms 2024-03-12T05:33:22.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 249939 25.97 MB/sec execute 365 sec latency 110.803 ms 2024-03-12T05:33:22.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:22 smithi156 ceph-mon[36270]: pgmap v671: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 509 op/s 2024-03-12T05:33:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:22 smithi161 ceph-mon[36043]: pgmap v671: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 509 op/s 2024-03-12T05:33:22.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:22 smithi100 ceph-mon[32210]: pgmap v671: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 509 op/s 2024-03-12T05:33:23.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 250418 25.95 MB/sec execute 366 sec latency 135.124 ms 2024-03-12T05:33:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 250901 25.97 MB/sec execute 367 sec latency 157.116 ms 2024-03-12T05:33:24.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:24 smithi156 ceph-mon[36270]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 502 op/s 2024-03-12T05:33:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:24 smithi161 ceph-mon[36043]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 502 op/s 2024-03-12T05:33:24.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:24 smithi100 ceph-mon[32210]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 502 op/s 2024-03-12T05:33:25.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 251465 26.00 MB/sec execute 368 sec latency 149.652 ms 2024-03-12T05:33:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 251774 25.98 MB/sec execute 369 sec latency 348.560 ms 2024-03-12T05:33:26.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:26 smithi156 ceph-mon[36270]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 499 op/s 2024-03-12T05:33:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:26 smithi161 ceph-mon[36043]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 499 op/s 2024-03-12T05:33:26.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:26 smithi100 ceph-mon[32210]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 499 op/s 2024-03-12T05:33:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 252150 25.96 MB/sec execute 370 sec latency 199.642 ms 2024-03-12T05:33:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 252450 25.93 MB/sec execute 371 sec latency 155.589 ms 2024-03-12T05:33:28.365 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:28 smithi100 ceph-mon[32210]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 610 op/s 2024-03-12T05:33:28.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:28 smithi156 ceph-mon[36270]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 610 op/s 2024-03-12T05:33:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:28 smithi161 ceph-mon[36043]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 610 op/s 2024-03-12T05:33:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 252784 25.90 MB/sec execute 372 sec latency 213.146 ms 2024-03-12T05:33:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 253238 25.88 MB/sec execute 373 sec latency 122.134 ms 2024-03-12T05:33:30.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:30 smithi156 ceph-mon[36270]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.5 MiB/s wr, 401 op/s 2024-03-12T05:33:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:30 smithi161 ceph-mon[36043]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.5 MiB/s wr, 401 op/s 2024-03-12T05:33:30.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:30 smithi100 ceph-mon[32210]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.5 MiB/s wr, 401 op/s 2024-03-12T05:33:30.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:33:30.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3147977452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc1035a0 msgr2=0x7f5abc103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3147977452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc1035a0 0x7f5abc103980 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009930 tx=0x7f5aac02bc30 comp rx=0 tx=0).stop 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3147977452 shutdown_connections 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3147977452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc108670 0x7f5abc110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3147977452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5abc103f50 0x7f5abc107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3147977452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc1035a0 0x7f5abc103980 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.749+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3147977452 >> 172.21.15.100:0/3147977452 conn(0x7f5abc0fee40 msgr2=0x7f5abc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.750+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3147977452 shutdown_connections 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.750+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3147977452 wait complete. 2024-03-12T05:33:30.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.750+0000 7f5ac0a81640 1 Processor -- start 2024-03-12T05:33:30.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.750+0000 7f5ac0a81640 1 -- start start 2024-03-12T05:33:30.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ac0a81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc1035a0 0x7f5abc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ac0a81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5abc108670 0x7f5abc19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ac0a81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 0x7f5abc199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ac0a81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc19a300 con 0x7f5abc199850 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ac0a81640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc19a470 con 0x7f5abc1035a0 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ac0a81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc19a5e0 con 0x7f5abc108670 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 0x7f5abc199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5aba59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc1035a0 0x7f5abc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 0x7f5abc199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33360/0 (socket says 172.21.15.100:33360) 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 -- 172.21.15.100:0/3895311753 learned_addr learned my addr 172.21.15.100:0/3895311753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5abc108670 0x7f5abc19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 -- 172.21.15.100:0/3895311753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5abc108670 msgr2=0x7f5abc19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5abc108670 0x7f5abc19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 -- 172.21.15.100:0/3895311753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc1035a0 msgr2=0x7f5abc19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc1035a0 0x7f5abc19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 -- 172.21.15.100:0/3895311753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac009560 con 0x7f5abc199850 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5aba59c640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc1035a0 0x7f5abc19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.751+0000 7f5abad9d640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 0x7f5abc199d00 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400eb10 tx=0x7f5ab400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.752+0000 7f5ab37fe640 1 -- 172.21.15.100:0/3895311753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004b20 con 0x7f5abc199850 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.752+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc1a6540 con 0x7f5abc199850 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.752+0000 7f5ab37fe640 1 -- 172.21.15.100:0/3895311753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab4004c80 con 0x7f5abc199850 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.752+0000 7f5ab37fe640 1 -- 172.21.15.100:0/3895311753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4010650 con 0x7f5abc199850 2024-03-12T05:33:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.752+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc1a6a20 con 0x7f5abc199850 2024-03-12T05:33:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.754+0000 7f5ab37fe640 1 -- 172.21.15.100:0/3895311753 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5ab4005020 con 0x7f5abc199850 2024-03-12T05:33:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.755+0000 7f5ac3655740 1 Processor -- start 2024-03-12T05:33:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.755+0000 7f5ac3655740 1 -- start start 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab37fe640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5aa80a1ba0 0x7f5aa80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 0x56453c5f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ac3655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c554280 con 0x56453c5e5bd0 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ac3655740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c5517b0 con 0x56453c5ee7b0 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ac3655740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c54cf90 con 0x56453c5e52b0 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5aa80a1ba0 0x7f5aa80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 0x56453c5f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab37fe640 1 -- 172.21.15.100:0/3895311753 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ab4014070 con 0x7f5abc199850 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 0x56453c5f0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44676/0 (socket says 172.21.15.100:44676) 2024-03-12T05:33:30.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33364/0 (socket says 172.21.15.100:33364) 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 -- 172.21.15.100:0/2384309468 learned_addr learned my addr 172.21.15.100:0/2384309468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5aba59c640 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 -- 172.21.15.100:0/2384309468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 msgr2=0x56453c5e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 -- 172.21.15.100:0/2384309468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 msgr2=0x56453c5ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 -- 172.21.15.100:0/2384309468 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56453c50b570 con 0x56453c5ee7b0 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5abad9d640 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 0x56453c5f0ba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4097350 tx=0x7f5ab4009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91f38911cfbebb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.756+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5aa80a1ba0 0x7f5aa80a4060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x56453c5e4d80 tx=0x7f5aa4004730 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.757+0000 7f5ab0ff9640 1 -- 172.21.15.100:0/2384309468 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400b510 con 0x56453c5ee7b0 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.757+0000 7f5ab0ff9640 1 -- 172.21.15.100:0/2384309468 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab409a980 con 0x56453c5ee7b0 2024-03-12T05:33:30.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.757+0000 7f5ab0ff9640 1 -- 172.21.15.100:0/2384309468 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40cb720 con 0x56453c5ee7b0 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 -- 172.21.15.100:0/2384309468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 msgr2=0x56453c5f0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 0x56453c5f0ba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4097350 tx=0x7f5ab4009a70 comp rx=0 tx=0).stop 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 -- 172.21.15.100:0/2384309468 shutdown_connections 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5ee7b0 0x56453c5f0ba0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2384309468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 -- 172.21.15.100:0/2384309468 >> 172.21.15.100:0/2384309468 conn(0x56453c5df990 msgr2=0x56453c5dfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 -- 172.21.15.100:0/2384309468 shutdown_connections 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 -- 172.21.15.100:0/2384309468 wait complete. 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 -- start start 2024-03-12T05:33:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.758+0000 7f5ac3655740 1 Processor -- start 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 0x56453c5ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c220430 con 0x56453c5ee7b0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c1dce00 con 0x56453c5e5bd0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c4d1fd0 con 0x56453c5e52b0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 10 client.0 _ll_register_callbacks cb 0x56453c527fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 0x56453c5ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 0x56453c5ebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33372/0 (socket says 172.21.15.100:33372) 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 -- 172.21.15.100:0/1055640713 learned_addr learned my addr 172.21.15.100:0/1055640713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5aba59c640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 -- 172.21.15.100:0/1055640713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 msgr2=0x56453c5ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 -- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 msgr2=0x56453c5eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 -- 172.21.15.100:0/1055640713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc1a6a20 con 0x56453c5ee7b0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5aba59c640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5abad9d640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 0x56453c5ebd40 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409a680 tx=0x7f5ab4098a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e218aca5fb81e98 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/1055640713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab409ead0 con 0x56453c5ee7b0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56453c24c7e0 con 0x56453c5ee7b0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.759+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/1055640713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ab40a3120 con 0x56453c5ee7b0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/1055640713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ab400b510 con 0x56453c5ee7b0 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 10 client.19950 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56453c54b610 con 0x56453c5ee7b0 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/1055640713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56453c24c7e0 con 0x56453c5ee7b0 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/1055640713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ab401f060 con 0x56453c5ee7b0 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 10 client.19950 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 10 client.19950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 20 client.19950 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c5fe660 0x56453c600a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 4 client.19950 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56453c289720 con 0x56453c5fe660 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.760+0000 7f5aba59c640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c5fe660 0x56453c600a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.761+0000 7f5aa1ffb640 10 client.19950 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.761+0000 7f5aba59c640 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c5fe660 0x56453c600a80 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d49668106b5810 server_cookie=8ee24abd99aaf162 in_seq=0 out_seq=0 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.761+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/1055640713 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56453c289720 con 0x56453c5fe660 2024-03-12T05:33:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5aa1ffb640 10 client.19950 handle_command_reply: tid=0 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 client.19950 shutdown 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 msgr2=0x56453c5ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 0x56453c5ebd40 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409a680 tx=0x7f5ab4098a20 comp rx=0 tx=0).stop 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 shutdown_connections 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c5fe660 0x56453c600a80 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5ee7b0 0x56453c5ebd40 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 --2- 172.21.15.100:0/1055640713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5e52b0 0x56453c5ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 >> 172.21.15.100:0/1055640713 conn(0x56453c5df990 msgr2=0x56453c5e1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5aa1ffb640 0 client.19950 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 shutdown_connections 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 1 -- 172.21.15.100:0/1055640713 wait complete. 2024-03-12T05:33:30.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.762+0000 7f5ac3655740 20 client.19950 trim_cache size 0 max 16384 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 Processor -- start 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 -- start start 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e52b0 0x56453c5edd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c5f11a0 con 0x56453c5e5bd0 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c5e4600 con 0x56453c5e52b0 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ac3655740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c5e0a30 con 0x56453c5ee7b0 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33374/0 (socket says 172.21.15.100:33374) 2024-03-12T05:33:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5aba59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e52b0 0x56453c5edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5ab3fff640 1 -- 172.21.15.100:0/3296232070 learned_addr learned my addr 172.21.15.100:0/3296232070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.781+0000 7f5abad9d640 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5ab3fff640 1 -- 172.21.15.100:0/3296232070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 msgr2=0x56453c5f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5ab3fff640 1 -- 172.21.15.100:0/3296232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e52b0 msgr2=0x56453c5edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e52b0 0x56453c5edd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5ab3fff640 1 -- 172.21.15.100:0/3296232070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56453c4e7ac0 con 0x56453c5e5bd0 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005d00 tx=0x7f5aa401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4153134a65180b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5aa3fff640 1 -- 172.21.15.100:0/3296232070 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4004020 con 0x56453c5e5bd0 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5aa3fff640 1 -- 172.21.15.100:0/3296232070 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aa4005500 con 0x56453c5e5bd0 2024-03-12T05:33:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.782+0000 7f5aa3fff640 1 -- 172.21.15.100:0/3296232070 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4004c40 con 0x56453c5e5bd0 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 -- 172.21.15.100:0/3296232070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 msgr2=0x56453c5ee270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005d00 tx=0x7f5aa401b000 comp rx=0 tx=0).stop 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 -- 172.21.15.100:0/3296232070 shutdown_connections 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e5bd0 0x56453c5ee270 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 --2- 172.21.15.100:0/3296232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e52b0 0x56453c5edd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 -- 172.21.15.100:0/3296232070 >> 172.21.15.100:0/3296232070 conn(0x56453c5df990 msgr2=0x56453c5e6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 -- 172.21.15.100:0/3296232070 shutdown_connections 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 -- 172.21.15.100:0/3296232070 wait complete. 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 -- start start 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.783+0000 7f5ac3655740 1 Processor -- start 2024-03-12T05:33:30.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 0x56453c5f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c5f4950 con 0x56453c5e52b0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c5f4ac0 con 0x56453c5e5bd0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56453c654b90 con 0x56453c5ee7b0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 0x56453c5f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 0x56453c5f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33376/0 (socket says 172.21.15.100:33376) 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ac3655740 10 client.0 _ll_register_callbacks cb 0x7f5ab4004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 -- 172.21.15.100:0/2704820151 learned_addr learned my addr 172.21.15.100:0/2704820151 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5abad9d640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 -- 172.21.15.100:0/2704820151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 msgr2=0x56453c5f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 -- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 msgr2=0x56453c5f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 -- 172.21.15.100:0/2704820151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa401a040 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aba59c640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 0x56453c5f59a0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005e00 tx=0x7f5aac005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e6bf967ec0bf77 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.784+0000 7f5aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/2704820151 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0040c0 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/2704820151 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5aac046040 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ab409a980 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/2704820151 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac004220 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5ac3655740 10 client.19956 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56453c50d6a0 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.785+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/2704820151 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aac05f080 con 0x56453c5e52b0 2024-03-12T05:33:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/2704820151 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5aac03c040 con 0x56453c5e52b0 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ac3655740 10 client.19956 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ac3655740 10 client.19956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ac3655740 20 client.19956 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c656330 0x56453c658750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ac3655740 4 client.19956 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56453c2d8890 con 0x56453c656330 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.786+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c656330 0x56453c658750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ab3fff640 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c656330 0x56453c658750 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6364155771db5e57 server_cookie=e3f8ca1cb55abdb7 in_seq=0 out_seq=0 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5aa1ffb640 10 client.19956 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5aa1ffb640 1 -- 172.21.15.100:0/2704820151 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56453c2d8890 con 0x56453c656330 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5aa1ffb640 10 client.19956 handle_command_reply: tid=0 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 client.19956 shutdown 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 msgr2=0x56453c5f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 0x56453c5f59a0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005e00 tx=0x7f5aac005850 comp rx=0 tx=0).stop 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 shutdown_connections 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x56453c656330 0x56453c658750 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56453c5ee7b0 0x56453c5f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56453c5e5bd0 0x56453c5f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 --2- 172.21.15.100:0/2704820151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56453c5e52b0 0x56453c5f59a0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 >> 172.21.15.100:0/2704820151 conn(0x56453c5df990 msgr2=0x56453c5eae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5aa1ffb640 0 client.19956 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 shutdown_connections 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 1 -- 172.21.15.100:0/2704820151 wait complete. 2024-03-12T05:33:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.787+0000 7f5ac3655740 20 client.19956 trim_cache size 0 max 16384 2024-03-12T05:33:30.787 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:33:30.787 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:33:30.787 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:33:30.787 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:33:30.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5aa80a1ba0 msgr2=0x7f5aa80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5aa80a1ba0 0x7f5aa80a4060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x56453c5e4d80 tx=0x7f5aa4004730 comp rx=0 tx=0).stop 2024-03-12T05:33:30.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 msgr2=0x7f5abc199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 0x7f5abc199d00 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400eb10 tx=0x7f5ab400c2d0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 shutdown_connections 2024-03-12T05:33:30.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5aa80a1ba0 0x7f5aa80a4060 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5abc199850 0x7f5abc199d00 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.788+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5abc108670 0x7f5abc19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.789+0000 7f5ac0a81640 1 --2- 172.21.15.100:0/3895311753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5abc1035a0 0x7f5abc19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.789+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 >> 172.21.15.100:0/3895311753 conn(0x7f5abc0fee40 msgr2=0x7f5abc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.789+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 shutdown_connections 2024-03-12T05:33:30.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.789+0000 7f5ac0a81640 1 -- 172.21.15.100:0/3895311753 wait complete. 2024-03-12T05:33:30.795 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:33:30.795 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 -- 172.21.15.100:0/3388616035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c0ffed0 msgr2=0x7f1c4c1002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 --2- 172.21.15.100:0/3388616035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c1002b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009930 tx=0x7f1c3c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 -- 172.21.15.100:0/3388616035 shutdown_connections 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 --2- 172.21.15.100:0/3388616035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c1076a0 0x7f1c4c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 --2- 172.21.15.100:0/3388616035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c100880 0x7f1c4c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 --2- 172.21.15.100:0/3388616035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c1002b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 -- 172.21.15.100:0/3388616035 >> 172.21.15.100:0/3388616035 conn(0x7f1c4c0753a0 msgr2=0x7f1c4c0757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:30.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 -- 172.21.15.100:0/3388616035 shutdown_connections 2024-03-12T05:33:30.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.924+0000 7f1c5118c640 1 -- 172.21.15.100:0/3388616035 wait complete. 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 Processor -- start 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 -- start start 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c1076a0 0x7f1c4c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c197610 0x7f1c4c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c198000 con 0x7f1c4c197610 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c198170 con 0x7f1c4c0ffed0 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.925+0000 7f1c5118c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c198330 con 0x7f1c4c1076a0 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c197610 0x7f1c4c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44724/0 (socket says 172.21.15.100:44724) 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4ad9d640 1 -- 172.21.15.100:0/1357625723 learned_addr learned my addr 172.21.15.100:0/1357625723 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4a59c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c1076a0 0x7f1c4c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 -- 172.21.15.100:0/1357625723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c1076a0 msgr2=0x7f1c4c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c1076a0 0x7f1c4c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 -- 172.21.15.100:0/1357625723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 msgr2=0x7f1c4c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 -- 172.21.15.100:0/1357625723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c009560 con 0x7f1c4c197610 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4a59c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c1076a0 0x7f1c4c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4ad9d640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c4b59e640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c197610 0x7f1c4c197ac0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009fd0 tx=0x7f1c4400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400ce20 con 0x7f1c4c197610 2024-03-12T05:33:30.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c44005020 con 0x7f1c4c197610 2024-03-12T05:33:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c1a4320 con 0x7f1c4c197610 2024-03-12T05:33:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.926+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44010690 con 0x7f1c4c197610 2024-03-12T05:33:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.927+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c1a4880 con 0x7f1c4c197610 2024-03-12T05:33:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.927+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0676e0 con 0x7f1c4c197610 2024-03-12T05:33:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.928+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1c44005500 con 0x7f1c4c197610 2024-03-12T05:33:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.929+0000 7f1c33fff640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1c340a1c20 0x7f1c340a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.929+0000 7f1c4ad9d640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1c340a1c20 0x7f1c340a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.929+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1c44014070 con 0x7f1c4c197610 2024-03-12T05:33:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.930+0000 7f1c4ad9d640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1c340a1c20 0x7f1c340a40e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c000c00 tx=0x7f1c3c0058d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:30.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:30.934+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c44096d10 con 0x7f1c4c197610 2024-03-12T05:33:31.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.220+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c4c109740 con 0x7f1c4c197610 2024-03-12T05:33:31.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.221+0000 7f1c33fff640 1 -- 172.21.15.100:0/1357625723 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f1c44005190 con 0x7f1c4c197610 2024-03-12T05:33:31.214 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:33:31.214 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:33:31.215 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:33:31.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1c340a1c20 msgr2=0x7f1c340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1c340a1c20 0x7f1c340a40e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c000c00 tx=0x7f1c3c0058d0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c197610 msgr2=0x7f1c4c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c197610 0x7f1c4c197ac0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009fd0 tx=0x7f1c4400ef10 comp rx=0 tx=0).stop 2024-03-12T05:33:31.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 shutdown_connections 2024-03-12T05:33:31.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f1c340a1c20 0x7f1c340a40e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c197610 0x7f1c4c197ac0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4c1076a0 0x7f1c4c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 --2- 172.21.15.100:0/1357625723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c0ffed0 0x7f1c4c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 >> 172.21.15.100:0/1357625723 conn(0x7f1c4c0753a0 msgr2=0x7f1c4c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 shutdown_connections 2024-03-12T05:33:31.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.224+0000 7f1c5118c640 1 -- 172.21.15.100:0/1357625723 wait complete. 2024-03-12T05:33:31.233 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:33:31.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:33:31.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 253684 25.88 MB/sec execute 374 sec latency 248.651 ms 2024-03-12T05:33:31.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.386+0000 7f46190bf640 1 -- 172.21.15.100:0/1702472550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461002cbe0 con 0x7f4614101d30 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 -- 172.21.15.100:0/1702472550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614101d30 msgr2=0x7f4614105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 --2- 172.21.15.100:0/1702472550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614101d30 0x7f4614105d70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f4610009930 tx=0x7f461002bc30 comp rx=0 tx=0).stop 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 -- 172.21.15.100:0/1702472550 shutdown_connections 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 --2- 172.21.15.100:0/1702472550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614106450 0x7f461410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 --2- 172.21.15.100:0/1702472550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614101d30 0x7f4614105d70 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 --2- 172.21.15.100:0/1702472550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4614101380 0x7f4614101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 -- 172.21.15.100:0/1702472550 >> 172.21.15.100:0/1702472550 conn(0x7f4614078580 msgr2=0x7f4614078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 -- 172.21.15.100:0/1702472550 shutdown_connections 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.387+0000 7f461c325640 1 -- 172.21.15.100:0/1702472550 wait complete. 2024-03-12T05:33:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 Processor -- start 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 -- start start 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614101380 0x7f4614073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 0x7f46140739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4614073f00 0x7f46141a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46141a5f90 con 0x7f4614106450 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46141a6100 con 0x7f4614101380 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461c325640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46141a62c0 con 0x7f4614073f00 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 0x7f46140739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614101380 0x7f4614073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 0x7f46140739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33426/0 (socket says 172.21.15.100:33426) 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 -- 172.21.15.100:0/867870834 learned_addr learned my addr 172.21.15.100:0/867870834 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f461a8c2640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4614073f00 0x7f46141a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 -- 172.21.15.100:0/867870834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4614073f00 msgr2=0x7f46141a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4614073f00 0x7f46141a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 -- 172.21.15.100:0/867870834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614101380 msgr2=0x7f4614073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614101380 0x7f4614073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 -- 172.21.15.100:0/867870834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009560 con 0x7f4614106450 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f46198c0640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 0x7f46140739c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4610009230 tx=0x7f4610004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.388+0000 7f460b7fe640 1 -- 172.21.15.100:0/867870834 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100043d0 con 0x7f4614106450 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.389+0000 7f460b7fe640 1 -- 172.21.15.100:0/867870834 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46100052d0 con 0x7f4614106450 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.389+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46141a6540 con 0x7f4614106450 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.389+0000 7f460b7fe640 1 -- 172.21.15.100:0/867870834 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610003da0 con 0x7f4614106450 2024-03-12T05:33:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.389+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46141a6a50 con 0x7f4614106450 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.391+0000 7f461efd2740 1 Processor -- start 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.391+0000 7f460b7fe640 1 -- 172.21.15.100:0/867870834 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4610005440 con 0x7f4614106450 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.391+0000 7f461efd2740 1 -- start start 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461efd2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 0x555aa52cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461efd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461efd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa52d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461efd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa5288960 con 0x555aa52d6750 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461efd2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa523c310 con 0x555aa52cd250 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461efd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa5239840 con 0x555aa52cdb70 2024-03-12T05:33:31.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 0x555aa52cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f46198c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa52d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 0x555aa52cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44748/0 (socket says 172.21.15.100:44748) 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 -- 172.21.15.100:0/4072934421 learned_addr learned my addr 172.21.15.100:0/4072934421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a0c1640 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 -- 172.21.15.100:0/4072934421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 msgr2=0x555aa52d6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 -- 172.21.15.100:0/4072934421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 msgr2=0x555aa52d8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa52d8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 -- 172.21.15.100:0/4072934421 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa51f7060 con 0x555aa52cd250 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f460b7fe640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f46000a1bd0 0x7f46000a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a8c2640 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 0x555aa52cd630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f460c009f90 tx=0x7f460c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ccad8e57ffed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f461a0c1640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f46000a1bd0 0x7f46000a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.392+0000 7f460b7fe640 1 -- 172.21.15.100:0/867870834 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f46100ef8a0 con 0x7f4614106450 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f4608bfe640 1 -- 172.21.15.100:0/4072934421 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c011700 con 0x555aa52cd250 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f4608bfe640 1 -- 172.21.15.100:0/4072934421 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f460c015070 con 0x555aa52cd250 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f4608bfe640 1 -- 172.21.15.100:0/4072934421 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c019cd0 con 0x555aa52cd250 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 -- 172.21.15.100:0/4072934421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 msgr2=0x555aa52cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461a0c1640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f46000a1bd0 0x7f46000a4090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x555aa52ccd20 tx=0x7f4604005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 0x555aa52cd630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f460c009f90 tx=0x7f460c010860 comp rx=0 tx=0).stop 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 -- 172.21.15.100:0/4072934421 shutdown_connections 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa52d8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 --2- 172.21.15.100:0/4072934421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cd250 0x555aa52cd630 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.393+0000 7f461efd2740 1 -- 172.21.15.100:0/4072934421 >> 172.21.15.100:0/4072934421 conn(0x555aa52c7980 msgr2=0x555aa52c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 -- 172.21.15.100:0/4072934421 shutdown_connections 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 -- 172.21.15.100:0/4072934421 wait complete. 2024-03-12T05:33:31.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 -- start start 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 Processor -- start 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52d6750 0x555aa52d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa5235020 con 0x555aa52cd250 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa4f08430 con 0x555aa52cdb70 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa4ec4e00 con 0x555aa52d6750 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.394+0000 7f461efd2740 10 client.0 _ll_register_callbacks cb 0x555aa51dccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33444/0 (socket says 172.21.15.100:33444) 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 -- 172.21.15.100:0/268148285 learned_addr learned my addr 172.21.15.100:0/268148285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a8c2640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52d6750 0x555aa52d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f46198c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 -- 172.21.15.100:0/268148285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52d6750 msgr2=0x555aa52d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52d6750 0x555aa52d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 -- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 msgr2=0x555aa52d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52d3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 -- 172.21.15.100:0/268148285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c009bf0 con 0x555aa52cd250 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f46198c0640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461a0c1640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d37e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4604003870 tx=0x7f4604004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d5f0a7c00155d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f45fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f45fdffb640 1 -- 172.21.15.100:0/268148285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604017070 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f45fdffb640 1 -- 172.21.15.100:0/268148285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4604003c70 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aa4f347e0 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f45fdffb640 1 -- 172.21.15.100:0/268148285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4604003e30 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461efd2740 10 client.19974 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.395+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555aa52336a0 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f45fdffb640 1 -- 172.21.15.100:0/268148285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555aa4f347e0 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f45fdffb640 1 -- 172.21.15.100:0/268148285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f460401eae0 con 0x555aa52cd250 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f461efd2740 10 client.19974 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f461efd2740 10 client.19974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f461efd2740 20 client.19974 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f461efd2740 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa52ef410 0x555aa52f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f461efd2740 4 client.19974 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:31.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aa4f71720 con 0x555aa52ef410 2024-03-12T05:33:31.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.396+0000 7f46198c0640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa52ef410 0x555aa52f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.397+0000 7f45fdffb640 10 client.19974 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:33:31.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.397+0000 7f46198c0640 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa52ef410 0x555aa52f1830 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b49d31f0a6014779 server_cookie=498eaf6847023e1c in_seq=0 out_seq=0 2024-03-12T05:33:31.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.398+0000 7f45fdffb640 1 -- 172.21.15.100:0/268148285 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555aa4f71720 con 0x555aa52ef410 2024-03-12T05:33:31.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.398+0000 7f45fdffb640 10 client.19974 handle_command_reply: tid=0 2024-03-12T05:33:31.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.398+0000 7f461efd2740 1 client.19974 shutdown 2024-03-12T05:33:31.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 msgr2=0x555aa52d37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d37e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4604003870 tx=0x7f4604004730 comp rx=0 tx=0).stop 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 shutdown_connections 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa52ef410 0x555aa52f1830 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52d6750 0x555aa52d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f45fdffb640 0 client.19974 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 --2- 172.21.15.100:0/268148285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d37e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 >> 172.21.15.100:0/268148285 conn(0x555aa52c7980 msgr2=0x555aa52c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 shutdown_connections 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 1 -- 172.21.15.100:0/268148285 wait complete. 2024-03-12T05:33:31.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.399+0000 7f461efd2740 20 client.19974 trim_cache size 0 max 16384 2024-03-12T05:33:31.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 Processor -- start 2024-03-12T05:33:31.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 -- start start 2024-03-12T05:33:31.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52d6750 0x555aa52d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa51ba130 con 0x555aa52cd250 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa52c89e0 con 0x555aa52d6750 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461efd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa52f6b00 con 0x555aa52cdb70 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33454/0 (socket says 172.21.15.100:33454) 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 -- 172.21.15.100:0/3232916350 learned_addr learned my addr 172.21.15.100:0/3232916350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f46198c0640 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a8c2640 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52d6750 0x555aa52d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 -- 172.21.15.100:0/3232916350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 msgr2=0x555aa52d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 -- 172.21.15.100:0/3232916350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52d6750 msgr2=0x555aa52d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52d6750 0x555aa52d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.412+0000 7f461a0c1640 1 -- 172.21.15.100:0/3232916350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa51cfc20 con 0x555aa52cd250 2024-03-12T05:33:31.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.413+0000 7f461a0c1640 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d5f10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4604005c20 tx=0x7f4604012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c4afadf8c431c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.413+0000 7f45fffff640 1 -- 172.21.15.100:0/3232916350 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040043d0 con 0x555aa52cd250 2024-03-12T05:33:31.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.413+0000 7f45fffff640 1 -- 172.21.15.100:0/3232916350 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4604030da0 con 0x555aa52cd250 2024-03-12T05:33:31.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.413+0000 7f45fffff640 1 -- 172.21.15.100:0/3232916350 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604005260 con 0x555aa52cd250 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 -- 172.21.15.100:0/3232916350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 msgr2=0x555aa52d5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d5f10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4604005c20 tx=0x7f4604012d30 comp rx=0 tx=0).stop 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 -- 172.21.15.100:0/3232916350 shutdown_connections 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52d6750 0x555aa52d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cdb70 0x555aa52d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 --2- 172.21.15.100:0/3232916350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52cd250 0x555aa52d5f10 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 -- 172.21.15.100:0/3232916350 >> 172.21.15.100:0/3232916350 conn(0x555aa52c7980 msgr2=0x555aa52cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 -- 172.21.15.100:0/3232916350 shutdown_connections 2024-03-12T05:33:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 -- 172.21.15.100:0/3232916350 wait complete. 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 -- start start 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.414+0000 7f461efd2740 1 Processor -- start 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 0x555aa52e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa533e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa533ee60 con 0x555aa52d6750 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa533efd0 con 0x555aa52cdb70 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa533f190 con 0x555aa52cd250 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461efd2740 10 client.0 _ll_register_callbacks cb 0x7f460c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 0x555aa52e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461a0c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 0x555aa52e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:55972/0 (socket says 172.21.15.100:55972) 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44776/0 (socket says 172.21.15.100:44776) 2024-03-12T05:33:31.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461a0c1640 1 -- 172.21.15.100:0/438251899 learned_addr learned my addr 172.21.15.100:0/438251899 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461a8c2640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa533e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 -- 172.21.15.100:0/438251899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 msgr2=0x555aa52e5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 0x555aa52e5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 -- 172.21.15.100:0/438251899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 msgr2=0x555aa533e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa533e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f46198c0640 1 -- 172.21.15.100:0/438251899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555aa52336a0 con 0x555aa52cdb70 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.415+0000 7f461a0c1640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 0x555aa52e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f46198c0640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52e58a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46100ed060 tx=0x7f46100f5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc5e49e9332ec36 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f45fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f45fdffb640 1 -- 172.21.15.100:0/438251899 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100f5040 con 0x555aa52cdb70 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f45fdffb640 1 -- 172.21.15.100:0/438251899 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f46100f2e40 con 0x555aa52cdb70 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f460c015070 con 0x555aa52cdb70 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f45fdffb640 1 -- 172.21.15.100:0/438251899 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46100bc080 con 0x555aa52cdb70 2024-03-12T05:33:31.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f461efd2740 10 client.29312 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555aa51f58c0 con 0x555aa52cdb70 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f45fdffb640 1 -- 172.21.15.100:0/438251899 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46100bbea0 con 0x555aa52cdb70 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.416+0000 7f45fdffb640 1 -- 172.21.15.100:0/438251899 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f461003c070 con 0x555aa52cdb70 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461efd2740 10 client.29312 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461efd2740 10 client.29312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461efd2740 20 client.29312 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461efd2740 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa5347960 0x555aa5349d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461efd2740 4 client.29312 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f460c015070 con 0x555aa5347960 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461a0c1640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa5347960 0x555aa5349d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f45fdffb640 10 client.29312 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:33:31.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.417+0000 7f461a0c1640 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa5347960 0x555aa5349d80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fda786560c2eeed4 server_cookie=942879b889fefc59 in_seq=0 out_seq=0 2024-03-12T05:33:31.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.419+0000 7f45fdffb640 1 -- 172.21.15.100:0/438251899 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f460c015070 con 0x555aa5347960 2024-03-12T05:33:31.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.419+0000 7f45fdffb640 10 client.29312 handle_command_reply: tid=0 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 client.29312 shutdown 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 msgr2=0x555aa52e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52e58a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46100ed060 tx=0x7f46100f5530 comp rx=0 tx=0).stop 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 shutdown_connections 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x555aa5347960 0x555aa5349d80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555aa52d6750 0x555aa533e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f45fdffb640 0 client.29312 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555aa52cdb70 0x555aa52e58a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 --2- 172.21.15.100:0/438251899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555aa52cd250 0x555aa52e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 >> 172.21.15.100:0/438251899 conn(0x555aa52c7980 msgr2=0x555aa52c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 shutdown_connections 2024-03-12T05:33:31.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 1 -- 172.21.15.100:0/438251899 wait complete. 2024-03-12T05:33:31.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.420+0000 7f461efd2740 20 client.29312 trim_cache size 0 max 16384 2024-03-12T05:33:31.417 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:33:31.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f46000a1bd0 msgr2=0x7f46000a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f46000a1bd0 0x7f46000a4090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x555aa52ccd20 tx=0x7f4604005f90 comp rx=0 tx=0).stop 2024-03-12T05:33:31.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 msgr2=0x7f46140739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 0x7f46140739c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4610009230 tx=0x7f4610004c00 comp rx=0 tx=0).stop 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 shutdown_connections 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f46000a1bd0 0x7f46000a4090 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4614073f00 0x7f46141a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4614106450 0x7f46140739c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 --2- 172.21.15.100:0/867870834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614101380 0x7f4614073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 >> 172.21.15.100:0/867870834 conn(0x7f4614078580 msgr2=0x7f46140ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 shutdown_connections 2024-03-12T05:33:31.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.421+0000 7f461c325640 1 -- 172.21.15.100:0/867870834 wait complete. 2024-03-12T05:33:31.426 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:33:31.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:33:31.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:31 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1357625723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:33:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.593+0000 7fce0628d640 1 -- 172.21.15.100:0/1414198536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 msgr2=0x7fce00101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.593+0000 7fce0628d640 1 --2- 172.21.15.100:0/1414198536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce00101760 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009930 tx=0x7fcdf002bc30 comp rx=0 tx=0).stop 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 -- 172.21.15.100:0/1414198536 shutdown_connections 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 --2- 172.21.15.100:0/1414198536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce00106450 0x7fce0010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 --2- 172.21.15.100:0/1414198536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00101d30 0x7fce00105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 --2- 172.21.15.100:0/1414198536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce00101760 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 -- 172.21.15.100:0/1414198536 >> 172.21.15.100:0/1414198536 conn(0x7fce00078540 msgr2=0x7fce00078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 -- 172.21.15.100:0/1414198536 shutdown_connections 2024-03-12T05:33:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 -- 172.21.15.100:0/1414198536 wait complete. 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.594+0000 7fce0628d640 1 Processor -- start 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 -- start start 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce0019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce00106450 0x7fce0019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce001976b0 0x7fce00197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce00198180 con 0x7fce00101380 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce001982f0 con 0x7fce001976b0 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0628d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce001984b0 con 0x7fce00106450 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce0019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce0019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33492/0 (socket says 172.21.15.100:33492) 2024-03-12T05:33:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fce0482a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce001976b0 0x7fce00197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fcdfffff640 1 -- 172.21.15.100:0/4050092049 learned_addr learned my addr 172.21.15.100:0/4050092049 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.595+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce00106450 0x7fce0019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfffff640 1 -- 172.21.15.100:0/4050092049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce00106450 msgr2=0x7fce0019db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfffff640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce00106450 0x7fce0019db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfffff640 1 -- 172.21.15.100:0/4050092049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce001976b0 msgr2=0x7fce00197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfffff640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce001976b0 0x7fce00197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfffff640 1 -- 172.21.15.100:0/4050092049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0009560 con 0x7fce00101380 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fce0482a640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce001976b0 0x7fce00197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfffff640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce0019d5f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009230 tx=0x7fcdf002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfd7fa640 1 -- 172.21.15.100:0/4050092049 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0004350 con 0x7fce00101380 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce001a43f0 con 0x7fce00101380 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfd7fa640 1 -- 172.21.15.100:0/4050092049 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdf00044b0 con 0x7fce00101380 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fcdfd7fa640 1 -- 172.21.15.100:0/4050092049 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003c60 con 0x7fce00101380 2024-03-12T05:33:31.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.596+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce001a48d0 con 0x7fce00101380 2024-03-12T05:33:31.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.598+0000 7fcdfd7fa640 1 -- 172.21.15.100:0/4050092049 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fcdf0005560 con 0x7fce00101380 2024-03-12T05:33:31.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 Processor -- start 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 -- start start 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d55054630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdfd7fa640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcde80a1ba0 0x7fcde80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 0x558d5505d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d5505fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5500f960 con 0x558d55054b70 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d54fc3310 con 0x558d55054250 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce08e5c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d54fc0840 con 0x558d5505d750 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 0x558d5505d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce0482a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d55054630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 0x558d5505d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33504/0 (socket says 172.21.15.100:33504) 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 -- 172.21.15.100:0/871765827 learned_addr learned my addr 172.21.15.100:0/871765827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce0482a640 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d5505fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdfd7fa640 1 -- 172.21.15.100:0/4050092049 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcdf00ef780 con 0x7fce00101380 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 -- 172.21.15.100:0/871765827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 msgr2=0x558d5505fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d5505fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 -- 172.21.15.100:0/871765827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 msgr2=0x558d55054630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d55054630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fcdff7fe640 1 -- 172.21.15.100:0/871765827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d54f7e060 con 0x558d55054b70 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.599+0000 7fce0482a640 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d55054630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.600+0000 7fcdfffff640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcde80a1ba0 0x7fcde80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.600+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 0x558d5505d210 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009a00 tx=0x7fcdf4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6600c799625eae19 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.600+0000 7fcde67fc640 1 -- 172.21.15.100:0/871765827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400eae0 con 0x558d55054b70 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.600+0000 7fcde67fc640 1 -- 172.21.15.100:0/871765827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdf400ec40 con 0x558d55054b70 2024-03-12T05:33:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.600+0000 7fcde67fc640 1 -- 172.21.15.100:0/871765827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4012630 con 0x558d55054b70 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 -- 172.21.15.100:0/871765827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 msgr2=0x558d5505d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fcdfffff640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcde80a1ba0 0x7fcde80a4060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf00dc040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 0x558d5505d210 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009a00 tx=0x7fcdf4009ec0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 -- 172.21.15.100:0/871765827 shutdown_connections 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d5505fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054b70 0x558d5505d210 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 --2- 172.21.15.100:0/871765827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d55054630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 -- 172.21.15.100:0/871765827 >> 172.21.15.100:0/871765827 conn(0x558d5504e980 msgr2=0x558d5504ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 -- 172.21.15.100:0/871765827 shutdown_connections 2024-03-12T05:33:31.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 -- 172.21.15.100:0/871765827 wait complete. 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 -- start start 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.601+0000 7fce08e5c740 1 Processor -- start 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d5505a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 0x558d5505ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d5505b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d54fbc020 con 0x558d5505d750 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d54c8f430 con 0x558d55054250 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d54c4be00 con 0x558d55054b70 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 10 client.0 _ll_register_callbacks cb 0x558d54f63cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d5505a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 0x558d5505ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d5505b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 0x558d5505ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:56006/0 (socket says 172.21.15.100:56006) 2024-03-12T05:33:31.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcdff7fe640 1 -- 172.21.15.100:0/1645155900 learned_addr learned my addr 172.21.15.100:0/1645155900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d5505a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44802/0 (socket says 172.21.15.100:44802) 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 -- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 msgr2=0x558d5505ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 0x558d5505ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 -- 172.21.15.100:0/1645155900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 msgr2=0x558d5505b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d5505b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 -- 172.21.15.100:0/1645155900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf4009630 con 0x558d55054250 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 0x558d5505ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce0482a640 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d5505a750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005c40 tx=0x7fcdf8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30fbd0630496a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcddf7fe640 1 -- 172.21.15.100:0/1645155900 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8004b20 con 0x558d55054250 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fcddf7fe640 1 -- 172.21.15.100:0/1645155900 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcdf8004c80 con 0x558d55054250 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.602+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d54cbb7e0 con 0x558d55054250 2024-03-12T05:33:31.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fcddf7fe640 1 -- 172.21.15.100:0/1645155900 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf8004e40 con 0x558d55054250 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 10 client.29321 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d54fba6a0 con 0x558d55054250 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fcddf7fe640 1 -- 172.21.15.100:0/1645155900 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdf8031020 con 0x558d55054250 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fcddf7fe640 1 -- 172.21.15.100:0/1645155900 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcdf8004fa0 con 0x558d55054250 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 10 client.29321 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 10 client.29321 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 20 client.29321 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d5506db10 0x558d5506ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 4 client.29321 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d54cf8720 con 0x558d5506db10 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.603+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d5506db10 0x558d5506ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fcddf7fe640 10 client.29321 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d5506db10 0x558d5506ff00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6090fe851c294cbe server_cookie=d7c1c3f5054c719f in_seq=0 out_seq=0 2024-03-12T05:33:31.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fcddf7fe640 1 -- 172.21.15.100:0/1645155900 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d54cf8720 con 0x558d5506db10 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fcddf7fe640 10 client.29321 handle_command_reply: tid=0 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 client.29321 shutdown 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 msgr2=0x558d5505a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d5505a750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005c40 tx=0x7fcdf8005bd0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 shutdown_connections 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d5505b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054b70 0x558d5505ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d5506db10 0x558d5506ff00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 --2- 172.21.15.100:0/1645155900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054250 0x558d5505a750 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 >> 172.21.15.100:0/1645155900 conn(0x558d5504e980 msgr2=0x558d550506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fcddf7fe640 0 client.29321 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 shutdown_connections 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 1 -- 172.21.15.100:0/1645155900 wait complete. 2024-03-12T05:33:31.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.604+0000 7fce08e5c740 20 client.29321 trim_cache size 0 max 16384 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 Processor -- start 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 -- start start 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 0x558d5505ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d5505aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d550579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d54f41130 con 0x558d55054250 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d55075150 con 0x558d55054b70 2024-03-12T05:33:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce08e5c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5504a080 con 0x558d5505d750 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 0x558d5505ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d5505aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 0x558d5505ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33526/0 (socket says 172.21.15.100:33526) 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 -- 172.21.15.100:0/648723359 learned_addr learned my addr 172.21.15.100:0/648723359 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d550579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 -- 172.21.15.100:0/648723359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 msgr2=0x558d550579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d550579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 -- 172.21.15.100:0/648723359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 msgr2=0x558d5505aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d5505aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.619+0000 7fce0482a640 1 -- 172.21.15.100:0/648723359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d54f56c20 con 0x558d55054250 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.620+0000 7fce0482a640 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 0x558d5505ce80 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800a010 tx=0x7fcdf80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be743d0a88e4e29 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.620+0000 7fcde5ffb640 1 -- 172.21.15.100:0/648723359 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800fb60 con 0x558d55054250 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.620+0000 7fcde5ffb640 1 -- 172.21.15.100:0/648723359 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdf8031040 con 0x558d55054250 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.620+0000 7fcde5ffb640 1 -- 172.21.15.100:0/648723359 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8036460 con 0x558d55054250 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- 172.21.15.100:0/648723359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 msgr2=0x558d5505ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 0x558d5505ce80 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800a010 tx=0x7fcdf80096c0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- 172.21.15.100:0/648723359 shutdown_connections 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d5505d750 0x558d550579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d5505aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- 172.21.15.100:0/648723359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d55054250 0x558d5505ce80 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- 172.21.15.100:0/648723359 >> 172.21.15.100:0/648723359 conn(0x558d5504e980 msgr2=0x558d55055a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- 172.21.15.100:0/648723359 shutdown_connections 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- 172.21.15.100:0/648723359 wait complete. 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- start start 2024-03-12T05:33:31.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 Processor -- start 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054250 0x558d550639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d55063ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d550be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d550be690 con 0x558d5505d750 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d550be7d0 con 0x558d55054b70 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d550be990 con 0x558d55054250 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce08e5c740 10 client.0 _ll_register_callbacks cb 0x7fcdf4009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.621+0000 7fce0482a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d550be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d550be150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33542/0 (socket says 172.21.15.100:33542) 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 -- 172.21.15.100:0/2537098536 learned_addr learned my addr 172.21.15.100:0/2537098536 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d55063ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054250 0x558d550639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 -- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054250 msgr2=0x558d550639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054250 0x558d550639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 -- 172.21.15.100:0/2537098536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 msgr2=0x558d55063ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d55063ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 -- 172.21.15.100:0/2537098536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d54fba6a0 con 0x558d5505d750 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce0482a640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d550be150 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8002230 tx=0x7fcdf8004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98779c1bc777dc08 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fcddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fcddf7fe640 1 -- 172.21.15.100:0/2537098536 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8005110 con 0x558d5505d750 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fcddf7fe640 1 -- 172.21.15.100:0/2537098536 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcdf8032080 con 0x558d5505d750 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdf400ec40 con 0x558d5505d750 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fcddf7fe640 1 -- 172.21.15.100:0/2537098536 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf8005570 con 0x558d5505d750 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce08e5c740 10 client.19998 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d54f7c8c0 con 0x558d5505d750 2024-03-12T05:33:31.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.622+0000 7fcddf7fe640 1 -- 172.21.15.100:0/2537098536 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdf400ec40 con 0x558d5505d750 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fcddf7fe640 1 -- 172.21.15.100:0/2537098536 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcdf8034040 con 0x558d5505d750 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fce08e5c740 10 client.19998 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fce08e5c740 10 client.19998 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fce08e5c740 20 client.19998 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fce08e5c740 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d550c5f10 0x558d550c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fce08e5c740 4 client.19998 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d54d47890 con 0x558d550c5f10 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d550c5f10 0x558d550c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fcdff7fe640 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d550c5f10 0x558d550c8300 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85e4dfe0d2baed47 server_cookie=4da804446cc19f8e in_seq=0 out_seq=0 2024-03-12T05:33:31.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.623+0000 7fcddf7fe640 10 client.19998 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fcddf7fe640 1 -- 172.21.15.100:0/2537098536 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d54d47890 con 0x558d550c5f10 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fcddf7fe640 10 client.19998 handle_command_reply: tid=0 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 client.19998 shutdown 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 msgr2=0x558d550be150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d550be150 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8002230 tx=0x7fcdf8004eb0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 shutdown_connections 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d5505d750 0x558d550be150 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x558d550c5f10 0x558d550c8300 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558d55054b70 0x558d55063ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 --2- 172.21.15.100:0/2537098536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d55054250 0x558d550639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fcddf7fe640 0 client.19998 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:33:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 >> 172.21.15.100:0/2537098536 conn(0x558d5504e980 msgr2=0x558d5504fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 shutdown_connections 2024-03-12T05:33:31.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.624+0000 7fce08e5c740 1 -- 172.21.15.100:0/2537098536 wait complete. 2024-03-12T05:33:31.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce08e5c740 20 client.19998 trim_cache size 0 max 16384 2024-03-12T05:33:31.625 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcde80a1ba0 msgr2=0x7fcde80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcde80a1ba0 0x7fcde80a4060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf00dc040 comp rx=0 tx=0).stop 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 msgr2=0x7fce0019d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce0019d5f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009230 tx=0x7fcdf002c6c0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 shutdown_connections 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcde80a1ba0 0x7fcde80a4060 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce001976b0 0x7fce00197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce00106450 0x7fce0019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 --2- 172.21.15.100:0/4050092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce00101380 0x7fce0019d5f0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 >> 172.21.15.100:0/4050092049 conn(0x7fce00078540 msgr2=0x7fce000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 shutdown_connections 2024-03-12T05:33:31.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.625+0000 7fce0628d640 1 -- 172.21.15.100:0/4050092049 wait complete. 2024-03-12T05:33:31.629 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:33:31.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:33:31.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:31 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1357625723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:33:31.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:31 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1357625723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:33:31.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 -- 172.21.15.100:0/952966011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda041064c0 msgr2=0x7fda0410e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 --2- 172.21.15.100:0/952966011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda041064c0 0x7fda0410e070 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009cd0 tx=0x7fd9fc02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 -- 172.21.15.100:0/952966011 shutdown_connections 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 --2- 172.21.15.100:0/952966011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda041064c0 0x7fda0410e070 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 --2- 172.21.15.100:0/952966011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda04068fa0 0x7fda04105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 --2- 172.21.15.100:0/952966011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda040685f0 0x7fda040689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 -- 172.21.15.100:0/952966011 >> 172.21.15.100:0/952966011 conn(0x7fda04077180 msgr2=0x7fda04077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 -- 172.21.15.100:0/952966011 shutdown_connections 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.795+0000 7fda0bf02640 1 -- 172.21.15.100:0/952966011 wait complete. 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 Processor -- start 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 -- start start 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 0x7fda0419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda041064c0 0x7fda0419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda04197600 0x7fda04197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04197ff0 con 0x7fda040685f0 2024-03-12T05:33:31.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04198160 con 0x7fda04197600 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0bf02640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04198320 con 0x7fda041064c0 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 0x7fda0419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 0x7fda0419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33568/0 (socket says 172.21.15.100:33568) 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda09c9e640 1 -- 172.21.15.100:0/438292983 learned_addr learned my addr 172.21.15.100:0/438292983 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0949d640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda041064c0 0x7fda0419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.796+0000 7fda0a49f640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda04197600 0x7fda04197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda09c9e640 1 -- 172.21.15.100:0/438292983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda041064c0 msgr2=0x7fda0419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda09c9e640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda041064c0 0x7fda0419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda09c9e640 1 -- 172.21.15.100:0/438292983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda04197600 msgr2=0x7fda04197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda09c9e640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda04197600 0x7fda04197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda09c9e640 1 -- 172.21.15.100:0/438292983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc009900 con 0x7fda040685f0 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda0949d640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda041064c0 0x7fda0419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda0a49f640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda04197600 0x7fda04197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda09c9e640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 0x7fda0419d450 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fda0000e9f0 tx=0x7fda0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fd9faffd640 1 -- 172.21.15.100:0/438292983 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00009700 con 0x7fda040685f0 2024-03-12T05:33:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda041a4310 con 0x7fda040685f0 2024-03-12T05:33:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fd9faffd640 1 -- 172.21.15.100:0/438292983 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda0000cd80 con 0x7fda040685f0 2024-03-12T05:33:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fd9faffd640 1 -- 172.21.15.100:0/438292983 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00018920 con 0x7fda040685f0 2024-03-12T05:33:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.797+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda041a4870 con 0x7fda040685f0 2024-03-12T05:33:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.800+0000 7fd9faffd640 1 -- 172.21.15.100:0/438292983 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fda00010430 con 0x7fda040685f0 2024-03-12T05:33:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.801+0000 7fd9faffd640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9f00a1ba0 0x7fd9f00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.801+0000 7fd9faffd640 1 -- 172.21.15.100:0/438292983 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fda00014070 con 0x7fda040685f0 2024-03-12T05:33:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.801+0000 7fda0949d640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9f00a1ba0 0x7fd9f00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.801+0000 7fda0949d640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9f00a1ba0 0x7fd9f00a4060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fda040ffb10 tx=0x7fd9f4005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 Processor -- start 2024-03-12T05:33:31.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 -- start start 2024-03-12T05:33:31.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a8c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a95210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a95750 0x55c976a97b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976a47960 con 0x55c976a8c250 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9769fb310 con 0x55c976a95750 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9769f8840 con 0x55c976a8cb70 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a8c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a8c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33576/0 (socket says 172.21.15.100:33576) 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda09c9e640 1 -- 172.21.15.100:0/4223694009 learned_addr learned my addr 172.21.15.100:0/4223694009 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0a49f640 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a95750 0x55c976a97b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.802+0000 7fda0949d640 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a95210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fda09c9e640 1 -- 172.21.15.100:0/4223694009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 msgr2=0x55c976a95210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fda09c9e640 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a95210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fda09c9e640 1 -- 172.21.15.100:0/4223694009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a95750 msgr2=0x55c976a97b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fda09c9e640 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a95750 0x55c976a97b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fda09c9e640 1 -- 172.21.15.100:0/4223694009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9769b6060 con 0x55c976a8c250 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fda09c9e640 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a8c630 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fda00097230 tx=0x7fda0009a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afcff8aa3e7110d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fd9effff640 1 -- 172.21.15.100:0/4223694009 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000a2f50 con 0x55c976a8c250 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fd9effff640 1 -- 172.21.15.100:0/4223694009 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda00010b80 con 0x55c976a8c250 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.803+0000 7fd9effff640 1 -- 172.21.15.100:0/4223694009 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00010940 con 0x55c976a8c250 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 -- 172.21.15.100:0/4223694009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 msgr2=0x55c976a8c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a8c630 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fda00097230 tx=0x7fda0009a760 comp rx=0 tx=0).stop 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 -- 172.21.15.100:0/4223694009 shutdown_connections 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a95750 0x55c976a97b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a95210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/4223694009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a8c630 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 -- 172.21.15.100:0/4223694009 >> 172.21.15.100:0/4223694009 conn(0x55c976a86980 msgr2=0x55c976a86d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 -- 172.21.15.100:0/4223694009 shutdown_connections 2024-03-12T05:33:31.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 -- 172.21.15.100:0/4223694009 wait complete. 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.804+0000 7fda0ebb1740 1 -- start start 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 Processor -- start 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a92790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a92cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a95750 0x55c976a93210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9769f4020 con 0x55c976a95750 2024-03-12T05:33:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9766c7430 con 0x55c976a8c250 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976683e00 con 0x55c976a8cb70 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0ebb1740 10 client.0 _ll_register_callbacks cb 0x55c97699bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a92790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0a49f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a95750 0x55c976a93210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a92790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:44876/0 (socket says 172.21.15.100:44876) 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda09c9e640 1 -- 172.21.15.100:0/503454608 learned_addr learned my addr 172.21.15.100:0/503454608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.805+0000 7fda0949d640 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a92cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda09c9e640 1 -- 172.21.15.100:0/503454608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 msgr2=0x55c976a92cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda09c9e640 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a92cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda09c9e640 1 -- 172.21.15.100:0/503454608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a95750 msgr2=0x55c976a93210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda09c9e640 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a95750 0x55c976a93210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda09c9e640 1 -- 172.21.15.100:0/503454608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda041a4870 con 0x55c976a8c250 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda09c9e640 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a92790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda0009a7f0 tx=0x7fda00097100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fba6af830bbb6b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fd9ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/503454608 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000108f0 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/503454608 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fda000b3200 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9766f37e0 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/503454608 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda00099cb0 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda0ebb1740 10 client.29339 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c9769f26a0 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.806+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/503454608 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c9766f37e0 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/503454608 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fda000ba050 con 0x55c976a8c250 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0ebb1740 10 client.29339 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0ebb1740 10 client.29339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0ebb1740 20 client.29339 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976aa5810 0x55c976aa7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0ebb1740 4 client.29339 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c976730720 con 0x55c976aa5810 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0949d640 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976aa5810 0x55c976aa7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fd9ed7fa640 10 client.29339 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.807+0000 7fda0949d640 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976aa5810 0x55c976aa7c30 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f668dbb4e2db5a0 server_cookie=d041e7efecc0417c in_seq=0 out_seq=0 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/503454608 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c976730720 con 0x55c976aa5810 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fd9ed7fa640 10 client.29339 handle_command_reply: tid=0 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 client.29339 shutdown 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 msgr2=0x55c976a92790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a92790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda0009a7f0 tx=0x7fda00097100 comp rx=0 tx=0).stop 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 shutdown_connections 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976aa5810 0x55c976aa7c30 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a95750 0x55c976a93210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fd9ed7fa640 0 client.29339 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a8cb70 0x55c976a92cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/503454608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a92790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 >> 172.21.15.100:0/503454608 conn(0x55c976a86980 msgr2=0x55c976a886d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 shutdown_connections 2024-03-12T05:33:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.808+0000 7fda0ebb1740 1 -- 172.21.15.100:0/503454608 wait complete. 2024-03-12T05:33:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.809+0000 7fda0ebb1740 20 client.29339 trim_cache size 0 max 16384 2024-03-12T05:33:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 Processor -- start 2024-03-12T05:33:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 -- start start 2024-03-12T05:33:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a94ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 0x55c976a93030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976979130 con 0x55c976a8cb70 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976a87550 con 0x55c976a8c250 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976a82080 con 0x55c976a95750 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.826+0000 7fda0949d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 0x55c976a93030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a8fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:56074/0 (socket says 172.21.15.100:56074) 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0949d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 0x55c976a93030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33592/0 (socket says 172.21.15.100:33592) 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 -- 172.21.15.100:0/2097566707 learned_addr learned my addr 172.21.15.100:0/2097566707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda09c9e640 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a94ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 -- 172.21.15.100:0/2097566707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 msgr2=0x55c976a94ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a94ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 -- 172.21.15.100:0/2097566707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 msgr2=0x55c976a93030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 0x55c976a93030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 -- 172.21.15.100:0/2097566707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c97698ec20 con 0x55c976a95750 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0949d640 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 0x55c976a93030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fda0a49f640 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a8fa10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc02c700 tx=0x7fd9fc032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7cff6a66d70ddd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fd9ef7fe640 1 -- 172.21.15.100:0/2097566707 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc034a90 con 0x55c976a95750 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.827+0000 7fd9ef7fe640 1 -- 172.21.15.100:0/2097566707 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc034bf0 con 0x55c976a95750 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fd9ef7fe640 1 -- 172.21.15.100:0/2097566707 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc004040 con 0x55c976a95750 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 -- 172.21.15.100:0/2097566707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 msgr2=0x55c976a8fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a8fa10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc02c700 tx=0x7fd9fc032e20 comp rx=0 tx=0).stop 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 -- 172.21.15.100:0/2097566707 shutdown_connections 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a8fa10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8cb70 0x55c976a93030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/2097566707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8c250 0x55c976a94ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 -- 172.21.15.100:0/2097566707 >> 172.21.15.100:0/2097566707 conn(0x55c976a86980 msgr2=0x55c976aac590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 -- 172.21.15.100:0/2097566707 shutdown_connections 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.828+0000 7fda0ebb1740 1 -- 172.21.15.100:0/2097566707 wait complete. 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.829+0000 7fda0ebb1740 1 -- start start 2024-03-12T05:33:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.829+0000 7fda0ebb1740 1 Processor -- start 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.829+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a9b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.829+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8cb70 0x55c976a9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.829+0000 7fda0ebb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976afbdc0 con 0x55c976a8c250 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976afbf30 con 0x55c976a8cb70 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0ebb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c976afc0f0 con 0x55c976a95750 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0ebb1740 10 client.0 _ll_register_callbacks cb 0x7fda00096b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a9b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a9b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33606/0 (socket says 172.21.15.100:33606) 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 -- 172.21.15.100:0/1959056222 learned_addr learned my addr 172.21.15.100:0/1959056222 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0949d640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8cb70 0x55c976a9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0a49f640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 -- 172.21.15.100:0/1959056222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 msgr2=0x55c976a9bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a9bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 -- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8cb70 msgr2=0x55c976a9b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8cb70 0x55c976a9b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 -- 172.21.15.100:0/1959056222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f4017070 con 0x55c976a8c250 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0949d640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8cb70 0x55c976a9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda09c9e640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a9b130 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fda00096cc0 tx=0x7fda0009a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358daee2fb7370d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fd9ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/1959056222 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00099720 con 0x55c976a8c250 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/1959056222 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fda000a2f50 con 0x55c976a8c250 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.830+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda00010b80 con 0x55c976a8c250 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/1959056222 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0009a3a0 con 0x55c976a8c250 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 10 client.20016 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fda0410e880 con 0x55c976a8c250 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/1959056222 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda00010b80 con 0x55c976a8c250 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/1959056222 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fda00099cb0 con 0x55c976a8c250 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 10 client.20016 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 10 client.20016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 20 client.20016 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976afd4a0 0x55c976aff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 4 client.20016 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c97677f890 con 0x55c976afd4a0 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.831+0000 7fda0949d640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976afd4a0 0x55c976aff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fd9ed7fa640 10 client.20016 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0949d640 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976afd4a0 0x55c976aff890 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63cb89460015ed4 server_cookie=5d3580165203f036 in_seq=0 out_seq=0 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fd9ed7fa640 1 -- 172.21.15.100:0/1959056222 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c97677f890 con 0x55c976afd4a0 2024-03-12T05:33:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fd9ed7fa640 10 client.20016 handle_command_reply: tid=0 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 client.20016 shutdown 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 msgr2=0x55c976a9b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a9b130 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fda00096cc0 tx=0x7fda0009a760 comp rx=0 tx=0).stop 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 shutdown_connections 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c976afd4a0 0x55c976aff890 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c976a95750 0x55c976a9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fd9ed7fa640 0 client.20016 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c976a8cb70 0x55c976a9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 --2- 172.21.15.100:0/1959056222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c976a8c250 0x55c976a9b130 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.832+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 >> 172.21.15.100:0/1959056222 conn(0x55c976a86980 msgr2=0x55c976a852f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.833+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 shutdown_connections 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.833+0000 7fda0ebb1740 1 -- 172.21.15.100:0/1959056222 wait complete. 2024-03-12T05:33:31.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.833+0000 7fda0ebb1740 20 client.20016 trim_cache size 0 max 16384 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9f00a1ba0 msgr2=0x7fd9f00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9f00a1ba0 0x7fd9f00a4060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fda040ffb10 tx=0x7fd9f4005f90 comp rx=0 tx=0).stop 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 msgr2=0x7fda0419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 0x7fda0419d450 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fda0000e9f0 tx=0x7fda0000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 shutdown_connections 2024-03-12T05:33:31.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd9f00a1ba0 0x7fd9f00a4060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda04197600 0x7fda04197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda041064c0 0x7fda0419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 --2- 172.21.15.100:0/438292983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda040685f0 0x7fda0419d450 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 >> 172.21.15.100:0/438292983 conn(0x7fda04077180 msgr2=0x7fda040fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 shutdown_connections 2024-03-12T05:33:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:31.834+0000 7fda0bf02640 1 -- 172.21.15.100:0/438292983 wait complete. 2024-03-12T05:33:32.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 254209 25.89 MB/sec execute 375 sec latency 133.035 ms 2024-03-12T05:33:32.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:32 smithi100 ceph-mon[32210]: pgmap v676: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-03-12T05:33:32.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:32 smithi156 ceph-mon[36270]: pgmap v676: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-03-12T05:33:32.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:32 smithi161 ceph-mon[36043]: pgmap v676: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-03-12T05:33:32.840 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-12T05:33:32.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:33:32.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-12T05:33:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 -- 172.21.15.100:0/2255780949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 msgr2=0x7f756410e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 --2- 172.21.15.100:0/2255780949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756410e070 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f755c009cd0 tx=0x7f755c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 -- 172.21.15.100:0/2255780949 shutdown_connections 2024-03-12T05:33:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 --2- 172.21.15.100:0/2255780949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756410e070 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 --2- 172.21.15.100:0/2255780949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7564068fa0 0x7f7564105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 --2- 172.21.15.100:0/2255780949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75640685f0 0x7f75640689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 -- 172.21.15.100:0/2255780949 >> 172.21.15.100:0/2255780949 conn(0x7f7564077180 msgr2=0x7f7564077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 -- 172.21.15.100:0/2255780949 shutdown_connections 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.961+0000 7f756ab6b640 1 -- 172.21.15.100:0/2255780949 wait complete. 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 Processor -- start 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 -- start start 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75640685f0 0x7f756419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7564197600 0x7f7564197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564197ff0 con 0x7f75641064c0 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564198160 con 0x7f75640685f0 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564198320 con 0x7f7564197600 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33618/0 (socket says 172.21.15.100:33618) 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 -- 172.21.15.100:0/556954558 learned_addr learned my addr 172.21.15.100:0/556954558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7568907640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75640685f0 0x7f756419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7569108640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7564197600 0x7f7564197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 -- 172.21.15.100:0/556954558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7564197600 msgr2=0x7f7564197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7564197600 0x7f7564197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 -- 172.21.15.100:0/556954558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75640685f0 msgr2=0x7f756419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75640685f0 0x7f756419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 -- 172.21.15.100:0/556954558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009900 con 0x7f75641064c0 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7563fff640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756419d990 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f755000e9c0 tx=0x7f755000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7561ffb640 1 -- 172.21.15.100:0/556954558 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75500096c0 con 0x7f75641064c0 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75641a4310 con 0x7f75641064c0 2024-03-12T05:33:32.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7561ffb640 1 -- 172.21.15.100:0/556954558 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7550005500 con 0x7f75641064c0 2024-03-12T05:33:32.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.962+0000 7f7561ffb640 1 -- 172.21.15.100:0/556954558 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550010430 con 0x7f75641064c0 2024-03-12T05:33:32.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.963+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75641a47d0 con 0x7f75641064c0 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.965+0000 7f7561ffb640 1 -- 172.21.15.100:0/556954558 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7550005020 con 0x7f75641064c0 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 Processor -- start 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7561ffb640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75540a1ba0 0x7f75540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 -- start start 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7569108640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75540a1ba0 0x7f75540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25500f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7561ffb640 1 -- 172.21.15.100:0/556954558 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7550014070 con 0x7f75641064c0 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c2550182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c25501ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c254f7df90 con 0x55c25500f380 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c254f7b4c0 con 0x55c25500fd30 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f756d80f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c254f76ca0 con 0x55c2550188b0 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25500f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25500f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33624/0 (socket says 172.21.15.100:33624) 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 -- 172.21.15.100:0/2687383706 learned_addr learned my addr 172.21.15.100:0/2687383706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7563fff640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c2550182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7569108640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c25501ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 -- 172.21.15.100:0/2687383706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 msgr2=0x55c25501ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c25501ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 -- 172.21.15.100:0/2687383706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 msgr2=0x55c2550182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c2550182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7568907640 1 -- 172.21.15.100:0/2687383706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c254f390b0 con 0x55c25500f380 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7563fff640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c2550182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7569108640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c25501ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.966+0000 7f7569108640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75540a1ba0 0x7f75540a4060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f755c005cc0 tx=0x7f755c032ef0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.967+0000 7f7568907640 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25500f760 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7558009cf0 tx=0x7f7558011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed993436140cf5e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.967+0000 7f754effd640 1 -- 172.21.15.100:0/2687383706 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558009ea0 con 0x55c25500f380 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.967+0000 7f754effd640 1 -- 172.21.15.100:0/2687383706 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7558004230 con 0x55c25500f380 2024-03-12T05:33:32.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.967+0000 7f754effd640 1 -- 172.21.15.100:0/2687383706 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558019880 con 0x55c25500f380 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- 172.21.15.100:0/2687383706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 msgr2=0x55c25500f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25500f760 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7558009cf0 tx=0x7f7558011810 comp rx=0 tx=0).stop 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- 172.21.15.100:0/2687383706 shutdown_connections 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c25501ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c2550182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- 172.21.15.100:0/2687383706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25500f760 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- 172.21.15.100:0/2687383706 >> 172.21.15.100:0/2687383706 conn(0x55c255009840 msgr2=0x55c255009c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- 172.21.15.100:0/2687383706 shutdown_connections 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- 172.21.15.100:0/2687383706 wait complete. 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- start start 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 Processor -- start 2024-03-12T05:33:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255015870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255015db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c2550162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c254f1bf40 con 0x55c2550188b0 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c254c4a4b0 con 0x55c25500fd30 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c254efc670 con 0x55c25500f380 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.968+0000 7f756d80f740 10 client.0 _ll_register_callbacks cb 0x55c254f69f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c2550162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c2550162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33628/0 (socket says 172.21.15.100:33628) 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 -- 172.21.15.100:0/3125314437 learned_addr learned my addr 172.21.15.100:0/3125314437 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7563fff640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255015db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7568907640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255015870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 -- 172.21.15.100:0/3125314437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 msgr2=0x55c255015870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255015870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 -- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 msgr2=0x55c255015db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255015db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 -- 172.21.15.100:0/3125314437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7558009950 con 0x55c2550188b0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7568907640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255015870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7569108640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c2550162f0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f755c034cc0 tx=0x7f755c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadc61ab29b27f79 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7543fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7543fff640 1 -- 172.21.15.100:0/3125314437 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c032ac0 con 0x55c2550188b0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.969+0000 7f7543fff640 1 -- 172.21.15.100:0/3125314437 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f755c003970 con 0x55c2550188b0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c254f50f40 con 0x55c2550188b0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f7543fff640 1 -- 172.21.15.100:0/3125314437 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755c003b30 con 0x55c2550188b0 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 10 client.20028 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c254f75320 con 0x55c2550188b0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f7543fff640 1 -- 172.21.15.100:0/3125314437 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f755c059030 con 0x55c2550188b0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f7543fff640 1 -- 172.21.15.100:0/3125314437 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f755c041440 con 0x55c2550188b0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 10 client.20028 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 10 client.20028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 20 client.20028 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25502b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 4 client.20028 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.970+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c254f6d310 con 0x55c255028ea0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.971+0000 7f7568907640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25502b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.971+0000 7f7568907640 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25502b2c0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebdc3d4be1264d2 server_cookie=3c5b6b30ba47d060 in_seq=0 out_seq=0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.971+0000 7f7543fff640 10 client.20028 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.972+0000 7f7543fff640 1 -- 172.21.15.100:0/3125314437 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c254f6d310 con 0x55c255028ea0 2024-03-12T05:33:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.972+0000 7f7543fff640 10 client.20028 handle_command_reply: tid=0 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 client.20028 shutdown 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 msgr2=0x55c2550162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c2550162f0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f755c034cc0 tx=0x7f755c02c0f0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 shutdown_connections 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25502b2c0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f7543fff640 0 client.20028 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c2550162f0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255015db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 --2- 172.21.15.100:0/3125314437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255015870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 >> 172.21.15.100:0/3125314437 conn(0x55c255009840 msgr2=0x55c25500b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 shutdown_connections 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.974+0000 7f756d80f740 1 -- 172.21.15.100:0/3125314437 wait complete. 2024-03-12T05:33:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:32.975+0000 7f756d80f740 20 client.20028 trim_cache size 0 max 16384 2024-03-12T05:33:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 Processor -- start 2024-03-12T05:33:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 -- start start 2024-03-12T05:33:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255017fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255016110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c255012af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2550093a0 con 0x55c2550188b0 2024-03-12T05:33:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25500e610 con 0x55c25500fd30 2024-03-12T05:33:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f756d80f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c255030500 con 0x55c25500f380 2024-03-12T05:33:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f7569108640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c255012af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.000+0000 7f7569108640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c255012af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33634/0 (socket says 172.21.15.100:33634) 2024-03-12T05:33:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 -- 172.21.15.100:0/399684648 learned_addr learned my addr 172.21.15.100:0/399684648 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 -- 172.21.15.100:0/399684648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 msgr2=0x55c255017fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T05:33:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255017fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 -- 172.21.15.100:0/399684648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 msgr2=0x55c255016110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255016110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 -- 172.21.15.100:0/399684648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c254f11c00 con 0x55c2550188b0 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.001+0000 7f7569108640 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c255012af0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f755c005c90 tx=0x7f755c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ec6800b191165d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.002+0000 7f754e7fc640 1 -- 172.21.15.100:0/399684648 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c032d70 con 0x55c2550188b0 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.002+0000 7f754e7fc640 1 -- 172.21.15.100:0/399684648 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f755c059040 con 0x55c2550188b0 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.002+0000 7f754e7fc640 1 -- 172.21.15.100:0/399684648 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0042a0 con 0x55c2550188b0 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 -- 172.21.15.100:0/399684648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 msgr2=0x55c255012af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c255012af0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f755c005c90 tx=0x7f755c009c20 comp rx=0 tx=0).stop 2024-03-12T05:33:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 -- 172.21.15.100:0/399684648 shutdown_connections 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2550188b0 0x55c255012af0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c255016110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 --2- 172.21.15.100:0/399684648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c25500f380 0x55c255017fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 -- 172.21.15.100:0/399684648 >> 172.21.15.100:0/399684648 conn(0x55c255009840 msgr2=0x55c255010ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 -- 172.21.15.100:0/399684648 shutdown_connections 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 -- 172.21.15.100:0/399684648 wait complete. 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 -- start start 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.003+0000 7f756d80f740 1 Processor -- start 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25501e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c25501ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c255087bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25501f5d0 con 0x55c25500f380 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c255088100 con 0x55c25500fd30 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c255088290 con 0x55c2550188b0 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f756d80f740 10 client.0 _ll_register_callbacks cb 0x7f75580097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f7568907640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25501e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f7568907640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25501e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33644/0 (socket says 172.21.15.100:33644) 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f7568907640 1 -- 172.21.15.100:0/1920641974 learned_addr learned my addr 172.21.15.100:0/1920641974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f7563fff640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c25501ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.004+0000 7f7568907640 1 -- 172.21.15.100:0/1920641974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 msgr2=0x55c255087bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7569108640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c255087bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7568907640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c255087bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7568907640 1 -- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 msgr2=0x55c25501ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7568907640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c25501ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7568907640 1 -- 172.21.15.100:0/1920641974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c254f75320 con 0x55c25500f380 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7563fff640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c25501ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7569108640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c255087bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7568907640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25501e950 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7558009c40 tx=0x7f7558005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d3cfb8aa861ad4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7543fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7543fff640 1 -- 172.21.15.100:0/1920641974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558005040 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7543fff640 1 -- 172.21.15.100:0/1920641974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f75580051a0 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c254f9c900 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f7543fff640 1 -- 172.21.15.100:0/1920641974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7558005380 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f756d80f740 10 client.20034 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.005+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c254f37910 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f7543fff640 1 -- 172.21.15.100:0/1920641974 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c254f9c900 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f7543fff640 1 -- 172.21.15.100:0/1920641974 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75580196a0 con 0x55c25500f380 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f756d80f740 10 client.20034 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f756d80f740 10 client.20034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:33.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f756d80f740 20 client.20034 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:33.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f756d80f740 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25508bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f756d80f740 4 client.20034 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-12T05:33:33.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.006+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c254c76860 con 0x55c255028ea0 2024-03-12T05:33:33.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.007+0000 7f7563fff640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25508bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.008+0000 7f7563fff640 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25508bad0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1aefc3bc7ab1554 server_cookie=23dca1fb44aaeba3 in_seq=0 out_seq=0 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.008+0000 7f7543fff640 10 client.20034 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f7543fff640 1 -- 172.21.15.100:0/1920641974 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c254c76860 con 0x55c255028ea0 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f7543fff640 10 client.20034 handle_command_reply: tid=0 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 client.20034 shutdown 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 msgr2=0x55c25501e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25501e950 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7558009c40 tx=0x7f7558005a70 comp rx=0 tx=0).stop 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 shutdown_connections 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55c255028ea0 0x55c25508bad0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2550188b0 0x55c255087bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c25500fd30 0x55c25501ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 --2- 172.21.15.100:0/1920641974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25500f380 0x55c25501e950 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 >> 172.21.15.100:0/1920641974 conn(0x55c255009840 msgr2=0x55c255008050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f7543fff640 0 client.20034 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 shutdown_connections 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.010+0000 7f756d80f740 1 -- 172.21.15.100:0/1920641974 wait complete. 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756d80f740 20 client.20034 trim_cache size 0 max 16384 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:33:33.009 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-03-12T05:33:33.010 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "0241771e-4350-4f1e-9f94-57d98b06e396", 2024-03-12T05:33:33.010 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-03-12T05:33:33.010 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75540a1ba0 msgr2=0x7f75540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75540a1ba0 0x7f75540a4060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f755c005cc0 tx=0x7f755c032ef0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 msgr2=0x7f756419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756419d990 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f755000e9c0 tx=0x7f755000ee80 comp rx=0 tx=0).stop 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 shutdown_connections 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f75540a1ba0 0x7f75540a4060 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7564197600 0x7f7564197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75641064c0 0x7f756419d990 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 --2- 172.21.15.100:0/556954558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75640685f0 0x7f756419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 >> 172.21.15.100:0/556954558 conn(0x7f7564077180 msgr2=0x7f75640fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 shutdown_connections 2024-03-12T05:33:33.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.011+0000 7f756ab6b640 1 -- 172.21.15.100:0/556954558 wait complete. 2024-03-12T05:33:33.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:33:33.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:33:33.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 -- 172.21.15.100:0/4213565819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff060073790 msgr2=0x7ff060073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 --2- 172.21.15.100:0/4213565819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff060073790 0x7ff060073b70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff050009900 tx=0x7ff05002bc60 comp rx=0 tx=0).stop 2024-03-12T05:33:33.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 -- 172.21.15.100:0/4213565819 shutdown_connections 2024-03-12T05:33:33.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 --2- 172.21.15.100:0/4213565819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff06010fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 --2- 172.21.15.100:0/4213565819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0600740b0 0x7ff06010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 --2- 172.21.15.100:0/4213565819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff060073790 0x7ff060073b70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 -- 172.21.15.100:0/4213565819 >> 172.21.15.100:0/4213565819 conn(0x7ff0600fcc20 msgr2=0x7ff0600ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 -- 172.21.15.100:0/4213565819 shutdown_connections 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.148+0000 7ff067102640 1 -- 172.21.15.100:0/4213565819 wait complete. 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.149+0000 7ff067102640 1 Processor -- start 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.149+0000 7ff067102640 1 -- start start 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.149+0000 7ff067102640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff060073790 0x7ff06019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.149+0000 7ff067102640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0600740b0 0x7ff06019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.149+0000 7ff067102640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff0601997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.149+0000 7ff067102640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff060199f50 con 0x7ff06010d930 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff067102640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06019a0c0 con 0x7ff060073790 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff067102640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06019a280 con 0x7ff0600740b0 2024-03-12T05:33:33.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff0601997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff0601997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33658/0 (socket says 172.21.15.100:33658) 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 -- 172.21.15.100:0/500477012 learned_addr learned my addr 172.21.15.100:0/500477012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff064e9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff060073790 0x7ff06019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff05ffff640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0600740b0 0x7ff06019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 -- 172.21.15.100:0/500477012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0600740b0 msgr2=0x7ff06019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0600740b0 0x7ff06019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 -- 172.21.15.100:0/500477012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff060073790 msgr2=0x7ff06019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff060073790 0x7ff06019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 -- 172.21.15.100:0/500477012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff050009560 con 0x7ff06010d930 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff06569f640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff0601997c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff05800eb10 tx=0x7ff05800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff05dffb640 1 -- 172.21.15.100:0/500477012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058004b20 con 0x7ff06010d930 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06019a560 con 0x7ff06010d930 2024-03-12T05:33:33.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff05dffb640 1 -- 172.21.15.100:0/500477012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff058004c80 con 0x7ff06010d930 2024-03-12T05:33:33.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.150+0000 7ff05dffb640 1 -- 172.21.15.100:0/500477012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058010650 con 0x7ff06010d930 2024-03-12T05:33:33.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.151+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0601a6630 con 0x7ff06010d930 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.153+0000 7ff05dffb640 1 -- 172.21.15.100:0/500477012 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff058005020 con 0x7ff06010d930 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 Processor -- start 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 -- start start 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff05dffb640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0480a1ba0 0x7ff0480a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a34270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a347b0 0x560290a36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029099a280 con 0x560290a2b2b0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602909977b0 con 0x560290a2bbd0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff069db3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290992f90 con 0x560290a347b0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff05dffb640 1 -- 172.21.15.100:0/500477012 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff058014070 con 0x7ff06010d930 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff064e9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff064e9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a2b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33672/0 (socket says 172.21.15.100:33672) 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff06569f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a347b0 0x560290a36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff064e9e640 1 -- 172.21.15.100:0/2737637676 learned_addr learned my addr 172.21.15.100:0/2737637676 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff05ffff640 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a34270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.154+0000 7ff05ffff640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0480a1ba0 0x7ff0480a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff06569f640 1 -- 172.21.15.100:0/2737637676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 msgr2=0x560290a34270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff06569f640 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a34270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff06569f640 1 -- 172.21.15.100:0/2737637676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 msgr2=0x560290a2b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff06569f640 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a2b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff06569f640 1 -- 172.21.15.100:0/2737637676 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560290951570 con 0x560290a347b0 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff064e9e640 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a2b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff05ffff640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0480a1ba0 0x7ff0480a4060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff054005d00 tx=0x7ff05400b7f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff06569f640 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a347b0 0x560290a36ba0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff0580cb4f0 tx=0x7ff05800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d11a7cfd014ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff046ffd640 1 -- 172.21.15.100:0/2737637676 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05809eb20 con 0x560290a347b0 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff046ffd640 1 -- 172.21.15.100:0/2737637676 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0580ba040 con 0x560290a347b0 2024-03-12T05:33:33.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.155+0000 7ff046ffd640 1 -- 172.21.15.100:0/2737637676 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058097040 con 0x560290a347b0 2024-03-12T05:33:33.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 -- 172.21.15.100:0/2737637676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a347b0 msgr2=0x560290a36ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a347b0 0x560290a36ba0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff0580cb4f0 tx=0x7ff05800cf70 comp rx=0 tx=0).stop 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 -- 172.21.15.100:0/2737637676 shutdown_connections 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a347b0 0x560290a36ba0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a34270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 --2- 172.21.15.100:0/2737637676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a2b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 -- 172.21.15.100:0/2737637676 >> 172.21.15.100:0/2737637676 conn(0x560290a25990 msgr2=0x560290a25d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.156+0000 7ff069db3740 1 -- 172.21.15.100:0/2737637676 shutdown_connections 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.157+0000 7ff069db3740 1 -- 172.21.15.100:0/2737637676 wait complete. 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.157+0000 7ff069db3740 1 -- start start 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.157+0000 7ff069db3740 1 Processor -- start 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.157+0000 7ff069db3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.157+0000 7ff069db3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.157+0000 7ff069db3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 0x560290a31a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff069db3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290666430 con 0x560290a347b0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff069db3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290622e00 con 0x560290a2bbd0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff069db3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290917fd0 con 0x560290a2b2b0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 0x560290a31a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff069db3740 10 client.0 _ll_register_callbacks cb 0x56029096dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 0x560290a31a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33676/0 (socket says 172.21.15.100:33676) 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 -- 172.21.15.100:0/470835523 learned_addr learned my addr 172.21.15.100:0/470835523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff064e9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff05ffff640 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 -- 172.21.15.100:0/470835523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 msgr2=0x560290a34130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a34130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 -- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 msgr2=0x560290a314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 -- 172.21.15.100:0/470835523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0601a6630 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff06569f640 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 0x560290a31a40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff0580cb5a0 tx=0x7ff0580a3170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1d33affde45d50 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff03ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff03ffff640 1 -- 172.21.15.100:0/470835523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a2ee0 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff03ffff640 1 -- 172.21.15.100:0/470835523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff05809eb20 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602906927e0 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.158+0000 7ff03ffff640 1 -- 172.21.15.100:0/470835523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff058095210 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 10 client.20046 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560290991610 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff03ffff640 1 -- 172.21.15.100:0/470835523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5602906927e0 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff03ffff640 1 -- 172.21.15.100:0/470835523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff05801f060 con 0x560290a347b0 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 10 client.20046 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 10 client.20046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:33.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 20 client.20046 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a445b0 0x560290a469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 4 client.20046 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602906cf720 con 0x560290a445b0 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.159+0000 7ff064e9e640 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a445b0 0x560290a469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.160+0000 7ff064e9e640 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a445b0 0x560290a469d0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3693a5f154d37dd server_cookie=c9178defae73f47d in_seq=0 out_seq=0 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.160+0000 7ff03ffff640 10 client.20046 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.160+0000 7ff03ffff640 1 -- 172.21.15.100:0/470835523 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602906cf720 con 0x560290a445b0 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.160+0000 7ff03ffff640 10 client.20046 handle_command_reply: tid=0 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.160+0000 7ff069db3740 1 client.20046 shutdown 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 msgr2=0x560290a31a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 0x560290a31a40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff0580cb5a0 tx=0x7ff0580a3170 comp rx=0 tx=0).stop 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 shutdown_connections 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a445b0 0x560290a469d0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a347b0 0x560290a31a40 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a2bbd0 0x560290a314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 --2- 172.21.15.100:0/470835523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a34130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff03ffff640 0 client.20046 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 >> 172.21.15.100:0/470835523 conn(0x560290a25990 msgr2=0x560290a27730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 shutdown_connections 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 1 -- 172.21.15.100:0/470835523 wait complete. 2024-03-12T05:33:33.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.161+0000 7ff069db3740 20 client.20046 trim_cache size 0 max 16384 2024-03-12T05:33:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 Processor -- start 2024-03-12T05:33:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 -- start start 2024-03-12T05:33:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a33bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 0x560290a34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290a2c320 con 0x560290a2bbd0 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290a443b0 con 0x560290a347b0 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff069db3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290a26a10 con 0x560290a2b2b0 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff05ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 0x560290a34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff05ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 0x560290a34130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33680/0 (socket says 172.21.15.100:33680) 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff05ffff640 1 -- 172.21.15.100:0/2795329334 learned_addr learned my addr 172.21.15.100:0/2795329334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff06569f640 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.176+0000 7ff064e9e640 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a33bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff05ffff640 1 -- 172.21.15.100:0/2795329334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 msgr2=0x560290a33bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff05ffff640 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a33bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff05ffff640 1 -- 172.21.15.100:0/2795329334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 msgr2=0x560290a2fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff05ffff640 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a2fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff05ffff640 1 -- 172.21.15.100:0/2795329334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56029092dac0 con 0x560290a2bbd0 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff05ffff640 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 0x560290a34130 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x560290a2ad80 tx=0x7ff05401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96d020ae1910392 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff0467fc640 1 -- 172.21.15.100:0/2795329334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054026070 con 0x560290a2bbd0 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff0467fc640 1 -- 172.21.15.100:0/2795329334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff054010ea0 con 0x560290a2bbd0 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.177+0000 7ff0467fc640 1 -- 172.21.15.100:0/2795329334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054003aa0 con 0x560290a2bbd0 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 -- 172.21.15.100:0/2795329334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 msgr2=0x560290a34130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 0x560290a34130 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x560290a2ad80 tx=0x7ff05401f000 comp rx=0 tx=0).stop 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 -- 172.21.15.100:0/2795329334 shutdown_connections 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a2fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2bbd0 0x560290a34130 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 --2- 172.21.15.100:0/2795329334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2b2b0 0x560290a33bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 -- 172.21.15.100:0/2795329334 >> 172.21.15.100:0/2795329334 conn(0x560290a25990 msgr2=0x560290a2c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 -- 172.21.15.100:0/2795329334 shutdown_connections 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 -- 172.21.15.100:0/2795329334 wait complete. 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 -- start start 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.178+0000 7ff069db3740 1 Processor -- start 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2bbd0 0x560290a38630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a38b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290a9aab0 con 0x560290a2b2b0 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290a9abf0 con 0x560290a347b0 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560290a9ad30 con 0x560290a2bbd0 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 10 client.0 _ll_register_callbacks cb 0x7ff050009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33690/0 (socket says 172.21.15.100:33690) 2024-03-12T05:33:33.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 -- 172.21.15.100:0/717425237 learned_addr learned my addr 172.21.15.100:0/717425237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff06569f640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a38b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff05ffff640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2bbd0 0x560290a38630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 -- 172.21.15.100:0/717425237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2bbd0 msgr2=0x560290a38630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2bbd0 0x560290a38630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 -- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 msgr2=0x560290a38b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a38b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 -- 172.21.15.100:0/717425237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05401a040 con 0x560290a2b2b0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff064e9e640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a380f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff05002c1a0 tx=0x7ff05002c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e122fcbf7b76c6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff03ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff03ffff640 1 -- 172.21.15.100:0/717425237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050044040 con 0x560290a2b2b0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff03ffff640 1 -- 172.21.15.100:0/717425237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff05002cb90 con 0x560290a2b2b0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.179+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0580ba040 con 0x560290a2b2b0 2024-03-12T05:33:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff03ffff640 1 -- 172.21.15.100:0/717425237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff050004050 con 0x560290a2b2b0 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 10 client.20052 fetch_fsmap learned FSMap version 30 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5602909536a0 con 0x560290a2b2b0 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff03ffff640 1 -- 172.21.15.100:0/717425237 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0580ba040 con 0x560290a2b2b0 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff03ffff640 1 -- 172.21.15.100:0/717425237 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff05003c040 con 0x560290a2b2b0 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 10 client.20052 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 10 client.20052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 20 client.20052 populate_metadata read hostname 'smithi100' 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a9c570 0x560290a9e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 4 client.20052 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56029071e890 con 0x560290a9c570 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff05ffff640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a9c570 0x560290a9e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:33:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff03ffff640 10 client.20052 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.180+0000 7ff05ffff640 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a9c570 0x560290a9e990 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49de283488cbcf0b server_cookie=ca1153ec46f765fa in_seq=0 out_seq=0 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff03ffff640 1 -- 172.21.15.100:0/717425237 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56029071e890 con 0x560290a9c570 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff03ffff640 10 client.20052 handle_command_reply: tid=0 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 client.20052 shutdown 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 msgr2=0x560290a380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a380f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff05002c1a0 tx=0x7ff05002c400 comp rx=0 tx=0).stop 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 shutdown_connections 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x560290a9c570 0x560290a9e990 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560290a347b0 0x560290a38b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff03ffff640 0 client.20052 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560290a2bbd0 0x560290a38630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 --2- 172.21.15.100:0/717425237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560290a2b2b0 0x560290a380f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 >> 172.21.15.100:0/717425237 conn(0x560290a25990 msgr2=0x560290a30d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 shutdown_connections 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 1 -- 172.21.15.100:0/717425237 wait complete. 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff069db3740 20 client.20052 trim_cache size 0 max 16384 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout: "0241771e-4350-4f1e-9f94-57d98b06e396": { 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "0241771e-4350-4f1e-9f94-57d98b06e396", 2024-03-12T05:33:33.182 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-03-12T05:33:33.183 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:33:33.183 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:33:33.183 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0480a1ba0 msgr2=0x7ff0480a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.181+0000 7ff067102640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0480a1ba0 0x7ff0480a4060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff054005d00 tx=0x7ff05400b7f0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 msgr2=0x7ff0601997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff0601997c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff05800eb10 tx=0x7ff05800c2d0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 shutdown_connections 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff0480a1ba0 0x7ff0480a4060 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06010d930 0x7ff0601997c0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0600740b0 0x7ff06019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 --2- 172.21.15.100:0/500477012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff060073790 0x7ff06019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 >> 172.21.15.100:0/500477012 conn(0x7ff0600fcc20 msgr2=0x7ff0600fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:33:33.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 shutdown_connections 2024-03-12T05:33:33.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:33:33.182+0000 7ff067102640 1 -- 172.21.15.100:0/500477012 wait complete. 2024-03-12T05:33:33.186 INFO:tasks.cephfs.filesystem:scrub status for tag:0241771e-4350-4f1e-9f94-57d98b06e396 - {'path': '/', 'tag': '0241771e-4350-4f1e-9f94-57d98b06e396', 'options': 'recursive,force'} 2024-03-12T05:33:33.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 254687 25.92 MB/sec execute 376 sec latency 239.156 ms 2024-03-12T05:33:33.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:33 smithi100 ceph-mon[32210]: scrub queued for path: / 2024-03-12T05:33:33.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:33 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:33.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:33 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:33.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:33 smithi156 ceph-mon[36270]: scrub queued for path: / 2024-03-12T05:33:33.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:33 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:33.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:33 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:33.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:33 smithi161 ceph-mon[36043]: scrub queued for path: / 2024-03-12T05:33:33.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:33 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:33.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:33 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:34.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 254954 25.87 MB/sec execute 377 sec latency 188.466 ms 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: active paths [/] 2024-03-12T05:33:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:34 smithi100 ceph-mon[32210]: pgmap v677: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 451 op/s 2024-03-12T05:33:34.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:34.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:34.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:34.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: active paths [/] 2024-03-12T05:33:34.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:34 smithi156 ceph-mon[36270]: pgmap v677: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 451 op/s 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: active paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: scrub summary: idle+waiting paths [/] 2024-03-12T05:33:34.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:34 smithi161 ceph-mon[36043]: pgmap v677: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 451 op/s 2024-03-12T05:33:35.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 255305 25.85 MB/sec execute 378 sec latency 153.072 ms 2024-03-12T05:33:36.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 255691 25.84 MB/sec execute 379 sec latency 264.936 ms 2024-03-12T05:33:36.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:36 smithi156 ceph-mon[36270]: pgmap v678: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.6 MiB/s wr, 366 op/s 2024-03-12T05:33:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:36 smithi161 ceph-mon[36043]: pgmap v678: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.6 MiB/s wr, 366 op/s 2024-03-12T05:33:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:36 smithi100 ceph-mon[32210]: pgmap v678: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.6 MiB/s wr, 366 op/s 2024-03-12T05:33:37.246 INFO:tasks.workunit.client.0.smithi100.stdout: 10 256208 25.85 MB/sec execute 380 sec latency 118.401 ms 2024-03-12T05:33:38.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 256623 25.83 MB/sec execute 381 sec latency 150.066 ms 2024-03-12T05:33:38.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:38 smithi156 ceph-mon[36270]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 597 op/s 2024-03-12T05:33:38.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:38 smithi161 ceph-mon[36043]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 597 op/s 2024-03-12T05:33:38.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:38 smithi100 ceph-mon[32210]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 597 op/s 2024-03-12T05:33:39.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 257120 25.83 MB/sec execute 382 sec latency 139.263 ms 2024-03-12T05:33:40.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 257460 25.80 MB/sec execute 383 sec latency 151.534 ms 2024-03-12T05:33:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:40 smithi156 ceph-mon[36270]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.9 MiB/s wr, 406 op/s 2024-03-12T05:33:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:40 smithi161 ceph-mon[36043]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.9 MiB/s wr, 406 op/s 2024-03-12T05:33:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:40 smithi100 ceph-mon[32210]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.9 MiB/s wr, 406 op/s 2024-03-12T05:33:41.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 257793 25.79 MB/sec execute 384 sec latency 288.688 ms 2024-03-12T05:33:42.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 258151 25.78 MB/sec execute 385 sec latency 135.293 ms 2024-03-12T05:33:42.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:42 smithi156 ceph-mon[36270]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 499 op/s 2024-03-12T05:33:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:42 smithi161 ceph-mon[36043]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 499 op/s 2024-03-12T05:33:42.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:42 smithi100 ceph-mon[32210]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 499 op/s 2024-03-12T05:33:43.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 258567 25.77 MB/sec execute 386 sec latency 187.956 ms 2024-03-12T05:33:44.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 258982 25.75 MB/sec execute 387 sec latency 177.264 ms 2024-03-12T05:33:44.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:44 smithi156 ceph-mon[36270]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 505 op/s 2024-03-12T05:33:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:44 smithi161 ceph-mon[36043]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 505 op/s 2024-03-12T05:33:44.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:44 smithi100 ceph-mon[32210]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 505 op/s 2024-03-12T05:33:45.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 259430 25.75 MB/sec execute 388 sec latency 147.381 ms 2024-03-12T05:33:45.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:45 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:33:45.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:45 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:33:45.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:45 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:33:46.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 259800 25.74 MB/sec execute 389 sec latency 332.239 ms 2024-03-12T05:33:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:46 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:33:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:46 smithi156 ceph-mon[36270]: scrub summary: idle 2024-03-12T05:33:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:46 smithi156 ceph-mon[36270]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.6 MiB/s wr, 413 op/s 2024-03-12T05:33:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:46 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:33:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:46 smithi161 ceph-mon[36043]: scrub summary: idle 2024-03-12T05:33:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:46 smithi161 ceph-mon[36043]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.6 MiB/s wr, 413 op/s 2024-03-12T05:33:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:46 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:33:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:46 smithi100 ceph-mon[32210]: scrub summary: idle 2024-03-12T05:33:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:46 smithi100 ceph-mon[32210]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.6 MiB/s wr, 413 op/s 2024-03-12T05:33:47.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 260165 25.72 MB/sec execute 390 sec latency 158.015 ms 2024-03-12T05:33:47.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:47 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:33:47.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:47 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:33:47.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:47 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:33:48.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 260615 25.73 MB/sec execute 391 sec latency 156.775 ms 2024-03-12T05:33:49.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:48 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:48 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:48 smithi100 ceph-mon[32210]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 591 op/s 2024-03-12T05:33:49.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:48 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:33:49.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:48 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:33:49.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:48 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:48 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:48 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:48 smithi156 ceph-mon[36270]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 591 op/s 2024-03-12T05:33:49.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:48 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:33:49.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:48 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:33:49.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:48 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:48 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:48 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:48 smithi161 ceph-mon[36043]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 591 op/s 2024-03-12T05:33:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:48 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:33:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:48 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:33:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:48 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:33:49.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 260929 25.73 MB/sec execute 392 sec latency 148.761 ms 2024-03-12T05:33:50.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 261175 25.72 MB/sec execute 393 sec latency 149.179 ms 2024-03-12T05:33:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:50 smithi156 ceph-mon[36270]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.1 MiB/s wr, 360 op/s 2024-03-12T05:33:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:50 smithi161 ceph-mon[36043]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.1 MiB/s wr, 360 op/s 2024-03-12T05:33:50.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:50 smithi100 ceph-mon[32210]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.1 MiB/s wr, 360 op/s 2024-03-12T05:33:51.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 261431 25.68 MB/sec execute 394 sec latency 343.048 ms 2024-03-12T05:33:52.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 261913 25.67 MB/sec execute 395 sec latency 359.772 ms 2024-03-12T05:33:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:52 smithi156 ceph-mon[36270]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 452 op/s 2024-03-12T05:33:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:52 smithi161 ceph-mon[36043]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 452 op/s 2024-03-12T05:33:52.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:52 smithi100 ceph-mon[32210]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 452 op/s 2024-03-12T05:33:53.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 262400 25.69 MB/sec execute 396 sec latency 183.796 ms 2024-03-12T05:33:54.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 262776 25.68 MB/sec execute 397 sec latency 140.049 ms 2024-03-12T05:33:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:54 smithi156 ceph-mon[36270]: pgmap v687: 97 pgs: 97 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-03-12T05:33:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:54 smithi161 ceph-mon[36043]: pgmap v687: 97 pgs: 97 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-03-12T05:33:54.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:54 smithi100 ceph-mon[32210]: pgmap v687: 97 pgs: 97 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-03-12T05:33:55.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 263199 25.65 MB/sec execute 398 sec latency 157.982 ms 2024-03-12T05:33:56.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 263608 25.64 MB/sec execute 399 sec latency 249.683 ms 2024-03-12T05:33:56.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:56 smithi156 ceph-mon[36270]: pgmap v688: 97 pgs: 97 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 340 op/s 2024-03-12T05:33:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:56 smithi161 ceph-mon[36043]: pgmap v688: 97 pgs: 97 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 340 op/s 2024-03-12T05:33:56.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:56 smithi100 ceph-mon[32210]: pgmap v688: 97 pgs: 97 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 340 op/s 2024-03-12T05:33:57.250 INFO:tasks.workunit.client.0.smithi100.stdout: 10 264041 25.63 MB/sec execute 400 sec latency 386.569 ms 2024-03-12T05:33:58.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 264425 25.63 MB/sec execute 401 sec latency 181.008 ms 2024-03-12T05:33:58.365 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:33:58 smithi100 ceph-mon[32210]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 532 op/s 2024-03-12T05:33:58.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:33:58 smithi156 ceph-mon[36270]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 532 op/s 2024-03-12T05:33:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:33:58 smithi161 ceph-mon[36043]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 532 op/s 2024-03-12T05:33:59.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 264888 25.62 MB/sec execute 402 sec latency 147.256 ms 2024-03-12T05:34:00.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 265400 25.65 MB/sec execute 403 sec latency 155.781 ms 2024-03-12T05:34:00.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:00 smithi156 ceph-mon[36270]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 354 op/s 2024-03-12T05:34:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:00 smithi161 ceph-mon[36043]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 354 op/s 2024-03-12T05:34:00.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:00 smithi100 ceph-mon[32210]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 354 op/s 2024-03-12T05:34:01.247 INFO:tasks.workunit.client.0.smithi100.stdout: 10 265400 25.59 MB/sec execute 404 sec latency 1155.826 ms 2024-03-12T05:34:02.026 INFO:tasks.workunit.client.0.smithi100.stdout:[467444] write failed on handle 11819 (Resource temporarily unavailable) 2024-03-12T05:34:02.026 INFO:tasks.workunit.client.0.smithi100.stdout:Child failed with status 1 2024-03-12T05:34:02.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:34:02.061 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-12T05:34:02.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-12T05:34:02.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:02 smithi100 ceph-mon[32210]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 438 op/s 2024-03-12T05:34:02.423 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-12T05:34:06.203 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=69450056edff49a29b93dd18f48af00f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-12T05:34:06.206 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-12T05:34:06.216 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-12T05:34:06.216 INFO:teuthology.task.exec:Executing custom commands... 2024-03-12T05:34:06.216 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:34:06.216 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-12T05:34:06.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-12T05:34:06.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-12T05:34:06.222 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:04 smithi100 ceph-mon[32210]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 446 op/s 2024-03-12T05:34:06.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:02 smithi161 ceph-mon[36043]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 438 op/s 2024-03-12T05:34:06.223 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:04 smithi161 ceph-mon[36043]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 446 op/s 2024-03-12T05:34:06.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:02 smithi156 ceph-mon[36270]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 438 op/s 2024-03-12T05:34:06.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:04 smithi156 ceph-mon[36270]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 446 op/s 2024-03-12T05:34:06.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-12T05:34:06.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-12T05:34:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.393+0000 7f3bc64f4640 1 -- 172.21.15.100:0/3644107116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 msgr2=0x7f3bc011cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.393+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/3644107116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 0x7f3bc011cd50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009be0 tx=0x7f3bb002bf30 comp rx=0 tx=0).stop 2024-03-12T05:34:06.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.395+0000 7f3bc64f4640 1 -- 172.21.15.100:0/3644107116 shutdown_connections 2024-03-12T05:34:06.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.395+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/3644107116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 0x7f3bc011cd50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.395+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/3644107116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc010fd10 0x7f3bc010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.395+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/3644107116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.395+0000 7f3bc64f4640 1 -- 172.21.15.100:0/3644107116 >> 172.21.15.100:0/3644107116 conn(0x7f3bc006c270 msgr2=0x7f3bc006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.401+0000 7f3bc64f4640 1 -- 172.21.15.100:0/3644107116 shutdown_connections 2024-03-12T05:34:06.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.402+0000 7f3bc64f4640 1 -- 172.21.15.100:0/3644107116 wait complete. 2024-03-12T05:34:06.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.402+0000 7f3bc64f4640 1 Processor -- start 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 -- start start 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc0117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc010fd10 0x7f3bc01182f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 0x7f3bc0110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01116e0 con 0x7f3bc010fd10 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0111850 con 0x7f3bc010f8a0 2024-03-12T05:34:06.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bc64f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0111a10 con 0x7f3bc011a960 2024-03-12T05:34:06.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc0117db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc0117db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:50156/0 (socket says 172.21.15.100:50156) 2024-03-12T05:34:06.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.403+0000 7f3bbffff640 1 -- 172.21.15.100:0/4122994268 learned_addr learned my addr 172.21.15.100:0/4122994268 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.404+0000 7f3bc4a91640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 0x7f3bc0110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.404+0000 7f3bbffff640 1 -- 172.21.15.100:0/4122994268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 msgr2=0x7f3bc0110f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.404+0000 7f3bbffff640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 0x7f3bc0110f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.404+0000 7f3bbffff640 1 -- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc010fd10 msgr2=0x7f3bc01182f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-12T05:34:06.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.404+0000 7f3bbffff640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc010fd10 0x7f3bc01182f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.405+0000 7f3bbffff640 1 -- 172.21.15.100:0/4122994268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb0009840 con 0x7f3bc010f8a0 2024-03-12T05:34:06.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.405+0000 7f3bbffff640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc0117db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800d950 tx=0x7f3bb800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.409+0000 7f3bbd7fa640 1 -- 172.21.15.100:0/4122994268 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8009c30 con 0x7f3bc010f8a0 2024-03-12T05:34:06.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.409+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc0111c90 con 0x7f3bc010f8a0 2024-03-12T05:34:06.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.409+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc0115c50 con 0x7f3bc010f8a0 2024-03-12T05:34:06.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.409+0000 7f3bbd7fa640 1 -- 172.21.15.100:0/4122994268 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bb8009d90 con 0x7f3bc010f8a0 2024-03-12T05:34:06.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.409+0000 7f3bbd7fa640 1 -- 172.21.15.100:0/4122994268 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800f740 con 0x7f3bc010f8a0 2024-03-12T05:34:06.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 Processor -- start 2024-03-12T05:34:06.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 -- start start 2024-03-12T05:34:06.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b56270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b567b0 0x561d46b58ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46abc280 con 0x561d46b4d2b0 2024-03-12T05:34:06.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46ab97b0 con 0x561d46b4dbd0 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.414+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46ab4f90 con 0x561d46b567b0 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b567b0 0x561d46b58ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b567b0 0x561d46b58ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35822/0 (socket says 172.21.15.100:35822) 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/1525740482 learned_addr learned my addr 172.21.15.100:0/1525740482 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bc4a91640 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbffff640 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b56270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbffff640 1 -- 172.21.15.100:0/1525740482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b567b0 msgr2=0x561d46b58ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbffff640 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b567b0 0x561d46b58ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbffff640 1 -- 172.21.15.100:0/1525740482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 msgr2=0x561d46b4d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbffff640 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b4d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.415+0000 7f3bbffff640 1 -- 172.21.15.100:0/1525740482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d46a73570 con 0x561d46b4dbd0 2024-03-12T05:34:06.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.416+0000 7f3bbffff640 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b56270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0115de0 tx=0x7f3bb8005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3509e9c6e77f223a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.420+0000 7f3ba67fc640 1 -- 172.21.15.100:0/1525740482 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb804e070 con 0x561d46b4dbd0 2024-03-12T05:34:06.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.420+0000 7f3ba67fc640 1 -- 172.21.15.100:0/1525740482 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bb80cf800 con 0x561d46b4dbd0 2024-03-12T05:34:06.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:06 smithi100 ceph-mon[32210]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 375 op/s 2024-03-12T05:34:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.421+0000 7f3bbd7fa640 1 -- 172.21.15.100:0/4122994268 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3bb80048d0 con 0x7f3bc010f8a0 2024-03-12T05:34:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.421+0000 7f3bc91a2740 1 -- 172.21.15.100:0/1525740482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 msgr2=0x561d46b56270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.421+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b56270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0115de0 tx=0x7f3bb8005620 comp rx=0 tx=0).stop 2024-03-12T05:34:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.422+0000 7f3bbd7fa640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3ba80a1db0 0x7f3ba80a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.422+0000 7f3bbd7fa640 1 -- 172.21.15.100:0/4122994268 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3bb80cee00 con 0x7f3bc010f8a0 2024-03-12T05:34:06.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.422+0000 7f3bc4a91640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3ba80a1db0 0x7f3ba80a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.424+0000 7f3bc91a2740 1 -- 172.21.15.100:0/1525740482 shutdown_connections 2024-03-12T05:34:06.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.424+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b567b0 0x561d46b58ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.424+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b56270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.424+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/1525740482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b4d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.424+0000 7f3bc91a2740 1 -- 172.21.15.100:0/1525740482 >> 172.21.15.100:0/1525740482 conn(0x561d46b47990 msgr2=0x561d46b47d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.425+0000 7f3bc4a91640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3ba80a1db0 0x7f3ba80a4270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3bb002c540 tx=0x7f3bb002e040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.427+0000 7f3bc91a2740 1 -- 172.21.15.100:0/1525740482 shutdown_connections 2024-03-12T05:34:06.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.428+0000 7f3bc91a2740 1 -- 172.21.15.100:0/1525740482 wait complete. 2024-03-12T05:34:06.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.429+0000 7f3bc91a2740 1 -- start start 2024-03-12T05:34:06.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.429+0000 7f3bc91a2740 1 Processor -- start 2024-03-12T05:34:06.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 0x561d46b53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b567b0 0x561d46b539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46788430 con 0x561d46b4d2b0 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46744e00 con 0x561d46b567b0 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46a39fd0 con 0x561d46b4dbd0 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bc91a2740 10 client.0 _ll_register_callbacks cb 0x561d46a8ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 0x561d46b53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 0x561d46b53470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35830/0 (socket says 172.21.15.100:35830) 2024-03-12T05:34:06.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.430+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/377864253 learned_addr learned my addr 172.21.15.100:0/377864253 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.431+0000 7f3bbffff640 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.431+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b567b0 msgr2=0x561d46b539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.431+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b567b0 0x561d46b539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.431+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/377864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 msgr2=0x561d46b560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.431+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.431+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/377864253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc0115c50 con 0x561d46b4dbd0 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.432+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 0x561d46b53470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400dd20 tx=0x7f3bb400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a79f33faeba7db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.432+0000 7f3b9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.432+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/377864253 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4011640 con 0x561d46b4dbd0 2024-03-12T05:34:06.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.432+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/377864253 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3bb40117a0 con 0x561d46b4dbd0 2024-03-12T05:34:06.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.432+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d467b47e0 con 0x561d46b4dbd0 2024-03-12T05:34:06.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.434+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/377864253 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb4011960 con 0x561d46b4dbd0 2024-03-12T05:34:06.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.434+0000 7f3bc91a2740 10 client.29326 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:06.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.434+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d46ab3610 con 0x561d46b4dbd0 2024-03-12T05:34:06.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.435+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/377864253 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561d467b47e0 con 0x561d46b4dbd0 2024-03-12T05:34:06.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.436+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/377864253 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3bb401b410 con 0x561d46b4dbd0 2024-03-12T05:34:06.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.437+0000 7f3bc91a2740 10 client.29326 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:06.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.437+0000 7f3bc91a2740 10 client.29326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:34:06.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.437+0000 7f3bc91a2740 20 client.29326 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:06.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.437+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46b668e0 0x561d46b68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.437+0000 7f3bc91a2740 4 client.29326 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:34:06.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.437+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d467f1720 con 0x561d46b668e0 2024-03-12T05:34:06.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.439+0000 7f3bbffff640 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46b668e0 0x561d46b68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.440+0000 7f3bbffff640 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46b668e0 0x561d46b68d20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48515a4a88e24ff server_cookie=96984ebb787472 in_seq=0 out_seq=0 2024-03-12T05:34:06.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.440+0000 7f3b9f7fe640 10 client.29326 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:06.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.441+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/377864253 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d467f1720 con 0x561d46b668e0 2024-03-12T05:34:06.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.441+0000 7f3b9f7fe640 10 client.29326 handle_command_reply: tid=0 2024-03-12T05:34:06.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.441+0000 7f3bc91a2740 1 client.29326 shutdown 2024-03-12T05:34:06.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.442+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 msgr2=0x561d46b53470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.442+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 0x561d46b53470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400dd20 tx=0x7f3bb400f730 comp rx=0 tx=0).stop 2024-03-12T05:34:06.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.442+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 shutdown_connections 2024-03-12T05:34:06.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46b668e0 0x561d46b68d20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b567b0 0x561d46b539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4dbd0 0x561d46b53470 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/377864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b4d2b0 0x561d46b560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 >> 172.21.15.100:0/377864253 conn(0x561d46b47990 msgr2=0x561d46b49730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3b9f7fe640 0 client.29326 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 shutdown_connections 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 1 -- 172.21.15.100:0/377864253 wait complete. 2024-03-12T05:34:06.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.443+0000 7f3bc91a2740 20 client.29326 trim_cache size 0 max 16384 2024-03-12T05:34:06.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:06 smithi156 ceph-mon[36270]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 375 op/s 2024-03-12T05:34:06.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 Processor -- start 2024-03-12T05:34:06.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 -- start start 2024-03-12T05:34:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b55ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46b51d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46b62cd0 con 0x561d46b567b0 2024-03-12T05:34:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46b48790 con 0x561d46b4dbd0 2024-03-12T05:34:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc91a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46b53fb0 con 0x561d46b4d2b0 2024-03-12T05:34:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc4a91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46b51d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b55ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:50174/0 (socket says 172.21.15.100:50174) 2024-03-12T05:34:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/167498001 learned_addr learned my addr 172.21.15.100:0/167498001 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/167498001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 msgr2=0x561d46b55ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b55ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/167498001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 msgr2=0x561d46b51d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46b51d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/167498001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d46a4fac0 con 0x561d46b4dbd0 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.468+0000 7f3bc4a91640 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46b51d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.469+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b560e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3bb401b820 tx=0x7f3bb400d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133aeaeab53e7e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.469+0000 7f3ba5ffb640 1 -- 172.21.15.100:0/167498001 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb401b410 con 0x561d46b4dbd0 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.469+0000 7f3ba5ffb640 1 -- 172.21.15.100:0/167498001 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bb401b570 con 0x561d46b4dbd0 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.469+0000 7f3ba5ffb640 1 -- 172.21.15.100:0/167498001 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40039d0 con 0x561d46b4dbd0 2024-03-12T05:34:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 -- 172.21.15.100:0/167498001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 msgr2=0x561d46b560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b560e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3bb401b820 tx=0x7f3bb400d2e0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 -- 172.21.15.100:0/167498001 shutdown_connections 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46b51d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b560e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/167498001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b55ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 -- 172.21.15.100:0/167498001 >> 172.21.15.100:0/167498001 conn(0x561d46b47990 msgr2=0x561d46b4e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 -- 172.21.15.100:0/167498001 shutdown_connections 2024-03-12T05:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 -- 172.21.15.100:0/167498001 wait complete. 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 -- start start 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 Processor -- start 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.470+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46bbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b50aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:50190/0 (socket says 172.21.15.100:50190) 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc4a91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46bbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b50560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.100:35860/0 (socket says 172.21.15.100:35860) 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/2554989748 learned_addr learned my addr 172.21.15.100:0/2554989748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46b51370 con 0x561d46b567b0 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46bbd380 con 0x561d46b4dbd0 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d46bbd4c0 con 0x561d46b4d2b0 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 10 client.0 _ll_register_callbacks cb 0x7f3bb801e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/2554989748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 msgr2=0x561d46b50560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b50560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/2554989748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 msgr2=0x561d46bbce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46bbce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc011d610 con 0x561d46b4dbd0 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbffff640 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bbf7fe640 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b50aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005850 tx=0x7f3bb4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9cefdf921cfe900 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3b9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-12T05:34:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/2554989748 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb401b410 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/2554989748 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3bb401b990 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.471+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bb80cf800 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/2554989748 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb40037f0 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3bc91a2740 10 client.29393 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d46ab3610 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/2554989748 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bb80cf800 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/2554989748 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3bb40118f0 con 0x561d46b4dbd0 2024-03-12T05:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3bc91a2740 10 client.29393 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3bc91a2740 10 client.29393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.472+0000 7f3bc91a2740 20 client.29393 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46bbe7a0 0x561d46bc0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bc91a2740 4 client.29393 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d46840890 con 0x561d46bbe7a0 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bbffff640 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46bbe7a0 0x561d46bc0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3b9f7fe640 10 client.29393 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bbffff640 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46bbe7a0 0x561d46bc0b90 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5782168add25058a server_cookie=eee6f9444055a0d8 in_seq=0 out_seq=0 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3b9f7fe640 1 -- 172.21.15.100:0/2554989748 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561d46840890 con 0x561d46bbe7a0 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3b9f7fe640 10 client.29393 handle_command_reply: tid=0 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bc91a2740 1 client.29393 shutdown 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 msgr2=0x561d46b50aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.473+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b50aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005850 tx=0x7f3bb4005740 comp rx=0 tx=0).stop 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 shutdown_connections 2024-03-12T05:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x561d46bbe7a0 0x561d46bc0b90 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3b9f7fe640 0 client.29393 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d46b567b0 0x561d46bbce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d46b4dbd0 0x561d46b50aa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 --2- 172.21.15.100:0/2554989748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d46b4d2b0 0x561d46b50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 >> 172.21.15.100:0/2554989748 conn(0x561d46b47990 msgr2=0x561d46b51cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 shutdown_connections 2024-03-12T05:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 1 -- 172.21.15.100:0/2554989748 wait complete. 2024-03-12T05:34:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc91a2740 20 client.29393 trim_cache size 0 max 16384 2024-03-12T05:34:06.480 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:34:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-03-12T05:34:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-03-12T05:34:06.481 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:34:06.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:06 smithi161 ceph-mon[36043]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 375 op/s 2024-03-12T05:34:06.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3ba80a1db0 msgr2=0x7f3ba80a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3ba80a1db0 0x7f3ba80a4270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3bb002c540 tx=0x7f3bb002e040 comp rx=0 tx=0).stop 2024-03-12T05:34:06.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 msgr2=0x7f3bc0117db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc0117db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800d950 tx=0x7f3bb800de10 comp rx=0 tx=0).stop 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 shutdown_connections 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3ba80a1db0 0x7f3ba80a4270 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc011a960 0x7f3bc0110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc010fd10 0x7f3bc01182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 --2- 172.21.15.100:0/4122994268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc010f8a0 0x7f3bc0117db0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 >> 172.21.15.100:0/4122994268 conn(0x7f3bc006c270 msgr2=0x7f3bc01193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.474+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 shutdown_connections 2024-03-12T05:34:06.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.475+0000 7f3bc64f4640 1 -- 172.21.15.100:0/4122994268 wait complete. 2024-03-12T05:34:06.485 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-12T05:34:06.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:34:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 -- 172.21.15.100:0/2657902473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 msgr2=0x7fab80107690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 --2- 172.21.15.100:0/2657902473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab80107690 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fab70009900 tx=0x7fab7002bcd0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 -- 172.21.15.100:0/2657902473 shutdown_connections 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 --2- 172.21.15.100:0/2657902473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab8006a360 0x7fab8010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 --2- 172.21.15.100:0/2657902473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab800699a0 0x7fab80069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 --2- 172.21.15.100:0/2657902473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab80107690 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 -- 172.21.15.100:0/2657902473 >> 172.21.15.100:0/2657902473 conn(0x7fab80076ee0 msgr2=0x7fab800772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 -- 172.21.15.100:0/2657902473 shutdown_connections 2024-03-12T05:34:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.523+0000 7fab854f4640 1 -- 172.21.15.100:0/2657902473 wait complete. 2024-03-12T05:34:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 Processor -- start 2024-03-12T05:34:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 -- start start 2024-03-12T05:34:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab800699a0 0x7fab80197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab8006a360 0x7fab80197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab8019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8019c530 con 0x7fab801072b0 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8019c6a0 con 0x7fab8006a360 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab854f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8019c860 con 0x7fab800699a0 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab8019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab8019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57018/0 (socket says 172.21.15.100:57018) 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 -- 172.21.15.100:0/2496411218 learned_addr learned my addr 172.21.15.100:0/2496411218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab77fff640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab8006a360 0x7fab80197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7effd640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab800699a0 0x7fab80197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 -- 172.21.15.100:0/2496411218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab800699a0 msgr2=0x7fab80197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab800699a0 0x7fab80197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 -- 172.21.15.100:0/2496411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab8006a360 msgr2=0x7fab80197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab8006a360 0x7fab80197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 -- 172.21.15.100:0/2496411218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70009560 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.524+0000 7fab7f7fe640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab8019bf30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fab7800da40 tx=0x7fab7800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.525+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7800bdd0 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.525+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab78009890 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.525+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8019cb40 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.525+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780106c0 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.525+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab801a43c0 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.526+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab80102880 con 0x7fab801072b0 2024-03-12T05:34:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.527+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fab780041b0 con 0x7fab801072b0 2024-03-12T05:34:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.528+0000 7fab7cff9640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab6c0a1cc0 0x7fab6c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.528+0000 7fab7effd640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab6c0a1cc0 0x7fab6c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.528+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fab780cd3b0 con 0x7fab801072b0 2024-03-12T05:34:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.528+0000 7fab7effd640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab6c0a1cc0 0x7fab6c0a4180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab70006010 tx=0x7fab70005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.533+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab780962f0 con 0x7fab801072b0 2024-03-12T05:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 -- 172.21.15.100:0/863294006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 msgr2=0x7fad8410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/863294006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8410b1f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fad74009900 tx=0x7fad7402bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 -- 172.21.15.100:0/863294006 shutdown_connections 2024-03-12T05:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/863294006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 0x7fad8410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/863294006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8410b1f0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/863294006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 0x7fad841027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 -- 172.21.15.100:0/863294006 >> 172.21.15.100:0/863294006 conn(0x7fad84077180 msgr2=0x7fad84077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 -- 172.21.15.100:0/863294006 shutdown_connections 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.631+0000 7fad8c3c7640 1 -- 172.21.15.100:0/863294006 wait complete. 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 Processor -- start 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 -- start start 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 0x7fad8419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 0x7fad84199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad84199f10 con 0x7fad84102d10 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8419a080 con 0x7fad841023f0 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8c3c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8419a240 con 0x7fad8410b7c0 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57038/0 (socket says 172.21.15.100:57038) 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 -- 172.21.15.100:0/89035337 learned_addr learned my addr 172.21.15.100:0/89035337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8a163640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 0x7fad8419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 -- 172.21.15.100:0/89035337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 msgr2=0x7fad84199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8a964640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 0x7fad84199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 0x7fad84199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 -- 172.21.15.100:0/89035337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 msgr2=0x7fad8419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 0x7fad8419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 -- 172.21.15.100:0/89035337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74009560 con 0x7fad84102d10 2024-03-12T05:34:06.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8a163640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 0x7fad8419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad8a964640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 0x7fad84199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.632+0000 7fad89962640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8419fbf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fad74005940 tx=0x7fad7402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.633+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74004350 con 0x7fad84102d10 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.633+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8419a4c0 con 0x7fad84102d10 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.633+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad740044b0 con 0x7fad84102d10 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.633+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74003d30 con 0x7fad84102d10 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.633+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad841a66e0 con 0x7fad84102d10 2024-03-12T05:34:06.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.634+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad84068b20 con 0x7fad84102d10 2024-03-12T05:34:06.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.635+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fad74004df0 con 0x7fad84102d10 2024-03-12T05:34:06.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.635+0000 7fad7b7fe640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad700a1ba0 0x7fad700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.635+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad740efe20 con 0x7fad84102d10 2024-03-12T05:34:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.635+0000 7fad8a163640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad700a1ba0 0x7fad700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.636+0000 7fad8a163640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad700a1ba0 0x7fad700a4060 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad80009b50 tx=0x7fad80005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.643+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad740b8dc0 con 0x7fad84102d10 2024-03-12T05:34:06.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.744+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 --> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fab80111290 con 0x7fab6c0a1cc0 2024-03-12T05:34:06.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.756+0000 7fab7cff9640 1 -- 172.21.15.100:0/2496411218 <== mgr.15432 v2:172.21.15.100:6800/2025021937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fab80111290 con 0x7fab6c0a1cc0 2024-03-12T05:34:06.751 INFO:teuthology.orchestra.run.smithi100.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-12T00:00:00", "created": "2024-03-12T05:12:06", "first": "2024-03-12T05:13:00", "last": "2024-03-12T05:34:00", "last_pruned": "2024-03-12T05:34:01", "created_count": 22, "pruned_count": 16, "active": true} 2024-03-12T05:34:06.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.758+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab6c0a1cc0 msgr2=0x7fab6c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.758+0000 7fab854f4640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab6c0a1cc0 0x7fab6c0a4180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab70006010 tx=0x7fab70005e40 comp rx=0 tx=0).stop 2024-03-12T05:34:06.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.758+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 msgr2=0x7fab8019bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.758+0000 7fab854f4640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab8019bf30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fab7800da40 tx=0x7fab7800df00 comp rx=0 tx=0).stop 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.758+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 shutdown_connections 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.758+0000 7fab854f4640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fab6c0a1cc0 0x7fab6c0a4180 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.759+0000 7fab854f4640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab801072b0 0x7fab8019bf30 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.759+0000 7fab854f4640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab8006a360 0x7fab80197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.759+0000 7fab854f4640 1 --2- 172.21.15.100:0/2496411218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab800699a0 0x7fab80197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.759+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 >> 172.21.15.100:0/2496411218 conn(0x7fab80076ee0 msgr2=0x7fab800fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.759+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 shutdown_connections 2024-03-12T05:34:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.759+0000 7fab854f4640 1 -- 172.21.15.100:0/2496411218 wait complete. 2024-03-12T05:34:06.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-12T05:34:06.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.862+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad84067f70 con 0x7fad84102d10 2024-03-12T05:34:06.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.863+0000 7fad7b7fe640 1 -- 172.21.15.100:0/89035337 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fad740bc480 con 0x7fad84102d10 2024-03-12T05:34:06.857 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:06.858 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:34:06.858 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:34:06.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad700a1ba0 msgr2=0x7fad700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad700a1ba0 0x7fad700a4060 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad80009b50 tx=0x7fad80005f90 comp rx=0 tx=0).stop 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 msgr2=0x7fad8419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8419fbf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fad74005940 tx=0x7fad7402c670 comp rx=0 tx=0).stop 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 shutdown_connections 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fad700a1ba0 0x7fad700a4060 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad8410b7c0 0x7fad84199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad84102d10 0x7fad8419fbf0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 --2- 172.21.15.100:0/89035337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad841023f0 0x7fad8419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 >> 172.21.15.100:0/89035337 conn(0x7fad84077180 msgr2=0x7fad84107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 shutdown_connections 2024-03-12T05:34:06.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.865+0000 7fad8c3c7640 1 -- 172.21.15.100:0/89035337 wait complete. 2024-03-12T05:34:06.872 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:34:06.872 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-12T05:34:06.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 -- 172.21.15.100:0/1921096527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 msgr2=0x7fcc74100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/1921096527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74100750 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009900 tx=0x7fcc6402bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:06.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 -- 172.21.15.100:0/1921096527 shutdown_connections 2024-03-12T05:34:06.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/1921096527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74100c90 0x7fcc7410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/1921096527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74100750 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/1921096527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc74105020 0x7fcc74105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 -- 172.21.15.100:0/1921096527 >> 172.21.15.100:0/1921096527 conn(0x7fcc74076ed0 msgr2=0x7fcc740772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 -- 172.21.15.100:0/1921096527 shutdown_connections 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.917+0000 7fcc7b70d640 1 -- 172.21.15.100:0/1921096527 wait complete. 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 Processor -- start 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 -- start start 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc74100c90 0x7fcc74196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74105020 0x7fcc74196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741a5b90 con 0x7fcc741002d0 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741a5d00 con 0x7fcc74105020 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc7b70d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741a5ec0 con 0x7fcc74100c90 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57048/0 (socket says 172.21.15.100:57048) 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 -- 172.21.15.100:0/3785008863 learned_addr learned my addr 172.21.15.100:0/3785008863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc78ca8640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc74100c90 0x7fcc74196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc79caa640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74105020 0x7fcc74196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 -- 172.21.15.100:0/3785008863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc74100c90 msgr2=0x7fcc74196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc74100c90 0x7fcc74196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 -- 172.21.15.100:0/3785008863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74105020 msgr2=0x7fcc74196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74105020 0x7fcc74196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 -- 172.21.15.100:0/3785008863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64009560 con 0x7fcc741002d0 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc79caa640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74105020 0x7fcc74196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:34:06.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.918+0000 7fcc794a9640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74196450 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000d950 tx=0x7fcc7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.919+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70009a30 con 0x7fcc741002d0 2024-03-12T05:34:06.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.919+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc70009b90 con 0x7fcc741002d0 2024-03-12T05:34:06.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.919+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc741a61a0 con 0x7fcc741002d0 2024-03-12T05:34:06.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.919+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70004a00 con 0x7fcc741002d0 2024-03-12T05:34:06.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.919+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc741a6680 con 0x7fcc741002d0 2024-03-12T05:34:06.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.920+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7404eee0 con 0x7fcc741002d0 2024-03-12T05:34:06.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.921+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fcc70009d00 con 0x7fcc741002d0 2024-03-12T05:34:06.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.921+0000 7fcc6a7fc640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc600a1c40 0x7fcc600a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.922+0000 7fcc78ca8640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc600a1c40 0x7fcc600a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.922+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc700cd940 con 0x7fcc741002d0 2024-03-12T05:34:06.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.922+0000 7fcc78ca8640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc600a1c40 0x7fcc600a4100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006010 tx=0x7fcc64005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.929+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc700967d0 con 0x7fcc741002d0 2024-03-12T05:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 -- 172.21.15.100:0/2091787641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70100c10 msgr2=0x7fac70104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 --2- 172.21.15.100:0/2091787641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70100c10 0x7fac70104c50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fac64009930 tx=0x7fac6402bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 -- 172.21.15.100:0/2091787641 shutdown_connections 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 --2- 172.21.15.100:0/2091787641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70105330 0x7fac7010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 --2- 172.21.15.100:0/2091787641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70100c10 0x7fac70104c50 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 --2- 172.21.15.100:0/2091787641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac70100260 0x7fac70100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 -- 172.21.15.100:0/2091787641 >> 172.21.15.100:0/2091787641 conn(0x7fac70077180 msgr2=0x7fac70077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 -- 172.21.15.100:0/2091787641 shutdown_connections 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.987+0000 7fac74adb640 1 -- 172.21.15.100:0/2091787641 wait complete. 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 Processor -- start 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 -- start start 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70100260 0x7fac7019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 0x7fac7019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac701975c0 0x7fac70197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac70197fb0 con 0x7fac70105330 2024-03-12T05:34:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac70198120 con 0x7fac701975c0 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac74adb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac701982e0 con 0x7fac70100260 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 0x7fac7019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 0x7fac7019d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57066/0 (socket says 172.21.15.100:57066) 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 -- 172.21.15.100:0/326753044 learned_addr learned my addr 172.21.15.100:0/326753044 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6e59c640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70100260 0x7fac7019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac701975c0 0x7fac70197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 -- 172.21.15.100:0/326753044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70100260 msgr2=0x7fac7019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70100260 0x7fac7019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 -- 172.21.15.100:0/326753044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac701975c0 msgr2=0x7fac70197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac701975c0 0x7fac70197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 -- 172.21.15.100:0/326753044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64009560 con 0x7fac70105330 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.988+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 0x7fac7019d950 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fac64009230 tx=0x7fac64004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.989+0000 7fac5f7fe640 1 -- 172.21.15.100:0/326753044 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac640043d0 con 0x7fac70105330 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.989+0000 7fac5f7fe640 1 -- 172.21.15.100:0/326753044 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac640052d0 con 0x7fac70105330 2024-03-12T05:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.989+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac701a42d0 con 0x7fac70105330 2024-03-12T05:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.989+0000 7fac5f7fe640 1 -- 172.21.15.100:0/326753044 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64003d80 con 0x7fac70105330 2024-03-12T05:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.989+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac701a4780 con 0x7fac70105330 2024-03-12T05:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.991+0000 7fac5f7fe640 1 -- 172.21.15.100:0/326753044 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fac64005440 con 0x7fac70105330 2024-03-12T05:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.991+0000 7fac77783740 1 Processor -- start 2024-03-12T05:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 -- start start 2024-03-12T05:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2250 0x5606fa4c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 0x5606fa4cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa47d960 con 0x5606fa4cb750 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa431310 con 0x5606fa4c2b70 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac77783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa42e840 con 0x5606fa4c2250 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac5f7fe640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac580a1ba0 0x7fac580a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 0x5606fa4cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2250 0x5606fa4c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 0x5606fa4cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.100:50296/0 (socket says 172.21.15.100:50296) 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57076/0 (socket says 172.21.15.100:57076) 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 -- 172.21.15.100:0/3641372577 learned_addr learned my addr 172.21.15.100:0/3641372577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac580a1ba0 0x7fac580a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac5f7fe640 1 -- 172.21.15.100:0/326753044 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fac640ef840 con 0x7fac70105330 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 -- 172.21.15.100:0/3641372577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2250 msgr2=0x5606fa4c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2250 0x5606fa4c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 -- 172.21.15.100:0/3641372577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 msgr2=0x5606fa4cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6e59c640 1 -- 172.21.15.100:0/3641372577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606fa3ec060 con 0x5606fa4c2b70 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.992+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2250 0x5606fa4c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.993+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac580a1ba0 0x7fac580a4060 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f60 tx=0x7fac60005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.993+0000 7fac6e59c640 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 0x5606fa4cb210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac68007f90 tx=0x7fac6800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac5a37fdfce2d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.993+0000 7fac5cff9640 1 -- 172.21.15.100:0/3641372577 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68004d20 con 0x5606fa4c2b70 2024-03-12T05:34:06.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.993+0000 7fac5cff9640 1 -- 172.21.15.100:0/3641372577 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac68004e80 con 0x5606fa4c2b70 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.993+0000 7fac5cff9640 1 -- 172.21.15.100:0/3641372577 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac680056c0 con 0x5606fa4c2b70 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- 172.21.15.100:0/3641372577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 msgr2=0x5606fa4cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 0x5606fa4cb210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac68007f90 tx=0x7fac6800f850 comp rx=0 tx=0).stop 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- 172.21.15.100:0/3641372577 shutdown_connections 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2b70 0x5606fa4cb210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- 172.21.15.100:0/3641372577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2250 0x5606fa4c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:06.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- 172.21.15.100:0/3641372577 >> 172.21.15.100:0/3641372577 conn(0x5606fa4bc980 msgr2=0x5606fa4bcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- 172.21.15.100:0/3641372577 shutdown_connections 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- 172.21.15.100:0/3641372577 wait complete. 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- start start 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 Processor -- start 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa42a020 con 0x5606fa4cb750 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa0fd430 con 0x5606fa4c2250 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa0b9e00 con 0x5606fa4c2b70 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.994+0000 7fac77783740 10 client.0 _ll_register_callbacks cb 0x5606fa3d1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57084/0 (socket says 172.21.15.100:57084) 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 -- 172.21.15.100:0/3645931022 learned_addr learned my addr 172.21.15.100:0/3645931022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6e59c640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 -- 172.21.15.100:0/3645931022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2b70 msgr2=0x5606fa4c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 -- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 msgr2=0x5606fa4c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 -- 172.21.15.100:0/3645931022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac68007bc0 con 0x5606fa4cb750 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4c91c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fac60011410 tx=0x7fac600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68c5c5397d539ab server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:34:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac55ffb640 1 -- 172.21.15.100:0/3645931022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac600037f0 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac55ffb640 1 -- 172.21.15.100:0/3645931022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fac60011e10 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606fa166720 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac55ffb640 1 -- 172.21.15.100:0/3645931022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac600044e0 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac77783740 10 client.20097 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5606fa4286a0 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.995+0000 7fac55ffb640 1 -- 172.21.15.100:0/3645931022 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5606fa166720 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac55ffb640 1 -- 172.21.15.100:0/3645931022 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac60004830 con 0x5606fa4cb750 2024-03-12T05:34:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac77783740 10 client.20097 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac77783740 10 client.20097 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac77783740 20 client.20097 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac77783740 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa4dbbb0 0x5606fa4ddfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac77783740 4 client.20097 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606fa1b5890 con 0x5606fa4dbbb0 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac6e59c640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa4dbbb0 0x5606fa4ddfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac55ffb640 10 client.20097 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:34:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.996+0000 7fac6e59c640 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa4dbbb0 0x5606fa4ddfd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ce0a5ae6db9e7de server_cookie=b77e4205a64af125 in_seq=0 out_seq=0 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac55ffb640 1 -- 172.21.15.100:0/3645931022 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606fa1b5890 con 0x5606fa4dbbb0 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac55ffb640 10 client.20097 handle_command_reply: tid=0 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 client.20097 shutdown 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 msgr2=0x5606fa4c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4c91c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fac60011410 tx=0x7fac600046c0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 shutdown_connections 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa4dbbb0 0x5606fa4ddfd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4cb750 0x5606fa4c91c0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac55ffb640 0 client.20097 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 --2- 172.21.15.100:0/3645931022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 >> 172.21.15.100:0/3645931022 conn(0x5606fa4bc980 msgr2=0x5606fa4be6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 shutdown_connections 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 1 -- 172.21.15.100:0/3645931022 wait complete. 2024-03-12T05:34:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:06.997+0000 7fac77783740 20 client.20097 trim_cache size 0 max 16384 2024-03-12T05:34:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.009+0000 7fac77783740 1 Processor -- start 2024-03-12T05:34:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.009+0000 7fac77783740 1 -- start start 2024-03-12T05:34:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac77783740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac77783740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac77783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac77783740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa3af130 con 0x5606fa4c2b70 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac77783740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa166720 con 0x5606fa4c2250 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac77783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa4aea40 con 0x5606fa4cb750 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57096/0 (socket says 172.21.15.100:57096) 2024-03-12T05:34:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 -- 172.21.15.100:0/4280063577 learned_addr learned my addr 172.21.15.100:0/4280063577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6e59c640 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 -- 172.21.15.100:0/4280063577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 msgr2=0x5606fa4c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 -- 172.21.15.100:0/4280063577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 msgr2=0x5606fa4cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 -- 172.21.15.100:0/4280063577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606fa3c4c20 con 0x5606fa4c2b70 2024-03-12T05:34:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6e59c640 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8fe0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fac64032040 tx=0x7fac6408f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73505f09f5df556b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac57fff640 1 -- 172.21.15.100:0/4280063577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac640bc080 con 0x5606fa4c2b70 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac57fff640 1 -- 172.21.15.100:0/4280063577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac640bbea0 con 0x5606fa4c2b70 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.010+0000 7fac57fff640 1 -- 172.21.15.100:0/4280063577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac640ba840 con 0x5606fa4c2b70 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 -- 172.21.15.100:0/4280063577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 msgr2=0x5606fa4c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8fe0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fac64032040 tx=0x7fac6408f7f0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 -- 172.21.15.100:0/4280063577 shutdown_connections 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4c8fe0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 --2- 172.21.15.100:0/4280063577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 -- 172.21.15.100:0/4280063577 >> 172.21.15.100:0/4280063577 conn(0x5606fa4bc980 msgr2=0x5606fa4c39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 -- 172.21.15.100:0/4280063577 shutdown_connections 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 -- 172.21.15.100:0/4280063577 wait complete. 2024-03-12T05:34:07.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.011+0000 7fac77783740 1 -- start start 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 Processor -- start 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa532180 con 0x5606fa4c2b70 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa5322f0 con 0x5606fa4c2250 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fa5324b0 con 0x5606fa4cb750 2024-03-12T05:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac77783740 10 client.0 _ll_register_callbacks cb 0x7fac680079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57110/0 (socket says 172.21.15.100:57110) 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 -- 172.21.15.100:0/2442476093 learned_addr learned my addr 172.21.15.100:0/2442476093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6e59c640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 -- 172.21.15.100:0/2442476093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 msgr2=0x5606fa4d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 -- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 msgr2=0x5606fa4d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 -- 172.21.15.100:0/2442476093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac701a4780 con 0x5606fa4c2b70 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6ed9d640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6e59c640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac6dd9b640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4d1730 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fac640ed550 tx=0x7fac640bc080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3945a86ab3522dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.012+0000 7fac55ffb640 1 -- 172.21.15.100:0/2442476093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac640bcc40 con 0x5606fa4c2b70 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac55ffb640 1 -- 172.21.15.100:0/2442476093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fac640c0690 con 0x5606fa4c2b70 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606fa1297e0 con 0x5606fa4c2b70 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac55ffb640 1 -- 172.21.15.100:0/2442476093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac640035f0 con 0x5606fa4c2b70 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 10 client.20103 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5606fa3ea8c0 con 0x5606fa4c2b70 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac55ffb640 1 -- 172.21.15.100:0/2442476093 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5606fa1297e0 con 0x5606fa4c2b70 2024-03-12T05:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac55ffb640 1 -- 172.21.15.100:0/2442476093 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac6403c070 con 0x5606fa4c2b70 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 10 client.20103 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 10 client.20103 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 20 client.20103 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa533820 0x5606fa535c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 4 client.20103 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 --> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5606fa0d5700 con 0x5606fa533820 2024-03-12T05:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.013+0000 7fac6e59c640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa533820 0x5606fa535c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac6e59c640 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa533820 0x5606fa535c40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72972523f7b3e631 server_cookie=1e30b24ca55151e1 in_seq=0 out_seq=0 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac55ffb640 10 client.20103 ms_handle_connect on v2:172.21.15.156:6836/2463295988 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac55ffb640 1 -- 172.21.15.100:0/2442476093 <== mds.2 v2:172.21.15.156:6836/2463295988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5606fa0d5700 con 0x5606fa533820 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac55ffb640 10 client.20103 handle_command_reply: tid=0 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 client.20103 shutdown 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 msgr2=0x5606fa4d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4d1730 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fac640ed550 tx=0x7fac640bc080 comp rx=0 tx=0).stop 2024-03-12T05:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 shutdown_connections 2024-03-12T05:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:6836/2463295988,v1:172.21.15.156:6837/2463295988] conn(0x5606fa533820 0x5606fa535c40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606fa4cb750 0x5606fa4d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac55ffb640 0 client.20103 ms_handle_reset on v2:172.21.15.156:6836/2463295988 2024-03-12T05:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606fa4c2b70 0x5606fa4d1730 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 --2- 172.21.15.100:0/2442476093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606fa4c2250 0x5606fa4d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 >> 172.21.15.100:0/2442476093 conn(0x5606fa4bc980 msgr2=0x5606fa4bb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 shutdown_connections 2024-03-12T05:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 1 -- 172.21.15.100:0/2442476093 wait complete. 2024-03-12T05:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac77783740 20 client.20103 trim_cache size 0 max 16384 2024-03-12T05:34:07.025 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac580a1ba0 msgr2=0x7fac580a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac74adb640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac580a1ba0 0x7fac580a4060 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f60 tx=0x7fac60005f90 comp rx=0 tx=0).stop 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 msgr2=0x7fac7019d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.014+0000 7fac74adb640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 0x7fac7019d950 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fac64009230 tx=0x7fac64004c00 comp rx=0 tx=0).stop 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 shutdown_connections 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fac580a1ba0 0x7fac580a4060 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac701975c0 0x7fac70197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac70105330 0x7fac7019d950 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 --2- 172.21.15.100:0/326753044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70100260 0x7fac7019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 >> 172.21.15.100:0/326753044 conn(0x7fac70077180 msgr2=0x7fac70068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 shutdown_connections 2024-03-12T05:34:07.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.015+0000 7fac74adb640 1 -- 172.21.15.100:0/326753044 wait complete. 2024-03-12T05:34:07.028 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:34:07.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-12T05:34:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.154+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 --> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcc741102c0 con 0x7fcc600a1c40 2024-03-12T05:34:07.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.167+0000 7fcc6a7fc640 1 -- 172.21.15.100:0/3785008863 <== mgr.15432 v2:172.21.15.100:6800/2025021937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fcc741102c0 con 0x7fcc600a1c40 2024-03-12T05:34:07.162 INFO:teuthology.orchestra.run.smithi100.stdout:/ 1m 6m3h 2024-03-12T05:34:07.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc600a1c40 msgr2=0x7fcc600a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc600a1c40 0x7fcc600a4100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006010 tx=0x7fcc64005e40 comp rx=0 tx=0).stop 2024-03-12T05:34:07.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 msgr2=0x7fcc74196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74196450 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000d950 tx=0x7fcc7000de10 comp rx=0 tx=0).stop 2024-03-12T05:34:07.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 shutdown_connections 2024-03-12T05:34:07.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fcc600a1c40 0x7fcc600a4100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74105020 0x7fcc74196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc74100c90 0x7fcc74196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 --2- 172.21.15.100:0/3785008863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc741002d0 0x7fcc74196450 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 >> 172.21.15.100:0/3785008863 conn(0x7fcc74076ed0 msgr2=0x7fcc740fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 shutdown_connections 2024-03-12T05:34:07.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.169+0000 7fcc7b70d640 1 -- 172.21.15.100:0/3785008863 wait complete. 2024-03-12T05:34:07.174 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-12T05:34:07.185 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-12T05:34:07.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 -- 172.21.15.100:0/2832964081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8964107380 msgr2=0x7f8964107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 --2- 172.21.15.100:0/2832964081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8964107380 0x7f8964107760 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8954009900 tx=0x7f895402bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:07.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 -- 172.21.15.100:0/2832964081 shutdown_connections 2024-03-12T05:34:07.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 --2- 172.21.15.100:0/2832964081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8964100400 0x7f896410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 --2- 172.21.15.100:0/2832964081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89640ffa40 0x7f89640ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 --2- 172.21.15.100:0/2832964081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8964107380 0x7f8964107760 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 -- 172.21.15.100:0/2832964081 >> 172.21.15.100:0/2832964081 conn(0x7f89640753c0 msgr2=0x7f89640757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 -- 172.21.15.100:0/2832964081 shutdown_connections 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.185+0000 7f8969d6c640 1 -- 172.21.15.100:0/2832964081 wait complete. 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 Processor -- start 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 -- start start 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 0x7f896419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8964100400 0x7f896419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8964107380 0x7f8964197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8964197e00 con 0x7f89640ffa40 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8964197f70 con 0x7f8964107380 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f8969d6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8964198130 con 0x7f8964100400 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f89637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 0x7f896419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f89637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 0x7f896419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57118/0 (socket says 172.21.15.100:57118) 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f89637fe640 1 -- 172.21.15.100:0/4132070446 learned_addr learned my addr 172.21.15.100:0/4132070446 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.186+0000 7f895affd640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8964100400 0x7f896419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89637fe640 1 -- 172.21.15.100:0/4132070446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8964100400 msgr2=0x7f896419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89637fe640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8964100400 0x7f896419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89637fe640 1 -- 172.21.15.100:0/4132070446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8964107380 msgr2=0x7f8964197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f8963fff640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8964107380 0x7f8964197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89637fe640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8964107380 0x7f8964197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89637fe640 1 -- 172.21.15.100:0/4132070446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954009560 con 0x7f89640ffa40 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89637fe640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 0x7f896419d5b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8954005940 tx=0x7f8954004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89617fa640 1 -- 172.21.15.100:0/4132070446 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954004440 con 0x7f89640ffa40 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89617fa640 1 -- 172.21.15.100:0/4132070446 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89540052d0 con 0x7f89640ffa40 2024-03-12T05:34:07.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89641983b0 con 0x7f89640ffa40 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f89617fa640 1 -- 172.21.15.100:0/4132070446 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954003da0 con 0x7f89640ffa40 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.187+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89641a4530 con 0x7f89640ffa40 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.189+0000 7f89617fa640 1 -- 172.21.15.100:0/4132070446 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8954005440 con 0x7f89640ffa40 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.189+0000 7f896ca18740 1 Processor -- start 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 -- start start 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b191960 con 0x55731b1df750 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b145310 con 0x55731b1d6250 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f896ca18740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b142840 con 0x55731b1d6b70 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f89617fa640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89500a1ba0 0x7f89500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f8963fff640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89500a1ba0 0x7f89500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57132/0 (socket says 172.21.15.100:57132) 2024-03-12T05:34:07.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 -- 172.21.15.100:0/2136348183 learned_addr learned my addr 172.21.15.100:0/2136348183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f8963fff640 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f89617fa640 1 -- 172.21.15.100:0/4132070446 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89540ef960 con 0x7f89640ffa40 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f89637fe640 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 -- 172.21.15.100:0/2136348183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 msgr2=0x55731b1df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1df210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 -- 172.21.15.100:0/2136348183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 msgr2=0x55731b1d6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1d6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 -- 172.21.15.100:0/2136348183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55731b100060 con 0x55731b1df750 2024-03-12T05:34:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f8963fff640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89500a1ba0 0x7f89500a4060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f895c005fd0 tx=0x7f895c0092b0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.190+0000 7f895affd640 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e1b40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f894c00bc90 tx=0x7f894c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3631d6cf2cd048 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f8959ffb640 1 -- 172.21.15.100:0/2136348183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c005020 con 0x55731b1df750 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f8959ffb640 1 -- 172.21.15.100:0/2136348183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f894c015070 con 0x55731b1df750 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f8959ffb640 1 -- 172.21.15.100:0/2136348183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c011680 con 0x55731b1df750 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 -- 172.21.15.100:0/2136348183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 msgr2=0x55731b1e1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e1b40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f894c00bc90 tx=0x7f894c0107a0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 -- 172.21.15.100:0/2136348183 shutdown_connections 2024-03-12T05:34:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e1b40 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 --2- 172.21.15.100:0/2136348183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 -- 172.21.15.100:0/2136348183 >> 172.21.15.100:0/2136348183 conn(0x55731b1d0980 msgr2=0x55731b1d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 -- 172.21.15.100:0/2136348183 shutdown_connections 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.191+0000 7f896ca18740 1 -- 172.21.15.100:0/2136348183 wait complete. 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 -- start start 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 Processor -- start 2024-03-12T05:34:07.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 0x55731b1dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6b70 0x55731b1dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1df750 0x55731b1dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b13e020 con 0x55731b1d6250 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731ae11430 con 0x55731b1d6b70 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731adcde00 con 0x55731b1df750 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f896ca18740 10 client.0 _ll_register_callbacks cb 0x55731b0e5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f89637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 0x55731b1dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f89637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 0x55731b1dc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57142/0 (socket says 172.21.15.100:57142) 2024-03-12T05:34:07.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f89637fe640 1 -- 172.21.15.100:0/1294625291 learned_addr learned my addr 172.21.15.100:0/1294625291 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f895affd640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6b70 0x55731b1dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f8963fff640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1df750 0x55731b1dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.192+0000 7f89637fe640 1 -- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1df750 msgr2=0x55731b1dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f89637fe640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1df750 0x55731b1dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f89637fe640 1 -- 172.21.15.100:0/1294625291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6b70 msgr2=0x55731b1dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f89637fe640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6b70 0x55731b1dccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f89637fe640 1 -- 172.21.15.100:0/1294625291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c00b8c0 con 0x55731b1d6250 2024-03-12T05:34:07.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f895affd640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6b70 0x55731b1dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8963fff640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1df750 0x55731b1dd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f89637fe640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 0x55731b1dc7b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f89540b8ac0 tx=0x7f8954000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5657eee9f57a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8942ffd640 1 -- 172.21.15.100:0/1294625291 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540bcd60 con 0x55731b1d6250 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8942ffd640 1 -- 172.21.15.100:0/1294625291 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89540bc660 con 0x55731b1d6250 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55731ae3d7e0 con 0x55731b1d6250 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8942ffd640 1 -- 172.21.15.100:0/1294625291 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89540c07b0 con 0x55731b1d6250 2024-03-12T05:34:07.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 10 client.20115 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55731b13c6a0 con 0x55731b1d6250 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8942ffd640 1 -- 172.21.15.100:0/1294625291 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55731ae3d7e0 con 0x55731b1d6250 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f8942ffd640 1 -- 172.21.15.100:0/1294625291 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f895403c070 con 0x55731b1d6250 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 10 client.20115 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 10 client.20115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 20 client.20115 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 4 client.20115 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.193+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55731ae7a720 con 0x55731b1ef9d0 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.194+0000 7f895affd640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.194+0000 7f8942ffd640 10 client.20115 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.194+0000 7f895affd640 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b1f1dc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23e0ebcaf7adb1c8 server_cookie=ef5df003c2fd34d3 in_seq=0 out_seq=0 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f8942ffd640 1 -- 172.21.15.100:0/1294625291 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55731ae7a720 con 0x55731b1ef9d0 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f8942ffd640 10 client.20115 handle_command_reply: tid=0 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 client.20115 shutdown 2024-03-12T05:34:07.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 msgr2=0x55731b1dc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 0x55731b1dc7b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f89540b8ac0 tx=0x7f8954000bc0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 shutdown_connections 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1df750 0x55731b1dd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b1f1dc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6b70 0x55731b1dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f8942ffd640 0 client.20115 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:34:07.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 --2- 172.21.15.100:0/1294625291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1d6250 0x55731b1dc7b0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 >> 172.21.15.100:0/1294625291 conn(0x55731b1d0980 msgr2=0x55731b1d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 shutdown_connections 2024-03-12T05:34:07.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 1 -- 172.21.15.100:0/1294625291 wait complete. 2024-03-12T05:34:07.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.195+0000 7f896ca18740 20 client.20115 trim_cache size 0 max 16384 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 Processor -- start 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 -- start start 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1deee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b0c3130 con 0x55731b1df750 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b1d1980 con 0x55731b1d6250 2024-03-12T05:34:07.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.206+0000 7f896ca18740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b1d5eb0 con 0x55731b1d6b70 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1d9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57158/0 (socket says 172.21.15.100:57158) 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 -- 172.21.15.100:0/668413615 learned_addr learned my addr 172.21.15.100:0/668413615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f895affd640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f89637fe640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1deee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 -- 172.21.15.100:0/668413615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 msgr2=0x55731b1dd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1dd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 -- 172.21.15.100:0/668413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 msgr2=0x55731b1deee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1deee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 -- 172.21.15.100:0/668413615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55731b0d8c20 con 0x55731b1df750 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f89637fe640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1deee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f895affd640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f8963fff640 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1d9a10 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f895c00ba40 tx=0x7f895c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68956047ab741eb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f89597fa640 1 -- 172.21.15.100:0/668413615 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c010bd0 con 0x55731b1df750 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f89597fa640 1 -- 172.21.15.100:0/668413615 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895c019070 con 0x55731b1df750 2024-03-12T05:34:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.207+0000 7f89597fa640 1 -- 172.21.15.100:0/668413615 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c028650 con 0x55731b1df750 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- 172.21.15.100:0/668413615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 msgr2=0x55731b1d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1d9a10 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f895c00ba40 tx=0x7f895c009240 comp rx=0 tx=0).stop 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- 172.21.15.100:0/668413615 shutdown_connections 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1d9a10 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- 172.21.15.100:0/668413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1deee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- 172.21.15.100:0/668413615 >> 172.21.15.100:0/668413615 conn(0x55731b1d0980 msgr2=0x55731b1db080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- 172.21.15.100:0/668413615 shutdown_connections 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- 172.21.15.100:0/668413615 wait complete. 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- start start 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 Processor -- start 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1e6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b1e5910 con 0x55731b1df750 2024-03-12T05:34:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b1e5a80 con 0x55731b1d6250 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b245e70 con 0x55731b1d6b70 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.208+0000 7f896ca18740 10 client.0 _ll_register_callbacks cb 0x7f894c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57160/0 (socket says 172.21.15.100:57160) 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 -- 172.21.15.100:0/681452705 learned_addr learned my addr 172.21.15.100:0/681452705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f895affd640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f89637fe640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 -- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 msgr2=0x55731b1e4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1e4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 -- 172.21.15.100:0/681452705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 msgr2=0x55731b1e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1e6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 -- 172.21.15.100:0/681452705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c00b700 con 0x55731b1df750 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f895affd640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8963fff640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e5210 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f895c01ecb0 tx=0x7f895c013840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419800ac3e497f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:34:07.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8942ffd640 1 -- 172.21.15.100:0/681452705 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c013250 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8942ffd640 1 -- 172.21.15.100:0/681452705 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f895c01e440 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f894c015070 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8942ffd640 1 -- 172.21.15.100:0/681452705 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f895c01e600 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f896ca18740 10 client.20121 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55731b0fe8c0 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.209+0000 7f8942ffd640 1 -- 172.21.15.100:0/681452705 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f894c015070 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f8942ffd640 1 -- 172.21.15.100:0/681452705 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f895c031970 con 0x55731b1df750 2024-03-12T05:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 10 client.20121 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 10 client.20121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 20 client.20121 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b249500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 4 client.20121 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 --> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55731aec9890 con 0x55731b1ef9d0 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f89637fe640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b249500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f8942ffd640 10 client.20121 ms_handle_connect on v2:172.21.15.161:6838/1712807460 2024-03-12T05:34:07.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f89637fe640 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b249500 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f653f9cc2d3126c server_cookie=86da190b95de75b1 in_seq=0 out_seq=0 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f8942ffd640 1 -- 172.21.15.100:0/681452705 <== mds.1 v2:172.21.15.161:6838/1712807460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55731aec9890 con 0x55731b1ef9d0 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f8942ffd640 10 client.20121 handle_command_reply: tid=0 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 1 client.20121 shutdown 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 msgr2=0x55731b1e5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.210+0000 7f896ca18740 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e5210 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f895c01ecb0 tx=0x7f895c013840 comp rx=0 tx=0).stop 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 shutdown_connections 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55731b1df750 0x55731b1e5210 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55731b1d6b70 0x55731b1e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.161:6838/1712807460,v1:172.21.15.161:6839/1712807460] conn(0x55731b1ef9d0 0x55731b249500 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 --2- 172.21.15.100:0/681452705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55731b1d6250 0x55731b1e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8942ffd640 0 client.20121 ms_handle_reset on v2:172.21.15.161:6838/1712807460 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 >> 172.21.15.100:0/681452705 conn(0x55731b1d0980 msgr2=0x55731b1db550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 shutdown_connections 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 1 -- 172.21.15.100:0/681452705 wait complete. 2024-03-12T05:34:07.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f896ca18740 20 client.20121 trim_cache size 0 max 16384 2024-03-12T05:34:07.227 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:34:07.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89500a1ba0 msgr2=0x7f89500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89500a1ba0 0x7f89500a4060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f895c005fd0 tx=0x7f895c0092b0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 msgr2=0x7f896419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 0x7f896419d5b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8954005940 tx=0x7f8954004040 comp rx=0 tx=0).stop 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 shutdown_connections 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f89500a1ba0 0x7f89500a4060 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8964107380 0x7f8964197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8964100400 0x7f896419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 --2- 172.21.15.100:0/4132070446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89640ffa40 0x7f896419d5b0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 >> 172.21.15.100:0/4132070446 conn(0x7f89640753c0 msgr2=0x7f89640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 shutdown_connections 2024-03-12T05:34:07.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.211+0000 7f8969d6c640 1 -- 172.21.15.100:0/4132070446 wait complete. 2024-03-12T05:34:07.230 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-12T05:34:07.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-12T05:34:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:07 smithi100 ceph-mon[32210]: from='client.20073 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:34:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:07 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/89035337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/1183680424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38100c50 msgr2=0x7f3e38104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/1183680424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38100c50 0x7f3e38104c90 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009930 tx=0x7f3e2802bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/1183680424 shutdown_connections 2024-03-12T05:34:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/1183680424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e38105370 0x7f3e3810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/1183680424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38100c50 0x7f3e38104c90 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/1183680424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e381002a0 0x7f3e38100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/1183680424 >> 172.21.15.100:0/1183680424 conn(0x7f3e38076400 msgr2=0x7f3e38076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/1183680424 shutdown_connections 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.378+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/1183680424 wait complete. 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 Processor -- start 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 -- start start 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e381002a0 0x7f3e3819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e38105370 0x7f3e3819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 0x7f3e38197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38198120 con 0x7f3e38197650 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38198290 con 0x7f3e38105370 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3cf7a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38198450 con 0x7f3e381002a0 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 0x7f3e38197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 0x7f3e38197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57180/0 (socket says 172.21.15.100:57180) 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 -- 172.21.15.100:0/3953120107 learned_addr learned my addr 172.21.15.100:0/3953120107 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e3659c640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e381002a0 0x7f3e3819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e38105370 0x7f3e3819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 -- 172.21.15.100:0/3953120107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e381002a0 msgr2=0x7f3e3819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e381002a0 0x7f3e3819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 -- 172.21.15.100:0/3953120107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e38105370 msgr2=0x7f3e3819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e38105370 0x7f3e3819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 -- 172.21.15.100:0/3953120107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e28009560 con 0x7f3e38197650 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.379+0000 7f3e36d9d640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 0x7f3e38197ad0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000eaa0 tx=0x7f3e3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.380+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3953120107 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000cdb0 con 0x7f3e38197650 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.380+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e381a43f0 con 0x7f3e38197650 2024-03-12T05:34:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.380+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3953120107 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e30005020 con 0x7f3e38197650 2024-03-12T05:34:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.380+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3953120107 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300116b0 con 0x7f3e38197650 2024-03-12T05:34:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.380+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e381a48d0 con 0x7f3e38197650 2024-03-12T05:34:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.382+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3953120107 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3e30005520 con 0x7f3e38197650 2024-03-12T05:34:07.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.382+0000 7f3e3fc2f740 1 Processor -- start 2024-03-12T05:34:07.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.382+0000 7f3e3fc2f740 1 -- start start 2024-03-12T05:34:07.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 0x55bf0c493630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493b70 0x55bf0c49c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c44e960 con 0x55bf0c493250 2024-03-12T05:34:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c402310 con 0x55bf0c49c750 2024-03-12T05:34:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c3ff840 con 0x55bf0c493b70 2024-03-12T05:34:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e2f7fe640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e240a1ba0 0x7f3e240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 0x55bf0c493630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493b70 0x55bf0c49c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 0x55bf0c493630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57182/0 (socket says 172.21.15.100:57182) 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e35d9b640 1 -- 172.21.15.100:0/712462042 learned_addr learned my addr 172.21.15.100:0/712462042 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e36d9d640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e240a1ba0 0x7f3e240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e2f7fe640 1 -- 172.21.15.100:0/3953120107 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3e300cd9a0 con 0x7f3e38197650 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 -- 172.21.15.100:0/712462042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 msgr2=0x55bf0c49eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 -- 172.21.15.100:0/712462042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 msgr2=0x55bf0c493630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 0x55bf0c493630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 -- 172.21.15.100:0/712462042 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0c3bd060 con 0x55bf0c493b70 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 0x55bf0c493630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e3659c640 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493b70 0x55bf0c49c210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009c00 tx=0x7f3e20011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239a0ba51e7d7483 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e36d9d640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e240a1ba0 0x7f3e240a4060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3e3009a5d0 tx=0x7f3e3001e900 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e2cff9640 1 -- 172.21.15.100:0/712462042 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20004230 con 0x55bf0c493b70 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e2cff9640 1 -- 172.21.15.100:0/712462042 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e20009e70 con 0x55bf0c493b70 2024-03-12T05:34:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.383+0000 7f3e2cff9640 1 -- 172.21.15.100:0/712462042 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20004c60 con 0x55bf0c493b70 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/712462042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493b70 msgr2=0x55bf0c49c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493b70 0x55bf0c49c210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009c00 tx=0x7f3e20011810 comp rx=0 tx=0).stop 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/712462042 shutdown_connections 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493b70 0x55bf0c49c210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/712462042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493250 0x55bf0c493630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/712462042 >> 172.21.15.100:0/712462042 conn(0x55bf0c48d980 msgr2=0x55bf0c48dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/712462042 shutdown_connections 2024-03-12T05:34:07.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.384+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/712462042 wait complete. 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 -- start start 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 Processor -- start 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c499780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c499cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c3fb020 con 0x55bf0c493b70 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c0ce430 con 0x55bf0c49c750 2024-03-12T05:34:07.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c08ae00 con 0x55bf0c493250 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3fc2f740 10 client.0 _ll_register_callbacks cb 0x55bf0c3a2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c499cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c499cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57190/0 (socket says 172.21.15.100:57190) 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 -- 172.21.15.100:0/2713365704 learned_addr learned my addr 172.21.15.100:0/2713365704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c499780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 -- 172.21.15.100:0/2713365704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 msgr2=0x55bf0c499780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c499780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 -- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 msgr2=0x55bf0c49a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 -- 172.21.15.100:0/2713365704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e20009830 con 0x55bf0c493b70 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e3659c640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c499cc0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005890 tx=0x7f3e20019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f4104b68e51782 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.385+0000 7f3e1dffb640 1 -- 172.21.15.100:0/2713365704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20019df0 con 0x55bf0c493b70 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e1dffb640 1 -- 172.21.15.100:0/2713365704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3e20024470 con 0x55bf0c493b70 2024-03-12T05:34:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf0c0fa7e0 con 0x55bf0c493b70 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e1dffb640 1 -- 172.21.15.100:0/2713365704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e20024630 con 0x55bf0c493b70 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 10 client.20133 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bf0c3f96a0 con 0x55bf0c493b70 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e1dffb640 1 -- 172.21.15.100:0/2713365704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bf0c0fa7e0 con 0x55bf0c493b70 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e1dffb640 1 -- 172.21.15.100:0/2713365704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e200238b0 con 0x55bf0c493b70 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 10 client.20133 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 10 client.20133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:34:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 20 client.20133 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c4acbb0 0x55bf0c4aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 4 client.20133 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf0c137720 con 0x55bf0c4acbb0 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.386+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c4acbb0 0x55bf0c4aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e1dffb640 10 client.20133 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c4acbb0 0x55bf0c4aefd0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588b0f694116d26b server_cookie=69553d1690e7e04b in_seq=0 out_seq=0 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e1dffb640 1 -- 172.21.15.100:0/2713365704 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf0c137720 con 0x55bf0c4acbb0 2024-03-12T05:34:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e1dffb640 10 client.20133 handle_command_reply: tid=0 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 client.20133 shutdown 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 msgr2=0x55bf0c499cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c499cc0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005890 tx=0x7f3e20019a80 comp rx=0 tx=0).stop 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 shutdown_connections 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c4acbb0 0x55bf0c4aefd0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c49a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c499cc0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e1dffb640 0 client.20133 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/2713365704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c499780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 >> 172.21.15.100:0/2713365704 conn(0x55bf0c48d980 msgr2=0x55bf0c48f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 shutdown_connections 2024-03-12T05:34:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.387+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/2713365704 wait complete. 2024-03-12T05:34:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.388+0000 7f3e3fc2f740 20 client.20133 trim_cache size 0 max 16384 2024-03-12T05:34:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 Processor -- start 2024-03-12T05:34:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 -- start start 2024-03-12T05:34:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c49beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c493b70 0x55bf0c49a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 0x55bf0c496a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c380130 con 0x55bf0c49c750 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c4892b0 con 0x55bf0c493b70 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c4a95f0 con 0x55bf0c493250 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 0x55bf0c496a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 0x55bf0c496a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57198/0 (socket says 172.21.15.100:57198) 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 -- 172.21.15.100:0/748539933 learned_addr learned my addr 172.21.15.100:0/748539933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e3659c640 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c493b70 0x55bf0c49a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 -- 172.21.15.100:0/748539933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 msgr2=0x55bf0c49beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c49beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 -- 172.21.15.100:0/748539933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c493b70 msgr2=0x55bf0c49a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c493b70 0x55bf0c49a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.399+0000 7f3e35d9b640 1 -- 172.21.15.100:0/748539933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0c395c20 con 0x55bf0c49c750 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 0x55bf0c496a00 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f3e28036040 tx=0x7f3e2802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49adc0353596007e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e1ffff640 1 -- 172.21.15.100:0/748539933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28054070 con 0x55bf0c49c750 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e1ffff640 1 -- 172.21.15.100:0/748539933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e2804b040 con 0x55bf0c49c750 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e1ffff640 1 -- 172.21.15.100:0/748539933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2804a040 con 0x55bf0c49c750 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/748539933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 msgr2=0x55bf0c496a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 0x55bf0c496a00 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f3e28036040 tx=0x7f3e2802bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/748539933 shutdown_connections 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c49c750 0x55bf0c496a00 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c493b70 0x55bf0c49a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/748539933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c49beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/748539933 >> 172.21.15.100:0/748539933 conn(0x55bf0c48d980 msgr2=0x55bf0c4b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/748539933 shutdown_connections 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.400+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/748539933 wait complete. 2024-03-12T05:34:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 -- start start 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 Processor -- start 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c4a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c4a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c4fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c4fc630 con 0x55bf0c493b70 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c4fc7a0 con 0x55bf0c49c750 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0c4fc960 con 0x55bf0c493250 2024-03-12T05:34:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3fc2f740 10 client.0 _ll_register_callbacks cb 0x7f3e20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c4a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c4a30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57208/0 (socket says 172.21.15.100:57208) 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 -- 172.21.15.100:0/4163230012 learned_addr learned my addr 172.21.15.100:0/4163230012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c4a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c4fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 -- 172.21.15.100:0/4163230012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 msgr2=0x55bf0c4a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c4a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 -- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 msgr2=0x55bf0c4fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c4fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 -- 172.21.15.100:0/4163230012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e28030070 con 0x55bf0c493b70 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c4fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e3659c640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c4a30a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3e20011850 tx=0x7f3e20018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc6c21b84cbf689 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e1dffb640 1 -- 172.21.15.100:0/4163230012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20011d40 con 0x55bf0c493b70 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e20009e70 con 0x55bf0c493b70 2024-03-12T05:34:07.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.401+0000 7f3e1dffb640 1 -- 172.21.15.100:0/4163230012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3e20000e40 con 0x55bf0c493b70 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e1dffb640 1 -- 172.21.15.100:0/4163230012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e200233d0 con 0x55bf0c493b70 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 10 client.20139 fetch_fsmap learned FSMap version 30 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bf0c3bb8c0 con 0x55bf0c493b70 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e1dffb640 1 -- 172.21.15.100:0/4163230012 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e20009e70 con 0x55bf0c493b70 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e1dffb640 1 -- 172.21.15.100:0/4163230012 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3e200035f0 con 0x55bf0c493b70 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 10 client.20139 fetch_fsmap finished waiting for FSMap version 30 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 10 client.20139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-12T05:34:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 20 client.20139 populate_metadata read hostname 'smithi100' 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c505100 0x55bf0c507520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 4 client.20139 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 --> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bf0c186890 con 0x55bf0c505100 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c505100 0x55bf0c507520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e1dffb640 10 client.20139 ms_handle_connect on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.402+0000 7f3e35d9b640 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c505100 0x55bf0c507520 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad8f51148223c88 server_cookie=eac3aa177c9cf43d in_seq=0 out_seq=0 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e1dffb640 1 -- 172.21.15.100:0/4163230012 <== mds.0 v2:172.21.15.156:6838/1209044969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bf0c186890 con 0x55bf0c505100 2024-03-12T05:34:07.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e1dffb640 10 client.20139 handle_command_reply: tid=0 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 client.20139 shutdown 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 msgr2=0x55bf0c4a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c4a30a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3e20011850 tx=0x7f3e20018400 comp rx=0 tx=0).stop 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 shutdown_connections 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:6838/1209044969,v1:172.21.15.156:6839/1209044969] conn(0x55bf0c505100 0x55bf0c507520 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0c49c750 0x55bf0c4fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf0c493b70 0x55bf0c4a30a0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e1dffb640 0 client.20139 ms_handle_reset on v2:172.21.15.156:6838/1209044969 2024-03-12T05:34:07.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 --2- 172.21.15.100:0/4163230012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf0c493250 0x55bf0c4a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 >> 172.21.15.100:0/4163230012 conn(0x55bf0c48d980 msgr2=0x55bf0c48c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 shutdown_connections 2024-03-12T05:34:07.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 1 -- 172.21.15.100:0/4163230012 wait complete. 2024-03-12T05:34:07.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3fc2f740 20 client.20139 trim_cache size 0 max 16384 2024-03-12T05:34:07.413 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-03-12T05:34:07.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e240a1ba0 msgr2=0x7f3e240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e240a1ba0 0x7f3e240a4060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3e3009a5d0 tx=0x7f3e3001e900 comp rx=0 tx=0).stop 2024-03-12T05:34:07.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 msgr2=0x7f3e38197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:07.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 0x7f3e38197ad0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000eaa0 tx=0x7f3e3000ef60 comp rx=0 tx=0).stop 2024-03-12T05:34:07.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 shutdown_connections 2024-03-12T05:34:07.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f3e240a1ba0 0x7f3e240a4060 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38197650 0x7f3e38197ad0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e38105370 0x7f3e3819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 --2- 172.21.15.100:0/3953120107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e381002a0 0x7f3e3819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:07.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 >> 172.21.15.100:0/3953120107 conn(0x7f3e38076400 msgr2=0x7f3e380fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:07.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 shutdown_connections 2024-03-12T05:34:07.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:07.403+0000 7f3e3cf7a640 1 -- 172.21.15.100:0/3953120107 wait complete. 2024-03-12T05:34:07.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:07 smithi156 ceph-mon[36270]: from='client.20073 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:34:07.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:07 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/89035337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:07.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:07 smithi161 ceph-mon[36043]: from='client.20073 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:34:07.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:07 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/89035337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:08.418 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-12T05:34:08.418 INFO:tasks.fwd_scrub:done joining 2024-03-12T05:34:08.418 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-12T05:34:08.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:34:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:08 smithi156 ceph-mon[36270]: from='client.20085 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:34:08.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:08 smithi156 ceph-mon[36270]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 433 op/s 2024-03-12T05:34:08.458 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:08 smithi100 ceph-mon[32210]: from='client.20085 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:34:08.458 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:08 smithi100 ceph-mon[32210]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 433 op/s 2024-03-12T05:34:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:08 smithi161 ceph-mon[36043]: from='client.20085 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T05:34:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:08 smithi161 ceph-mon[36043]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 433 op/s 2024-03-12T05:34:08.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 -- 172.21.15.100:0/2433181606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 msgr2=0x7fef6c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2433181606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c104500 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009900 tx=0x7fef5c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:08.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 -- 172.21.15.100:0/2433181606 shutdown_connections 2024-03-12T05:34:08.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2433181606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef6c10a260 0x7fef6c10c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2433181606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c104500 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2433181606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c0676e0 0x7fef6c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 -- 172.21.15.100:0/2433181606 >> 172.21.15.100:0/2433181606 conn(0x7fef6c0761e0 msgr2=0x7fef6c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 -- 172.21.15.100:0/2433181606 shutdown_connections 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.551+0000 7fef73f1e640 1 -- 172.21.15.100:0/2433181606 wait complete. 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 Processor -- start 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 -- start start 2024-03-12T05:34:08.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c0676e0 0x7fef6c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef6c10a260 0x7fef6c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c197c80 con 0x7fef6c0ffef0 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c197df0 con 0x7fef6c10a260 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef73f1e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c197fb0 con 0x7fef6c0676e0 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef714b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef714b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57216/0 (socket says 172.21.15.100:57216) 2024-03-12T05:34:08.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef714b9640 1 -- 172.21.15.100:0/2300859386 learned_addr learned my addr 172.21.15.100:0/2300859386 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.552+0000 7fef71cba640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c0676e0 0x7fef6c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef724bb640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef6c10a260 0x7fef6c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef714b9640 1 -- 172.21.15.100:0/2300859386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c0676e0 msgr2=0x7fef6c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef714b9640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c0676e0 0x7fef6c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef714b9640 1 -- 172.21.15.100:0/2300859386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef6c10a260 msgr2=0x7fef6c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef714b9640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef6c10a260 0x7fef6c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef714b9640 1 -- 172.21.15.100:0/2300859386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c009560 con 0x7fef6c0ffef0 2024-03-12T05:34:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef714b9640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c19d970 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005940 tx=0x7fef5c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004440 con 0x7fef6c0ffef0 2024-03-12T05:34:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c198230 con 0x7fef6c0ffef0 2024-03-12T05:34:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef5c0052d0 con 0x7fef6c0ffef0 2024-03-12T05:34:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c003da0 con 0x7fef6c0ffef0 2024-03-12T05:34:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.553+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c1a44d0 con 0x7fef6c0ffef0 2024-03-12T05:34:08.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.554+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0683c0 con 0x7fef6c0ffef0 2024-03-12T05:34:08.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.555+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fef5c005440 con 0x7fef6c0ffef0 2024-03-12T05:34:08.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.556+0000 7fef62ffd640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef580a1ba0 0x7fef580a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.556+0000 7fef71cba640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef580a1ba0 0x7fef580a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.556+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fef5c0ef930 con 0x7fef6c0ffef0 2024-03-12T05:34:08.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.556+0000 7fef71cba640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef580a1ba0 0x7fef580a4060 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fef68005fd0 tx=0x7fef6800b390 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:08.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.560+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef5c0b88d0 con 0x7fef6c0ffef0 2024-03-12T05:34:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.734+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef6c10f920 con 0x7fef6c0ffef0 2024-03-12T05:34:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.736+0000 7fef62ffd640 1 -- 172.21.15.100:0/2300859386 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+23209 (secure 0 0 0) 0x7fef5c0bbf90 con 0x7fef6c0ffef0 2024-03-12T05:34:08.731 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:08.731 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":119,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:33:15.961343+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/327651532":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1404323639":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/929302438":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2828798630":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/576676871":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/3916551108":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6800/327651532":"2024-03-13T05:12:05.720783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:34:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef580a1ba0 msgr2=0x7fef580a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef580a1ba0 0x7fef580a4060 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fef68005fd0 tx=0x7fef6800b390 comp rx=0 tx=0).stop 2024-03-12T05:34:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 msgr2=0x7fef6c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c19d970 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005940 tx=0x7fef5c004040 comp rx=0 tx=0).stop 2024-03-12T05:34:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 shutdown_connections 2024-03-12T05:34:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fef580a1ba0 0x7fef580a4060 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef6c10a260 0x7fef6c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0ffef0 0x7fef6c19d970 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 --2- 172.21.15.100:0/2300859386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c0676e0 0x7fef6c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 >> 172.21.15.100:0/2300859386 conn(0x7fef6c0761e0 msgr2=0x7fef6c103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 shutdown_connections 2024-03-12T05:34:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.738+0000 7fef73f1e640 1 -- 172.21.15.100:0/2300859386 wait complete. 2024-03-12T05:34:08.745 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 118, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:34:08.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:34:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 -- 172.21.15.100:0/2272636312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 msgr2=0x7fae64100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 --2- 172.21.15.100:0/2272636312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64100680 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fae58009930 tx=0x7fae5802bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 -- 172.21.15.100:0/2272636312 shutdown_connections 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 --2- 172.21.15.100:0/2272636312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae64105370 0x7fae6410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 --2- 172.21.15.100:0/2272636312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64100c50 0x7fae64104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 --2- 172.21.15.100:0/2272636312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64100680 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 -- 172.21.15.100:0/2272636312 >> 172.21.15.100:0/2272636312 conn(0x7fae64076400 msgr2=0x7fae64076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 -- 172.21.15.100:0/2272636312 shutdown_connections 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.894+0000 7fae6abac640 1 -- 172.21.15.100:0/2272636312 wait complete. 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 Processor -- start 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 -- start start 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae64105370 0x7fae64102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae641027b0 0x7fae641a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641a6060 con 0x7fae641002a0 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641a61d0 con 0x7fae64105370 2024-03-12T05:34:08.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641a6390 con 0x7fae641027b0 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57234/0 (socket says 172.21.15.100:57234) 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 -- 172.21.15.100:0/3160354635 learned_addr learned my addr 172.21.15.100:0/3160354635 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae69149640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae641027b0 0x7fae641a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 -- 172.21.15.100:0/3160354635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae641027b0 msgr2=0x7fae641a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae641027b0 0x7fae641a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 -- 172.21.15.100:0/3160354635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae64105370 msgr2=0x7fae64102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae64105370 0x7fae64102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 -- 172.21.15.100:0/3160354635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009560 con 0x7fae641002a0 2024-03-12T05:34:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae68948640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64101d30 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fae58009230 tx=0x7fae58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58004440 con 0x7fae641002a0 2024-03-12T05:34:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae641a6610 con 0x7fae641002a0 2024-03-12T05:34:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.895+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae580052d0 con 0x7fae641002a0 2024-03-12T05:34:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.896+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58003d70 con 0x7fae641002a0 2024-03-12T05:34:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.896+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641a6b20 con 0x7fae641002a0 2024-03-12T05:34:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.897+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae64101770 con 0x7fae641002a0 2024-03-12T05:34:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.898+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fae58005440 con 0x7fae641002a0 2024-03-12T05:34:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.898+0000 7fae61ffb640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae540a1bd0 0x7fae540a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.898+0000 7fae63fff640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae540a1bd0 0x7fae540a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.898+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fae580f0110 con 0x7fae641002a0 2024-03-12T05:34:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.899+0000 7fae63fff640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae540a1bd0 0x7fae540a4090 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fae50005fd0 tx=0x7fae50005850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:08.902+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae580b9150 con 0x7fae641002a0 2024-03-12T05:34:09.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.096+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae64100c50 con 0x7fae641002a0 2024-03-12T05:34:09.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.096+0000 7fae61ffb640 1 -- 172.21.15.100:0/3160354635 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae58047e90 con 0x7fae641002a0 2024-03-12T05:34:09.090 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:34:09.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae540a1bd0 msgr2=0x7fae540a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae540a1bd0 0x7fae540a4090 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fae50005fd0 tx=0x7fae50005850 comp rx=0 tx=0).stop 2024-03-12T05:34:09.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 msgr2=0x7fae64101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64101d30 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fae58009230 tx=0x7fae58004040 comp rx=0 tx=0).stop 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 shutdown_connections 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fae540a1bd0 0x7fae540a4090 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae641027b0 0x7fae641a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae64105370 0x7fae64102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 --2- 172.21.15.100:0/3160354635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae641002a0 0x7fae64101d30 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 >> 172.21.15.100:0/3160354635 conn(0x7fae64076400 msgr2=0x7fae64104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 shutdown_connections 2024-03-12T05:34:09.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.099+0000 7fae6abac640 1 -- 172.21.15.100:0/3160354635 wait complete. 2024-03-12T05:34:09.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:34:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 -- 172.21.15.100:0/988318379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8100260 msgr2=0x7f32a8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 --2- 172.21.15.100:0/988318379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8100260 0x7f32a8100640 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f3298009cd0 tx=0x7f329802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 -- 172.21.15.100:0/988318379 shutdown_connections 2024-03-12T05:34:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 --2- 172.21.15.100:0/988318379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a8105330 0x7f32a810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 --2- 172.21.15.100:0/988318379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a8100c10 0x7f32a8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 --2- 172.21.15.100:0/988318379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8100260 0x7f32a8100640 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 -- 172.21.15.100:0/988318379 >> 172.21.15.100:0/988318379 conn(0x7f32a8077180 msgr2=0x7f32a8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:09.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 -- 172.21.15.100:0/988318379 shutdown_connections 2024-03-12T05:34:09.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 -- 172.21.15.100:0/988318379 wait complete. 2024-03-12T05:34:09.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.227+0000 7f32ad92c640 1 Processor -- start 2024-03-12T05:34:09.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 -- start start 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a8100260 0x7f32a819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 0x7f32a819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a81975d0 0x7f32a8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8197fc0 con 0x7f32a8105330 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8198130 con 0x7f32a8100260 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32ad92c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a81982f0 con 0x7f32a81975d0 2024-03-12T05:34:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 0x7f32a819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 0x7f32a819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57244/0 (socket says 172.21.15.100:57244) 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 -- 172.21.15.100:0/3044146878 learned_addr learned my addr 172.21.15.100:0/3044146878 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a6ffd640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a8100260 0x7f32a819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a77fe640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a81975d0 0x7f32a8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 -- 172.21.15.100:0/3044146878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a81975d0 msgr2=0x7f32a8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a81975d0 0x7f32a8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 -- 172.21.15.100:0/3044146878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a8100260 msgr2=0x7f32a819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a8100260 0x7f32a819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a67fc640 1 -- 172.21.15.100:0/3044146878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298009900 con 0x7f32a8105330 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.228+0000 7f32a77fe640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a81975d0 0x7f32a8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.229+0000 7f32a67fc640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 0x7f32a819d960 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9c0 tx=0x7f329c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.229+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c009700 con 0x7f32a8105330 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.229+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a81a42e0 con 0x7f32a8105330 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.229+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f329c00cd80 con 0x7f32a8105330 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.229+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c010430 con 0x7f32a8105330 2024-03-12T05:34:09.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.229+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a81a4840 con 0x7f32a8105330 2024-03-12T05:34:09.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.231+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a8067f70 con 0x7f32a8105330 2024-03-12T05:34:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.232+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f329c005000 con 0x7f32a8105330 2024-03-12T05:34:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.232+0000 7f32ac92a640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f32900a1ba0 0x7f32900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.232+0000 7f32a6ffd640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f32900a1ba0 0x7f32900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.232+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f329c014070 con 0x7f32a8105330 2024-03-12T05:34:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.233+0000 7f32a6ffd640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f32900a1ba0 0x7f32900a4060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3298004af0 tx=0x7f3298005c50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:09.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.239+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f329c096b70 con 0x7f32a8105330 2024-03-12T05:34:09.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.434+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32a8108560 con 0x7f32a8105330 2024-03-12T05:34:09.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.434+0000 7f32ac92a640 1 -- 172.21.15.100:0/3044146878 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f329c09a230 con 0x7f32a8105330 2024-03-12T05:34:09.428 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:34:09.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f32900a1ba0 msgr2=0x7f32900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f32900a1ba0 0x7f32900a4060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3298004af0 tx=0x7f3298005c50 comp rx=0 tx=0).stop 2024-03-12T05:34:09.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 msgr2=0x7f32a819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 0x7f32a819d960 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9c0 tx=0x7f329c00ee80 comp rx=0 tx=0).stop 2024-03-12T05:34:09.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 shutdown_connections 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f32900a1ba0 0x7f32900a4060 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a81975d0 0x7f32a8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32a8105330 0x7f32a819d960 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 --2- 172.21.15.100:0/3044146878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a8100260 0x7f32a819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 >> 172.21.15.100:0/3044146878 conn(0x7f32a8077180 msgr2=0x7f32a80fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 shutdown_connections 2024-03-12T05:34:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.436+0000 7f32ad92c640 1 -- 172.21.15.100:0/3044146878 wait complete. 2024-03-12T05:34:09.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:09 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2300859386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:09.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:09 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3160354635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:34:09.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:34:09.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:09 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2300859386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:09.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:09 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3160354635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:34:09.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:09 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2300859386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:09.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:09 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3160354635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:34:09.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 -- 172.21.15.100:0/2033132229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101d30 msgr2=0x7f834c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 --2- 172.21.15.100:0/2033132229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101d30 0x7f834c105d70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8334009930 tx=0x7f833402bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 -- 172.21.15.100:0/2033132229 shutdown_connections 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 --2- 172.21.15.100:0/2033132229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c106450 0x7f834c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 --2- 172.21.15.100:0/2033132229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101d30 0x7f834c105d70 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 --2- 172.21.15.100:0/2033132229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834c101380 0x7f834c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 -- 172.21.15.100:0/2033132229 >> 172.21.15.100:0/2033132229 conn(0x7f834c0fcc20 msgr2=0x7f834c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 -- 172.21.15.100:0/2033132229 shutdown_connections 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.552+0000 7f8351a83640 1 -- 172.21.15.100:0/2033132229 wait complete. 2024-03-12T05:34:09.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 Processor -- start 2024-03-12T05:34:09.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 -- start start 2024-03-12T05:34:09.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 0x7f834c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834c106450 0x7f834c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c1976d0 0x7f834c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1981a0 con 0x7f834c101380 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c198310 con 0x7f834c1976d0 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8351a83640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1984d0 con 0x7f834c106450 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f834b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 0x7f834c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f834b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 0x7f834c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57268/0 (socket says 172.21.15.100:57268) 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f834b7fe640 1 -- 172.21.15.100:0/3633188243 learned_addr learned my addr 172.21.15.100:0/3633188243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f8342ffd640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834c106450 0x7f834c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.553+0000 7f834bfff640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c1976d0 0x7f834c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f834b7fe640 1 -- 172.21.15.100:0/3633188243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834c106450 msgr2=0x7f834c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f834b7fe640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834c106450 0x7f834c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f834b7fe640 1 -- 172.21.15.100:0/3633188243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c1976d0 msgr2=0x7f834c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f834b7fe640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c1976d0 0x7f834c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f834b7fe640 1 -- 172.21.15.100:0/3633188243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334009560 con 0x7f834c101380 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f834b7fe640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 0x7f834c19d5c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c970 tx=0x7f833c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c007bb0 con 0x7f834c101380 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c1a4420 con 0x7f834c101380 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f833c007d10 con 0x7f834c101380 2024-03-12T05:34:09.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00f450 con 0x7f834c101380 2024-03-12T05:34:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.554+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c1a4900 con 0x7f834c101380 2024-03-12T05:34:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.555+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c102850 con 0x7f834c101380 2024-03-12T05:34:09.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.556+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f833c004f70 con 0x7f834c101380 2024-03-12T05:34:09.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.557+0000 7f83497fa640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83380a1ba0 0x7f83380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.557+0000 7f8342ffd640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83380a1ba0 0x7f83380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.557+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f833c0cd590 con 0x7f834c101380 2024-03-12T05:34:09.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.557+0000 7f8342ffd640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83380a1ba0 0x7f83380a4060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8334000c00 tx=0x7f8334005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.561+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f833c096420 con 0x7f834c101380 2024-03-12T05:34:09.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.735+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f834c109680 con 0x7f834c101380 2024-03-12T05:34:09.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.736+0000 7f83497fa640 1 -- 172.21.15.100:0/3633188243 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f833c01d020 con 0x7f834c101380 2024-03-12T05:34:09.729 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:34:09.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83380a1ba0 msgr2=0x7f83380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83380a1ba0 0x7f83380a4060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8334000c00 tx=0x7f8334005fd0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 msgr2=0x7f834c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 0x7f834c19d5c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c970 tx=0x7f833c00ce30 comp rx=0 tx=0).stop 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 shutdown_connections 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f83380a1ba0 0x7f83380a4060 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c1976d0 0x7f834c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834c106450 0x7f834c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 --2- 172.21.15.100:0/3633188243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f834c101380 0x7f834c19d5c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 >> 172.21.15.100:0/3633188243 conn(0x7f834c0fcc20 msgr2=0x7f834c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 shutdown_connections 2024-03-12T05:34:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.738+0000 7f8351a83640 1 -- 172.21.15.100:0/3633188243 wait complete. 2024-03-12T05:34:09.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-12T05:34:09.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 -- 172.21.15.100:0/2381570551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460100410 msgr2=0x7ff46010e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 --2- 172.21.15.100:0/2381570551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460100410 0x7ff46010e010 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff458009cd0 tx=0x7ff45802c180 comp rx=0 tx=0).stop 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 -- 172.21.15.100:0/2381570551 shutdown_connections 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 --2- 172.21.15.100:0/2381570551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460100410 0x7ff46010e010 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 --2- 172.21.15.100:0/2381570551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4600ffa50 0x7ff4600ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 --2- 172.21.15.100:0/2381570551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff460107390 0x7ff460107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 -- 172.21.15.100:0/2381570551 >> 172.21.15.100:0/2381570551 conn(0x7ff4600753d0 msgr2=0x7ff4600757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 -- 172.21.15.100:0/2381570551 shutdown_connections 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.850+0000 7ff465b77640 1 -- 172.21.15.100:0/2381570551 wait complete. 2024-03-12T05:34:09.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 Processor -- start 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 -- start start 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4600ffa50 0x7ff46019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460100410 0x7ff46019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 0x7ff460197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460197c60 con 0x7ff460107390 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460197dd0 con 0x7ff4600ffa50 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff465b77640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460197f90 con 0x7ff460100410 2024-03-12T05:34:09.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 0x7ff460197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff45effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460100410 0x7ff46019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 0x7ff460197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57292/0 (socket says 172.21.15.100:57292) 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.851+0000 7ff45ffff640 1 -- 172.21.15.100:0/2066962883 learned_addr learned my addr 172.21.15.100:0/2066962883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45f7fe640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4600ffa50 0x7ff46019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45effd640 1 -- 172.21.15.100:0/2066962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4600ffa50 msgr2=0x7ff46019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45effd640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4600ffa50 0x7ff46019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45effd640 1 -- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 msgr2=0x7ff460197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45effd640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 0x7ff460197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:09.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45effd640 1 -- 172.21.15.100:0/2066962883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff458009900 con 0x7ff460100410 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45ffff640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 0x7ff460197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45effd640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460100410 0x7ff46019d950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e9f0 tx=0x7ff45400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45400ccc0 con 0x7ff460100410 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff460198270 con 0x7ff460100410 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff45400ce20 con 0x7ff460100410 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454004a00 con 0x7ff460100410 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.852+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601a44b0 con 0x7ff460100410 2024-03-12T05:34:09.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.853+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4600683c0 con 0x7ff460100410 2024-03-12T05:34:09.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.854+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff4540050b0 con 0x7ff460100410 2024-03-12T05:34:09.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.855+0000 7ff45cff9640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4480a1ba0 0x7ff4480a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:09.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.855+0000 7ff45f7fe640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4480a1ba0 0x7ff4480a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:09.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.855+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff454014070 con 0x7ff460100410 2024-03-12T05:34:09.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.855+0000 7ff45f7fe640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4480a1ba0 0x7ff4480a4060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff450005d00 tx=0x7ff45000b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:09.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:09.858+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff454096760 con 0x7ff460100410 2024-03-12T05:34:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.080+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7ff460198af0 con 0x7ff460100410 2024-03-12T05:34:10.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.088+0000 7ff45cff9640 1 -- 172.21.15.100:0/2066962883 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+735009 (secure 0 0 0) 0x7ff454099e20 con 0x7ff460100410 2024-03-12T05:34:10.092 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:10.092 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 26, 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "active_gid": 15432, 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "active_name": "x", 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "active_addrs": { 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.093 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2025021937 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2025021937 2024-03-12T05:34:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: "active_addr": "172.21.15.100:6801/2025021937", 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: "active_change": "2024-03-12T05:12:05.721002+0000", 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: "active_mgr_features": 4540701547738038271, 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: "available": true, 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: "standbys": [ 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.095 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 24875, 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "y", 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_features": 4540701547738038271, 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.096 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.097 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.098 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.099 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.100 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.101 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.102 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.103 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.104 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.105 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.107 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to", 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-03-12T05:34:10.108 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-03-12T05:34:10.109 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.110 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.111 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.112 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-03-12T05:34:10.112 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-03-12T05:34:10.113 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.114 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.115 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.116 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.119 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.120 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.122 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.123 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.124 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.125 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-03-12T05:34:10.126 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.127 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.128 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.129 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.130 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-03-12T05:34:10.131 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-03-12T05:34:10.132 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T05:34:10.133 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.134 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.136 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.138 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.139 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.139 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.139 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.139 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.139 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.140 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.140 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.140 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.140 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.140 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.140 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.141 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.142 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.142 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.147 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-03-12T05:34:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-03-12T05:34:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-03-12T05:34:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-03-12T05:34:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-03-12T05:34:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-03-12T05:34:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-03-12T05:34:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-03-12T05:34:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-12T05:34:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.165 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-03-12T05:34:10.166 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.167 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-03-12T05:34:10.171 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-03-12T05:34:10.172 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.174 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.176 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-03-12T05:34:10.177 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.178 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.179 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-12T05:34:10.181 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.182 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.183 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.184 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.185 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.185 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-12T05:34:10.185 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.185 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.185 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.185 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.186 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.187 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-12T05:34:10.188 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.189 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.190 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.191 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.192 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.193 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-03-12T05:34:10.196 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.197 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.198 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.199 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.200 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.202 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-03-12T05:34:10.203 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-03-12T05:34:10.204 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.205 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.206 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.209 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-03-12T05:34:10.210 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-03-12T05:34:10.211 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.212 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.213 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.214 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-03-12T05:34:10.215 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.216 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.217 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.218 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.219 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.221 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.222 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-03-12T05:34:10.223 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.224 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.225 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.226 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.227 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.228 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.229 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.230 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-12T05:34:10.231 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.232 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.233 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-12T05:34:10.234 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.235 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.237 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-03-12T05:34:10.238 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.239 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.240 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-03-12T05:34:10.241 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-03-12T05:34:10.242 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-12T05:34:10.243 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.244 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.246 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-03-12T05:34:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.248 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-03-12T05:34:10.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-12T05:34:10.250 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.251 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.252 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.253 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.255 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-03-12T05:34:10.256 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-12T05:34:10.257 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.258 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.260 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.262 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-03-12T05:34:10.263 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.264 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.265 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.266 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.267 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.268 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.270 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.271 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.272 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.273 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.275 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-03-12T05:34:10.276 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-12T05:34:10.277 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.278 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.280 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.281 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-03-12T05:34:10.282 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.283 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.284 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.285 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.286 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.288 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-12T05:34:10.289 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.290 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.291 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.294 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.295 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-12T05:34:10.296 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.297 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.298 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.299 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.300 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.302 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-12T05:34:10.303 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.304 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.305 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.306 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.307 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.309 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-03-12T05:34:10.310 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.311 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.312 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.313 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.314 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.316 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-12T05:34:10.317 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.318 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-12T05:34:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.320 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.321 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.322 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-12T05:34:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.326 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.327 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.328 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-12T05:34:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-12T05:34:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-12T05:34:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-12T05:34:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-03-12T05:34:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-03-12T05:34:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-03-12T05:34:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-03-12T05:34:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-03-12T05:34:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-03-12T05:34:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-03-12T05:34:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.432 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:10.433 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.434 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.435 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.436 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.438 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.439 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.440 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.441 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.442 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.444 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-03-12T05:34:10.445 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.447 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.448 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.449 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.451 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-03-12T05:34:10.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T05:34:10.453 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-03-12T05:34:10.454 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.455 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.456 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.457 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.458 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.459 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.460 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.461 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.462 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.465 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.466 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.467 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.469 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.470 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.471 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.472 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "k8sevents", 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.473 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_event_retention_days": { 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ceph_event_retention_days", 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-03-12T05:34:10.474 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "config_check_secs": { 2024-03-12T05:34:10.475 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_check_secs", 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "10", 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.476 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.477 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.478 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.480 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.481 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.482 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.483 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.484 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.485 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.486 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.487 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.488 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.489 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.490 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.491 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.493 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.494 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.495 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.496 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.497 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.498 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-03-12T05:34:10.499 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.501 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-03-12T05:34:10.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-03-12T05:34:10.503 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.504 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.506 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.507 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.508 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.509 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.510 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.512 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.513 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.514 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.515 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.516 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.517 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.518 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.518 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.518 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.518 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.518 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.519 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.519 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.519 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.519 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.519 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.520 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.520 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.520 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.520 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.520 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.520 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.521 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.521 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.521 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.521 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.521 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.523 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.524 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.527 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.528 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.529 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.530 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.532 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.533 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.534 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.535 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-03-12T05:34:10.536 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.537 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.538 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.539 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.540 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.541 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.542 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.543 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.544 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.545 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.547 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-03-12T05:34:10.548 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-03-12T05:34:10.549 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.550 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.551 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.552 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.553 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.555 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.556 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.557 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.558 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.559 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.561 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.562 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.563 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.564 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.565 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.567 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.568 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.569 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.570 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.571 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.572 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.573 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.574 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.577 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.578 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.579 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.580 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-03-12T05:34:10.581 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.582 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.583 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.584 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.585 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.586 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.587 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.590 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.591 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.592 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.593 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.594 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:10.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.596 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.597 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.598 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.599 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.600 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.601 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.602 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.603 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.604 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.605 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.606 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.607 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.608 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.609 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.610 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.612 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-03-12T05:34:10.613 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.614 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.615 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.616 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-03-12T05:34:10.617 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.618 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.619 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-03-12T05:34:10.620 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.621 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-03-12T05:34:10.622 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-03-12T05:34:10.623 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.625 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.626 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.627 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.628 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.629 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.630 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.632 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-03-12T05:34:10.633 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.634 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.635 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.636 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.637 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.639 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.640 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.641 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.642 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.643 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.644 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.645 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.646 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.646 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.646 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.646 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.646 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.646 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.647 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.648 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.649 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.650 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.651 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.652 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.653 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.654 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.656 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.657 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.659 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.660 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.661 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.662 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.663 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-03-12T05:34:10.664 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.665 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.666 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.667 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.668 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.669 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.670 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.672 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-03-12T05:34:10.673 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.674 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.675 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.676 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.677 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.678 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.681 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.682 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.683 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.685 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-03-12T05:34:10.686 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.687 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.688 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-03-12T05:34:10.689 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.690 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.691 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.693 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-03-12T05:34:10.694 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.695 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.696 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.697 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.698 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-03-12T05:34:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.702 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.703 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.704 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-03-12T05:34:10.705 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.706 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.707 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.708 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.709 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.710 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.713 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.714 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.716 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.718 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.719 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.720 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.721 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.722 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.723 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.724 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.725 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.726 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.727 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.728 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.729 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.730 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.731 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.732 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.733 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.734 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.735 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.736 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.737 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.738 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.739 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.740 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.741 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.742 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.743 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.744 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.745 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.746 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.747 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-03-12T05:34:10.748 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.749 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.751 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-03-12T05:34:10.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.753 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.754 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.756 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-03-12T05:34:10.757 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.758 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.759 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.760 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-12T05:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.762 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.763 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.764 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.765 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.768 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-03-12T05:34:10.769 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.770 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.772 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.773 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.774 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.775 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.776 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.777 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.778 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.780 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-03-12T05:34:10.781 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.782 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.783 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.784 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.785 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.786 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.787 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.788 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.789 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.790 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.791 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.792 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.793 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.795 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.797 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.800 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.801 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.802 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.803 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-03-12T05:34:10.804 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.805 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.806 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.807 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.809 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-03-12T05:34:10.810 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.811 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.812 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:10.813 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.814 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.815 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.816 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.818 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.819 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.821 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.822 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.823 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.824 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.825 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.826 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.827 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.828 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 25009, 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "z", 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_features": 4540701547738038271, 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to", 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-03-12T05:34:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-03-12T05:34:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-03-12T05:34:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-03-12T05:34:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T05:34:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-03-12T05:34:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-12T05:34:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-03-12T05:34:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-03-12T05:34:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-03-12T05:34:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-03-12T05:34:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-12T05:34:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-03-12T05:34:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-03-12T05:34:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-03-12T05:34:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-12T05:34:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-03-12T05:34:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-03-12T05:34:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-03-12T05:34:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-03-12T05:34:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-03-12T05:34:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-12T05:34:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-03-12T05:34:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-03-12T05:34:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-03-12T05:34:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-03-12T05:34:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-12T05:34:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.938 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.939 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.940 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-03-12T05:34:10.941 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.942 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.943 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.944 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.945 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-03-12T05:34:10.946 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.947 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.948 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.949 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.950 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.951 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.952 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.955 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.956 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.959 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.960 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.962 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-03-12T05:34:10.963 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.964 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.966 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.967 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-03-12T05:34:10.968 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-03-12T05:34:10.969 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-03-12T05:34:10.970 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.971 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-12T05:34:10.972 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-03-12T05:34:10.973 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.975 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.977 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-03-12T05:34:10.978 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.979 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.980 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.981 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-03-12T05:34:10.981 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-03-12T05:34:10.981 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-03-12T05:34:10.983 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.984 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.985 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.986 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.987 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.988 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-03-12T05:34:10.989 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.990 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.991 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.992 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.993 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:10.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:10.995 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:10.996 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:10.998 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:10.999 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.000 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.001 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.002 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.003 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.004 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-03-12T05:34:11.005 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.006 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.007 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.008 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-12T05:34:11.011 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.012 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.013 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.014 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.015 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-03-12T05:34:11.016 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.017 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-12T05:34:11.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.019 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.020 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.022 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-12T05:34:11.023 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.024 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.025 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.026 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.027 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.028 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.029 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-12T05:34:11.030 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.034 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-03-12T05:34:11.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.036 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.037 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.039 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.040 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.041 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.042 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.043 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.044 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.045 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.046 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.047 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.048 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.049 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-12T05:34:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.051 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.052 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.053 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.054 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-12T05:34:11.055 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.056 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.057 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.058 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.059 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-12T05:34:11.060 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.061 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.062 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.064 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.065 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.066 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-03-12T05:34:11.067 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.068 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.069 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-12T05:34:11.070 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.071 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.072 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.074 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-12T05:34:11.075 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.076 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.077 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.078 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:11.079 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-03-12T05:34:11.080 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.081 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.083 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.084 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-03-12T05:34:11.085 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.086 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.087 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.089 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.090 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.091 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.092 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.093 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.095 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.096 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.097 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.098 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.099 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.100 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.101 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:11.102 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.103 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.104 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.105 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-03-12T05:34:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-03-12T05:34:11.107 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.108 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-03-12T05:34:11.109 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-03-12T05:34:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.113 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.114 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.115 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.116 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.117 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.118 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.120 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.121 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.122 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.123 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.125 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.126 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.127 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-03-12T05:34:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.130 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.131 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.132 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.133 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-03-12T05:34:11.134 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.135 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.136 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.137 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.139 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.140 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.141 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.142 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.143 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.144 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-03-12T05:34:11.145 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.146 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.147 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.148 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.149 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.151 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-03-12T05:34:11.153 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.154 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.155 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.156 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.158 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.159 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.160 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.161 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.162 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.163 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.164 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.166 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-03-12T05:34:11.167 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.168 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-03-12T05:34:11.169 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T05:34:11.170 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.171 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.173 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-03-12T05:34:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.175 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.176 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.177 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.178 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.179 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.180 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.181 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.182 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.183 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.184 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-03-12T05:34:11.185 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.186 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.187 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.188 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.189 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.190 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.191 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.193 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "k8sevents", 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_event_retention_days": { 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ceph_event_retention_days", 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.195 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.196 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: "config_check_secs": { 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_check_secs", 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "10", 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.198 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.199 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.200 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.201 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.203 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.204 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.205 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.206 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-03-12T05:34:11.207 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.208 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.209 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.210 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.211 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.212 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.213 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.214 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.215 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.217 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-03-12T05:34:11.218 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-03-12T05:34:11.219 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.220 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-03-12T05:34:11.222 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.223 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.224 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-03-12T05:34:11.225 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-12T05:34:11.226 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.228 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.229 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.231 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.232 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.233 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.235 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.237 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.238 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.239 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.240 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.242 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.243 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.244 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.245 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.246 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.247 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.250 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.251 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.253 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.254 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.256 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.257 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.258 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.259 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.260 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.261 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.262 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.263 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.264 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.268 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.269 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.270 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.271 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.272 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.273 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.274 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.275 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.276 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.277 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.278 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.279 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.280 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.281 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.282 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.284 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.285 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.286 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.287 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-03-12T05:34:11.288 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.289 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.290 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.292 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.294 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.295 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.296 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.297 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.299 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-12T05:34:11.300 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.301 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.302 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.303 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.304 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.305 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.306 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.308 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.309 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.310 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.311 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.312 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.313 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.314 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.315 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.316 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.318 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.319 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.320 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.322 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.323 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.325 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.326 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.327 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.329 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-03-12T05:34:11.330 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.331 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.332 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.333 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-03-12T05:34:11.334 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.335 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.336 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:11.337 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-12T05:34:11.338 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.339 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.341 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.342 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-03-12T05:34:11.343 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.345 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.346 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.347 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.348 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.349 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.350 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.351 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.352 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-03-12T05:34:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.357 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.358 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.359 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.360 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-03-12T05:34:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.375 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.376 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.377 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.378 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.379 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.380 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.381 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.382 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.383 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.384 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.385 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.388 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.389 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.390 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.391 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.392 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.393 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.394 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.395 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.396 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.397 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.398 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.399 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.401 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.402 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-03-12T05:34:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-03-12T05:34:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-03-12T05:34:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-03-12T05:34:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-03-12T05:34:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-03-12T05:34:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-12T05:34:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-03-12T05:34:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-03-12T05:34:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-03-12T05:34:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.518 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.520 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.521 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.522 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.523 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.524 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-03-12T05:34:11.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-12T05:34:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-03-12T05:34:11.529 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.530 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.531 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.532 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:11.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.534 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.535 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.536 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.537 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.538 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.540 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.541 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.542 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-03-12T05:34:11.543 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.544 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.545 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.546 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.547 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.548 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: "modules": [ 2024-03-12T05:34:11.549 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: "dashboard", 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: "iostat", 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: "nfs", 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: "restful", 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_schedule" 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-03-12T05:34:11.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.552 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.553 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.554 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.555 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.557 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.558 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.559 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.560 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-03-12T05:34:11.561 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.562 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to", 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.563 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.564 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.565 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.566 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.568 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.569 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.570 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.572 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.573 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-03-12T05:34:11.574 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.575 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.576 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-03-12T05:34:11.577 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.578 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-03-12T05:34:11.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T05:34:11.580 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.581 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.583 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-03-12T05:34:11.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-12T05:34:11.585 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.586 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.587 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-03-12T05:34:11.588 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.590 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.591 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.593 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.593 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.593 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.593 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.594 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.595 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.596 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.597 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.598 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.599 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.602 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.603 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:11.605 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.606 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.607 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:11.608 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.609 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.610 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.611 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-03-12T05:34:11.612 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-12T05:34:11.613 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.614 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.615 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.616 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.617 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.618 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-03-12T05:34:11.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-12T05:34:11.620 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.621 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-03-12T05:34:11.622 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.623 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.624 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.625 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.626 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.627 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-03-12T05:34:11.628 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:11.629 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.630 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.631 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-12T05:34:11.632 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.633 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.634 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.635 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.636 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-12T05:34:11.637 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.638 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.639 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-03-12T05:34:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-03-12T05:34:11.641 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.642 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.643 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-12T05:34:11.644 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-03-12T05:34:11.645 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.646 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.647 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-03-12T05:34:11.648 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.649 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.650 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.651 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-03-12T05:34:11.652 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-03-12T05:34:11.653 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.656 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-03-12T05:34:11.657 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-12T05:34:11.658 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.659 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.660 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-03-12T05:34:11.661 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.662 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.663 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.664 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.666 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-03-12T05:34:11.667 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.668 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.669 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.671 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-12T05:34:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-12T05:34:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.686 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.687 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.689 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-03-12T05:34:11.690 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.691 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.693 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-03-12T05:34:11.694 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.695 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.696 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.697 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.698 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.700 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-03-12T05:34:11.701 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-12T05:34:11.702 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.704 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.705 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.706 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.707 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.709 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.710 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.712 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-03-12T05:34:11.713 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.714 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.717 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.718 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.720 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.721 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.722 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.724 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.725 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-03-12T05:34:11.726 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.727 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.728 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.729 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.730 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.731 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-12T05:34:11.732 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.733 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.734 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.737 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.738 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-12T05:34:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.740 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.741 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.743 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-12T05:34:11.744 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.745 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.746 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.747 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.748 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.749 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.750 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-12T05:34:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.752 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.754 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.755 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.756 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.757 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.758 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.759 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.760 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-12T05:34:11.761 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.762 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.763 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.764 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.765 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-03-12T05:34:11.766 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.767 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.768 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.769 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.770 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-12T05:34:11.771 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.772 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.773 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.774 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.775 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-12T05:34:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.777 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.780 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-03-12T05:34:11.781 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.782 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.783 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.784 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.785 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.787 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-12T05:34:11.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.789 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.791 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-03-12T05:34:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-03-12T05:34:11.793 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.794 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.795 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.795 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.795 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.795 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-03-12T05:34:11.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-12T05:34:11.796 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.796 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.796 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.796 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.796 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:11.798 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.800 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.801 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.801 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.801 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.801 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.801 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.801 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:11.802 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.803 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.803 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.803 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.803 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.803 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.804 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.805 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.808 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-03-12T05:34:11.809 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.810 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.811 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.812 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.813 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.814 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.815 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.816 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.817 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.818 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.819 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.822 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.823 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.824 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.825 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.827 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-03-12T05:34:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.829 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-03-12T05:34:11.830 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.831 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-03-12T05:34:11.832 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-03-12T05:34:11.833 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.834 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-03-12T05:34:11.835 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.836 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.837 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.838 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.839 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.840 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.841 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.842 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.843 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.844 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.845 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.846 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.848 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-03-12T05:34:11.849 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.850 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.851 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.852 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:11.853 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.854 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.855 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.856 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.857 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-03-12T05:34:11.858 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.859 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.860 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.861 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.862 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.863 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.864 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.866 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-03-12T05:34:11.867 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.868 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.869 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.871 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.873 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.874 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-03-12T05:34:11.875 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-03-12T05:34:11.876 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-03-12T05:34:11.877 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.878 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.879 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.880 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.881 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.882 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.883 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.884 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.885 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.885 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.885 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.885 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.885 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.885 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.886 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.887 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-03-12T05:34:11.888 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-03-12T05:34:11.889 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.892 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-03-12T05:34:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.894 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.895 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.896 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.897 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.900 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.901 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.902 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.904 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.905 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.906 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.907 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.908 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.909 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.910 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.911 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.912 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.913 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.914 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.915 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.916 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "k8sevents", 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.917 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_event_retention_days": { 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ceph_event_retention_days", 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-03-12T05:34:11.918 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.919 INFO:teuthology.orchestra.run.smithi100.stdout: "config_check_secs": { 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_check_secs", 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "10", 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.920 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.921 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.922 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.924 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.926 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.927 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.928 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-03-12T05:34:11.930 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-12T05:34:11.931 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.932 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.934 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.935 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.936 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.937 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.938 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.939 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.940 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.941 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.944 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-03-12T05:34:11.945 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-03-12T05:34:11.946 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.947 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.948 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.949 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.950 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.951 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.952 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.953 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.954 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.955 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.957 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.959 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.960 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.962 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.963 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.964 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.965 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.966 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.967 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.968 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.969 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.970 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.971 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.972 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.973 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.974 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.975 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.976 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.977 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.978 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.979 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.980 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.981 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.982 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.983 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.984 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.985 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.986 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.987 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.988 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.989 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:11.990 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.991 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.992 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.993 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.994 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:11.995 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:11.996 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:11.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:11.998 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-03-12T05:34:11.999 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.000 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.001 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.002 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.003 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.004 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.005 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.007 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.009 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.010 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.011 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.012 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.013 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.014 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.015 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-03-12T05:34:12.019 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-03-12T05:34:12.020 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.021 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.022 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.023 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.024 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.025 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.026 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.027 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.028 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.029 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.030 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.031 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.032 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-03-12T05:34:12.033 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.034 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:12.035 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.036 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.037 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.038 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-03-12T05:34:12.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-12T05:34:12.040 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.042 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.043 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.044 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.046 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.047 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.050 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-12T05:34:12.051 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.052 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.053 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.054 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-03-12T05:34:12.055 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.056 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.057 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-03-12T05:34:12.058 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.059 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.060 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.061 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.062 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.063 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.064 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.065 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.066 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.067 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.068 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.069 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-03-12T05:34:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-03-12T05:34:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-03-12T05:34:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-03-12T05:34:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.116 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.117 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.118 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.119 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.120 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.121 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.123 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.124 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.125 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.126 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.128 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-03-12T05:34:12.129 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.130 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.131 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.133 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-03-12T05:34:12.134 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.135 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.136 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-03-12T05:34:12.138 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-03-12T05:34:12.139 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.140 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.140 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.140 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.140 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.140 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.141 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.141 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.141 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.141 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-03-12T05:34:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-03-12T05:34:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.143 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.143 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.143 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.143 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.143 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.144 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.144 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.144 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.144 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.145 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.145 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-03-12T05:34:12.145 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-03-12T05:34:12.145 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.145 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.146 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.146 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.146 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.146 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.147 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.147 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.147 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.147 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.147 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.148 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.148 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.149 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-03-12T05:34:12.149 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.149 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.149 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.149 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-03-12T05:34:12.150 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-03-12T05:34:12.150 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.150 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.150 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.151 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.151 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.151 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.151 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.151 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-03-12T05:34:12.152 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.152 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.152 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.152 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-03-12T05:34:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-03-12T05:34:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.154 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.155 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.156 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.157 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.159 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.160 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.161 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.162 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.163 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.164 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.165 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.166 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.167 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.169 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.170 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.171 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.172 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.173 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.174 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.175 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.177 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.178 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.179 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.180 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.182 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.184 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.185 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.186 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.187 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.188 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.190 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.191 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.192 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.193 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.194 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.195 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.196 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.198 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.200 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-03-12T05:34:12.201 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.202 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.203 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.204 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.205 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.206 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.207 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.208 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.209 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.210 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.211 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-03-12T05:34:12.212 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-03-12T05:34:12.213 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.214 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.215 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.217 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.218 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.219 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.220 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.221 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.222 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.224 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.225 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.226 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.227 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.228 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.229 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.231 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.232 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.233 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.234 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.235 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.236 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.237 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.238 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.239 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.240 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.241 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.242 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.243 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.244 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.245 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.246 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.247 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.248 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-03-12T05:34:12.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-12T05:34:12.250 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.251 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.252 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.253 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-03-12T05:34:12.255 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.256 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.257 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.259 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-03-12T05:34:12.260 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.261 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.262 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.264 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.265 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.266 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.267 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.268 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-03-12T05:34:12.269 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.270 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.271 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-03-12T05:34:12.272 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-03-12T05:34:12.273 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: "services": { 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: "dashboard": "https://172.21.15.100:8443/" 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.274 INFO:teuthology.orchestra.run.smithi100.stdout: "always_on_modules": { 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "octopus": [ 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-03-12T05:34:12.275 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "pacific": [ 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-03-12T05:34:12.276 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "quincy": [ 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-03-12T05:34:12.277 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-03-12T05:34:12.278 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "reef": [ 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-03-12T05:34:12.279 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "squid": [ 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-03-12T05:34:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-03-12T05:34:12.281 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: "last_failure_osd_epoch": 89, 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: "active_clients": [ 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.282 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3202065849 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "libcephsqlite", 2024-03-12T05:34:12.283 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 395427041 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1794715027 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 342666172 2024-03-12T05:34:12.286 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:12.287 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 602245019 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-03-12T05:34:12.288 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2936939018 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-12T05:34:12.289 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:34:12.291 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:10 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3044146878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:34:12.291 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:10 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3633188243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:10 smithi161 ceph-mon[36043]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:10 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2066962883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:11 smithi161 ceph-mon[36043]: osdmap e120: 12 total, 12 up, 12 in 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:10 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3044146878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:10 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3633188243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:10 smithi156 ceph-mon[36270]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:10 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2066962883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-12T05:34:12.292 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:11 smithi156 ceph-mon[36270]: osdmap e120: 12 total, 12 up, 12 in 2024-03-12T05:34:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4480a1ba0 msgr2=0x7ff4480a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4480a1ba0 0x7ff4480a4060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff450005d00 tx=0x7ff45000b410 comp rx=0 tx=0).stop 2024-03-12T05:34:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460100410 msgr2=0x7ff46019d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460100410 0x7ff46019d950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e9f0 tx=0x7ff45400eeb0 comp rx=0 tx=0).stop 2024-03-12T05:34:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 shutdown_connections 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7ff4480a1ba0 0x7ff4480a4060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.093+0000 7ff465b77640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff460107390 0x7ff460197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.094+0000 7ff465b77640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460100410 0x7ff46019d950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.094+0000 7ff465b77640 1 --2- 172.21.15.100:0/2066962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4600ffa50 0x7ff46019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.094+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 >> 172.21.15.100:0/2066962883 conn(0x7ff4600753d0 msgr2=0x7ff4600fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.094+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 shutdown_connections 2024-03-12T05:34:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:10.094+0000 7ff465b77640 1 -- 172.21.15.100:0/2066962883 wait complete. 2024-03-12T05:34:12.300 DEBUG:tasks.check_counter:Getting stats from a 2024-03-12T05:34:12.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-12T05:34:12.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:10 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3044146878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:34:12.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:10 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3633188243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:34:12.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:10 smithi100 ceph-mon[32210]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 242 op/s 2024-03-12T05:34:12.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:10 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2066962883' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-12T05:34:12.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:11 smithi100 ceph-mon[32210]: osdmap e120: 12 total, 12 up, 12 in 2024-03-12T05:34:12.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:12 smithi156 ceph-mon[36270]: pgmap v697: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 231 op/s 2024-03-12T05:34:12.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:12 smithi161 ceph-mon[36043]: pgmap v697: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 231 op/s 2024-03-12T05:34:12.538 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:34:12.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:12 smithi100 ceph-mon[32210]: pgmap v697: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 231 op/s 2024-03-12T05:34:14.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:14 smithi100 ceph-mon[32210]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 4.3 MiB/s wr, 146 op/s 2024-03-12T05:34:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 0, 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2, 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 0, 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 0, 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.001185147, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000000000, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000234116, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 7, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 301, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 3504, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 285536, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.018664610, 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.013181546, 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.002378200, 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 299, 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.015536570, 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000051961 2024-03-12T05:34:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 285536 2024-03-12T05:34:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 400, 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 389, 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1614928, 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 234864, 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-03-12T05:34:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.058896940, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.020662043, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.038371230, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.006824751, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 388, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014028986, 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000036157 2024-03-12T05:34:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1614928, 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 234864 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148714, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 30, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 403, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1558459, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 403, 2024-03-12T05:34:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 403, 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1558459, 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:14.335 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:34:14.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:14 smithi156 ceph-mon[36270]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 4.3 MiB/s wr, 146 op/s 2024-03-12T05:34:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:14 smithi161 ceph-mon[36043]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 4.3 MiB/s wr, 146 op/s 2024-03-12T05:34:15.023 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-12T05:34:15.023 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-12T05:34:15.023 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-12T05:34:15.023 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-12T05:34:15.023 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-12T05:34:15.023 DEBUG:tasks.check_counter:Getting stats from d 2024-03-12T05:34:15.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-12T05:34:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:34:17.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:16 smithi100 ceph-mon[32210]: osdmap e121: 12 total, 12 up, 12 in 2024-03-12T05:34:17.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:16 smithi100 ceph-mon[32210]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 2.7 MiB/s wr, 95 op/s 2024-03-12T05:34:17.197 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:34:17.197 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 0, 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2, 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 0, 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 0, 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-03-12T05:34:17.198 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.001059137, 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000000000, 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000115708, 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:17.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:17.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 411, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 398, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1619952, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 239360, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-03-12T05:34:17.201 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.060937641, 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.020487682, 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.039976617, 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.007125886, 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 398, 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014142657, 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000035534 2024-03-12T05:34:17.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1619952, 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 239360 2024-03-12T05:34:17.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 304, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 480, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 285968, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.019187545, 2024-03-12T05:34:17.204 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.014334784, 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.001852665, 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 301, 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.018996705, 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000063111 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:17.205 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 285968 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-12T05:34:17.206 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:17.207 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:17.208 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:17.209 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:17.210 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148670, 2024-03-12T05:34:17.211 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 30, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:17.212 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:17.213 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-12T05:34:17.214 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 410, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1559472, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 410, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 410, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1559472, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:17.215 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:17.216 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:17.216 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:17.216 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:17.216 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:34:17.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:16 smithi156 ceph-mon[36270]: osdmap e121: 12 total, 12 up, 12 in 2024-03-12T05:34:17.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:16 smithi156 ceph-mon[36270]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 2.7 MiB/s wr, 95 op/s 2024-03-12T05:34:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:16 smithi161 ceph-mon[36043]: osdmap e121: 12 total, 12 up, 12 in 2024-03-12T05:34:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:16 smithi161 ceph-mon[36043]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 2.7 MiB/s wr, 95 op/s 2024-03-12T05:34:17.813 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-12T05:34:17.813 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-12T05:34:17.813 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-12T05:34:17.813 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-12T05:34:17.813 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-12T05:34:17.813 DEBUG:tasks.check_counter:Getting stats from g 2024-03-12T05:34:17.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-12T05:34:18.056 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:34:18.414 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:18 smithi100 ceph-mon[32210]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 5.4 MiB/s wr, 187 op/s 2024-03-12T05:34:18.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:18 smithi156 ceph-mon[36270]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 5.4 MiB/s wr, 187 op/s 2024-03-12T05:34:18.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:18 smithi161 ceph-mon[36043]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 5.4 MiB/s wr, 187 op/s 2024-03-12T05:34:19.710 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 4, 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 4, 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 3024, 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 448, 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.001448802, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000040796, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000487513, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000763129, 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000254376 2024-03-12T05:34:19.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:19.713 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 402, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 391, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1615472, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 236112, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.056434304, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.019704237, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.036871939, 2024-03-12T05:34:19.714 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.006896210, 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 390, 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.012789087, 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000032792 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:19.715 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1615472, 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 236112 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3, 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 300, 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 480, 2024-03-12T05:34:19.716 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 285264, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.018652674, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.013817068, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.001903429, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 297, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.018883511, 2024-03-12T05:34:19.717 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000063580 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 285264 2024-03-12T05:34:19.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:19.719 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:19.720 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:19.721 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-12T05:34:19.722 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148650, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 28, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-12T05:34:19.723 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:19.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-12T05:34:19.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 405, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1558719, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:19.726 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 405, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 405, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1558719, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:19.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:19.728 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:19.728 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:34:20.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:20 smithi100 ceph-mon[32210]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 4.3 MiB/s wr, 151 op/s 2024-03-12T05:34:20.360 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-12T05:34:20.361 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-12T05:34:20.361 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-12T05:34:20.361 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-12T05:34:20.361 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-12T05:34:20.361 DEBUG:tasks.check_counter:Getting stats from j 2024-03-12T05:34:20.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-12T05:34:20.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:20 smithi156 ceph-mon[36270]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 4.3 MiB/s wr, 151 op/s 2024-03-12T05:34:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:20 smithi161 ceph-mon[36043]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 4.3 MiB/s wr, 151 op/s 2024-03-12T05:34:20.601 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.a/config 2024-03-12T05:34:22.105 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:22 smithi161 ceph-mon[36043]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 5.1 MiB/s wr, 141 op/s 2024-03-12T05:34:22.144 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:22 smithi100 ceph-mon[32210]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 5.1 MiB/s wr, 141 op/s 2024-03-12T05:34:22.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:22 smithi156 ceph-mon[36270]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 5.1 MiB/s wr, 141 op/s 2024-03-12T05:34:22.545 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-12T05:34:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 0, 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2, 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 0, 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 0, 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-03-12T05:34:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.001234832, 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000000000, 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000391609, 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 305, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 480, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 286144, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.021127077, 2024-03-12T05:34:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.015433272, 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.002620194, 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 302, 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.013964005, 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000046238 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 286144 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 413, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 400, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1620496, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 240608, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.058940749, 2024-03-12T05:34:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.020712139, 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.037762245, 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.006996398, 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 400, 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.013876247, 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000034690 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.553 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1620496, 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 240608 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.554 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:22.555 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:22.556 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:22.557 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:22.558 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148754, 2024-03-12T05:34:22.559 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 31, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:22.560 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-03-12T05:34:22.561 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-12T05:34:22.562 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-12T05:34:22.563 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-12T05:34:22.563 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 412, 2024-03-12T05:34:22.563 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1559732, 2024-03-12T05:34:22.563 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 412, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 412, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1559732, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-12T05:34:22.564 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-12T05:34:22.565 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-12T05:34:22.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:22.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-12T05:34:22.565 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-12T05:34:23.411 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-12T05:34:23.411 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-12T05:34:23.411 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-12T05:34:23.411 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-12T05:34:23.411 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-12T05:34:23.412 DEBUG:tasks.check_counter:Getting stats from b 2024-03-12T05:34:23.412 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-12T05:34:24.155 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:34:24.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:24 smithi156 ceph-mon[36270]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 5.7 MiB/s wr, 131 op/s 2024-03-12T05:34:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:24 smithi161 ceph-mon[36043]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 5.7 MiB/s wr, 131 op/s 2024-03-12T05:34:24.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:24 smithi100 ceph-mon[32210]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 5.7 MiB/s wr, 131 op/s 2024-03-12T05:34:26.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:26 smithi156 ceph-mon[36270]: pgmap v705: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 5.7 MiB/s wr, 130 op/s 2024-03-12T05:34:26.292 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-12T05:34:26.292 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:26.292 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 93692, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 52381, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 47150208, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 14464131, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 10, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 3.906507046, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 1.454855037, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 3.157879077, 2024-03-12T05:34:26.293 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.062600310, 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 52382, 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.329866468, 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000025387 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 141916, 2024-03-12T05:34:26.294 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.018324882, 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000129 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1723952, 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 878912 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 74838, 2024-03-12T05:34:26.295 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 86597, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 20222190, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 33509207, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 328, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 10, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 3.533387288, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 1.548738657, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 2.326483647, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.166504884, 2024-03-12T05:34:26.296 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 86595, 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.862342800, 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000021506 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 84650, 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.026146800, 2024-03-12T05:34:26.297 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000308 2024-03-12T05:34:26.298 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 779008, 2024-03-12T05:34:26.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 390144 2024-03-12T05:34:26.298 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.298 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:26.298 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 442697, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 223033, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 210970051, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 49994272, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 25, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 18.396846912, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 8.160135480, 2024-03-12T05:34:26.299 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 14.086142562, 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.118174613, 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 223029, 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.232454738, 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000014493 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 658640, 2024-03-12T05:34:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.073291048, 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000111 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-12T05:34:26.301 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5863, 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.590567640, 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000612411 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.302 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3250, 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.180950706, 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000055677 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.303 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 59864, 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 7201, 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7201, 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 32.252665305, 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004478914 2024-03-12T05:34:26.304 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.305 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-12T05:34:26.305 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 52663, 2024-03-12T05:34:26.305 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-03-12T05:34:26.305 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 12, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 126, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 16, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 89, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 21, 2024-03-12T05:34:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 102, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 165, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 23, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 24, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 8, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 68390, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 8724, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 52663, 2024-03-12T05:34:26.307 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 19, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 381, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 9, 2024-03-12T05:34:26.308 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 159, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 1, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 11, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 760, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 729, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 191556457, 2024-03-12T05:34:26.309 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 760, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 403, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 41, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 362, 2024-03-12T05:34:26.310 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 357, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 3, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 7081, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 12210, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 8283, 2024-03-12T05:34:26.311 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 709, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 551, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 6517 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 47, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 31, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 43, 2024-03-12T05:34:26.312 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 18, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 4, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 1306, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 1302, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-12T05:34:26.313 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 41, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 10, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-12T05:34:26.314 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-03-12T05:34:26.315 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-12T05:34:26.315 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-12T05:34:26.315 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.315 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-12T05:34:26.315 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-12T05:34:26.315 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 7624, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 7624, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 11, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-03-12T05:34:26.316 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 11, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 25227548, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-03-12T05:34:26.317 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2480, 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 10.287243624, 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004148082 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-12T05:34:26.318 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 118, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 1545, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 1427, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 179, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 248, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 69, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 126, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 1857, 2024-03-12T05:34:26.319 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 1731, 2024-03-12T05:34:26.320 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 24, 2024-03-12T05:34:26.320 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 1431, 2024-03-12T05:34:26.320 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 1407, 2024-03-12T05:34:26.320 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 67856, 2024-03-12T05:34:26.320 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 215420 2024-03-12T05:34:26.320 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 65532, 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 12, 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 59864, 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 108, 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 1482, 2024-03-12T05:34:26.321 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1390, 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 8.593991776, 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006182727 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 231, 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.053560121, 2024-03-12T05:34:26.322 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000231861 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.323 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1765, 2024-03-12T05:34:26.324 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 5.771329369, 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003269874 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-12T05:34:26.325 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000206501, 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000103250 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.326 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-12T05:34:26.327 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.328 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.328 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.328 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.328 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-12T05:34:26.328 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 22, 2024-03-12T05:34:26.329 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.005027746, 2024-03-12T05:34:26.329 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000228533 2024-03-12T05:34:26.329 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.329 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-12T05:34:26.329 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 20, 2024-03-12T05:34:26.329 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.020323632, 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001016181 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.330 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 22, 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 5.857894614, 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.266267937 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.331 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1964, 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.477143427, 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001261274 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 441, 2024-03-12T05:34:26.332 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.295444918, 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.007472664 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.333 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-12T05:34:26.334 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.010205528, 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.010205528 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 16, 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.436266723, 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.214766670 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.335 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001025228, 2024-03-12T05:34:26.336 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001025228 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-12T05:34:26.337 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 12, 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.003740129, 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000311677 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.338 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8, 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.023886388, 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002985798 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-12T05:34:26.339 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.341 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.341 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.341 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1306, 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.702619205, 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002069386 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-12T05:34:26.342 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.343 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 5, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 3, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 775, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 387, 2024-03-12T05:34:26.344 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 1319, 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:26.345 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:26.346 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:26.347 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:26.348 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:26.349 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 311685, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 208, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-12T05:34:26.350 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8352, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 190, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:26.351 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 775142, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 6181, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.352 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 6531, 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21364068, 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 31, 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 6531, 2024-03-12T05:34:26.353 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6531, 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 39.656418857, 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006072028 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-12T05:34:26.354 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6531, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7037 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 6531, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 1202, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 5329, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-12T05:34:26.355 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 11, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 401, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 2840, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 279, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-12T05:34:26.356 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 1819, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 801, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 22, 2024-03-12T05:34:26.357 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T05:34:26.358 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-12T05:34:26.359 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-12T05:34:26.360 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 121, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 36, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-03-12T05:34:26.361 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 677, 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 187 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.362 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-03-12T05:34:26.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 10, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 10, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 1302, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 1302, 2024-03-12T05:34:26.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 611223, 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 224346950, 2024-03-12T05:34:26.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 611223, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 611223, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 224346950, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6531, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 21495698, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6531, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 21495698, 2024-03-12T05:34:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6531, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 6531, 2024-03-12T05:34:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6531, 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 6531, 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.371 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 1302, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 131630, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 1302, 2024-03-12T05:34:26.372 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 360, 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 131630, 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.373 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55b2a79f66e0": { 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 574, 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 7624, 2024-03-12T05:34:26.374 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 21033244, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 7624, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2480, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 21032670, 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:26.375 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:26.376 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:26.376 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:26.376 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.376 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:26.376 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-12T05:34:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:26 smithi161 ceph-mon[36043]: pgmap v705: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 5.7 MiB/s wr, 130 op/s 2024-03-12T05:34:26.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:26 smithi100 ceph-mon[32210]: pgmap v705: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 5.7 MiB/s wr, 130 op/s 2024-03-12T05:34:27.400 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-03-12T05:34:27.400 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-03-12T05:34:27.400 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-12T05:34:27.401 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=22 2024-03-12T05:34:27.401 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=16 2024-03-12T05:34:27.401 DEBUG:tasks.check_counter:Getting stats from e 2024-03-12T05:34:27.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-12T05:34:27.656 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:34:28.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:28 smithi100 ceph-mon[32210]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 174 op/s 2024-03-12T05:34:28.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:28 smithi156 ceph-mon[36270]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 174 op/s 2024-03-12T05:34:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:28 smithi161 ceph-mon[36043]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 174 op/s 2024-03-12T05:34:29.567 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-12T05:34:29.567 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2120, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2099, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 635493181, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 763555, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.490366405, 2024-03-12T05:34:29.568 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.050819283, 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.379074650, 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.058966422, 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2099, 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.046849188, 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000022319 2024-03-12T05:34:29.569 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1689120, 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 247600 2024-03-12T05:34:29.570 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 3526, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 3828, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1376651176, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1791281, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 9, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 1.009531088, 2024-03-12T05:34:29.571 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.063821924, 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.829030855, 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.092627525, 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3825, 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.096934547, 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000025342 2024-03-12T05:34:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 695312 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.573 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 4450, 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 4452, 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1133751390, 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1384008, 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2024-03-12T05:34:29.574 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.818664489, 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.089519616, 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.601529504, 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.101614539, 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4450, 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.121164778, 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000027228 2024-03-12T05:34:29.575 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:29.576 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13547, 2024-03-12T05:34:29.577 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.526389047, 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000038856 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.578 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.579 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2024-03-12T05:34:29.580 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 108752, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 66444, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 42308, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 13832, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-03-12T05:34:29.581 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 16, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-12T05:34:29.582 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-03-12T05:34:29.583 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-12T05:34:29.584 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T05:34:29.585 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2024-03-12T05:34:29.586 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-03-12T05:34:29.587 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-12T05:34:29.588 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-03-12T05:34:29.589 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 486114, 2024-03-12T05:34:29.590 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 133, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 41, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-12T05:34:29.591 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 2610310347, 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 3145714603, 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 3145699271, 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.592 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 486114 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 4808, 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 108592, 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 103784, 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 259, 2024-03-12T05:34:29.593 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 664, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 405, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 108804, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 109024, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 220, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2024-03-12T05:34:29.594 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 158128, 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 215420 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2024-03-12T05:34:29.595 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.596 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-12T05:34:29.597 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.598 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-12T05:34:29.599 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.600 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.601 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.602 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.603 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-12T05:34:29.603 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-12T05:34:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.605 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.606 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.607 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.608 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.609 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-12T05:34:29.610 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.611 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.612 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.613 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.613 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-12T05:34:29.613 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.613 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.613 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.614 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.614 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-12T05:34:29.614 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.614 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.614 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.617 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.617 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-12T05:34:29.617 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.618 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 5, 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 3, 2024-03-12T05:34:29.619 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 3, 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 3567, 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-12T05:34:29.620 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:29.621 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:29.622 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:29.623 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:29.624 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 454801, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 5900, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:29.625 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:29.626 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8352, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 190, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 88984304, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 599289, 2024-03-12T05:34:29.627 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-12T05:34:29.628 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 9633, 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 9633, 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 9633, 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 14.348792876, 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001489545 2024-03-12T05:34:29.629 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 9633, 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 9634 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 9633, 2024-03-12T05:34:29.630 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 9633, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 6320, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 3311, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2024-03-12T05:34:29.631 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-12T05:34:29.632 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-12T05:34:29.633 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-12T05:34:29.634 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-12T05:34:29.635 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 121, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 35, 2024-03-12T05:34:29.636 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 11, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 11, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-12T05:34:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.638 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 0, 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2024-03-12T05:34:29.639 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 10092, 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 3144961142, 2024-03-12T05:34:29.640 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 10092, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 10092, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 3144961142, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.641 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:29.642 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 9633, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 3141613517, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 9633, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 3141613517, 2024-03-12T05:34:29.643 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:29.644 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 9633, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 9633, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 9633, 2024-03-12T05:34:29.645 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 9633, 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.646 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:29.647 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.648 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x564241082960": { 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:29.649 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:29.650 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:29.651 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:29.651 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:29.651 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.651 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:29.651 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-12T05:34:30.358 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:30 smithi156 ceph-mon[36270]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.8 MiB/s wr, 112 op/s 2024-03-12T05:34:30.385 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-12T05:34:30.385 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-12T05:34:30.385 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-12T05:34:30.385 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-12T05:34:30.386 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T05:34:30.386 DEBUG:tasks.check_counter:Getting stats from h 2024-03-12T05:34:30.386 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-12T05:34:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:30 smithi161 ceph-mon[36043]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.8 MiB/s wr, 112 op/s 2024-03-12T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:30 smithi100 ceph-mon[32210]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.8 MiB/s wr, 112 op/s 2024-03-12T05:34:30.628 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:34:32.278 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:32 smithi156 ceph-mon[36270]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 5.4 MiB/s wr, 132 op/s 2024-03-12T05:34:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:32 smithi161 ceph-mon[36043]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 5.4 MiB/s wr, 132 op/s 2024-03-12T05:34:32.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:32 smithi100 ceph-mon[32210]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 5.4 MiB/s wr, 132 op/s 2024-03-12T05:34:32.601 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-12T05:34:32.601 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:32.601 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 379996, 2024-03-12T05:34:32.601 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 476286, 2024-03-12T05:34:32.601 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 110470467, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 842558179, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 15, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 23.183360363, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 9.695158185, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 12.601532812, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 2.247076504, 2024-03-12T05:34:32.602 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 476285, 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 17.173939317, 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000036058 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 736027, 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.115352160, 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000156 2024-03-12T05:34:32.603 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1710448, 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 854480 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 122941, 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 123049, 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 31714258, 2024-03-12T05:34:32.604 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1518013452, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 89, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 8, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 11.214724474, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 4.491269869, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 2.006165623, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 3.997994115, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 123047, 2024-03-12T05:34:32.605 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 6.473490914, 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000052609 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 198, 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000028495, 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000143 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 9632, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 159392 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1317961, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2046924, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 357811727, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 2459701550, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 12, 2024-03-12T05:34:32.607 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 71.034385971, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 37.056342880, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 40.229110293, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 3.897108235, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2046922, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 69.210425002, 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000033811 2024-03-12T05:34:32.608 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2045801, 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.347257657, 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000169 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:32.609 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 278575, 2024-03-12T05:34:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 358.201127335, 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001285833 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 166067, 2024-03-12T05:34:32.611 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 48.760282833, 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000293618 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 740547, 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 724448, 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 724448, 2024-03-12T05:34:32.612 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3813.696666012, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.005264279 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 16099, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 1056, 2024-03-12T05:34:32.613 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 58, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 62, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 11304, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 6157, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 5145, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 2, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 2508, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 254, 2024-03-12T05:34:32.614 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 517, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 517, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 21, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 1081261, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 658463, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 16087, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 28, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-12T05:34:32.615 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 9273, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 23, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 222, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 10, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 164, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-12T05:34:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 10532, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 677, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 211510042, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 10532, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T05:34:32.617 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 1554, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 1554, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 8978, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 8, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 50461, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 393189, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T05:34:32.618 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 193592, 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 83257, 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 40067, 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 504314 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-12T05:34:32.619 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 15, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 12, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 8, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 32, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 71, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 56, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 103770, 2024-03-12T05:34:32.620 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 103714, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-12T05:34:32.621 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 23, 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 120, 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-12T05:34:32.622 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 486532, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 432836, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 430151, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 56381, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 2685, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 852, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 724, 2024-03-12T05:34:32.623 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 718, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 134, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 5, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 6, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 2610310347, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 3148711101, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-03-12T05:34:32.624 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 153547, 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 837.673364717, 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.005455485 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 4772, 2024-03-12T05:34:32.625 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 108699, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 103927, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 338, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 829, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 491, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 11243, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 213130, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 201887, 2024-03-12T05:34:32.626 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 454, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 111924, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 111470, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 97640, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207228 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 1075269, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 2954, 2024-03-12T05:34:32.627 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 740547, 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 14480, 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 6637, 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 104375, 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 484.185797054, 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004638905 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.628 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 20216, 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 199.775346487, 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.009882041 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.629 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-12T05:34:32.630 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 216601, 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 795.114406372, 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003670871 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.631 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-12T05:34:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.634 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.634 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.634 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.634 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 99, 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.687443904, 2024-03-12T05:34:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006943877 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-12T05:34:32.636 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 9, 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.012462510, 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001384723 2024-03-12T05:34:32.637 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 254090, 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1721.572835306, 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006775445 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 21719, 2024-03-12T05:34:32.638 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 102.663696753, 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004726907 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.640 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-12T05:34:32.641 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 185, 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4.261331680, 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.023034225 2024-03-12T05:34:32.642 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3384, 2024-03-12T05:34:32.643 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 12.086739773, 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003571731 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-12T05:34:32.644 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.645 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 103770, 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 493.336606173, 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004754135 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-12T05:34:32.646 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-12T05:34:32.647 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 5, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 3, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 55255, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 27627, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 1302, 2024-03-12T05:34:32.648 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:32.649 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:32.650 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:32.652 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:32.653 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 2828469, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 2787, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 88, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 1, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:32.654 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8352, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 190, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:32.655 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 20847064, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 225646, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-12T05:34:32.656 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 278010, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 3174609208, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 27, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 278010, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 278010, 2024-03-12T05:34:32.657 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1236.226242421, 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004446697 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 278010, 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 283360 2024-03-12T05:34:32.658 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 278010, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 12829, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 265181, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 440, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 537, 2024-03-12T05:34:32.659 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 2297, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 155844, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 1027, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 57, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 105419, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-12T05:34:32.660 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 10532, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 1074, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-12T05:34:32.661 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-12T05:34:32.662 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-12T05:34:32.663 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 121, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 35, 2024-03-12T05:34:32.664 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 4521, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 1612 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-12T05:34:32.665 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 6 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.666 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 10, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 10, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 103714, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 103714, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.667 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 1820894, 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 352046410, 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:32.668 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 1820894, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1820894, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 352046410, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.669 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:32.670 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 278010, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 3185098994, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 278010, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 3185098994, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.671 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:32.672 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 278010, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 278010, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 278010, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 278010, 2024-03-12T05:34:32.673 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-12T05:34:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 103714, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 10489786, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 103714, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2297, 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 10489786, 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:32.676 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x562bbb9986e0": { 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 2155140, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 486532, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 3144516797, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 486532, 2024-03-12T05:34:32.677 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 153547, 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 3142361657, 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:32.678 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:32.679 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.679 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:32.679 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-12T05:34:33.986 INFO:tasks.check_counter:Daemon mds.h mds.exported=23 2024-03-12T05:34:33.987 INFO:tasks.check_counter:Daemon mds.h mds.imported=10 2024-03-12T05:34:33.987 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-12T05:34:33.987 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-12T05:34:33.987 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T05:34:33.987 DEBUG:tasks.check_counter:Getting stats from k 2024-03-12T05:34:33.987 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-12T05:34:34.276 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.b/config 2024-03-12T05:34:34.397 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:34 smithi156 ceph-mon[36270]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 4.4 MiB/s wr, 126 op/s 2024-03-12T05:34:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:34 smithi161 ceph-mon[36043]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 4.4 MiB/s wr, 126 op/s 2024-03-12T05:34:34.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:34 smithi100 ceph-mon[32210]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 4.4 MiB/s wr, 126 op/s 2024-03-12T05:34:35.995 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-12T05:34:35.996 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:35.996 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 0, 2024-03-12T05:34:35.996 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2, 2024-03-12T05:34:35.996 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 0, 2024-03-12T05:34:35.996 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 0, 2024-03-12T05:34:35.996 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001089167, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000000000, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000288960, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:35.997 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:35.998 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 7, 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 310, 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 3504, 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 287120, 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:35.999 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.015426732, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.010498948, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.002139091, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 308, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.018153298, 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000058939 2024-03-12T05:34:36.000 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-12T05:34:36.001 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 287120 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 413, 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 402, 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1618560, 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 242656, 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2024-03-12T05:34:36.002 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.057124806, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.018256489, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.039158832, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.007983820, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 401, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.014202739, 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000035418 2024-03-12T05:34:36.003 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1618560, 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 242656 2024-03-12T05:34:36.004 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:36.005 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:36.006 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:36.007 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:36.008 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:36.009 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148744, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 30, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:36.010 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-12T05:34:36.011 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:36.012 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 416, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1560252, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 416, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-12T05:34:36.013 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 416, 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1560252, 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-12T05:34:36.014 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-12T05:34:36.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:36 smithi156 ceph-mon[36270]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 3.2 MiB/s wr, 105 op/s 2024-03-12T05:34:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:36 smithi161 ceph-mon[36043]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 3.2 MiB/s wr, 105 op/s 2024-03-12T05:34:36.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:36 smithi100 ceph-mon[32210]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 3.2 MiB/s wr, 105 op/s 2024-03-12T05:34:36.698 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-12T05:34:36.698 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-12T05:34:36.698 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-12T05:34:36.698 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-12T05:34:36.698 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-12T05:34:36.698 DEBUG:tasks.check_counter:Getting stats from c 2024-03-12T05:34:36.699 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-12T05:34:37.049 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:34:38.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:38 smithi156 ceph-mon[36270]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.8 MiB/s wr, 129 op/s 2024-03-12T05:34:38.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:38 smithi161 ceph-mon[36043]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.8 MiB/s wr, 129 op/s 2024-03-12T05:34:38.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:38 smithi100 ceph-mon[32210]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.8 MiB/s wr, 129 op/s 2024-03-12T05:34:38.905 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-12T05:34:38.905 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:38.905 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2449, 2024-03-12T05:34:38.905 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2451, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 9135095, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 761649, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.195469197, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.057298432, 2024-03-12T05:34:38.906 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.055263113, 2024-03-12T05:34:38.907 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.065450779, 2024-03-12T05:34:38.907 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:38.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2449, 2024-03-12T05:34:38.907 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.078448267, 2024-03-12T05:34:38.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000032032 2024-03-12T05:34:38.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:38.908 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:38.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.909 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:38.909 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1167, 2024-03-12T05:34:38.909 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1468, 2024-03-12T05:34:38.909 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 8683414, 2024-03-12T05:34:38.909 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1060273, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.097581996, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.032730858, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.026150791, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.027467758, 2024-03-12T05:34:38.910 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1466, 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.055769968, 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000038042 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.911 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 699136 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1737, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1718, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 6378323, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 645395, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.156112071, 2024-03-12T05:34:38.913 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.054166190, 2024-03-12T05:34:38.914 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.063437245, 2024-03-12T05:34:38.914 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.038205256, 2024-03-12T05:34:38.914 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:38.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1717, 2024-03-12T05:34:38.914 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.051724650, 2024-03-12T05:34:38.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000030125 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1686528, 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 247936 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-03-12T05:34:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8213, 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.186280763, 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000022681 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.918 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-03-12T05:34:38.920 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 1383, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1383, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 1383, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-03-12T05:34:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 4, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-03-12T05:34:38.922 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-03-12T05:34:38.923 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 729, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 212041823, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-03-12T05:34:38.924 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-03-12T05:34:38.925 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-03-12T05:34:38.926 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-03-12T05:34:38.927 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-03-12T05:34:38.928 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-03-12T05:34:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.930 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 7634, 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-03-12T05:34:38.931 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 11, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-03-12T05:34:38.932 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 25340376, 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 25334031, 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 7634 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 73, 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 1470, 2024-03-12T05:34:38.934 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 1397, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 23, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 34, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 11, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 1383, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 1530, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 147, 2024-03-12T05:34:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 54212, 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215420 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-03-12T05:34:38.936 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.937 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.939 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.941 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-03-12T05:34:38.943 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.945 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-03-12T05:34:38.945 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.945 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.946 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.946 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-03-12T05:34:38.946 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.946 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.946 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-03-12T05:34:38.947 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.949 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-03-12T05:34:38.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-03-12T05:34:38.953 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.954 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.955 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.956 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.957 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-03-12T05:34:38.957 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.957 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.957 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 5, 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 3, 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-03-12T05:34:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 5740, 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:38.961 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:38.962 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:38.963 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:38.964 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 203366, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 175, 2024-03-12T05:34:38.965 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:38.966 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8352, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 190, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-12T05:34:38.967 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 1318349, 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 8399, 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.968 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 4888, 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 26, 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 4, 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 4888, 2024-03-12T05:34:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4888, 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.627900901, 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000946788 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4888, 2024-03-12T05:34:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4889 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 4888, 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 4888, 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-03-12T05:34:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 2928, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1957, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-03-12T05:34:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 2, 2024-03-12T05:34:38.973 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-03-12T05:34:38.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T05:34:38.975 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-03-12T05:34:38.975 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-03-12T05:34:38.975 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-03-12T05:34:38.975 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-03-12T05:34:38.977 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-03-12T05:34:38.977 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-03-12T05:34:38.978 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 121, 2024-03-12T05:34:38.979 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 35, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 6, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 7, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-03-12T05:34:38.980 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-03-12T05:34:38.981 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-03-12T05:34:38.982 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.983 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 5349, 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 23688999, 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:38.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 5349, 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5349, 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 23688999, 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.985 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:38.986 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 4888, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 21146082, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4888, 2024-03-12T05:34:38.987 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 21146082, 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.988 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:38.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 4888, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 4888, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4888, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 4888, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.990 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:38.991 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-12T05:34:38.992 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:38.993 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.993 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.993 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:38.993 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5604d4b34960": { 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:38.994 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:38.995 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:38.996 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:38.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:38.996 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-12T05:34:39.830 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-12T05:34:39.831 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-12T05:34:39.831 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-12T05:34:39.831 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-12T05:34:39.831 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T05:34:39.831 DEBUG:tasks.check_counter:Getting stats from f 2024-03-12T05:34:39.831 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-12T05:34:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:34:40.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:40 smithi156 ceph-mon[36270]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-03-12T05:34:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:40 smithi161 ceph-mon[36043]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-03-12T05:34:40.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:40 smithi100 ceph-mon[32210]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-03-12T05:34:42.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:42 smithi161 ceph-mon[36043]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-03-12T05:34:42.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:42 smithi156 ceph-mon[36270]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-03-12T05:34:42.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:42 smithi100 ceph-mon[32210]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-03-12T05:34:42.573 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4267, 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4269, 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 336359849, 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1327044, 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-03-12T05:34:42.574 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.430252646, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.090208729, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.216001069, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.101534702, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4267, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.108627517, 2024-03-12T05:34:42.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000025457 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:42.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 733, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1038, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 243458178, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 928043, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 9, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.193764089, 2024-03-12T05:34:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.021350833, 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.149171943, 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.016156584, 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1035, 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.039216660, 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000037890 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.578 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 700640 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.579 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1491, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1469, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 341711066, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 568407, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.294675958, 2024-03-12T05:34:42.580 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.040206717, 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.226163172, 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.030592067, 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1469, 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.039236055, 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000026709 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:42.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1690272, 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 249632 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-12T05:34:42.582 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9421, 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.262524575, 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000027865 2024-03-12T05:34:42.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.585 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 44237, 2024-03-12T05:34:42.586 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 17180, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 27057, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 19694, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 16, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-03-12T05:34:42.587 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-03-12T05:34:42.588 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-03-12T05:34:42.589 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T05:34:42.590 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-03-12T05:34:42.591 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-03-12T05:34:42.592 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-03-12T05:34:42.593 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-03-12T05:34:42.594 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 218023, 2024-03-12T05:34:42.595 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 132, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 19, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-03-12T05:34:42.596 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 408992428, 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 922355166, 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 922354567, 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 218023 2024-03-12T05:34:42.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 1952, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 44031, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 42079, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 142, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 281, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 139, 2024-03-12T05:34:42.598 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 44240, 2024-03-12T05:34:42.599 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 44245, 2024-03-12T05:34:42.599 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 5, 2024-03-12T05:34:42.599 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-03-12T05:34:42.599 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-03-12T05:34:42.599 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-03-12T05:34:42.599 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 95476, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215420 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-03-12T05:34:42.600 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.602 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.605 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-03-12T05:34:42.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.607 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.607 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.607 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-03-12T05:34:42.607 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.609 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.610 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-03-12T05:34:42.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.610 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.611 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.612 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.614 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-03-12T05:34:42.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-03-12T05:34:42.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.617 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 5, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 4, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-03-12T05:34:42.620 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 5763, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:42.621 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:42.622 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:42.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:42.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 323616, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 2373, 2024-03-12T05:34:42.625 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8352, 2024-03-12T05:34:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 190, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 36242633, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 236616, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:42.627 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 6023, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-03-12T05:34:42.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 6023, 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6023, 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 7.308844098, 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001213488 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-03-12T05:34:42.629 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6023, 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6024 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 6023, 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 6023, 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-03-12T05:34:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 3841, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 2180, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-03-12T05:34:42.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-03-12T05:34:42.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-03-12T05:34:42.633 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-03-12T05:34:42.634 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 121, 2024-03-12T05:34:42.635 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 35, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 11, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-03-12T05:34:42.636 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.637 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-03-12T05:34:42.638 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:42.639 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 6487, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 920918730, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 6487, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 6487, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 920918730, 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:42.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:42.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 6023, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 918178667, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 6023, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 918178667, 2024-03-12T05:34:42.642 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:42.643 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 6023, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 6023, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 6023, 2024-03-12T05:34:42.644 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 6023, 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:42.645 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-12T05:34:42.646 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x556f0f826960": { 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:42.647 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:42.648 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:42.649 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-12T05:34:43.530 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-12T05:34:43.530 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-12T05:34:43.530 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-12T05:34:43.531 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-12T05:34:43.531 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T05:34:43.531 DEBUG:tasks.check_counter:Getting stats from i 2024-03-12T05:34:43.531 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-12T05:34:43.773 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:34:44.146 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:44 smithi161 ceph-mon[36043]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.5 MiB/s wr, 55 op/s 2024-03-12T05:34:44.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:44 smithi156 ceph-mon[36270]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.5 MiB/s wr, 55 op/s 2024-03-12T05:34:44.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:44 smithi100 ceph-mon[32210]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.5 MiB/s wr, 55 op/s 2024-03-12T05:34:45.471 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 83204, 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 125049, 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 17348846, 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 419270748, 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 46, 2024-03-12T05:34:45.472 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 7.241362732, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 3.211996822, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 2.588530551, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 1.388129189, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 125047, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.810934941, 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000038473 2024-03-12T05:34:45.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 141403, 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.020093277, 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000142 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 571200 2024-03-12T05:34:45.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 474801, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 674100, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 137707851, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 777796908, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 62, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 22.877284265, 2024-03-12T05:34:45.475 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 11.147148899, 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 13.408093285, 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 1.802248711, 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 674098, 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 24.321456213, 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000036080 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.476 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 583623, 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.110470256, 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000189 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 8992, 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 137424 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:45.477 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 453625, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 357553, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 198265935, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 407482350, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 13, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 22.709728417, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 9.461067014, 2024-03-12T05:34:45.478 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 15.547550116, 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 1.269139665, 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 357552, 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 10.029019330, 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000028049 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:45.479 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 738319, 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.090993526, 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000123 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1705216, 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 268976 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-12T05:34:45.480 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 108897, 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 162.727101048, 2024-03-12T05:34:45.481 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001494321 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 71677, 2024-03-12T05:34:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 14.617980469, 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000203942 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 322063, 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 293524, 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 293524, 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1306.682277783, 2024-03-12T05:34:45.483 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004451705 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 28539, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 379, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 25, 2024-03-12T05:34:45.484 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 23, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 4855, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 2314, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 2522, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 19, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 1549, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 17, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 226, 2024-03-12T05:34:45.485 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 226, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 27, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 469828, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 269171, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 28535, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 38, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-03-12T05:34:45.486 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 3796, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 1, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 5, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 22, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 211, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-03-12T05:34:45.487 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 3917, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 1, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 677, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 211510042, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 3917, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-03-12T05:34:45.488 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 448, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 448, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 3469, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 4, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 12675, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 158374, 2024-03-12T05:34:45.489 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 77872, 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 36477, 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 17183, 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 202370 2024-03-12T05:34:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 15, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 32, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 22, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 52, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 38, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-03-12T05:34:45.491 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 42076, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 42076, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-03-12T05:34:45.492 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 1, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 48, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-03-12T05:34:45.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 218036, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 126391, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 122901, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 95135, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-03-12T05:34:45.494 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 3490, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 314, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 186, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 182, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 132, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 8, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 4, 2024-03-12T05:34:45.495 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 408992428, 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 922368549, 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 61506, 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 430.780695828, 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007003880 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.496 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 1973, 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 44060, 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 42087, 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 220, 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 418, 2024-03-12T05:34:45.497 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 198, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 4790, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 87436, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 82646, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 226, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 47594, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 47368, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 89056, 2024-03-12T05:34:45.498 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207228 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 451356, 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 2208, 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 322063, 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 6523, 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 7371, 2024-03-12T05:34:45.499 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42116, 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 196.673525028, 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004669805 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8163, 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 94.334451281, 2024-03-12T05:34:45.500 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.011556345 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.501 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 89636, 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 192.694820283, 2024-03-12T05:34:45.502 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002149748 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-03-12T05:34:45.503 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.504 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.505 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.506 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.506 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-03-12T05:34:45.506 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.506 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.506 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.507 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-03-12T05:34:45.508 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 101426, 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 580.393995432, 2024-03-12T05:34:45.509 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005722339 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8723, 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 45.438338031, 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005209026 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-03-12T05:34:45.510 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.512 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 32, 2024-03-12T05:34:45.513 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.380393500, 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.011887296 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.514 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1352, 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.539559120, 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001878372 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.516 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42076, 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 194.227195108, 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004616104 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-03-12T05:34:45.517 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-03-12T05:34:45.518 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 5, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 4, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 47014, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 47014, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 1353, 2024-03-12T05:34:45.519 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:45.520 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:45.521 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:45.522 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-12T05:34:45.523 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 953623, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 1102, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:45.524 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8352, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 190, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:45.525 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 8869030, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 88878, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-12T05:34:45.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 111668, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 931177448, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 28, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 111668, 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-03-12T05:34:45.527 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 111668, 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 700.882133097, 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006276481 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 111668, 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 113778 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.528 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 111668, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 4944, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 106724, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 114, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 220, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1027, 2024-03-12T05:34:45.529 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 62533, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 511, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 25, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 42518, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-03-12T05:34:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 3917, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 440, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-03-12T05:34:45.531 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-03-12T05:34:45.532 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-03-12T05:34:45.533 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 121, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 35, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 16, 2024-03-12T05:34:45.534 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 4, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 1812, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 661 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-03-12T05:34:45.535 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 1 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-03-12T05:34:45.536 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 10, 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 10, 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 42076, 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 42076, 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:45.537 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 1011626, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 269100022, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 1011626, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:45.538 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1011626, 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 269100022, 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:45.540 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 111668, 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 935432804, 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 111668, 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 935432804, 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.541 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-12T05:34:45.542 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 111668, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 111668, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 111668, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 111668, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.543 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-12T05:34:45.544 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 42076, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 4255356, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 42076, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1027, 2024-03-12T05:34:45.545 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 4255356, 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55e4d13186e0": { 2024-03-12T05:34:45.546 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 218036, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 918174245, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 218036, 2024-03-12T05:34:45.547 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 61506, 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 918174245, 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.548 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:45.549 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-12T05:34:46.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:46 smithi161 ceph-mon[36043]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 708 KiB/s wr, 35 op/s 2024-03-12T05:34:46.208 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-12T05:34:46.208 INFO:tasks.check_counter:Daemon mds.i mds.imported=22 2024-03-12T05:34:46.208 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-12T05:34:46.208 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-12T05:34:46.208 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-12T05:34:46.209 DEBUG:tasks.check_counter:Getting stats from l 2024-03-12T05:34:46.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-12T05:34:46.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:46 smithi156 ceph-mon[36270]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 708 KiB/s wr, 35 op/s 2024-03-12T05:34:46.452 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/mon.c/config 2024-03-12T05:34:46.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:46 smithi100 ceph-mon[32210]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 708 KiB/s wr, 35 op/s 2024-03-12T05:34:48.053 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-12T05:34:48.053 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001018620, 2024-03-12T05:34:48.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000202788, 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:48.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 422, 2024-03-12T05:34:48.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 409, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1623008, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 245760, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.064644357, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.021445911, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.042287202, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.008218100, 2024-03-12T05:34:48.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 409, 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.015258948, 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000037307 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:48.058 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1623008, 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 245760 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3, 2024-03-12T05:34:48.059 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 311, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 480, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 287200, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.015862415, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.011151544, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001725515, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-12T05:34:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 308, 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.013971827, 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000045363 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:48.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 287200 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-12T05:34:48.062 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-12T05:34:48.063 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-12T05:34:48.064 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-12T05:34:48.065 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-12T05:34:48.066 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148722, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-12T05:34:48.067 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-12T05:34:48.068 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-12T05:34:48.069 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 421, 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1560966, 2024-03-12T05:34:48.070 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 421, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 421, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1560966, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-12T05:34:48.071 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-12T05:34:48.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-12T05:34:48.072 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:48.072 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-12T05:34:48.072 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-12T05:34:48.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:48 smithi156 ceph-mon[36270]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 737 KiB/s wr, 40 op/s 2024-03-12T05:34:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:48 smithi161 ceph-mon[36043]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 737 KiB/s wr, 40 op/s 2024-03-12T05:34:48.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:48 smithi100 ceph-mon[32210]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 737 KiB/s wr, 40 op/s 2024-03-12T05:34:48.864 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-12T05:34:48.864 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-12T05:34:48.864 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-12T05:34:48.864 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-12T05:34:48.864 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-12T05:34:48.865 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-12T05:34:48.875 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-12T05:34:48.875 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:34:48.875 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T05:34:48.891 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-12T05:34:48.892 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T05:34:48.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:34:49.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 -- 172.21.15.100:0/2187632355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c1023f0 msgr2=0x7fe46c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 --2- 172.21.15.100:0/2187632355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c1023f0 0x7fe46c1027d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009900 tx=0x7fe45c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:49.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 -- 172.21.15.100:0/2187632355 shutdown_connections 2024-03-12T05:34:49.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 --2- 172.21.15.100:0/2187632355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe46c10b7c0 0x7fe46c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 --2- 172.21.15.100:0/2187632355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c102d10 0x7fe46c10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 --2- 172.21.15.100:0/2187632355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c1023f0 0x7fe46c1027d0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 -- 172.21.15.100:0/2187632355 >> 172.21.15.100:0/2187632355 conn(0x7fe46c077180 msgr2=0x7fe46c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 -- 172.21.15.100:0/2187632355 shutdown_connections 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.103+0000 7fe472165640 1 -- 172.21.15.100:0/2187632355 wait complete. 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 Processor -- start 2024-03-12T05:34:49.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 -- start start 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe46c1023f0 0x7fe46c100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 0x7fe46c100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c10b7c0 0x7fe46c100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1a5c30 con 0x7fe46c102d10 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1a5da0 con 0x7fe46c1023f0 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe472165640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1a5f60 con 0x7fe46c10b7c0 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.104+0000 7fe46affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 0x7fe46c100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 0x7fe46c100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54730/0 (socket says 172.21.15.100:54730) 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 -- 172.21.15.100:0/4121355033 learned_addr learned my addr 172.21.15.100:0/4121355033 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46b7fe640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe46c1023f0 0x7fe46c100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46bfff640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c10b7c0 0x7fe46c100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 -- 172.21.15.100:0/4121355033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c10b7c0 msgr2=0x7fe46c100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c10b7c0 0x7fe46c100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 -- 172.21.15.100:0/4121355033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe46c1023f0 msgr2=0x7fe46c100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe46c1023f0 0x7fe46c100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 -- 172.21.15.100:0/4121355033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c009560 con 0x7fe46c102d10 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe46affd640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 0x7fe46c100580 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe46000e9f0 tx=0x7fe46000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460009720 con 0x7fe46c102d10 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe46000cd80 con 0x7fe46c102d10 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c1a6240 con 0x7fe46c102d10 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460010690 con 0x7fe46c102d10 2024-03-12T05:34:49.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.105+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c1a6700 con 0x7fe46c102d10 2024-03-12T05:34:49.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.106+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c067f70 con 0x7fe46c102d10 2024-03-12T05:34:49.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.108+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe4600046c0 con 0x7fe46c102d10 2024-03-12T05:34:49.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.109+0000 7fe468ff9640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe4540a1ba0 0x7fe4540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.109+0000 7fe46b7fe640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe4540a1ba0 0x7fe4540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.109+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe460014070 con 0x7fe46c102d10 2024-03-12T05:34:49.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.109+0000 7fe46b7fe640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe4540a1ba0 0x7fe4540a4060 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c005e40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:49.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.113+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe460096a50 con 0x7fe46c102d10 2024-03-12T05:34:49.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.295+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe46c101570 con 0x7fe46c102d10 2024-03-12T05:34:49.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.297+0000 7fe468ff9640 1 -- 172.21.15.100:0/4121355033 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+23209 (secure 0 0 0) 0x7fe46009a110 con 0x7fe46c102d10 2024-03-12T05:34:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:49.291 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":121,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:34:15.975040+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/327651532":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1404323639":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/929302438":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2828798630":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/576676871":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/3916551108":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6800/327651532":"2024-03-13T05:12:05.720783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:34:49.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe4540a1ba0 msgr2=0x7fe4540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe4540a1ba0 0x7fe4540a4060 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c005e40 comp rx=0 tx=0).stop 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 msgr2=0x7fe46c100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 0x7fe46c100580 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe46000e9f0 tx=0x7fe46000edf0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 shutdown_connections 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fe4540a1ba0 0x7fe4540a4060 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c10b7c0 0x7fe46c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe46c102d10 0x7fe46c100580 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 --2- 172.21.15.100:0/4121355033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe46c1023f0 0x7fe46c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 >> 172.21.15.100:0/4121355033 conn(0x7fe46c077180 msgr2=0x7fe46c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:49.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 shutdown_connections 2024-03-12T05:34:49.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.299+0000 7fe472165640 1 -- 172.21.15.100:0/4121355033 wait complete. 2024-03-12T05:34:49.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:49 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:34:49.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:49 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:49 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:49 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:49 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T05:05:27.979515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T05:09:52.867267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T05:09:53.487567+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 120, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T05:34:49.305 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T05:34:49.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 -- 172.21.15.100:0/2162216256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c100310 msgr2=0x7f944c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 --2- 172.21.15.100:0/2162216256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c100310 0x7f944c1006f0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f943c009930 tx=0x7f943c02bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:49.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 -- 172.21.15.100:0/2162216256 shutdown_connections 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 --2- 172.21.15.100:0/2162216256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c1053e0 0x7f944c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 --2- 172.21.15.100:0/2162216256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c100cc0 0x7f944c104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 --2- 172.21.15.100:0/2162216256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c100310 0x7f944c1006f0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 -- 172.21.15.100:0/2162216256 >> 172.21.15.100:0/2162216256 conn(0x7f944c0771e0 msgr2=0x7f944c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 -- 172.21.15.100:0/2162216256 shutdown_connections 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.416+0000 7f945182e640 1 -- 172.21.15.100:0/2162216256 wait complete. 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 Processor -- start 2024-03-12T05:34:49.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 -- start start 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c100310 0x7f944c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 0x7f944c19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c1976a0 0x7f944c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c198170 con 0x7f944c1053e0 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c1982e0 con 0x7f944c1976a0 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f945182e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c1984a0 con 0x7f944c100310 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 0x7f944c19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 0x7f944c19db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54754/0 (socket says 172.21.15.100:54754) 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 -- 172.21.15.100:0/3228532255 learned_addr learned my addr 172.21.15.100:0/3228532255 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944b7fe640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c1976a0 0x7f944c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944affd640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c100310 0x7f944c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 -- 172.21.15.100:0/3228532255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c100310 msgr2=0x7f944c19d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c100310 0x7f944c19d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 -- 172.21.15.100:0/3228532255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c1976a0 msgr2=0x7f944c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c1976a0 0x7f944c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.417+0000 7f944a7fc640 1 -- 172.21.15.100:0/3228532255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c009560 con 0x7f944c1053e0 2024-03-12T05:34:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.418+0000 7f944a7fc640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 0x7f944c19db20 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:49.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.418+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400096c0 con 0x7f944c1053e0 2024-03-12T05:34:49.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.418+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9440005500 con 0x7f944c1053e0 2024-03-12T05:34:49.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.418+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440010430 con 0x7f944c1053e0 2024-03-12T05:34:49.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.418+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c1a4440 con 0x7f944c1053e0 2024-03-12T05:34:49.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.418+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c1a48d0 con 0x7f944c1053e0 2024-03-12T05:34:49.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.419+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c1017e0 con 0x7f944c1053e0 2024-03-12T05:34:49.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:49 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:34:49.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:49 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:49 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:49 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:49 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:49 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:49 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:49 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:49 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.491 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:49 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.420+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9440005020 con 0x7f944c1053e0 2024-03-12T05:34:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.420+0000 7f945082c640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94340a1ba0 0x7f94340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.420+0000 7f944affd640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94340a1ba0 0x7f94340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.420+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9440014070 con 0x7f944c1053e0 2024-03-12T05:34:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.421+0000 7f944affd640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94340a1ba0 0x7f94340a4060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f943c005ab0 tx=0x7f943c005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:49.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.424+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9440096a40 con 0x7f944c1053e0 2024-03-12T05:34:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.597+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f944c108610 con 0x7f944c1053e0 2024-03-12T05:34:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.597+0000 7f945082c640 1 -- 172.21.15.100:0/3228532255 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f944009a100 con 0x7f944c1053e0 2024-03-12T05:34:49.591 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-03-12T05:34:49.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94340a1ba0 msgr2=0x7f94340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94340a1ba0 0x7f94340a4060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f943c005ab0 tx=0x7f943c005fd0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 msgr2=0x7f944c19db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 0x7f944c19db20 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000eeb0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 shutdown_connections 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f94340a1ba0 0x7f94340a4060 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c1976a0 0x7f944c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c1053e0 0x7f944c19db20 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 --2- 172.21.15.100:0/3228532255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c100310 0x7f944c19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.599+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 >> 172.21.15.100:0/3228532255 conn(0x7f944c0771e0 msgr2=0x7f944c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:49.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.600+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 shutdown_connections 2024-03-12T05:34:49.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.600+0000 7f945182e640 1 -- 172.21.15.100:0/3228532255 wait complete. 2024-03-12T05:34:49.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T05:34:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 -- 172.21.15.100:0/1909429311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c101170 msgr2=0x7f5d2c10cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 --2- 172.21.15.100:0/1909429311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c101170 0x7f5d2c10cea0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009ca0 tx=0x7f5d2402c140 comp rx=0 tx=0).stop 2024-03-12T05:34:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 -- 172.21.15.100:0/1909429311 shutdown_connections 2024-03-12T05:34:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 --2- 172.21.15.100:0/1909429311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c101170 0x7f5d2c10cea0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 --2- 172.21.15.100:0/1909429311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c1007d0 0x7f5d2c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 --2- 172.21.15.100:0/1909429311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c105030 0x7f5d2c105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 -- 172.21.15.100:0/1909429311 >> 172.21.15.100:0/1909429311 conn(0x7f5d2c076640 msgr2=0x7f5d2c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 -- 172.21.15.100:0/1909429311 shutdown_connections 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.716+0000 7f5d313f6640 1 -- 172.21.15.100:0/1909429311 wait complete. 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 Processor -- start 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 -- start start 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c1007d0 0x7f5d2c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c101170 0x7f5d2c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 0x7f5d2c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c197c40 con 0x7f5d2c105030 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c197db0 con 0x7f5d2c1007d0 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d313f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c197f70 con 0x7f5d2c101170 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d2b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 0x7f5d2c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d2b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 0x7f5d2c1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54758/0 (socket says 172.21.15.100:54758) 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.717+0000 7f5d2b7fe640 1 -- 172.21.15.100:0/2557099152 learned_addr learned my addr 172.21.15.100:0/2557099152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2b7fe640 1 -- 172.21.15.100:0/2557099152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c101170 msgr2=0x7f5d2c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2b7fe640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c101170 0x7f5d2c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2b7fe640 1 -- 172.21.15.100:0/2557099152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c1007d0 msgr2=0x7f5d2c19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2affd640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c1007d0 0x7f5d2c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2b7fe640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c1007d0 0x7f5d2c19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2b7fe640 1 -- 172.21.15.100:0/2557099152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24009900 con 0x7f5d2c105030 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d2b7fe640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 0x7f5d2c1975f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5d24037040 tx=0x7f5d24035cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24032ab0 con 0x7f5d2c105030 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d24003e50 con 0x7f5d2c105030 2024-03-12T05:34:49.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1981f0 con 0x7f5d2c105030 2024-03-12T05:34:49.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d240048a0 con 0x7f5d2c105030 2024-03-12T05:34:49.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.718+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c1a4490 con 0x7f5d2c105030 2024-03-12T05:34:49.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.719+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c0683c0 con 0x7f5d2c105030 2024-03-12T05:34:49.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.720+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5d2403b040 con 0x7f5d2c105030 2024-03-12T05:34:49.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.721+0000 7f5d13fff640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d140a1ba0 0x7f5d140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:49.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.721+0000 7f5d2affd640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d140a1ba0 0x7f5d140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:49.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.721+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5d240f0760 con 0x7f5d2c105030 2024-03-12T05:34:49.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.721+0000 7f5d2affd640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d140a1ba0 0x7f5d140a4060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005fd0 tx=0x7f5d1c005850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:49.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.724+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d240b9700 con 0x7f5d2c105030 2024-03-12T05:34:49.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.906+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d2c1113b0 con 0x7f5d2c105030 2024-03-12T05:34:49.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.906+0000 7f5d13fff640 1 -- 172.21.15.100:0/2557099152 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d240bcdc0 con 0x7f5d2c105030 2024-03-12T05:34:49.900 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d140a1ba0 msgr2=0x7f5d140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d140a1ba0 0x7f5d140a4060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005fd0 tx=0x7f5d1c005850 comp rx=0 tx=0).stop 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 msgr2=0x7f5d2c1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 0x7f5d2c1975f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5d24037040 tx=0x7f5d24035cd0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 shutdown_connections 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5d140a1ba0 0x7f5d140a4060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2c105030 0x7f5d2c1975f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d2c101170 0x7f5d2c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 --2- 172.21.15.100:0/2557099152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c1007d0 0x7f5d2c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:49.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 >> 172.21.15.100:0/2557099152 conn(0x7f5d2c076640 msgr2=0x7f5d2c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:49.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 shutdown_connections 2024-03-12T05:34:49.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:49.908+0000 7f5d313f6640 1 -- 172.21.15.100:0/2557099152 wait complete. 2024-03-12T05:34:49.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T05:34:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 -- 172.21.15.100:0/4150021468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 msgr2=0x7f9814105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 --2- 172.21.15.100:0/4150021468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f9814105450 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f9804009900 tx=0x7f980402bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 -- 172.21.15.100:0/4150021468 shutdown_connections 2024-03-12T05:34:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 --2- 172.21.15.100:0/4150021468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814101150 0x7f981410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 --2- 172.21.15.100:0/4150021468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814100790 0x7f9814100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 --2- 172.21.15.100:0/4150021468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f9814105450 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 -- 172.21.15.100:0/4150021468 >> 172.21.15.100:0/4150021468 conn(0x7f9814076640 msgr2=0x7f98140fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 -- 172.21.15.100:0/4150021468 shutdown_connections 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.025+0000 7f981a96b640 1 -- 172.21.15.100:0/4150021468 wait complete. 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 Processor -- start 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 -- start start 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814100790 0x7f98141984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814101150 0x7f98141989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f98141a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814199290 con 0x7f9814105070 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98141a3a70 con 0x7f9814101150 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f981a96b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98141a3bb0 con 0x7f9814100790 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f9818f08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f98141a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f9818f08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f98141a3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54764/0 (socket says 172.21.15.100:54764) 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f9818f08640 1 -- 172.21.15.100:0/237006913 learned_addr learned my addr 172.21.15.100:0/237006913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f9813fff640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814100790 0x7f98141984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.026+0000 7f98137fe640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814101150 0x7f98141989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f9818f08640 1 -- 172.21.15.100:0/237006913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814100790 msgr2=0x7f98141984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f9818f08640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814100790 0x7f98141984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f9818f08640 1 -- 172.21.15.100:0/237006913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814101150 msgr2=0x7f98141989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f9818f08640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814101150 0x7f98141989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f9818f08640 1 -- 172.21.15.100:0/237006913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804009560 con 0x7f9814105070 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f98137fe640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814101150 0x7f98141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f9818f08640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f98141a3530 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f980c00eb40 tx=0x7f980c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c004b20 con 0x7f9814105070 2024-03-12T05:34:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f980c004c80 con 0x7f9814105070 2024-03-12T05:34:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98141a3e60 con 0x7f9814105070 2024-03-12T05:34:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c010670 con 0x7f9814105070 2024-03-12T05:34:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.027+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98141a4340 con 0x7f9814105070 2024-03-12T05:34:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.028+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98140683c0 con 0x7f9814105070 2024-03-12T05:34:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.029+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f980c0107d0 con 0x7f9814105070 2024-03-12T05:34:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.029+0000 7f98117fa640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97fc0a1ba0 0x7f97fc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.029+0000 7f9813fff640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97fc0a1ba0 0x7f97fc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.030+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f980c014070 con 0x7f9814105070 2024-03-12T05:34:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.030+0000 7f9813fff640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97fc0a1ba0 0x7f97fc0a4060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f980402c7b0 tx=0x7f9804000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.033+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f980c0971c0 con 0x7f9814105070 2024-03-12T05:34:50.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.208+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9814110310 con 0x7f9814105070 2024-03-12T05:34:50.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.209+0000 7f98117fa640 1 -- 172.21.15.100:0/237006913 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f980c09a880 con 0x7f9814105070 2024-03-12T05:34:50.202 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-03-12T05:34:50.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97fc0a1ba0 msgr2=0x7f97fc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97fc0a1ba0 0x7f97fc0a4060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f980402c7b0 tx=0x7f9804000bc0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 msgr2=0x7f98141a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f98141a3530 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f980c00eb40 tx=0x7f980c00c2d0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 shutdown_connections 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f97fc0a1ba0 0x7f97fc0a4060 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9814105070 0x7f98141a3530 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9814101150 0x7f98141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 --2- 172.21.15.100:0/237006913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9814100790 0x7f98141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 >> 172.21.15.100:0/237006913 conn(0x7f9814076640 msgr2=0x7f9814102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 shutdown_connections 2024-03-12T05:34:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.211+0000 7f981a96b640 1 -- 172.21.15.100:0/237006913 wait complete. 2024-03-12T05:34:50.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/4121355033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:50.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3228532255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:34:50.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:34:50.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:34:50.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:50.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2557099152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:34:50.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:50 smithi100 ceph-mon[32210]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 117 KiB/s wr, 16 op/s 2024-03-12T05:34:50.216 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T05:34:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2306903093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c406a450 msgr2=0x7f07c4105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2306903093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c406a450 0x7f07c4105e00 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009930 tx=0x7f07c002bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2306903093 shutdown_connections 2024-03-12T05:34:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2306903093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07c41064e0 0x7f07c410e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2306903093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c406a450 0x7f07c4105e00 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2306903093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07c4069aa0 0x7f07c4069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2306903093 >> 172.21.15.100:0/2306903093 conn(0x7f07c4077150 msgr2=0x7f07c4077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2306903093 shutdown_connections 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.325+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2306903093 wait complete. 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 Processor -- start 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 -- start start 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07c4069aa0 0x7f07c4100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07c41064e0 0x7f07c41005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 0x7f07c41a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41a5f00 con 0x7f07c4100ae0 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41a6070 con 0x7f07c41064e0 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07cbb0f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41a6230 con 0x7f07c4069aa0 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 0x7f07c41a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 0x7f07c41a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54792/0 (socket says 172.21.15.100:54792) 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 -- 172.21.15.100:0/2093278764 learned_addr learned my addr 172.21.15.100:0/2093278764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07c90aa640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07c41064e0 0x7f07c41005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07c98ab640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07c4069aa0 0x7f07c4100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 -- 172.21.15.100:0/2093278764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07c4069aa0 msgr2=0x7f07c4100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07c4069aa0 0x7f07c4100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 -- 172.21.15.100:0/2093278764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07c41064e0 msgr2=0x7f07c41005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07c41064e0 0x7f07c41005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 -- 172.21.15.100:0/2093278764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0009560 con 0x7f07c4100ae0 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.326+0000 7f07ca0ac640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 0x7f07c41a5930 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00cce0 tx=0x7f07bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.327+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc00f450 con 0x7f07c4100ae0 2024-03-12T05:34:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.327+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07bc00fa90 con 0x7f07c4100ae0 2024-03-12T05:34:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.327+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c41a6510 con 0x7f07c4100ae0 2024-03-12T05:34:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.327+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc018b90 con 0x7f07c4100ae0 2024-03-12T05:34:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.327+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c41a69a0 con 0x7f07c4100ae0 2024-03-12T05:34:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.328+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c4102870 con 0x7f07c4100ae0 2024-03-12T05:34:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.329+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f07bc00fc00 con 0x7f07c4100ae0 2024-03-12T05:34:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.329+0000 7f07baffd640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07b00a1bd0 0x7f07b00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.329+0000 7f07c98ab640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07b00a1bd0 0x7f07b00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.329+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f07bc0cdf70 con 0x7f07c4100ae0 2024-03-12T05:34:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.330+0000 7f07c98ab640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07b00a1bd0 0x7f07b00a4090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07b4005fd0 tx=0x7f07b4005850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.333+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07bc096e00 con 0x7f07c4100ae0 2024-03-12T05:34:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/4121355033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3228532255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:34:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:34:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:34:50.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2557099152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:34:50.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:50 smithi156 ceph-mon[36270]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 117 KiB/s wr, 16 op/s 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/4121355033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3228532255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2557099152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-12T05:34:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:50 smithi161 ceph-mon[36043]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 117 KiB/s wr, 16 op/s 2024-03-12T05:34:50.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.519+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f07c4109740 con 0x7f07c4100ae0 2024-03-12T05:34:50.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.519+0000 7f07baffd640 1 -- 172.21.15.100:0/2093278764 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v121) v1 ==== 81+0+1011 (secure 0 0 0) 0x7f07bc09a4c0 con 0x7f07c4100ae0 2024-03-12T05:34:50.513 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/327651532 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.513 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3130631042 2024-03-13T05:03:03.632804+0000 2024-03-12T05:34:50.513 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/981252995 2024-03-13T05:02:37.449022+0000 2024-03-12T05:34:50.513 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2732470606 2024-03-13T05:02:37.449022+0000 2024-03-12T05:34:50.513 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/981252995 2024-03-13T05:02:37.449022+0000 2024-03-12T05:34:50.514 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4124706748 2024-03-13T05:03:03.632804+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/1254888591 2024-03-13T05:03:03.632804+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/1404323639 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3056681718 2024-03-13T05:02:37.449022+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/929302438 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2462375416 2024-03-13T05:02:37.449022+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2828798630 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.515 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/16620000 2024-03-13T05:03:03.632804+0000 2024-03-12T05:34:50.516 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/576676871 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.516 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/16620000 2024-03-13T05:03:03.632804+0000 2024-03-12T05:34:50.516 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3916551108 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.516 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/327651532 2024-03-13T05:12:05.720783+0000 2024-03-12T05:34:50.517 INFO:teuthology.orchestra.run.smithi100.stderr:listed 17 entries 2024-03-12T05:34:50.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07b00a1bd0 msgr2=0x7f07b00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07b00a1bd0 0x7f07b00a4090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07b4005fd0 tx=0x7f07b4005850 comp rx=0 tx=0).stop 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 msgr2=0x7f07c41a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 0x7f07c41a5930 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00cce0 tx=0x7f07bc00a620 comp rx=0 tx=0).stop 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 shutdown_connections 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f07b00a1bd0 0x7f07b00a4090 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07c4100ae0 0x7f07c41a5930 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07c41064e0 0x7f07c41005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 --2- 172.21.15.100:0/2093278764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07c4069aa0 0x7f07c4100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 >> 172.21.15.100:0/2093278764 conn(0x7f07c4077150 msgr2=0x7f07c40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:50.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 shutdown_connections 2024-03-12T05:34:50.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:50.521+0000 7f07cbb0f640 1 -- 172.21.15.100:0/2093278764 wait complete. 2024-03-12T05:34:50.526 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-12T05:34:50.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:34:50.526 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-12T05:34:50.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:34:50.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:34:50.756 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-12T05:34:50.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:34:50.805 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-03-12T05:34:50.845 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-12T05:34:50.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:34:50.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T05:34:50.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T05:34:50.943 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-03-12T05:34:51.036 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-12T05:34:51.047 INFO:tasks.cephadm:Teardown begin 2024-03-12T05:34:51.048 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-12T05:34:51.048 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:34:51.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 -- 172.21.15.100:0/2129224513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c4100ca0 msgr2=0x7fd1c4104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 --2- 172.21.15.100:0/2129224513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c4100ca0 0x7fd1c4104ce0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009930 tx=0x7fd1b002bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:51.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 -- 172.21.15.100:0/2129224513 shutdown_connections 2024-03-12T05:34:51.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 --2- 172.21.15.100:0/2129224513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c41053c0 0x7fd1c410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 --2- 172.21.15.100:0/2129224513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c4100ca0 0x7fd1c4104ce0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 --2- 172.21.15.100:0/2129224513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c41002f0 0x7fd1c41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 -- 172.21.15.100:0/2129224513 >> 172.21.15.100:0/2129224513 conn(0x7fd1c4077180 msgr2=0x7fd1c4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 -- 172.21.15.100:0/2129224513 shutdown_connections 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 -- 172.21.15.100:0/2129224513 wait complete. 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.206+0000 7fd1cae23640 1 Processor -- start 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 -- start start 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c41002f0 0x7fd1c419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 0x7fd1c419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4197660 0x7fd1c4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4198130 con 0x7fd1c41053c0 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c41982a0 con 0x7fd1c41002f0 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1cae23640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4198460 con 0x7fd1c4197660 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 0x7fd1c419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 0x7fd1c419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47468/0 (socket says 172.21.15.100:47468) 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 -- 172.21.15.100:0/1416731625 learned_addr learned my addr 172.21.15.100:0/1416731625 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c93c0640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4197660 0x7fd1c4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c8bbf640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c41002f0 0x7fd1c419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 -- 172.21.15.100:0/1416731625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4197660 msgr2=0x7fd1c4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4197660 0x7fd1c4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 -- 172.21.15.100:0/1416731625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c41002f0 msgr2=0x7fd1c419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c41002f0 0x7fd1c419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 -- 172.21.15.100:0/1416731625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b0009560 con 0x7fd1c41053c0 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c3fff640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 0x7fd1c419dae0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009230 tx=0x7fd1b002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.207+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0004350 con 0x7fd1c41053c0 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.208+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b00044b0 con 0x7fd1c41053c0 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.208+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c41a43a0 con 0x7fd1c41053c0 2024-03-12T05:34:51.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.208+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0003da0 con 0x7fd1c41053c0 2024-03-12T05:34:51.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.208+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c41a48d0 con 0x7fd1c41053c0 2024-03-12T05:34:51.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.209+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c41017c0 con 0x7fd1c41053c0 2024-03-12T05:34:51.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.210+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd1b0004df0 con 0x7fd1c41053c0 2024-03-12T05:34:51.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.211+0000 7fd1c1ffb640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1c20 0x7fd1b40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.211+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd1b00efe10 con 0x7fd1c41053c0 2024-03-12T05:34:51.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.211+0000 7fd1c8bbf640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1c20 0x7fd1b40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.212+0000 7fd1c8bbf640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1c20 0x7fd1b40a40e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009bd0 tx=0x7fd1b8005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:51.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.214+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1b00b8db0 con 0x7fd1c41053c0 2024-03-12T05:34:51.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/237006913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:34:51.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:51 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2093278764' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T05:34:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.428+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1c4109600 con 0x7fd1c41053c0 2024-03-12T05:34:51.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.429+0000 7fd1c1ffb640 1 -- 172.21.15.100:0/1416731625 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fd1b00bc470 con 0x7fd1c41053c0 2024-03-12T05:34:51.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:51.423 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:34:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:34:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1c20 msgr2=0x7fd1b40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1c20 0x7fd1b40a40e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009bd0 tx=0x7fd1b8005f90 comp rx=0 tx=0).stop 2024-03-12T05:34:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 msgr2=0x7fd1c419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 0x7fd1c419dae0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009230 tx=0x7fd1b002c680 comp rx=0 tx=0).stop 2024-03-12T05:34:51.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 shutdown_connections 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fd1b40a1c20 0x7fd1b40a40e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4197660 0x7fd1c4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c41053c0 0x7fd1c419dae0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 --2- 172.21.15.100:0/1416731625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c41002f0 0x7fd1c419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 >> 172.21.15.100:0/1416731625 conn(0x7fd1c4077180 msgr2=0x7fd1c4068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 shutdown_connections 2024-03-12T05:34:51.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.431+0000 7fd1cae23640 1 -- 172.21.15.100:0/1416731625 wait complete. 2024-03-12T05:34:51.436 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T05:34:51.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/237006913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:34:51.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:51 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2093278764' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T05:34:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/237006913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-12T05:34:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:51 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2093278764' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-12T05:34:51.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 -- 172.21.15.100:0/1582974600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 msgr2=0x7f300810dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 --2- 172.21.15.100:0/1582974600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f300810dc60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3000009ca0 tx=0x7f300002c0f0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 -- 172.21.15.100:0/1582974600 shutdown_connections 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 --2- 172.21.15.100:0/1582974600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f300810dc60 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 --2- 172.21.15.100:0/1582974600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3008102d10 0x7f300810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 --2- 172.21.15.100:0/1582974600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30081023f0 0x7f30081027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.544+0000 7f300e148640 1 -- 172.21.15.100:0/1582974600 >> 172.21.15.100:0/1582974600 conn(0x7f3008077180 msgr2=0x7f3008077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:51.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 -- 172.21.15.100:0/1582974600 shutdown_connections 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 -- 172.21.15.100:0/1582974600 wait complete. 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 Processor -- start 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 -- start start 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30081023f0 0x7f300819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3008102d10 0x7f300819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f30081975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008197bf0 con 0x7f300810b7c0 2024-03-12T05:34:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008197d60 con 0x7f30081023f0 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.545+0000 7f300e148640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008197f20 con 0x7f3008102d10 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f30081975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f30081975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47472/0 (socket says 172.21.15.100:47472) 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 -- 172.21.15.100:0/1941214685 learned_addr learned my addr 172.21.15.100:0/1941214685 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f30077fe640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30081023f0 0x7f300819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3006ffd640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3008102d10 0x7f300819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 -- 172.21.15.100:0/1941214685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3008102d10 msgr2=0x7f300819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3008102d10 0x7f300819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 -- 172.21.15.100:0/1941214685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30081023f0 msgr2=0x7f300819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30081023f0 0x7f300819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 -- 172.21.15.100:0/1941214685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009900 con 0x7f300810b7c0 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3007fff640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f30081975a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f300002c700 tx=0x7f3000034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000032ac0 con 0x7f300810b7c0 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3000032eb0 con 0x7f300810b7c0 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30081981a0 con 0x7f300810b7c0 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000040450 con 0x7f300810b7c0 2024-03-12T05:34:51.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.546+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30081a4490 con 0x7f300810b7c0 2024-03-12T05:34:51.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.547+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3008067f70 con 0x7f300810b7c0 2024-03-12T05:34:51.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.548+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f300000a040 con 0x7f300810b7c0 2024-03-12T05:34:51.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.549+0000 7f3004ff9640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2ff00a1ba0 0x7f2ff00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.549+0000 7f30077fe640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2ff00a1ba0 0x7f2ff00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.549+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30000f04b0 con 0x7f300810b7c0 2024-03-12T05:34:51.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.550+0000 7f30077fe640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2ff00a1ba0 0x7f2ff00a4060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800bbe0 tx=0x7f2ff800b390 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:51.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.554+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30000b9450 con 0x7f300810b7c0 2024-03-12T05:34:51.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.761+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3008111300 con 0x7f300810b7c0 2024-03-12T05:34:51.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.762+0000 7f3004ff9640 1 -- 172.21.15.100:0/1941214685 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f30000bcb10 con 0x7f300810b7c0 2024-03-12T05:34:51.756 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:51.756 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3900408401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3900408401},{"type":"v1","addr":"172.21.15.100:6835","nonce":3900408401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1446338988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1446338988},{"type":"v1","addr":"172.21.15.100:6837","nonce":1446338988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2856725609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2856725609},{"type":"v1","addr":"172.21.15.100:6839","nonce":2856725609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3942756808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3942756808},{"type":"v1","addr":"172.21.15.100:6841","nonce":3942756808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/692677489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":692677489},{"type":"v1","addr":"172.21.15.156:6833","nonce":692677489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2628600051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2628600051},{"type":"v1","addr":"172.21.15.161:6833","nonce":2628600051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-12T05:09:54.274771+0000","modified":"2024-03-12T05:28:08.580068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24451,"mds_2":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24401":{"gid":24401,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2592644588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2592644588},{"type":"v1","addr":"172.21.15.156:6835","nonce":2592644588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/692407308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":692407308},{"type":"v1","addr":"172.21.15.161:6835","nonce":692407308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/854337146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":854337146},{"type":"v1","addr":"172.21.15.161:6837","nonce":854337146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24439":{"gid":24439,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/2463295988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":2463295988},{"type":"v1","addr":"172.21.15.156:6837","nonce":2463295988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/1712807460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1712807460},{"type":"v1","addr":"172.21.15.161:6839","nonce":1712807460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1209044969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1209044969},{"type":"v1","addr":"172.21.15.156:6839","nonce":1209044969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T05:34:51.756 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-03-12T05:34:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2ff00a1ba0 msgr2=0x7f2ff00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2ff00a1ba0 0x7f2ff00a4060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800bbe0 tx=0x7f2ff800b390 comp rx=0 tx=0).stop 2024-03-12T05:34:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 msgr2=0x7f30081975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f30081975a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f300002c700 tx=0x7f3000034d40 comp rx=0 tx=0).stop 2024-03-12T05:34:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 shutdown_connections 2024-03-12T05:34:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f2ff00a1ba0 0x7f2ff00a4060 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300810b7c0 0x7f30081975a0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3008102d10 0x7f300819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 --2- 172.21.15.100:0/1941214685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30081023f0 0x7f300819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 >> 172.21.15.100:0/1941214685 conn(0x7f3008077180 msgr2=0x7f30080fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 shutdown_connections 2024-03-12T05:34:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.764+0000 7f300e148640 1 -- 172.21.15.100:0/1941214685 wait complete. 2024-03-12T05:34:51.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T05:34:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 -- 172.21.15.100:0/2852299640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 msgr2=0x7f31d010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2852299640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d010cf90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009cd0 tx=0x7f31c802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 -- 172.21.15.100:0/2852299640 shutdown_connections 2024-03-12T05:34:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2852299640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d010cf90 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2852299640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0100ca0 0x7f31d0104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2852299640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d01002f0 0x7f31d01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 -- 172.21.15.100:0/2852299640 >> 172.21.15.100:0/2852299640 conn(0x7f31d0077180 msgr2=0x7f31d0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 -- 172.21.15.100:0/2852299640 shutdown_connections 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 -- 172.21.15.100:0/2852299640 wait complete. 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 Processor -- start 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.878+0000 7f31d63d7640 1 -- start start 2024-03-12T05:34:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d01002f0 0x7f31d0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0102800 0x7f31d01a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d01a5fc0 con 0x7f31d01053c0 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d01a6130 con 0x7f31d01002f0 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d01a62f0 con 0x7f31d0102800 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d01022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47486/0 (socket says 172.21.15.100:47486) 2024-03-12T05:34:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 -- 172.21.15.100:0/2462846832 learned_addr learned my addr 172.21.15.100:0/2462846832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d4974640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0102800 0x7f31d01a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cffff640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d01002f0 0x7f31d0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 -- 172.21.15.100:0/2462846832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0102800 msgr2=0x7f31d01a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0102800 0x7f31d01a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 -- 172.21.15.100:0/2462846832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d01002f0 msgr2=0x7f31d0101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d01002f0 0x7f31d0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 -- 172.21.15.100:0/2462846832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c8009900 con 0x7f31d01053c0 2024-03-12T05:34:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cf7fe640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d01022c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f31c400d770 tx=0x7f31c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4009c30 con 0x7f31d01053c0 2024-03-12T05:34:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d01a65d0 con 0x7f31d01053c0 2024-03-12T05:34:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31c400de80 con 0x7f31d01053c0 2024-03-12T05:34:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400f6b0 con 0x7f31d01053c0 2024-03-12T05:34:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.879+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d01a6ae0 con 0x7f31d01053c0 2024-03-12T05:34:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.880+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d0067f70 con 0x7f31d01053c0 2024-03-12T05:34:51.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.882+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f31c4010460 con 0x7f31d01053c0 2024-03-12T05:34:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.882+0000 7f31cd7fa640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f31b80a1bd0 0x7f31b80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.882+0000 7f31cffff640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f31b80a1bd0 0x7f31b80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.882+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f31c40cd4e0 con 0x7f31d01053c0 2024-03-12T05:34:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.883+0000 7f31cffff640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f31b80a1bd0 0x7f31b80a4090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f31c0005fd0 tx=0x7f31c0005850 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:51.886+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31c4096370 con 0x7f31d01053c0 2024-03-12T05:34:52.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.063+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31d0109660 con 0x7f31d01053c0 2024-03-12T05:34:52.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.064+0000 7f31cd7fa640 1 -- 172.21.15.100:0/2462846832 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+23209 (secure 0 0 0) 0x7f31c4021090 con 0x7f31d01053c0 2024-03-12T05:34:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-12T05:34:52.058 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":121,"fsid":"8a1680ae-e02d-11ee-95c8-87774f69a715","created":"2024-03-12T05:02:08.927382+0000","modified":"2024-03-12T05:34:15.975040+0000","last_up_change":"2024-03-12T05:09:05.665165+0000","last_in_change":"2024-03-12T05:08:44.985128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T05:05:27.979515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T05:09:52.867267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T05:09:53.487567+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df3ee95d-798a-4fd9-82ef-81f981a5461d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6803","nonce":774764839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6805","nonce":774764839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6809","nonce":774764839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":774764839},{"type":"v1","addr":"172.21.15.100:6807","nonce":774764839}]},"public_addr":"172.21.15.100:6803/774764839","cluster_addr":"172.21.15.100:6805/774764839","heartbeat_back_addr":"172.21.15.100:6809/774764839","heartbeat_front_addr":"172.21.15.100:6807/774764839","state":["exists","up"]},{"osd":1,"uuid":"95723563-e515-4042-ac3c-bb298e215da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6801","nonce":3629499261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6803","nonce":3629499261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6807","nonce":3629499261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":3629499261},{"type":"v1","addr":"172.21.15.156:6805","nonce":3629499261}]},"public_addr":"172.21.15.156:6801/3629499261","cluster_addr":"172.21.15.156:6803/3629499261","heartbeat_back_addr":"172.21.15.156:6807/3629499261","heartbeat_front_addr":"172.21.15.156:6805/3629499261","state":["exists","up"]},{"osd":2,"uuid":"79302fd1-469d-440f-b520-378f0f3a459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6801","nonce":623458559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6803","nonce":623458559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6807","nonce":623458559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":623458559},{"type":"v1","addr":"172.21.15.161:6805","nonce":623458559}]},"public_addr":"172.21.15.161:6801/623458559","cluster_addr":"172.21.15.161:6803/623458559","heartbeat_back_addr":"172.21.15.161:6807/623458559","heartbeat_front_addr":"172.21.15.161:6805/623458559","state":["exists","up"]},{"osd":3,"uuid":"037d713e-354a-437d-9f75-3b506f6e4ed0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6811","nonce":2837761202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6813","nonce":2837761202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6817","nonce":2837761202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2837761202},{"type":"v1","addr":"172.21.15.100:6815","nonce":2837761202}]},"public_addr":"172.21.15.100:6811/2837761202","cluster_addr":"172.21.15.100:6813/2837761202","heartbeat_back_addr":"172.21.15.100:6817/2837761202","heartbeat_front_addr":"172.21.15.100:6815/2837761202","state":["exists","up"]},{"osd":4,"uuid":"206a171a-ec8a-474b-9f06-40508adf5df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6809","nonce":551000361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6811","nonce":551000361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6815","nonce":551000361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":551000361},{"type":"v1","addr":"172.21.15.156:6813","nonce":551000361}]},"public_addr":"172.21.15.156:6809/551000361","cluster_addr":"172.21.15.156:6811/551000361","heartbeat_back_addr":"172.21.15.156:6815/551000361","heartbeat_front_addr":"172.21.15.156:6813/551000361","state":["exists","up"]},{"osd":5,"uuid":"a41244b3-befa-47fa-809e-fdc7b1a23f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6809","nonce":3085464556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6811","nonce":3085464556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6815","nonce":3085464556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3085464556},{"type":"v1","addr":"172.21.15.161:6813","nonce":3085464556}]},"public_addr":"172.21.15.161:6809/3085464556","cluster_addr":"172.21.15.161:6811/3085464556","heartbeat_back_addr":"172.21.15.161:6815/3085464556","heartbeat_front_addr":"172.21.15.161:6813/3085464556","state":["exists","up"]},{"osd":6,"uuid":"4349c6aa-c535-40fb-a462-cf066f57ea26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6819","nonce":125600956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6821","nonce":125600956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6825","nonce":125600956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":125600956},{"type":"v1","addr":"172.21.15.100:6823","nonce":125600956}]},"public_addr":"172.21.15.100:6819/125600956","cluster_addr":"172.21.15.100:6821/125600956","heartbeat_back_addr":"172.21.15.100:6825/125600956","heartbeat_front_addr":"172.21.15.100:6823/125600956","state":["exists","up"]},{"osd":7,"uuid":"1fd43ecf-765c-412a-b351-a8b7fd3b7381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6817","nonce":3327014468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6819","nonce":3327014468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6823","nonce":3327014468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3327014468},{"type":"v1","addr":"172.21.15.156:6821","nonce":3327014468}]},"public_addr":"172.21.15.156:6817/3327014468","cluster_addr":"172.21.15.156:6819/3327014468","heartbeat_back_addr":"172.21.15.156:6823/3327014468","heartbeat_front_addr":"172.21.15.156:6821/3327014468","state":["exists","up"]},{"osd":8,"uuid":"2fc82b38-b2e7-47c3-9130-8c921510b06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6817","nonce":2263371293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6819","nonce":2263371293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6823","nonce":2263371293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2263371293},{"type":"v1","addr":"172.21.15.161:6821","nonce":2263371293}]},"public_addr":"172.21.15.161:6817/2263371293","cluster_addr":"172.21.15.161:6819/2263371293","heartbeat_back_addr":"172.21.15.161:6823/2263371293","heartbeat_front_addr":"172.21.15.161:6821/2263371293","state":["exists","up"]},{"osd":9,"uuid":"2f17e94a-a270-4e9e-837e-e0c5dfb2e68c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6827","nonce":917772860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6829","nonce":917772860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6833","nonce":917772860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":917772860},{"type":"v1","addr":"172.21.15.100:6831","nonce":917772860}]},"public_addr":"172.21.15.100:6827/917772860","cluster_addr":"172.21.15.100:6829/917772860","heartbeat_back_addr":"172.21.15.100:6833/917772860","heartbeat_front_addr":"172.21.15.100:6831/917772860","state":["exists","up"]},{"osd":10,"uuid":"359d96d0-445b-4e93-8811-5e0407d42957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6825","nonce":3731703036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6827","nonce":3731703036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6831","nonce":3731703036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3731703036},{"type":"v1","addr":"172.21.15.156:6829","nonce":3731703036}]},"public_addr":"172.21.15.156:6825/3731703036","cluster_addr":"172.21.15.156:6827/3731703036","heartbeat_back_addr":"172.21.15.156:6831/3731703036","heartbeat_front_addr":"172.21.15.156:6829/3731703036","state":["exists","up"]},{"osd":11,"uuid":"bcaf5ff6-d025-4fdb-b818-fcc0cc3d6230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6825","nonce":2757536197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6827","nonce":2757536197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6831","nonce":2757536197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2757536197},{"type":"v1","addr":"172.21.15.161:6829","nonce":2757536197}]},"public_addr":"172.21.15.161:6825/2757536197","cluster_addr":"172.21.15.161:6827/2757536197","heartbeat_back_addr":"172.21.15.161:6831/2757536197","heartbeat_front_addr":"172.21.15.161:6829/2757536197","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:04:37.861138+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:01.257462+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:24.896107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:05:49.423111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:13.376240+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:06:37.321168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:01.215527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:25.530417+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:07:49.346910+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:13.955905+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:08:38.260962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T05:09:03.240871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/327651532":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/3130631042":"2024-03-13T05:03:03.632804+0000","172.21.15.100:6801/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2732470606":"2024-03-13T05:02:37.449022+0000","172.21.15.100:6800/981252995":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/4124706748":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1254888591":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/1404323639":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/3056681718":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/929302438":"2024-03-13T05:12:05.720783+0000","172.21.15.100:0/2462375416":"2024-03-13T05:02:37.449022+0000","172.21.15.100:0/2828798630":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6801/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/576676871":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6800/16620000":"2024-03-13T05:03:03.632804+0000","172.21.15.100:0/3916551108":"2024-03-13T05:12:05.720783+0000","172.21.15.100:6800/327651532":"2024-03-13T05:12:05.720783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T05:34:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f31b80a1bd0 msgr2=0x7f31b80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f31b80a1bd0 0x7f31b80a4090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f31c0005fd0 tx=0x7f31c0005850 comp rx=0 tx=0).stop 2024-03-12T05:34:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 msgr2=0x7f31d01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d01022c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f31c400d770 tx=0x7f31c400dc30 comp rx=0 tx=0).stop 2024-03-12T05:34:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 shutdown_connections 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f31b80a1bd0 0x7f31b80a4090 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0102800 0x7f31d01a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d01053c0 0x7f31d01022c0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 --2- 172.21.15.100:0/2462846832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d01002f0 0x7f31d0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 >> 172.21.15.100:0/2462846832 conn(0x7f31d0077180 msgr2=0x7f31d00685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 shutdown_connections 2024-03-12T05:34:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.066+0000 7f31d63d7640 1 -- 172.21.15.100:0/2462846832 wait complete. 2024-03-12T05:34:52.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-12T05:34:52.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 -- 172.21.15.100:0/1480529398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101d30 msgr2=0x7fb700105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 --2- 172.21.15.100:0/1480529398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101d30 0x7fb700105d70 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009930 tx=0x7fb6f002bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 -- 172.21.15.100:0/1480529398 shutdown_connections 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 --2- 172.21.15.100:0/1480529398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700106450 0x7fb70010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 --2- 172.21.15.100:0/1480529398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101d30 0x7fb700105d70 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 --2- 172.21.15.100:0/1480529398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb700101380 0x7fb700101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 -- 172.21.15.100:0/1480529398 >> 172.21.15.100:0/1480529398 conn(0x7fb7000fcc20 msgr2=0x7fb7000ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:52.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 -- 172.21.15.100:0/1480529398 shutdown_connections 2024-03-12T05:34:52.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 -- 172.21.15.100:0/1480529398 wait complete. 2024-03-12T05:34:52.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 Processor -- start 2024-03-12T05:34:52.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.184+0000 7fb7065ac640 1 -- start start 2024-03-12T05:34:52.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 0x7fb700073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb700106450 0x7fb7000739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700073f00 0x7fb7001a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001a5f60 con 0x7fb700101380 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001a60d0 con 0x7fb700106450 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001a6290 con 0x7fb700073f00 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 0x7fb700073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 0x7fb700073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47492/0 (socket says 172.21.15.100:47492) 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 -- 172.21.15.100:0/3281768666 learned_addr learned my addr 172.21.15.100:0/3281768666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb704b49640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700073f00 0x7fb7001a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6ff7fe640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb700106450 0x7fb7000739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 -- 172.21.15.100:0/3281768666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700073f00 msgr2=0x7fb7001a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700073f00 0x7fb7001a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 -- 172.21.15.100:0/3281768666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb700106450 msgr2=0x7fb7000739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb700106450 0x7fb7000739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 -- 172.21.15.100:0/3281768666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009560 con 0x7fb700101380 2024-03-12T05:34:52.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb704b49640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700073f00 0x7fb7001a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T05:34:52.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fffff640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 0x7fb700073480 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400d950 tx=0x7fb6f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:52.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4009c30 con 0x7fb700101380 2024-03-12T05:34:52.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7001a6570 con 0x7fb700101380 2024-03-12T05:34:52.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6f4009d90 con 0x7fb700101380 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400f7e0 con 0x7fb700101380 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.185+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7001a6a50 con 0x7fb700101380 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.187+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb700067f70 con 0x7fb700101380 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.187+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb6f4010ad0 con 0x7fb700101380 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.188+0000 7fb6fd7fa640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb6e80a1ba0 0x7fb6e80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.188+0000 7fb6ff7fe640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb6e80a1ba0 0x7fb6e80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.188+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb6f40cd5a0 con 0x7fb700101380 2024-03-12T05:34:52.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.188+0000 7fb6ff7fe640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb6e80a1ba0 0x7fb6e80a4060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6f002c170 tx=0x7fb6f0000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:52.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.192+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6f4096430 con 0x7fb700101380 2024-03-12T05:34:52.349 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1416731625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.349 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1941214685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.349 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:52 smithi100 ceph-mon[32210]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 243 KiB/s wr, 19 op/s 2024-03-12T05:34:52.350 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:52 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2462846832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.426+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb700109680 con 0x7fb700101380 2024-03-12T05:34:52.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.430+0000 7fb6fd7fa640 1 -- 172.21.15.100:0/3281768666 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb6f4099af0 con 0x7fb700101380 2024-03-12T05:34:52.423 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb6e80a1ba0 msgr2=0x7fb6e80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb6e80a1ba0 0x7fb6e80a4060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6f002c170 tx=0x7fb6f0000bc0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 msgr2=0x7fb700073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 0x7fb700073480 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400d950 tx=0x7fb6f400de10 comp rx=0 tx=0).stop 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 shutdown_connections 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7fb6e80a1ba0 0x7fb6e80a4060 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb700073f00 0x7fb7001a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb700106450 0x7fb7000739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 --2- 172.21.15.100:0/3281768666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb700101380 0x7fb700073480 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 >> 172.21.15.100:0/3281768666 conn(0x7fb7000fcc20 msgr2=0x7fb7000fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 shutdown_connections 2024-03-12T05:34:52.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.432+0000 7fb7065ac640 1 -- 172.21.15.100:0/3281768666 wait complete. 2024-03-12T05:34:52.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-12T05:34:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1416731625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1941214685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mon[36270]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 243 KiB/s wr, 19 op/s 2024-03-12T05:34:52.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2462846832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1416731625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1941214685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mon[36043]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 243 KiB/s wr, 19 op/s 2024-03-12T05:34:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2462846832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T05:34:52.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 -- 172.21.15.100:0/4253489027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2770105330 msgr2=0x7f277010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 --2- 172.21.15.100:0/4253489027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2770105330 0x7f277010cf00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2768009cd0 tx=0x7f276802c0f0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 -- 172.21.15.100:0/4253489027 shutdown_connections 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 --2- 172.21.15.100:0/4253489027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2770105330 0x7f277010cf00 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 --2- 172.21.15.100:0/4253489027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770100c10 0x7f2770104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 --2- 172.21.15.100:0/4253489027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770100260 0x7f2770100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.554+0000 7f2776a22640 1 -- 172.21.15.100:0/4253489027 >> 172.21.15.100:0/4253489027 conn(0x7f2770077180 msgr2=0x7f2770077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 -- 172.21.15.100:0/4253489027 shutdown_connections 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 -- 172.21.15.100:0/4253489027 wait complete. 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 Processor -- start 2024-03-12T05:34:52.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 -- start start 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770100260 0x7f2770196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770105330 0x7f2770196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 0x7f277019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277019b930 con 0x7f277019af40 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277019baa0 con 0x7f2770105330 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.555+0000 7f2776a22640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277019bc10 con 0x7f2770100260 2024-03-12T05:34:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 0x7f277019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 0x7f277019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47516/0 (socket says 172.21.15.100:47516) 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 -- 172.21.15.100:0/2980094939 learned_addr learned my addr 172.21.15.100:0/2980094939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f276f7fe640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770105330 0x7f2770196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f276ffff640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770100260 0x7f2770196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 -- 172.21.15.100:0/2980094939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770100260 msgr2=0x7f2770196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770100260 0x7f2770196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 -- 172.21.15.100:0/2980094939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770105330 msgr2=0x7f2770196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770105330 0x7f2770196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:52.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 -- 172.21.15.100:0/2980094939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009900 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f276f7fe640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770105330 0x7f2770196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2774fbf640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 0x7f277019b3f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2768004af0 tx=0x7f2768034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680328c0 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27701a31f0 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701a3670 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2768003e50 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.556+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768004860 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.557+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2770067f70 con 0x7f277019af40 2024-03-12T05:34:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.559+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f276800a040 con 0x7f277019af40 2024-03-12T05:34:52.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.559+0000 7f276d7fa640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27580a1ba0 0x7f27580a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:52.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.559+0000 7f276ffff640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27580a1ba0 0x7f27580a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:52.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.559+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8799+0+0 (secure 0 0 0) 0x7f27680f09b0 con 0x7f277019af40 2024-03-12T05:34:52.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.560+0000 7f276ffff640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27580a1ba0 0x7f27580a4060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2760009bd0 tx=0x7f2760005f90 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:52.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.566+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27680b9880 con 0x7f277019af40 2024-03-12T05:34:52.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:52.784+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f27701085c0 con 0x7f277019af40 2024-03-12T05:34:52.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a[32206]: 2024-03-12T05:34:52.427+0000 7f706559a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T05:34:52.951 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e[66259]: ignoring --setuser ceph since I am not root 2024-03-12T05:34:52.951 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e[66259]: ignoring --setgroup ceph since I am not root 2024-03-12T05:34:52.951 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e[66259]: starting mds.e at 2024-03-12T05:34:52.952 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: ignoring --setuser ceph since I am not root 2024-03-12T05:34:52.952 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: ignoring --setgroup ceph since I am not root 2024-03-12T05:34:52.952 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mds[66881]: -- 172.21.15.156:0/4231094121 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5579abb8b080 con 0x5579abb54800 2024-03-12T05:34:52.952 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mds[66881]: -- 172.21.15.156:0/4231094121 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579abb7e1a0 con 0x5579abb54800 2024-03-12T05:34:52.952 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: starting mds.b at 2024-03-12T05:34:52.953 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h[66572]: ignoring --setuser ceph since I am not root 2024-03-12T05:34:52.953 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h[66572]: ignoring --setgroup ceph since I am not root 2024-03-12T05:34:52.953 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-mds[66576]: -- 172.21.15.156:0/2310009972 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55972814a1a0 con 0x559728120800 2024-03-12T05:34:52.953 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:34:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h[66572]: starting mds.h at 2024-03-12T05:34:52.969 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f[66705]: ignoring --setuser ceph since I am not root 2024-03-12T05:34:52.969 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f[66705]: ignoring --setgroup ceph since I am not root 2024-03-12T05:34:52.969 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mds[66709]: -- 172.21.15.161:0/4242295318 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b217a61a0 con 0x564b2177c800 2024-03-12T05:34:52.969 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f[66705]: starting mds.f at 2024-03-12T05:34:52.970 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i[67336]: ignoring --setuser ceph since I am not root 2024-03-12T05:34:52.970 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i[67336]: ignoring --setgroup ceph since I am not root 2024-03-12T05:34:52.970 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mds[67340]: -- 172.21.15.161:0/3428461455 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602f85561a0 con 0x5602f852cc00 2024-03-12T05:34:52.970 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i[67336]: starting mds.i at 2024-03-12T05:34:52.971 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c[67019]: ignoring --setuser ceph since I am not root 2024-03-12T05:34:52.971 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c[67019]: ignoring --setgroup ceph since I am not root 2024-03-12T05:34:52.971 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-mds[67023]: -- 172.21.15.161:0/3420087013 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561a277d01a0 con 0x561a277a6800 2024-03-12T05:34:52.971 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:34:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c[67019]: starting mds.c at 2024-03-12T05:34:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.432+0000 7f276d7fa640 1 -- 172.21.15.100:0/2980094939 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f27680bcf40 con 0x7f277019af40 2024-03-12T05:34:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.434+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27580a1ba0 msgr2=0x7f27580a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.434+0000 7f2776a22640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27580a1ba0 0x7f27580a4060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2760009bd0 tx=0x7f2760005f90 comp rx=0 tx=0).stop 2024-03-12T05:34:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 msgr2=0x7f277019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 0x7f277019b3f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2768004af0 tx=0x7f2768034cc0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 shutdown_connections 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f27580a1ba0 0x7f27580a4060 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277019af40 0x7f277019b3f0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2770105330 0x7f2770196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 --2- 172.21.15.100:0/2980094939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770100260 0x7f2770196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 >> 172.21.15.100:0/2980094939 conn(0x7f2770077180 msgr2=0x7f27700fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:53.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 shutdown_connections 2024-03-12T05:34:53.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.435+0000 7f2776a22640 1 -- 172.21.15.100:0/2980094939 wait complete. 2024-03-12T05:34:53.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3281768666' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-12T05:34:53.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-12T05:34:53.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T05:34:53.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3281768666' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-12T05:34:53.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: osdmap e122: 12 total, 12 up, 12 in 2024-03-12T05:34:53.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-12T05:34:53.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:53 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2980094939' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:34:53.442 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-12T05:34:53.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3281768666' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-12T05:34:53.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-12T05:34:53.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T05:34:53.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3281768666' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-12T05:34:53.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: osdmap e122: 12 total, 12 up, 12 in 2024-03-12T05:34:53.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-12T05:34:53.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:53 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2980094939' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:34:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3281768666' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-12T05:34:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-12T05:34:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T05:34:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3281768666' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-12T05:34:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: osdmap e122: 12 total, 12 up, 12 in 2024-03-12T05:34:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-12T05:34:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:53 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2980094939' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T05:34:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 -- 172.21.15.100:0/4143267857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100c10 msgr2=0x7f5f24104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 --2- 172.21.15.100:0/4143267857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100c10 0x7f5f24104c50 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009930 tx=0x7f5f1802bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 -- 172.21.15.100:0/4143267857 shutdown_connections 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 --2- 172.21.15.100:0/4143267857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f24105330 0x7f5f2410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 --2- 172.21.15.100:0/4143267857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100c10 0x7f5f24104c50 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 --2- 172.21.15.100:0/4143267857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f24100260 0x7f5f24100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 -- 172.21.15.100:0/4143267857 >> 172.21.15.100:0/4143267857 conn(0x7f5f24077180 msgr2=0x7f5f24077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 -- 172.21.15.100:0/4143267857 shutdown_connections 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.553+0000 7f5f291d0640 1 -- 172.21.15.100:0/4143267857 wait complete. 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.554+0000 7f5f291d0640 1 Processor -- start 2024-03-12T05:34:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.554+0000 7f5f291d0640 1 -- start start 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 0x7f5f241966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f24105330 0x7f5f24196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2419af90 0x7f5f2419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2419b980 con 0x7f5f24100260 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2419baf0 con 0x7f5f24105330 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2419bc60 con 0x7f5f2419af90 2024-03-12T05:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 0x7f5f241966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 0x7f5f241966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47542/0 (socket says 172.21.15.100:47542) 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 -- 172.21.15.100:0/1636369429 learned_addr learned my addr 172.21.15.100:0/1636369429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f2259c640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f24105330 0x7f5f24196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f2359e640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2419af90 0x7f5f2419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 -- 172.21.15.100:0/1636369429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2419af90 msgr2=0x7f5f2419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2419af90 0x7f5f2419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 -- 172.21.15.100:0/1636369429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f24105330 msgr2=0x7f5f24196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f24105330 0x7f5f24196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 -- 172.21.15.100:0/1636369429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009560 con 0x7f5f24100260 2024-03-12T05:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f22d9d640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 0x7f5f241966d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400d950 tx=0x7f5f1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14009c30 con 0x7f5f24100260 2024-03-12T05:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f241a3240 con 0x7f5f24100260 2024-03-12T05:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f14009d90 con 0x7f5f24100260 2024-03-12T05:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400f7e0 con 0x7f5f24100260 2024-03-12T05:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.555+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f241a3710 con 0x7f5f24100260 2024-03-12T05:34:53.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.556+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24067f70 con 0x7f5f24100260 2024-03-12T05:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.557+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5f14010ad0 con 0x7f5f24100260 2024-03-12T05:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.558+0000 7f5f0bfff640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f0c0a1ba0 0x7f5f0c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.558+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8799+0+0 (secure 0 0 0) 0x7f5f140cd600 con 0x7f5f24100260 2024-03-12T05:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.558+0000 7f5f2259c640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f0c0a1ba0 0x7f5f0c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.558+0000 7f5f2259c640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f0c0a1ba0 0x7f5f0c0a4060 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009900 tx=0x7f5f18005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.561+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f140963c0 con 0x7f5f24100260 2024-03-12T05:34:53.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:53.734+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f24197a20 con 0x7f5f24100260 2024-03-12T05:34:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.439+0000 7f5f0bfff640 1 -- 172.21.15.100:0/1636369429 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5f140170d0 con 0x7f5f24100260 2024-03-12T05:34:54.432 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-03-12T05:34:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f0c0a1ba0 msgr2=0x7f5f0c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f0c0a1ba0 0x7f5f0c0a4060 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009900 tx=0x7f5f18005fd0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 msgr2=0x7f5f241966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 0x7f5f241966d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400d950 tx=0x7f5f1400de10 comp rx=0 tx=0).stop 2024-03-12T05:34:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 shutdown_connections 2024-03-12T05:34:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f5f0c0a1ba0 0x7f5f0c0a4060 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f2419af90 0x7f5f2419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f24105330 0x7f5f24196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 --2- 172.21.15.100:0/1636369429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f24100260 0x7f5f241966d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 >> 172.21.15.100:0/1636369429 conn(0x7f5f24077180 msgr2=0x7f5f240fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.441+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 shutdown_connections 2024-03-12T05:34:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.442+0000 7f5f291d0640 1 -- 172.21.15.100:0/1636369429 wait complete. 2024-03-12T05:34:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-12T05:34:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-12T05:34:54.451 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: Cluster is now healthy 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/2980094939' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6836/2253333455,v1:172.21.15.161:6839/2253333455] up:boot 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6835/3251689231,v1:172.21.15.161:6838/3251689231] up:boot 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.161:6834/2943110252,v1:172.21.15.161:6837/2943110252] up:boot 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6835/481159515,v1:172.21.15.156:6839/481159515] up:boot 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6836/880713976,v1:172.21.15.156:6838/880713976] up:boot 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: mds.? [v2:172.21.15.156:6834/4080183965,v1:172.21.15.156:6837/4080183965] up:boot 2024-03-12T05:34:54.452 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: fsmap 12 up:standby 2024-03-12T05:34:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:34:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:34:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:34:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:34:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:34:54.453 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:34:54.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1636369429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:54.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:54 smithi156 ceph-mon[36270]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 232 KiB/s wr, 13 op/s 2024-03-12T05:34:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-12T05:34:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-12T05:34:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: Cluster is now healthy 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/2980094939' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6836/2253333455,v1:172.21.15.161:6839/2253333455] up:boot 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6835/3251689231,v1:172.21.15.161:6838/3251689231] up:boot 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.161:6834/2943110252,v1:172.21.15.161:6837/2943110252] up:boot 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6835/481159515,v1:172.21.15.156:6839/481159515] up:boot 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6836/880713976,v1:172.21.15.156:6838/880713976] up:boot 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: mds.? [v2:172.21.15.156:6834/4080183965,v1:172.21.15.156:6837/4080183965] up:boot 2024-03-12T05:34:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: fsmap 12 up:standby 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1636369429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:54.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:54 smithi100 ceph-mon[32210]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 232 KiB/s wr, 13 op/s 2024-03-12T05:34:54.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-12T05:34:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-12T05:34:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-12T05:34:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: Cluster is now healthy 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/2980094939' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6836/2253333455,v1:172.21.15.161:6839/2253333455] up:boot 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6835/3251689231,v1:172.21.15.161:6838/3251689231] up:boot 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.161:6834/2943110252,v1:172.21.15.161:6837/2943110252] up:boot 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6835/481159515,v1:172.21.15.156:6839/481159515] up:boot 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6836/880713976,v1:172.21.15.156:6838/880713976] up:boot 2024-03-12T05:34:54.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: mds.? [v2:172.21.15.156:6834/4080183965,v1:172.21.15.156:6837/4080183965] up:boot 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: fsmap 12 up:standby 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='mgr.15432 172.21.15.100:0/1533424291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1636369429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:54.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:54 smithi161 ceph-mon[36043]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 232 KiB/s wr, 13 op/s 2024-03-12T05:34:54.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3224705250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad00685d0 msgr2=0x7f7ad00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:54.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3224705250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad00685d0 0x7f7ad00689b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009930 tx=0x7f7ac002bc30 comp rx=0 tx=0).stop 2024-03-12T05:34:54.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3224705250 shutdown_connections 2024-03-12T05:34:54.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3224705250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad0106430 0x7f7ad010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3224705250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0068f80 0x7f7ad0105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3224705250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad00685d0 0x7f7ad00689b0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3224705250 >> 172.21.15.100:0/3224705250 conn(0x7f7ad0077180 msgr2=0x7f7ad0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3224705250 shutdown_connections 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.596+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3224705250 wait complete. 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 Processor -- start 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 -- start start 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad00685d0 0x7f7ad019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 0x7f7ad019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:54.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0197630 0x7f7ad0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0198100 con 0x7f7ad0106430 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0198270 con 0x7f7ad00685d0 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ad4edf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0198430 con 0x7f7ad0197630 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ac7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 0x7f7ad019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ace59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad00685d0 0x7f7ad019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ac7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 0x7f7ad019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47560/0 (socket says 172.21.15.100:47560) 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ac7fff640 1 -- 172.21.15.100:0/3933740501 learned_addr learned my addr 172.21.15.100:0/3933740501 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-12T05:34:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7aced9d640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0197630 0x7f7ad0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ace59c640 1 -- 172.21.15.100:0/3933740501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0197630 msgr2=0x7f7ad0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ace59c640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0197630 0x7f7ad0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ace59c640 1 -- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 msgr2=0x7f7ad019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ace59c640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 0x7f7ad019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ace59c640 1 -- 172.21.15.100:0/3933740501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac0009560 con 0x7f7ad00685d0 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.597+0000 7f7ac7fff640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 0x7f7ad019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.598+0000 7f7ace59c640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad00685d0 0x7f7ad019d570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009230 tx=0x7f7ac002c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:54.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.598+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00043d0 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.598+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ac00052d0 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.598+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad01a4370 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.598+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0003d00 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.598+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad01a4850 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.599+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad0102830 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.600+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7ac0005440 con 0x7f7ad00685d0 2024-03-12T05:34:54.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.600+0000 7f7ac77fe640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7abc0a1ba0 0x7f7abc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T05:34:54.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.601+0000 7f7ac7fff640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7abc0a1ba0 0x7f7abc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T05:34:54.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.601+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8303+0+0 (secure 0 0 0) 0x7f7ac00ef740 con 0x7f7ad00685d0 2024-03-12T05:34:54.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.601+0000 7f7ac7fff640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7abc0a1ba0 0x7f7abc0a4060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0198e80 tx=0x7f7ab800a300 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T05:34:54.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.604+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ac00b8920 con 0x7f7ad00685d0 2024-03-12T05:34:54.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:54.780+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ad0109660 con 0x7f7ad00685d0 2024-03-12T05:34:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.444+0000 7f7ac77fe640 1 -- 172.21.15.100:0/3933740501 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7ac0043020 con 0x7f7ad00685d0 2024-03-12T05:34:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-03-12T05:34:55.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7abc0a1ba0 msgr2=0x7f7abc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:55.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7abc0a1ba0 0x7f7abc0a4060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0198e80 tx=0x7f7ab800a300 comp rx=0 tx=0).stop 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad00685d0 msgr2=0x7f7ad019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad00685d0 0x7f7ad019d570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009230 tx=0x7f7ac002c710 comp rx=0 tx=0).stop 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 shutdown_connections 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:6800/2025021937,v1:172.21.15.100:6801/2025021937] conn(0x7f7abc0a1ba0 0x7f7abc0a4060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0197630 0x7f7ad0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad0106430 0x7f7ad019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 --2- 172.21.15.100:0/3933740501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ad00685d0 0x7f7ad019d570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T05:34:55.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 >> 172.21.15.100:0/3933740501 conn(0x7f7ad0077180 msgr2=0x7f7ad00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T05:34:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 shutdown_connections 2024-03-12T05:34:55.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-12T05:34:55.448+0000 7f7ad4edf640 1 -- 172.21.15.100:0/3933740501 wait complete. 2024-03-12T05:34:55.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:34:55.497 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:34:55.538 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:34:55.592 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-12T05:34:55.592 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T05:34:55.607 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T05:34:55.622 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T05:34:55.637 INFO:tasks.cephadm:Stopping all daemons... 2024-03-12T05:34:55.637 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-12T05:34:55.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a 2024-03-12T05:34:55.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/1636369429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-12T05:34:55.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:55 smithi156 ceph-mon[36270]: osdmap e123: 12 total, 12 up, 12 in 2024-03-12T05:34:55.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:55 smithi156 ceph-mon[36270]: from='client.? 172.21.15.100:0/3933740501' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:55.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:55 smithi156 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:55.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/1636369429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-12T05:34:55.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:55 smithi161 ceph-mon[36043]: osdmap e123: 12 total, 12 up, 12 in 2024-03-12T05:34:55.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:55 smithi161 ceph-mon[36043]: from='client.? 172.21.15.100:0/3933740501' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:55.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:55 smithi161 ceph-mon[36043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:55.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/1636369429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-12T05:34:55.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 ceph-mon[32210]: osdmap e123: 12 total, 12 up, 12 in 2024-03-12T05:34:55.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 ceph-mon[32210]: from='client.? 172.21.15.100:0/3933740501' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:55.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 ceph-mon[32210]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-12T05:34:55.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 systemd[1]: Stopping Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:34:56.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a[32206]: 2024-03-12T05:34:55.960+0000 7f706ada5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:34:56.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a[32206]: 2024-03-12T05:34:55.960+0000 7f706ada5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-12T05:34:56.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 podman[108751]: 2024-03-12 05:34:56.165965749 +0000 UTC m=+0.330614656 container died 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-03-12T05:34:56.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 podman[108751]: 2024-03-12 05:34:56.505589009 +0000 UTC m=+0.670237907 container cleanup 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-12T05:34:56.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 bash[108751]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a 2024-03-12T05:34:56.913 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service' 2024-03-12T05:34:56.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 podman[108765]: 2024-03-12 05:34:56.681759783 +0000 UTC m=+0.509309835 container remove 3a78a7de34afb0ac7a4978da97905e53c4f6a144c9ee5f71ffdbd7c2f57b2161 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-12T05:34:56.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-12T05:34:56.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 systemd[1]: Stopped Ceph mon.a for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:34:56.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Mar 12 05:34:56 smithi100 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.a.service: Consumed 42.592s CPU time. 2024-03-12T05:34:56.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:34:56.967 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-12T05:34:56.967 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-12T05:34:56.967 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.b 2024-03-12T05:34:57.307 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:57 smithi156 systemd[1]: Stopping Ceph mon.b for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:34:57.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-b[36266]: 2024-03-12T05:34:57.314+0000 7f805baf1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:34:57.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-b[36266]: 2024-03-12T05:34:57.314+0000 7f805baf1700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-12T05:34:57.702 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:57 smithi156 podman[75445]: 2024-03-12 05:34:57.508141243 +0000 UTC m=+0.292534008 container died 4d7eda35f161c5fa59bb9a08e5224901c0fbe0b3d3e28c1e27154cd7367a9f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-12T05:34:58.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:57 smithi156 podman[75445]: 2024-03-12 05:34:57.964313318 +0000 UTC m=+0.748706083 container cleanup 4d7eda35f161c5fa59bb9a08e5224901c0fbe0b3d3e28c1e27154cd7367a9f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:34:58.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 12 05:34:57 smithi156 bash[75445]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-b 2024-03-12T05:34:58.384 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.b.service' 2024-03-12T05:34:58.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:34:58.454 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-12T05:34:58.455 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-12T05:34:58.455 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.c 2024-03-12T05:34:58.806 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:58 smithi161 systemd[1]: Stopping Ceph mon.c for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:34:59.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c[36039]: 2024-03-12T05:34:58.812+0000 7f69f37f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:34:59.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c[36039]: 2024-03-12T05:34:58.813+0000 7f69f37f8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-12T05:34:59.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:59 smithi161 podman[75729]: 2024-03-12 05:34:59.038102131 +0000 UTC m=+0.346147850 container died 72c9153137b7742899fc2ad622863835b350570d3347af46c109748d2d19804c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-12T05:34:59.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:59 smithi161 podman[75729]: 2024-03-12 05:34:59.496410378 +0000 UTC m=+0.804456107 container cleanup 72c9153137b7742899fc2ad622863835b350570d3347af46c109748d2d19804c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-12T05:34:59.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 12 05:34:59 smithi161 bash[75729]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mon-c 2024-03-12T05:34:59.904 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mon.c.service' 2024-03-12T05:34:59.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:34:59.953 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-12T05:34:59.954 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-12T05:34:59.954 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x 2024-03-12T05:35:00.270 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:35:00 smithi100 systemd[1]: Stopping Ceph mgr.x for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:00.564 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:35:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:35:00.275+0000 7fc55848e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:00.564 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:35:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x[32441]: 2024-03-12T05:35:00.275+0000 7fc55848e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-12T05:35:00.565 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:35:00 smithi100 podman[108873]: 2024-03-12 05:35:00.30806259 +0000 UTC m=+0.154346257 container died 481c1b521344bdcbaeaf670c27a9fee1dfc50dd0da062a20d80f3400edb5ad8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397) 2024-03-12T05:35:00.930 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:35:00 smithi100 podman[108873]: 2024-03-12 05:35:00.686306278 +0000 UTC m=+0.532589997 container cleanup 481c1b521344bdcbaeaf670c27a9fee1dfc50dd0da062a20d80f3400edb5ad8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:35:00.931 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Mar 12 05:35:00 smithi100 bash[108873]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-x 2024-03-12T05:35:01.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.x.service' 2024-03-12T05:35:01.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:01.091 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-12T05:35:01.092 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-12T05:35:01.092 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.y 2024-03-12T05:35:01.449 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:35:01 smithi156 systemd[1]: Stopping Ceph mgr.y for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:01.701 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:35:01 smithi156 podman[75579]: 2024-03-12 05:35:01.456095281 +0000 UTC m=+0.148288359 container died 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-12T05:35:02.134 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:35:01 smithi156 podman[75579]: 2024-03-12 05:35:01.793693558 +0000 UTC m=+0.485886685 container cleanup 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:35:02.135 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:35:01 smithi156 bash[75579]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y 2024-03-12T05:35:02.135 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:35:01 smithi156 podman[75593]: 2024-03-12 05:35:01.961196669 +0000 UTC m=+0.494707457 container remove 8c9a0e5552445d4b064d9cc55d94279f3add01f65a58ab4f9c9c34b0047c8047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:35:02.135 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 12 05:35:01 smithi156 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-12T05:35:02.156 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.y.service' 2024-03-12T05:35:02.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:02.195 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-12T05:35:02.195 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-12T05:35:02.196 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.z 2024-03-12T05:35:02.531 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:35:02 smithi161 systemd[1]: Stopping Ceph mgr.z for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:02.819 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:35:02 smithi161 podman[75851]: 2024-03-12 05:35:02.538172453 +0000 UTC m=+0.142053257 container died a29f183d81ad0e4e2cda2d0360e755f9cba7c094ead92839f9f048b596564a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-12T05:35:03.137 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:35:02 smithi161 podman[75851]: 2024-03-12 05:35:02.926770621 +0000 UTC m=+0.530651390 container cleanup a29f183d81ad0e4e2cda2d0360e755f9cba7c094ead92839f9f048b596564a7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-12T05:35:03.137 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 12 05:35:02 smithi161 bash[75851]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mgr-z 2024-03-12T05:35:03.270 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mgr.z.service' 2024-03-12T05:35:03.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:03.308 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-12T05:35:03.309 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-12T05:35:03.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.0 2024-03-12T05:35:03.609 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:03 smithi100 systemd[1]: Stopping Ceph osd.0 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:04.064 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0[44602]: 2024-03-12T05:35:03.615+0000 7f0878e08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:04.064 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0[44602]: 2024-03-12T05:35:03.615+0000 7f0878e08700 -1 osd.0 124 *** Got signal Terminated *** 2024-03-12T05:35:04.065 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0[44602]: 2024-03-12T05:35:03.615+0000 7f0878e08700 -1 osd.0 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:09.501 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:09 smithi100 podman[108995]: 2024-03-12 05:35:09.230179445 +0000 UTC m=+5.721213052 container died a0dda6e7fb638a50e94c42f627a3ce90dc5836fab429d1d86210fc2bc8343d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-12T05:35:09.814 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:09 smithi100 podman[108995]: 2024-03-12 05:35:09.581868223 +0000 UTC m=+6.072901836 container cleanup a0dda6e7fb638a50e94c42f627a3ce90dc5836fab429d1d86210fc2bc8343d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-12T05:35:09.815 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:09 smithi100 bash[108995]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0 2024-03-12T05:35:10.193 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:09 smithi100 podman[109013]: 2024-03-12 05:35:09.858395826 +0000 UTC m=+0.626297514 container remove a0dda6e7fb638a50e94c42f627a3ce90dc5836fab429d1d86210fc2bc8343d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-12T05:35:10.443 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:10 smithi100 podman[109105]: 2024-03-12 05:35:10.1031266 +0000 UTC m=+0.017658499 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:10.443 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:10 smithi100 podman[109105]: 2024-03-12 05:35:10.234254197 +0000 UTC m=+0.148786084 container create ce4239fc6f2fb888b8a3092de08e2609ccef978c1766350fd4037629cfda43d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:10.697 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:10 smithi100 podman[109105]: 2024-03-12 05:35:10.612254787 +0000 UTC m=+0.526786679 container init ce4239fc6f2fb888b8a3092de08e2609ccef978c1766350fd4037629cfda43d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-12T05:35:10.697 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:10 smithi100 podman[109105]: 2024-03-12 05:35:10.615862328 +0000 UTC m=+0.530394216 container start ce4239fc6f2fb888b8a3092de08e2609ccef978c1766350fd4037629cfda43d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-12T05:35:11.064 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:10 smithi100 podman[109105]: 2024-03-12 05:35:10.703914574 +0000 UTC m=+0.618446493 container attach ce4239fc6f2fb888b8a3092de08e2609ccef978c1766350fd4037629cfda43d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-12T05:35:11.065 INFO:journalctl@ceph.osd.0.smithi100.stdout:Mar 12 05:35:10 smithi100 podman[109105]: 2024-03-12 05:35:10.902729849 +0000 UTC m=+0.817261728 container died ce4239fc6f2fb888b8a3092de08e2609ccef978c1766350fd4037629cfda43d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-0-deactivate, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-12T05:35:11.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.0.service' 2024-03-12T05:35:11.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:11.597 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-12T05:35:11.598 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-12T05:35:11.598 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.1 2024-03-12T05:35:11.924 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:11 smithi156 systemd[1]: Stopping Ceph osd.1 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:12.202 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1[40892]: 2024-03-12T05:35:11.929+0000 7f0ddcc4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:12.202 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1[40892]: 2024-03-12T05:35:11.929+0000 7f0ddcc4c700 -1 osd.1 124 *** Got signal Terminated *** 2024-03-12T05:35:12.202 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1[40892]: 2024-03-12T05:35:11.929+0000 7f0ddcc4c700 -1 osd.1 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:17.758 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:17 smithi156 podman[75702]: 2024-03-12 05:35:17.475867099 +0000 UTC m=+5.671430250 container died bce56cfd8fe3187ad7b3a8d5e0dd01352bece68a1beb5d353c6ce5092e90e843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-12T05:35:18.108 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:17 smithi156 podman[75702]: 2024-03-12 05:35:17.838349458 +0000 UTC m=+6.033912611 container cleanup bce56cfd8fe3187ad7b3a8d5e0dd01352bece68a1beb5d353c6ce5092e90e843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-12T05:35:18.108 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:17 smithi156 bash[75702]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1 2024-03-12T05:35:18.419 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:18 smithi156 podman[75719]: 2024-03-12 05:35:18.114654497 +0000 UTC m=+0.637135306 container remove bce56cfd8fe3187ad7b3a8d5e0dd01352bece68a1beb5d353c6ce5092e90e843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-12T05:35:18.672 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:18 smithi156 podman[75810]: 2024-03-12 05:35:18.328992177 +0000 UTC m=+0.017678313 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:18.673 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:18 smithi156 podman[75810]: 2024-03-12 05:35:18.465477153 +0000 UTC m=+0.154163314 container create 05223fc15bd78e9ab61c3d0b1476bc3cd5db2fcf86d35b9af4e56a4ee55c9aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-12T05:35:18.673 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:18 smithi156 podman[75810]: 2024-03-12 05:35:18.676141337 +0000 UTC m=+0.364827458 container init 05223fc15bd78e9ab61c3d0b1476bc3cd5db2fcf86d35b9af4e56a4ee55c9aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-12T05:35:18.673 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:18 smithi156 podman[75810]: 2024-03-12 05:35:18.679231074 +0000 UTC m=+0.367917197 container start 05223fc15bd78e9ab61c3d0b1476bc3cd5db2fcf86d35b9af4e56a4ee55c9aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-03-12T05:35:18.952 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:18 smithi156 podman[75810]: 2024-03-12 05:35:18.834493506 +0000 UTC m=+0.523179651 container attach 05223fc15bd78e9ab61c3d0b1476bc3cd5db2fcf86d35b9af4e56a4ee55c9aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-12T05:35:19.549 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 12 05:35:19 smithi156 podman[75810]: 2024-03-12 05:35:19.291268559 +0000 UTC m=+0.979954689 container died 05223fc15bd78e9ab61c3d0b1476bc3cd5db2fcf86d35b9af4e56a4ee55c9aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-12T05:35:19.894 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.1.service' 2024-03-12T05:35:19.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:19.937 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-12T05:35:19.937 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-12T05:35:19.937 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.2 2024-03-12T05:35:20.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:19 smithi161 systemd[1]: Stopping Ceph osd.2 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:20.557 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2[41374]: 2024-03-12T05:35:20.288+0000 7f312d521700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:20.557 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2[41374]: 2024-03-12T05:35:20.288+0000 7f312d521700 -1 osd.2 124 *** Got signal Terminated *** 2024-03-12T05:35:20.557 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2[41374]: 2024-03-12T05:35:20.288+0000 7f312d521700 -1 osd.2 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:26.033 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:25 smithi161 podman[75974]: 2024-03-12 05:35:25.783079407 +0000 UTC m=+5.637762642 container died 5298349a5efcc290cef405cbb57a5446c345d272c8af7be197ef597da3635288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:26.399 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 podman[75974]: 2024-03-12 05:35:26.130143007 +0000 UTC m=+5.984826242 container cleanup 5298349a5efcc290cef405cbb57a5446c345d272c8af7be197ef597da3635288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-12T05:35:26.399 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 bash[75974]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2 2024-03-12T05:35:26.720 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 podman[76003]: 2024-03-12 05:35:26.406254542 +0000 UTC m=+0.621130374 container remove 5298349a5efcc290cef405cbb57a5446c345d272c8af7be197ef597da3635288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-12T05:35:26.994 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 podman[76090]: 2024-03-12 05:35:26.732009144 +0000 UTC m=+0.109853756 container create 5a7861d3b0b11fef236fcff1672f3eabb15ace126e805aaddeb7f8662dc3ec32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-12T05:35:26.995 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 podman[76090]: 2024-03-12 05:35:26.639561647 +0000 UTC m=+0.017406262 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:26.995 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 podman[76090]: 2024-03-12 05:35:26.926024458 +0000 UTC m=+0.303869071 container init 5a7861d3b0b11fef236fcff1672f3eabb15ace126e805aaddeb7f8662dc3ec32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-12T05:35:26.995 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:26 smithi161 podman[76090]: 2024-03-12 05:35:26.929274591 +0000 UTC m=+0.307119199 container start 5a7861d3b0b11fef236fcff1672f3eabb15ace126e805aaddeb7f8662dc3ec32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-12T05:35:26.995 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:27 smithi161 podman[76090]: 2024-03-12 05:35:27.000951909 +0000 UTC m=+0.378796552 container attach 5a7861d3b0b11fef236fcff1672f3eabb15ace126e805aaddeb7f8662dc3ec32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2-deactivate, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-12T05:35:27.614 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 12 05:35:27 smithi161 podman[76090]: 2024-03-12 05:35:27.310856654 +0000 UTC m=+0.688701266 container died 5a7861d3b0b11fef236fcff1672f3eabb15ace126e805aaddeb7f8662dc3ec32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-12T05:35:27.898 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.2.service' 2024-03-12T05:35:27.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:27.940 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-12T05:35:27.941 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-12T05:35:27.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.3 2024-03-12T05:35:28.236 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:27 smithi100 systemd[1]: Stopping Ceph osd.3 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:28.564 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:28 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:35:28.241+0000 7f28182ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:28.565 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:28 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:35:28.241+0000 7f28182ea700 -1 osd.3 124 *** Got signal Terminated *** 2024-03-12T05:35:28.565 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:28 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:35:28.241+0000 7f28182ea700 -1 osd.3 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:31.550 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:35:31.217+0000 7f28110d7700 -1 osd.3 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:04.712852+0000 front 2024-03-12T05:35:04.712827+0000 (oldest deadline 2024-03-12T05:35:30.612961+0000) 2024-03-12T05:35:31.551 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:31.298+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:31.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:31 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:31.556+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:31.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:31 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:31.563+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:31.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:31.725+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:32.509 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:32.278+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:32.509 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:35:32.237+0000 7f28110d7700 -1 osd.3 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:04.712852+0000 front 2024-03-12T05:35:04.712827+0000 (oldest deadline 2024-03-12T05:35:30.612961+0000) 2024-03-12T05:35:32.814 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:32 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:32.514+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:32.951 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:32 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:32.610+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:32.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:32.752+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:32.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:32.723+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:33.476 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:33 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3[51605]: 2024-03-12T05:35:33.220+0000 7f28110d7700 -1 osd.3 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:04.712852+0000 front 2024-03-12T05:35:04.712827+0000 (oldest deadline 2024-03-12T05:35:30.612961+0000) 2024-03-12T05:35:33.477 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:33 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:33.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:33.815 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:33 smithi100 podman[109214]: 2024-03-12 05:35:33.645636971 +0000 UTC m=+5.506163231 container died bda9dc92dea3cced0b99ef1f90f025b9706298233602e9ab716c7efde2891de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-12T05:35:33.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:33 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:33.518+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:33.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:33 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:33.602+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:34.178 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:33 smithi100 podman[109214]: 2024-03-12 05:35:33.942625522 +0000 UTC m=+5.803151824 container cleanup bda9dc92dea3cced0b99ef1f90f025b9706298233602e9ab716c7efde2891de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-12T05:35:34.179 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:33 smithi100 bash[109214]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3 2024-03-12T05:35:34.221 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:33.761+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:34.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:33.731+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:34.503 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:34 smithi100 podman[109246]: 2024-03-12 05:35:34.185387931 +0000 UTC m=+0.536100837 container remove bda9dc92dea3cced0b99ef1f90f025b9706298233602e9ab716c7efde2891de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2) 2024-03-12T05:35:34.503 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:34.233+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:34.765 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:34 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:34.543+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:34.766 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:34 smithi100 podman[109337]: 2024-03-12 05:35:34.412176125 +0000 UTC m=+0.017636589 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:34.766 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:34 smithi100 podman[109337]: 2024-03-12 05:35:34.527905621 +0000 UTC m=+0.133366064 container create f40966d8f697e0411a46c3f468ad2139f261a6e255be88dcf398c551ff0faf49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-12T05:35:34.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:34 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:34.647+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:34.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:34.750+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:34.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:34.712+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:35.056 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:34 smithi100 podman[109337]: 2024-03-12 05:35:34.772110222 +0000 UTC m=+0.377570667 container init f40966d8f697e0411a46c3f468ad2139f261a6e255be88dcf398c551ff0faf49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-12T05:35:35.056 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:34 smithi100 podman[109337]: 2024-03-12 05:35:34.775285108 +0000 UTC m=+0.380745555 container start f40966d8f697e0411a46c3f468ad2139f261a6e255be88dcf398c551ff0faf49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-12T05:35:35.056 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:34 smithi100 podman[109337]: 2024-03-12 05:35:34.839056835 +0000 UTC m=+0.444517293 container attach f40966d8f697e0411a46c3f468ad2139f261a6e255be88dcf398c551ff0faf49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-12T05:35:35.315 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:35 smithi100 podman[109337]: 2024-03-12 05:35:35.062511695 +0000 UTC m=+0.667972143 container died f40966d8f697e0411a46c3f468ad2139f261a6e255be88dcf398c551ff0faf49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-12T05:35:35.315 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:35 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:35.214+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:35.603 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 12 05:35:35 smithi100 podman[109363]: 2024-03-12 05:35:35.592578201 +0000 UTC m=+0.521474745 container remove f40966d8f697e0411a46c3f468ad2139f261a6e255be88dcf398c551ff0faf49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-12T05:35:35.604 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:35 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:35.537+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:35.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.3.service' 2024-03-12T05:35:35.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:35.653 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-12T05:35:35.653 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-12T05:35:35.654 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.4 2024-03-12T05:35:35.950 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:35 smithi156 systemd[1]: Stopping Ceph osd.4 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:35.951 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:35 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:35.627+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:35.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:35.709+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:35.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:35.668+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:36.202 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:35 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4[47631]: 2024-03-12T05:35:35.955+0000 7f8721a5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:36.202 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:35 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4[47631]: 2024-03-12T05:35:35.955+0000 7f8721a5e700 -1 osd.4 124 *** Got signal Terminated *** 2024-03-12T05:35:36.202 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:35 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4[47631]: 2024-03-12T05:35:35.955+0000 7f8721a5e700 -1 osd.4 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:36.510 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:36 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:36.219+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:36.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:36 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:36.516+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:36.951 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:36 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:36.612+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:36.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:36 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:36.715+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:36.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:36 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:36.636+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:37.496 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:37 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:37.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:37.673 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:37 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:37.594+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:37.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:37 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:37.502+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:37.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:37 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:37.588+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:37.969 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:37 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:37.679+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:38.491 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:38 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:38.241+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:38.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:38 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:38.497+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:38.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:38 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:38.543+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:38.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:38 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:38.543+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:38.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:38 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:38.562+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:38.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:38 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:38.664+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:38.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:38 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:38.608+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:38.971 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:38 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:38.767+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:39.497 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:39 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:39.212+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:39.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:39 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:39.503+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:39.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:39 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:39.503+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:39.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:39 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:39.594+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:39.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:39 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:39.535+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:39.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:39 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:39.535+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:39.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:39 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:39.741+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:39.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:39 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:39.633+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:39.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:39 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:39.675+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:39.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:39 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:39.675+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:40.504 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:40 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:40.245+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:40.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:40 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:40.510+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:40.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:40 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:40.510+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:40.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:40 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:40.573+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:40.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:40 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:40.574+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:40.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:40 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:40.574+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:40.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:40 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:40.729+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:40.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:40 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:40.652+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:40.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:40 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:40.642+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:40.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:40 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:40.642+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:41.550 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:41 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:41.242+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:41.551 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:41 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:41.242+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:41.749 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:41 smithi156 podman[75928]: 2024-03-12 05:35:41.482540497 +0000 UTC m=+5.624645786 container died 4d39381bde0abca8a65f07c4f2e7c95f4a1c637ea58db21bfad260f44b85d9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:41.749 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:41 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:41.575+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:41.750 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:41 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:41.588+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:41.750 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:41 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:41.588+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:41.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:41 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:41.556+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:41.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:41 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:41.556+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:41.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:41 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:41.707+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:41.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:41 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:41.646+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:41.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:41 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:41.613+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:41.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:41 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:41.613+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:42.009 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:41 smithi156 podman[75928]: 2024-03-12 05:35:41.755735021 +0000 UTC m=+5.897840316 container cleanup 4d39381bde0abca8a65f07c4f2e7c95f4a1c637ea58db21bfad260f44b85d9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-12T05:35:42.009 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:41 smithi156 bash[75928]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4 2024-03-12T05:35:42.364 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[75945]: 2024-03-12 05:35:42.015591423 +0000 UTC m=+0.527250154 container remove 4d39381bde0abca8a65f07c4f2e7c95f4a1c637ea58db21bfad260f44b85d9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:42.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:42.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:42.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:42.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:42.625 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:42 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:42.534+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:42.626 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[76032]: 2024-03-12 05:35:42.273379214 +0000 UTC m=+0.017651803 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:42.626 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[76032]: 2024-03-12 05:35:42.382965067 +0000 UTC m=+0.127237637 container create 52ff417c77140997114feae8da736957bb7467d9bb43326fa7238766d52c6f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-12T05:35:42.921 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:42 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:42.632+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:42.921 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:42 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:42.632+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:42.921 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[76032]: 2024-03-12 05:35:42.635366579 +0000 UTC m=+0.379639160 container init 52ff417c77140997114feae8da736957bb7467d9bb43326fa7238766d52c6f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-12T05:35:42.922 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[76032]: 2024-03-12 05:35:42.638748922 +0000 UTC m=+0.383021495 container start 52ff417c77140997114feae8da736957bb7467d9bb43326fa7238766d52c6f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-12T05:35:42.922 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[76032]: 2024-03-12 05:35:42.726907698 +0000 UTC m=+0.471180285 container attach 52ff417c77140997114feae8da736957bb7467d9bb43326fa7238766d52c6f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-12T05:35:42.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:42 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:42.724+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:42.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:42 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:42.587+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:42.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:42 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:42.587+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:42.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:42 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:42.595+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:42.976 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:42.603+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:42.976 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:42 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:42.603+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:43.189 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:42 smithi156 podman[76032]: 2024-03-12 05:35:42.927648731 +0000 UTC m=+0.671921314 container died 52ff417c77140997114feae8da736957bb7467d9bb43326fa7238766d52c6f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-12T05:35:43.441 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 12 05:35:43 smithi156 podman[76058]: 2024-03-12 05:35:43.429904877 +0000 UTC m=+0.496781210 container remove 52ff417c77140997114feae8da736957bb7467d9bb43326fa7238766d52c6f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-12T05:35:43.446 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.4.service' 2024-03-12T05:35:43.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:43.486 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-12T05:35:43.486 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-12T05:35:43.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.5 2024-03-12T05:35:43.554 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:43.288+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:43.554 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:43.288+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:43.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:43 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:43.505+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:43.702 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:43 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:43.629+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:43.702 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:43 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:43.629+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:43.792 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:43.728+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:43.793 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:43.591+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:43.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:43 smithi161 systemd[1]: Stopping Ceph osd.5 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:43.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:43.595+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:43.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:43.595+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:43.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:43.798+0000 7f442e573700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:43.793 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:43.798+0000 7f442e573700 -1 osd.5 124 *** Got signal Terminated *** 2024-03-12T05:35:43.794 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:43 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:43.798+0000 7f442e573700 -1 osd.5 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:43.814 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:43.593+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:43.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:43.593+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:44.560 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:44 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:44.277+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:44.560 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:44 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:44.277+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:44.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:44 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:44.565+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:44.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:44 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:44.565+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:44.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:44 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:44.480+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:44.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:44 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:44.639+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:44.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:44 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:44.639+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:44.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:44 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:44.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:44.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:44 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:44.697+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:44.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:44 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:44.638+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:44.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:44 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:44.638+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:45.524 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:45 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:45.271+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:45.524 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:45 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:45.271+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:45.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:45 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:45.530+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:45.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:45 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:45.530+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:45.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:45 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:45.529+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:45.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:45 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:45.629+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:45.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:45 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:45.629+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:45.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:45 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:45.601+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:45.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:45 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:45.685+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:45.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:45 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:45.687+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:45.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:45 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:45.687+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:46.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:46 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:46.277+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:46.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:46 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:46.277+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:46.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:46 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:46.502+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:46.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:46 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:46.657+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:46.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:46 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:46.657+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:46.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:46 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:46.729+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:46.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:46 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:46.648+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:46.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:46 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:46.700+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:46.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:46 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:46.700+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:47.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:46 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:46.573+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:47.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:46 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:46.573+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:47.550 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:47 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:47.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:47.551 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:47 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:47.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:47.551 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:47 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:47.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:47.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:47 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:47.556+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:47.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:47 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:47.556+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:47.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:47 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:47.556+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:47.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:47 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:47.472+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:47.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:47 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:47.699+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:47.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:47 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:47.699+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:47.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:47 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:47.699+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:47.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:47 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:47.679+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:47.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:47 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:47.667+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:47.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:47 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:47.667+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:47.971 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:47 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:47.765+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:48.553 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:48 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:48.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:48.553 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:48 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:48.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:48.553 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:48 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:48.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:48.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:48 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:48.559+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:48.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:48 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:48.559+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:48.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:48 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:48.559+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:48.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:48 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:48.517+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:48.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:48 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:48.517+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:48.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:48 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:48.733+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:48.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:48 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:48.733+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:48.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:48 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:48.733+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:48.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:48 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:48.773+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:48.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:48 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:48.655+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:48.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:48 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:48.678+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:07.236602+0000 front 2024-03-12T05:35:07.236449+0000 (oldest deadline 2024-03-12T05:35:32.536480+0000) 2024-03-12T05:35:48.971 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:48 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5[47949]: 2024-03-12T05:35:48.678+0000 7f442b368700 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.037590+0000 front 2024-03-12T05:35:13.037688+0000 (oldest deadline 2024-03-12T05:35:38.838370+0000) 2024-03-12T05:35:49.470 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:49 smithi161 podman[76195]: 2024-03-12 05:35:49.223701166 +0000 UTC m=+5.539349235 container stop b5132ae8050cdb2846c92c0405feef60ab07a134dc3893d34e37c041c38b689a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:35:49.471 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:49 smithi161 podman[76195]: 2024-03-12 05:35:49.253010687 +0000 UTC m=+5.568658747 container died b5132ae8050cdb2846c92c0405feef60ab07a134dc3893d34e37c041c38b689a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-12T05:35:49.530 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:49.214+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:49.530 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:49.214+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:49.531 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:49.214+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:49.746 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:49 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:49.482+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:49.747 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:49 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:49.482+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:49.750 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:49 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:49.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:49.751 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:49 smithi161 podman[76195]: 2024-03-12 05:35:49.556010044 +0000 UTC m=+5.871658093 container cleanup b5132ae8050cdb2846c92c0405feef60ab07a134dc3893d34e37c041c38b689a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-12T05:35:49.751 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:49 smithi161 bash[76195]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5 2024-03-12T05:35:49.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:49.536+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:49.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:49.536+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:49.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:49.536+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:50.154 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:49 smithi161 podman[76213]: 2024-03-12 05:35:49.757089216 +0000 UTC m=+0.526716649 container remove b5132ae8050cdb2846c92c0405feef60ab07a134dc3893d34e37c041c38b689a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-12T05:35:50.154 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:49 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:49.816+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:50.154 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:49 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:49.816+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:50.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:49 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:49.751+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:50.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:49 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:49.751+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:50.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:49 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:49.751+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:50.405 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:50 smithi161 podman[76308]: 2024-03-12 05:35:50.062965682 +0000 UTC m=+0.018013447 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:50.405 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:50 smithi161 podman[76308]: 2024-03-12 05:35:50.18307471 +0000 UTC m=+0.138122530 container create caf6fc20b22435c2b39a790804639b37bf3857d3eb50222c5b75ef3ae9bc1597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:50.405 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:50 smithi161 podman[76308]: 2024-03-12 05:35:50.4107894 +0000 UTC m=+0.365837167 container init caf6fc20b22435c2b39a790804639b37bf3857d3eb50222c5b75ef3ae9bc1597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True) 2024-03-12T05:35:50.553 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:50 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:50.250+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:50.554 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:50 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:50.250+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:50.554 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:50 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:50.250+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:50.697 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:50 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:50.644+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:50.698 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:50 smithi161 podman[76308]: 2024-03-12 05:35:50.413963967 +0000 UTC m=+0.369011717 container start caf6fc20b22435c2b39a790804639b37bf3857d3eb50222c5b75ef3ae9bc1597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-12T05:35:50.698 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:50 smithi161 podman[76308]: 2024-03-12 05:35:50.502066876 +0000 UTC m=+0.457114662 container attach caf6fc20b22435c2b39a790804639b37bf3857d3eb50222c5b75ef3ae9bc1597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:50.778 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:50 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:50.532+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:50.778 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:50 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:50.532+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:50.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:50 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:50.559+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:50.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:50 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:50.559+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:50.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:50 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:50.559+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:50.970 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:50 smithi161 podman[76308]: 2024-03-12 05:35:50.704283368 +0000 UTC m=+0.659331117 container died caf6fc20b22435c2b39a790804639b37bf3857d3eb50222c5b75ef3ae9bc1597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:35:50.971 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:50 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:50.806+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:50.971 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:50 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:50.806+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:51.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:50 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:50.783+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:51.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:50 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:50.783+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:51.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:50 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:50.783+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:51.236 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.5.service' 2024-03-12T05:35:51.260 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:51 smithi161 podman[76334]: 2024-03-12 05:35:51.213736429 +0000 UTC m=+0.503738413 container remove caf6fc20b22435c2b39a790804639b37bf3857d3eb50222c5b75ef3ae9bc1597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-12T05:35:51.261 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:51 smithi161 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-12T05:35:51.261 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:51 smithi161 systemd[1]: Stopped Ceph osd.5 for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:35:51.261 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 12 05:35:51 smithi161 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.5.service: Consumed 5min 49.805s CPU time. 2024-03-12T05:35:51.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:51.277 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-12T05:35:51.277 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-12T05:35:51.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.6 2024-03-12T05:35:51.563 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 systemd[1]: Stopping Ceph osd.6 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:51.564 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:51.204+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:51.564 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:51.204+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:51.564 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:51.204+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:51.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:51.569+0000 7f2a54615700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:51.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:51.569+0000 7f2a54615700 -1 osd.6 124 *** Got signal Terminated *** 2024-03-12T05:35:51.816 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:51.569+0000 7f2a54615700 -1 osd.6 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:51.816 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:51.583+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:51.816 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:51.583+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:51.816 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:51 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:51.583+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:51.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:51 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:51.573+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:51.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:51 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:51.573+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:51.952 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:51 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:51.782+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:51.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:51 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:51.782+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:51.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:51 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:51.782+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:51.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:51 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:51.815+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:51.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:51 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:51.815+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:51.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:51 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:51.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:51.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:51 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:51.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:52.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:52.166+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:52.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:52.166+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:52.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:52.166+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:52.720 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:52.573+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:52.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:52.573+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:52.824 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:52.529+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:52.825 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:52.529+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:53.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:52.628+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:53.066 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:52.628+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:53.066 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:52 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:52.628+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:53.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:52.830+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:53.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:52.830+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:53.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:52 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:52.830+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:53.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:52.781+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:53.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:52 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:52.781+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:53.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:53.167+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:53.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:53.167+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:53.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:53.167+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:53.850 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:53 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:53.480+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:53.850 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:53 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:53.480+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:53.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:53 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:53.737+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:53.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:53 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:53.737+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:53.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:53 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:53.570+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:53.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:53 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:53.570+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:54.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:53.640+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:54.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:53.640+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:54.066 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:53.640+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:54.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:53 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:53.853+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:54.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:53 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:53.853+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:54.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:53 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:53.853+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:54.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:54.152+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:54.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:54.152+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:54.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:54.152+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:54.839 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:54 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:54.477+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:54.839 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:54 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:54.477+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:54.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:54 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:54.781+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:54.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:54 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:54.781+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:54.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:54 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:54.596+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:54.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:54 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:54.596+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:55.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:54.671+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:55.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:54.671+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:55.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:54.671+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:55.066 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:54 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:54.671+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:28.029597+0000 front 2024-03-12T05:35:28.029684+0000 (oldest deadline 2024-03-12T05:35:53.929733+0000) 2024-03-12T05:35:55.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:54 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:54.845+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:55.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:54 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:54.845+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:55.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:54 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:54.845+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:55.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:54 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:54.845+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:35:55.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:55.190+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:55.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:55.190+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:55.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:55.190+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:55.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:55 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:55.455+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:55.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:55 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:55.455+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:55.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:55 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:55.740+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:55.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:55 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:55.740+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:55.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:55 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:55.599+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:55.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:55 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:55.599+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:56.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:55.678+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:08.324838+0000 front 2024-03-12T05:35:08.324885+0000 (oldest deadline 2024-03-12T05:35:30.625007+0000) 2024-03-12T05:35:56.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:55.678+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:13.526333+0000 front 2024-03-12T05:35:13.526197+0000 (oldest deadline 2024-03-12T05:35:38.826216+0000) 2024-03-12T05:35:56.065 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:55.678+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.428308+0000 front 2024-03-12T05:35:23.428252+0000 (oldest deadline 2024-03-12T05:35:47.528324+0000) 2024-03-12T05:35:56.066 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:55 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6[58652]: 2024-03-12T05:35:55.678+0000 7f2a4d402700 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:28.029597+0000 front 2024-03-12T05:35:28.029684+0000 (oldest deadline 2024-03-12T05:35:53.929733+0000) 2024-03-12T05:35:56.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:55 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:55.833+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:56.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:55 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:55.833+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:56.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:55 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:55.833+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:56.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:55 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:55.833+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:35:56.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:56 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:56.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:56.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:56 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:56.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:56.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:56 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:56.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:56.837 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:56 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:56.459+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:56.838 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:56 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:56.459+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:56.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:56 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:56.751+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:56.971 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:56 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:56.751+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:56.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:56 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:56.593+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:56.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:56 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:56.593+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:57.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:56 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:56.843+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:57.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:56 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:56.843+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:57.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:56 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:56.843+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:57.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:56 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:56.843+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:35:57.315 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:57 smithi100 podman[109441]: 2024-03-12 05:35:57.064122604 +0000 UTC m=+5.587124905 container died c9c95753f4094799e060958a5e37570e59df1df07387b50055d45c12ffeb9f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-12T05:35:57.315 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:57 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:57.231+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:57.316 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:57 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:57.231+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:57.316 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:57 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:57.231+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:57.316 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:57 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:57.231+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:35:57.681 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:57 smithi100 podman[109441]: 2024-03-12 05:35:57.427739065 +0000 UTC m=+5.950741371 container cleanup c9c95753f4094799e060958a5e37570e59df1df07387b50055d45c12ffeb9f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:35:57.681 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:57 smithi100 bash[109441]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6 2024-03-12T05:35:57.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:57.427+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:57.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:57.427+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:57.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:57.427+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:35:57.791 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:57 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:57.546+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:57.791 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:57 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:57.546+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:57.792 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:57 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:57.546+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:35:58.024 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:57 smithi100 podman[109458]: 2024-03-12 05:35:57.687599905 +0000 UTC m=+0.624658576 container remove c9c95753f4094799e060958a5e37570e59df1df07387b50055d45c12ffeb9f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-12T05:35:58.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:57.813+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:58.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:57.813+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:58.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:57.813+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:58.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:57 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:57.813+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:35:58.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:57 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:57.796+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:58.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:57 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:57.796+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:58.275 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:58 smithi100 podman[109551]: 2024-03-12 05:35:57.93312254 +0000 UTC m=+0.018250880 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:35:58.275 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:58 smithi100 podman[109551]: 2024-03-12 05:35:58.046572932 +0000 UTC m=+0.131701269 container create eafbc27ef58d47c4bc8bc71c7e944e18bf47fd4673d8f690975615bba963ebdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:35:58.275 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:58 smithi100 podman[109551]: 2024-03-12 05:35:58.258173931 +0000 UTC m=+0.343302282 container init eafbc27ef58d47c4bc8bc71c7e944e18bf47fd4673d8f690975615bba963ebdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-03-12T05:35:58.275 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:58 smithi100 podman[109551]: 2024-03-12 05:35:58.261404143 +0000 UTC m=+0.346532491 container start eafbc27ef58d47c4bc8bc71c7e944e18bf47fd4673d8f690975615bba963ebdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T05:35:58.541 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:58 smithi100 podman[109551]: 2024-03-12 05:35:58.357753389 +0000 UTC m=+0.442881747 container attach eafbc27ef58d47c4bc8bc71c7e944e18bf47fd4673d8f690975615bba963ebdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-12T05:35:58.541 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:58.280+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:58.541 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:58.280+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:58.541 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:58.280+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:58.542 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:58 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:58.280+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:35:58.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:58.429+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:58.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:58.429+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:58.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:58.429+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:35:58.808 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:58.531+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:58.808 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:58.531+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:58.808 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:58.531+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:35:58.815 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:58 smithi100 podman[109551]: 2024-03-12 05:35:58.547595644 +0000 UTC m=+0.632723983 container died eafbc27ef58d47c4bc8bc71c7e944e18bf47fd4673d8f690975615bba963ebdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:35:59.081 INFO:journalctl@ceph.osd.6.smithi100.stdout:Mar 12 05:35:59 smithi100 podman[109577]: 2024-03-12 05:35:59.071919739 +0000 UTC m=+0.518795748 container remove eafbc27ef58d47c4bc8bc71c7e944e18bf47fd4673d8f690975615bba963ebdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:35:59.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.6.service' 2024-03-12T05:35:59.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:35:59.126 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-12T05:35:59.126 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-12T05:35:59.127 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.7 2024-03-12T05:35:59.155 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:58.858+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:35:59.155 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:58.858+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:35:59.156 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:58.858+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:35:59.156 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:58.858+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:35:59.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:58.813+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:59.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:58.813+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:59.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:58 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:58.813+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:35:59.418 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 systemd[1]: Stopping Ceph osd.7 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:35:59.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:59.259+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:35:59.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:59.259+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:35:59.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:59.259+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:35:59.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:35:59 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:35:59.259+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:35:59.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:59.423+0000 7fd70fbdb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:35:59.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:59.423+0000 7fd70fbdb700 -1 osd.7 124 *** Got signal Terminated *** 2024-03-12T05:35:59.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:59.423+0000 7fd70fbdb700 -1 osd.7 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:35:59.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:59.424+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:35:59.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:59.424+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:35:59.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:35:59.424+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:35:59.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:59.785+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:35:59.970 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:59.785+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:35:59.971 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:35:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:35:59.785+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:35:59.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:59.536+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:35:59.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:59.536+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:35:59.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:35:59 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:35:59.536+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:00.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:59.852+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:00.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:59.852+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:00.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:59.852+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:00.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:35:59 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:35:59.852+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:00.471 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:00.444+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:36:00.471 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:00.444+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:36:00.471 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:00.444+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:36:00.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:00.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:00.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:00.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:00.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:00.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:00.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:00 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:00.234+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:00.806 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:00.510+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:00.806 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:00.510+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:00.806 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:00.510+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:01.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:00.864+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:01.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:00.864+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:01.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:00.864+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:01.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:00 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:00.864+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:01.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:00.809+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:01.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:00.809+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:01.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:00 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:00.809+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:01.466 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:01.186+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:01.467 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:01.186+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:01.467 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:01.186+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:01.467 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:01 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:01.186+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:01.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:01.441+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:36:01.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:01.441+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:36:01.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:01.441+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:36:01.837 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:01.540+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:01.837 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:01.540+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:01.837 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:01.540+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:02.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:01.825+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:02.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:01.825+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:02.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:01.825+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:02.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:01.825+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:02.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:01.842+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:02.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:01.842+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:02.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:01 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:01.842+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:02.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:02.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:02.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:02.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:02.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:02.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:02.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:02.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:02.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:02 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:02.220+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:02.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:02.437+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:36:02.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:02.437+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:36:02.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:02.437+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:36:02.858 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:02.544+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:02.859 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:02.544+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:02.859 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:02.544+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:03.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:02.849+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:03.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:02.849+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:03.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:02.849+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:03.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:02 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:02.849+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:03.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:02.863+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:03.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:02.863+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:03.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:02 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:02.863+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:03.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:03.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:03.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:03.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:03.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:03.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:03.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:03.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:03.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:03 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:03.243+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:03.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:03.391+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:36:03.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:03.391+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:36:03.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:03.391+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:36:03.883 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:03.534+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:03.883 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:03.534+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:03.883 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:03.534+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:03.883 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:03.534+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:04.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:03.829+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:04.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:03.829+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:04.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:03.829+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:04.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:03 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:03.829+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:04.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:03.888+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:04.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:03.888+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:04.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:03 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:03.888+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:04.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:04.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:04.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:04.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:04.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:04.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:04.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:04.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:04.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:04 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:04.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:04.702 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:04.385+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:12.625460+0000 front 2024-03-12T05:35:12.625484+0000 (oldest deadline 2024-03-12T05:35:38.525578+0000) 2024-03-12T05:36:04.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:04.385+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.327223+0000 front 2024-03-12T05:35:21.327192+0000 (oldest deadline 2024-03-12T05:35:48.327943+0000) 2024-03-12T05:36:04.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:04.385+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.730103+0000 front 2024-03-12T05:35:31.730007+0000 (oldest deadline 2024-03-12T05:35:57.029954+0000) 2024-03-12T05:36:04.703 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7[54246]: 2024-03-12T05:36:04.385+0000 7fd70c9d0700 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:40.531021+0000 front 2024-03-12T05:35:40.530978+0000 (oldest deadline 2024-03-12T05:36:03.431075+0000) 2024-03-12T05:36:04.922 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:04.539+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:04.922 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:04.539+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:04.922 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:04.539+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:04.923 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:04.539+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:05.202 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:05 smithi156 podman[76136]: 2024-03-12 05:36:05.002471388 +0000 UTC m=+5.677128467 container died d8de97200528cb87a68e7754ede4648a7a2cf6f27b94d2898b247caddca85509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-12T05:36:05.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:04.823+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:05.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:04.823+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:05.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:04.823+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:05.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:04 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:04.823+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:05.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:04.927+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:05.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:04.927+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:05.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:04 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:04.927+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:05.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:05.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:05.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:05.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:05.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:05.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:05.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:05.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:05.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:05 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:05.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:05.626 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:05 smithi156 podman[76136]: 2024-03-12 05:36:05.35640343 +0000 UTC m=+6.031060514 container cleanup d8de97200528cb87a68e7754ede4648a7a2cf6f27b94d2898b247caddca85509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:05.626 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:05 smithi156 bash[76136]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7 2024-03-12T05:36:05.874 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:05.573+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:05.874 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:05.573+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:05.874 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:05.573+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:05.874 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:05.573+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:05.943 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:05 smithi156 podman[76167]: 2024-03-12 05:36:05.632240895 +0000 UTC m=+0.625455356 container remove d8de97200528cb87a68e7754ede4648a7a2cf6f27b94d2898b247caddca85509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-12T05:36:05.944 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:05.861+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:05.944 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:05.861+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:05.944 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:05.861+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:05.944 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:05.861+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:05.944 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:05.861+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:06.202 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:05 smithi156 podman[76255]: 2024-03-12 05:36:05.94978154 +0000 UTC m=+0.099742126 container create 2b42d6602de2ff517b1a95c44121be6d2f298c7fec18511ef607f1b3582baad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:36:06.203 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:05 smithi156 podman[76255]: 2024-03-12 05:36:05.867550579 +0000 UTC m=+0.017511177 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:36:06.203 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:06 smithi156 podman[76255]: 2024-03-12 05:36:06.168663685 +0000 UTC m=+0.318624288 container init 2b42d6602de2ff517b1a95c44121be6d2f298c7fec18511ef607f1b3582baad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-12T05:36:06.203 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:06 smithi156 podman[76255]: 2024-03-12 05:36:06.171643124 +0000 UTC m=+0.321603719 container start 2b42d6602de2ff517b1a95c44121be6d2f298c7fec18511ef607f1b3582baad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-12T05:36:06.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:05.879+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:06.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:05.879+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:06.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:05.879+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:06.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:05 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:05.879+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:06.453 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:06 smithi156 podman[76255]: 2024-03-12 05:36:06.2770975 +0000 UTC m=+0.427058099 container attach 2b42d6602de2ff517b1a95c44121be6d2f298c7fec18511ef607f1b3582baad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-03-12T05:36:06.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:06 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:06.208+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:06.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:06 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:06.208+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:06.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:06 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:06.208+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:06.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:06 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:06.208+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:06.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:06 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:06.208+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:06.748 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 12 05:36:06 smithi156 podman[76255]: 2024-03-12 05:36:06.458838212 +0000 UTC m=+0.608798798 container died 2b42d6602de2ff517b1a95c44121be6d2f298c7fec18511ef607f1b3582baad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-12T05:36:06.921 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:06.600+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:06.921 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:06.600+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:06.921 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:06.600+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:06.921 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:06.600+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:07.007 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:06 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:06.844+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:07.008 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:06 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:06.844+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:07.008 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:06 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:06.844+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:07.008 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:06 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:06.844+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:07.008 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:06 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:06.844+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:07.022 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.7.service' 2024-03-12T05:36:07.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:07.104 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-12T05:36:07.104 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-12T05:36:07.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.8 2024-03-12T05:36:07.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:06.925+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:07.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:06.925+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:07.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:06.926+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:07.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:06 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:06.926+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:07.222 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 systemd[1]: Stopping Ceph osd.8 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:07.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:07 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:07.254+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:07.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:07 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:07.254+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:07.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:07 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:07.254+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:07.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:07 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:07.254+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:07.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:07 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:07.254+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:07.720 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.398+0000 7f8f18e77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:07.721 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.398+0000 7f8f18e77700 -1 osd.8 124 *** Got signal Terminated *** 2024-03-12T05:36:07.721 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.398+0000 7f8f18e77700 -1 osd.8 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:36:07.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:07.622+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:07.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:07.622+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:07.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:07.622+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:07.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:07.622+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:08.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:07 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:07.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:08.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:07 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:07.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:08.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:07 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:07.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:08.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:07 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:07.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:08.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:07 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:07.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:08.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.940+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:08.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.940+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:08.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.940+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:08.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:07 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:07.940+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:08.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:08 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:08.240+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:08.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:08 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:08.240+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:08.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:08 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:08.240+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:08.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:08 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:08.240+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:08.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:08 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:08.240+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:08.943 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:08.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:08.943 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:08.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:08.943 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:08.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:08.944 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:08.604+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:09.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:08 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:08.817+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:09.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:08 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:08.817+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:09.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:08 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:08.817+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:09.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:08 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:08.817+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:09.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:08 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:08.817+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:09.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:08.948+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:09.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:08.948+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:09.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:08.948+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:09.221 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:08 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:08.948+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:09.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:09 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:09.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:09.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:09 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:09.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:09.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:09 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:09.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:09.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:09 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:09.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:09.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:09 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:09.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:09.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:09.566+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:09.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:09.566+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:09.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:09.566+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:09.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:09.566+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:10.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:09.776+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:10.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:09.776+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:10.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:09.776+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:10.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:09.776+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:10.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:09.776+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:10.470 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:09.996+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:10.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:09.996+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:10.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:09.996+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:10.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:09.996+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:10.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:09 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:09.996+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.132711+0000 front 2024-03-12T05:35:45.133046+0000 (oldest deadline 2024-03-12T05:36:09.232909+0000) 2024-03-12T05:36:10.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:10 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:10.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:10.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:10 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:10.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:10.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:10 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:10.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:10.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:10 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:10.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:10.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:10 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:10.237+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:10.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:10 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:10.608+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:10.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:10 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:10.608+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:10.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:10 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:10.608+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:10.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:10 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:10.608+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:10.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:10 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:10.608+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:11.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:10 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:10.805+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:11.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:10 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:10.805+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:11.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:10 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:10.805+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:11.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:10 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:10.805+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:11.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:10 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:10.805+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:11.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:10 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:10.805+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:11.470 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:11.021+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:11.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:11.021+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:11.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:11.021+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:11.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:11.021+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:11.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:11.021+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.132711+0000 front 2024-03-12T05:35:45.133046+0000 (oldest deadline 2024-03-12T05:36:09.232909+0000) 2024-03-12T05:36:11.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:11 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:11.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:11.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:11 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:11.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:11.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:11 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:11.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:11.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:11 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:11.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:11.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:11 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:11.226+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:11.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:11.563+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:11.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:11.563+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:11.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:11.563+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:11.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:11.563+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:11.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:11.563+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:12.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:11.839+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:12.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:11.839+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:12.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:11.839+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:12.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:11.839+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:12.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:11.839+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:12.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:11 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:11.839+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:12.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:12.024+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.728218+0000 front 2024-03-12T05:35:14.728343+0000 (oldest deadline 2024-03-12T05:35:38.228351+0000) 2024-03-12T05:36:12.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:12.024+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:22.929184+0000 front 2024-03-12T05:35:22.929361+0000 (oldest deadline 2024-03-12T05:35:48.829323+0000) 2024-03-12T05:36:12.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:12.024+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.330602+0000 front 2024-03-12T05:35:32.330484+0000 (oldest deadline 2024-03-12T05:35:58.230661+0000) 2024-03-12T05:36:12.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:12.024+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.832571+0000 front 2024-03-12T05:35:39.832386+0000 (oldest deadline 2024-03-12T05:36:05.132339+0000) 2024-03-12T05:36:12.471 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8[54718]: 2024-03-12T05:36:12.024+0000 7f8f15c6c700 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.132711+0000 front 2024-03-12T05:35:45.133046+0000 (oldest deadline 2024-03-12T05:36:09.232909+0000) 2024-03-12T05:36:12.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:12 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:12.211+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:12.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:12 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:12.211+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:12.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:12 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:12.211+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:12.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:12 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:12.211+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:12.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:12 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:12.211+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:12.939 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:12.567+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:12.940 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:12.567+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:12.940 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:12.567+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:12.940 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:12.567+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:12.940 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:12 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:12.567+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:13.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:12 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:12.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:13.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:12 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:12.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:13.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:12 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:12.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:13.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:12 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:12.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:13.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:12 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:12.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:13.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:12 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:12.832+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:13.220 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:12 smithi161 podman[76426]: 2024-03-12 05:36:12.945376719 +0000 UTC m=+5.641844430 container died 93368c0643e1fc3e97f9832dc242da3f8399246b0ad2e0018ba9ad61063016bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-12T05:36:13.540 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:13 smithi161 podman[76426]: 2024-03-12 05:36:13.30731586 +0000 UTC m=+6.003783570 container cleanup 93368c0643e1fc3e97f9832dc242da3f8399246b0ad2e0018ba9ad61063016bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-03-12T05:36:13.540 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:13 smithi161 bash[76426]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8 2024-03-12T05:36:13.565 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:13.222+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:13.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:13.222+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:13.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:13.222+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:13.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:13.222+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:13.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:13.222+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:13.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:13 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:13.222+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:13.895 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:13 smithi161 podman[76443]: 2024-03-12 05:36:13.575063905 +0000 UTC m=+0.621872689 container remove 93368c0643e1fc3e97f9832dc242da3f8399246b0ad2e0018ba9ad61063016bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-12T05:36:13.895 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:13.545+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:13.895 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:13.545+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:13.895 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:13.545+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:13.896 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:13.545+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:13.896 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:13 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:13.545+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:14.189 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:13 smithi161 podman[76536]: 2024-03-12 05:36:13.900961502 +0000 UTC m=+0.105098077 container create 18b3f9d459bcd97a3e90384c9eb63b9f385729adea8c1a7242b5ee4976c26677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:14.189 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:13 smithi161 podman[76536]: 2024-03-12 05:36:13.813476084 +0000 UTC m=+0.017612678 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:36:14.189 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:14 smithi161 podman[76536]: 2024-03-12 05:36:14.119984224 +0000 UTC m=+0.324120810 container init 18b3f9d459bcd97a3e90384c9eb63b9f385729adea8c1a7242b5ee4976c26677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-12T05:36:14.190 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:14 smithi161 podman[76536]: 2024-03-12 05:36:14.123273103 +0000 UTC m=+0.327409679 container start 18b3f9d459bcd97a3e90384c9eb63b9f385729adea8c1a7242b5ee4976c26677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-12T05:36:14.190 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:14 smithi161 podman[76536]: 2024-03-12 05:36:14.194788725 +0000 UTC m=+0.398925319 container attach 18b3f9d459bcd97a3e90384c9eb63b9f385729adea8c1a7242b5ee4976c26677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-12T05:36:14.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:13 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:13.878+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:14.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:13 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:13.878+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:14.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:13 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:13.878+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:14.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:13 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:13.878+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:14.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:13 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:13.878+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:14.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:13 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:13.878+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:14.470 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 12 05:36:14 smithi161 podman[76536]: 2024-03-12 05:36:14.416468496 +0000 UTC m=+0.620605075 container died 18b3f9d459bcd97a3e90384c9eb63b9f385729adea8c1a7242b5ee4976c26677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-12T05:36:14.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:14 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:14.268+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:14.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:14 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:14.268+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:14.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:14 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:14.268+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:14.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:14 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:14.268+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:14.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:14 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:14.268+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:14.567 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:14 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:14.268+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:14.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:14.511+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:14.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:14.511+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:14.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:14.511+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:14.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:14.511+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:14.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:14 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:14.511+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:15.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.8.service' 2024-03-12T05:36:15.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:15.059 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-12T05:36:15.059 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-12T05:36:15.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.9 2024-03-12T05:36:15.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:14.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:15.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:14.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:15.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:14.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:15.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:14.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:15.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:14.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:15.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:14 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:14.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:15.368 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 systemd[1]: Stopping Ceph osd.9 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:15.368 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:15.368 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:15.369 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:15.369 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:15.369 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:15.369 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.265+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:15.369 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.373+0000 7fd85ae84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:15.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:15.494+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:15.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:15.494+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:15.721 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:15.494+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:15.722 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:15.494+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:15.722 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:15 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:15.494+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:15.815 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.373+0000 7fd85ae84700 -1 osd.9 124 *** Got signal Terminated *** 2024-03-12T05:36:15.815 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:15 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:15.373+0000 7fd85ae84700 -1 osd.9 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:36:16.202 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:15 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:15.837+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:16.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:15 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:15.837+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:16.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:15 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:15.837+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:16.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:15 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:15.837+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:16.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:15 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:15.837+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:16.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:15 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:15.837+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:16.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:16 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:16.275+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:16.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:16 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:16.275+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:16.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:16 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:16.275+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:16.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:16 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:16.275+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:16.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:16 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:16.275+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:16.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:16 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:16.275+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:16.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:16.523+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:16.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:16.523+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:16.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:16.523+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:16.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:16.523+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:16.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:16.523+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:17.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:16.867+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:17.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:16.867+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:17.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:16.867+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:17.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:16.867+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:17.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:16.867+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:17.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:16 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:16.867+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:17.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:17 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:17.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:17.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:17 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:17.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:17.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:17 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:17.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:17.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:17 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:17.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:17.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:17 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:17.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:17.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:17 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:17.232+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:17.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:17.549+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:17.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:17.549+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:17.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:17.549+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:17.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:17.549+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:17.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:17 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:17.549+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:18.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:17 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:17.862+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:18.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:18.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:18.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:18.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:18.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:18.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:18.567 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:18 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:18.246+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.284790+0000 front 2024-03-12T05:35:52.284811+0000 (oldest deadline 2024-03-12T05:36:18.184847+0000) 2024-03-12T05:36:18.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:18.564+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:18.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:18.564+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:18.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:18.564+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:18.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:18.564+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:18.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:18.564+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:18.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:18 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:18.564+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:19.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:19.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:19.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:19.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:19.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:19.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:19.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:18 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:18.884+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:19.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:19.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:19.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:19.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:19.567 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:19.567 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:19.567 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:19 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:19.213+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.284790+0000 front 2024-03-12T05:35:52.284811+0000 (oldest deadline 2024-03-12T05:36:18.184847+0000) 2024-03-12T05:36:19.872 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:19.588+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:19.872 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:19.588+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:19.872 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:19.588+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:19.872 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:19.588+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:19.873 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:19.588+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:19.873 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:19 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:19.588+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:20.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:20.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:20.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:20.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:20.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:20.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:20.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:19 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:19.840+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:20.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.676209+0000 front 2024-03-12T05:35:05.676179+0000 (oldest deadline 2024-03-12T05:35:30.376246+0000) 2024-03-12T05:36:20.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.978457+0000 front 2024-03-12T05:35:14.978390+0000 (oldest deadline 2024-03-12T05:35:40.779156+0000) 2024-03-12T05:36:20.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:23.680464+0000 front 2024-03-12T05:35:23.680525+0000 (oldest deadline 2024-03-12T05:35:47.180453+0000) 2024-03-12T05:36:20.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.982100+0000 front 2024-03-12T05:35:32.982153+0000 (oldest deadline 2024-03-12T05:35:57.082214+0000) 2024-03-12T05:36:20.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:37.082815+0000 front 2024-03-12T05:35:37.082843+0000 (oldest deadline 2024-03-12T05:36:01.782738+0000) 2024-03-12T05:36:20.566 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:46.384336+0000 front 2024-03-12T05:35:46.384324+0000 (oldest deadline 2024-03-12T05:36:12.284277+0000) 2024-03-12T05:36:20.567 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9[65730]: 2024-03-12T05:36:20.202+0000 7fd853c71700 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.284790+0000 front 2024-03-12T05:35:52.284811+0000 (oldest deadline 2024-03-12T05:36:18.184847+0000) 2024-03-12T05:36:20.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:20.627+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:20.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:20.627+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:20.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:20.627+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:20.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:20.627+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:20.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:20.627+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:20.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:20 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:20.627+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:21.084 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:20 smithi100 podman[109669]: 2024-03-12 05:36:20.838278531 +0000 UTC m=+5.580452089 container died 271d39443757be85a1d0002abaea62c2a0637073f96fc76ffc4ff35bbfc906da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-12T05:36:21.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:21.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:21.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:21.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:21.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:21.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:21.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:20 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:20.795+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:21.352 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 podman[109669]: 2024-03-12 05:36:21.181336544 +0000 UTC m=+5.923510111 container cleanup 271d39443757be85a1d0002abaea62c2a0637073f96fc76ffc4ff35bbfc906da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-12T05:36:21.352 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 bash[109669]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9 2024-03-12T05:36:21.777 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 podman[109686]: 2024-03-12 05:36:21.357700426 +0000 UTC m=+0.518851283 container remove 271d39443757be85a1d0002abaea62c2a0637073f96fc76ffc4ff35bbfc906da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:36:21.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:21.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:21.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:21.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:21.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:21.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:21.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:21.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:21.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:21.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:21.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:21.662+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:22.066 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 podman[109777]: 2024-03-12 05:36:21.783424542 +0000 UTC m=+0.108507053 container create d603c1e3c475f5279841f86322c8e5866a671a8bb5109347c235c87503c6e0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9-deactivate, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-12T05:36:22.066 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 podman[109777]: 2024-03-12 05:36:21.692373056 +0000 UTC m=+0.017455580 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:36:22.066 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 podman[109777]: 2024-03-12 05:36:21.960981585 +0000 UTC m=+0.286064098 container init d603c1e3c475f5279841f86322c8e5866a671a8bb5109347c235c87503c6e0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9-deactivate, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-12T05:36:22.066 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:21 smithi100 podman[109777]: 2024-03-12 05:36:21.963970923 +0000 UTC m=+0.289053436 container start d603c1e3c475f5279841f86322c8e5866a671a8bb5109347c235c87503c6e0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-12T05:36:22.066 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:22 smithi100 podman[109777]: 2024-03-12 05:36:22.019150649 +0000 UTC m=+0.344233167 container attach d603c1e3c475f5279841f86322c8e5866a671a8bb5109347c235c87503c6e0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:22.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:22.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:22.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:22.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:22.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:22.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:22.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:21 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:21.757+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:22.533 INFO:journalctl@ceph.osd.9.smithi100.stdout:Mar 12 05:36:22 smithi100 podman[109777]: 2024-03-12 05:36:22.249691724 +0000 UTC m=+0.574774235 container died d603c1e3c475f5279841f86322c8e5866a671a8bb5109347c235c87503c6e0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-12T05:36:22.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.9.service' 2024-03-12T05:36:22.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:22.872 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-12T05:36:22.873 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-12T05:36:22.873 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.10 2024-03-12T05:36:22.970 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:22 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:22.677+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:22.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:22 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:22.677+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:22.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:22 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:22.677+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:22.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:22 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:22.677+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:22.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:22 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:22.677+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:22.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:22 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:22.677+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:23.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:23.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:23.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:23.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:22.794+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:22 smithi156 systemd[1]: Stopping Ceph osd.10 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.197+0000 7f553b0de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.197+0000 7f553b0de700 -1 osd.10 124 *** Got signal Terminated *** 2024-03-12T05:36:23.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.197+0000 7f553b0de700 -1 osd.10 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:36:23.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:23.710+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:23.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:23.710+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:23.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:23.710+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:23.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:23.710+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:23.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:23.710+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:23.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:23 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:23.710+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:24.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:24.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:24.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:24.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:24.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:24.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:24.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:23 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:23.761+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:25.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:25.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:25.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:25.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:25.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:25.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:25.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:24 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:24.740+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:25.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:24 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:24.748+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:25.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:24 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:24.748+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:25.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:24 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:24.748+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:25.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:24 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:24.748+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:25.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:24 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:24.748+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:25.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:24 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:24.749+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:26.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:26.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:26.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.787+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:26.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.787+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:26.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.787+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:26.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.787+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:26.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:25 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:25.787+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:26.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:25.772+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:26.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:25.772+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:26.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:25.772+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:26.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:25.772+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:26.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:25.772+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:26.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:25.772+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:27.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:27.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:27.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:27.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:27.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:27.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:27.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:26 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:26.786+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:27.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:26 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:26.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:27.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:26 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:26.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:27.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:26 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:26.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:27.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:26 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:26.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:27.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:26 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:26.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:27.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:26 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:26.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:28.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:05.730311+0000 front 2024-03-12T05:35:05.730385+0000 (oldest deadline 2024-03-12T05:35:31.030469+0000) 2024-03-12T05:36:28.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-12T05:35:14.432188+0000 front 2024-03-12T05:35:14.432209+0000 (oldest deadline 2024-03-12T05:35:38.532408+0000) 2024-03-12T05:36:28.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:20.833579+0000 front 2024-03-12T05:35:20.833545+0000 (oldest deadline 2024-03-12T05:35:46.733576+0000) 2024-03-12T05:36:28.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:32.435780+0000 front 2024-03-12T05:35:32.435799+0000 (oldest deadline 2024-03-12T05:35:54.735926+0000) 2024-03-12T05:36:28.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:39.337617+0000 front 2024-03-12T05:35:39.337667+0000 (oldest deadline 2024-03-12T05:36:05.237728+0000) 2024-03-12T05:36:28.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:45.238442+0000 front 2024-03-12T05:35:45.238627+0000 (oldest deadline 2024-03-12T05:36:10.538195+0000) 2024-03-12T05:36:28.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:27 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10[61189]: 2024-03-12T05:36:27.807+0000 7f5537ed3700 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:53.339747+0000 front 2024-03-12T05:35:53.339646+0000 (oldest deadline 2024-03-12T05:36:17.439627+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:28.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:27 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:27.738+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:28.878 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:28 smithi156 podman[76360]: 2024-03-12 05:36:28.608852447 +0000 UTC m=+5.540093997 container died 18f4d3a6817802ab9eb382979e1ed7f908c015e58917e567a8333ebd41a7127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:28.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:28.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:28.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:28.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:28.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:28.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:28.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:28 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:28.723+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:29.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:28 smithi156 podman[76360]: 2024-03-12 05:36:28.972650865 +0000 UTC m=+5.903892415 container cleanup 18f4d3a6817802ab9eb382979e1ed7f908c015e58917e567a8333ebd41a7127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:29.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:28 smithi156 bash[76360]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10 2024-03-12T05:36:29.555 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:29 smithi156 podman[76377]: 2024-03-12 05:36:29.248893971 +0000 UTC m=+0.643111110 container remove 18f4d3a6817802ab9eb382979e1ed7f908c015e58917e567a8333ebd41a7127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:36:29.863 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:29 smithi156 podman[76468]: 2024-03-12 05:36:29.464099843 +0000 UTC m=+0.018031619 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:36:29.863 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:29 smithi156 podman[76468]: 2024-03-12 05:36:29.591403689 +0000 UTC m=+0.145335452 container create eeb7478e269f2301ae5d00e64f87da938f2f2e2b378956df12a065caafbed6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-12T05:36:29.863 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:29 smithi156 podman[76468]: 2024-03-12 05:36:29.793786486 +0000 UTC m=+0.347718250 container init eeb7478e269f2301ae5d00e64f87da938f2f2e2b378956df12a065caafbed6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , ceph=True) 2024-03-12T05:36:29.863 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:29 smithi156 podman[76468]: 2024-03-12 05:36:29.796828877 +0000 UTC m=+0.350760639 container start eeb7478e269f2301ae5d00e64f87da938f2f2e2b378956df12a065caafbed6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:29.863 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:29 smithi156 podman[76468]: 2024-03-12 05:36:29.868456275 +0000 UTC m=+0.422388058 container attach eeb7478e269f2301ae5d00e64f87da938f2f2e2b378956df12a065caafbed6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:29.924 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:29 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:29.675+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:30.203 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 12 05:36:30 smithi156 podman[76468]: 2024-03-12 05:36:30.092366771 +0000 UTC m=+0.646298540 container died eeb7478e269f2301ae5d00e64f87da938f2f2e2b378956df12a065caafbed6e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-94a5397) 2024-03-12T05:36:30.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.10.service' 2024-03-12T05:36:30.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:30.737 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-12T05:36:30.737 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-12T05:36:30.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.11 2024-03-12T05:36:30.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:30.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:30.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:30.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:30.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:30.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:30.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:30.696+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:30.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:30 smithi161 systemd[1]: Stopping Ceph osd.11 for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:31.471 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.108+0000 7f27d230e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:31.471 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.108+0000 7f27d230e700 -1 osd.11 124 *** Got signal Terminated *** 2024-03-12T05:36:31.471 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.108+0000 7f27d230e700 -1 osd.11 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-12T05:36:31.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:31.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:31.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:31.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:31.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:31.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:31.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:31 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:31.724+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:33.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:33.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:33.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:33.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:33.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:33.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:33.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:32 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:32.741+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:33.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:33.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:33.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:33.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:33.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:33.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:33.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:33 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:33.694+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:34.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:34.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:34.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:34.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:34.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:34.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:34.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:34.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:34 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:34.716+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-03-12T05:36:11.088737+0000 front 2024-03-12T05:36:11.088779+0000 (oldest deadline 2024-03-12T05:36:34.588778+0000) 2024-03-12T05:36:36.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-03-12T05:35:06.176089+0000 front 2024-03-12T05:35:06.176054+0000 (oldest deadline 2024-03-12T05:35:31.475982+0000) 2024-03-12T05:36:36.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-12T05:35:21.580444+0000 front 2024-03-12T05:35:21.580545+0000 (oldest deadline 2024-03-12T05:35:51.581362+0000) 2024-03-12T05:36:36.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-03-12T05:35:31.581986+0000 front 2024-03-12T05:35:31.582006+0000 (oldest deadline 2024-03-12T05:35:57.481969+0000) 2024-03-12T05:36:36.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-12T05:35:38.483801+0000 front 2024-03-12T05:35:38.483702+0000 (oldest deadline 2024-03-12T05:36:03.183715+0000) 2024-03-12T05:36:36.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-12T05:35:47.284616+0000 front 2024-03-12T05:35:47.284663+0000 (oldest deadline 2024-03-12T05:36:10.085405+0000) 2024-03-12T05:36:36.222 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-03-12T05:35:52.986569+0000 front 2024-03-12T05:35:52.986576+0000 (oldest deadline 2024-03-12T05:36:18.286479+0000) 2024-03-12T05:36:36.223 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-12T05:36:02.987536+0000 front 2024-03-12T05:36:02.987552+0000 (oldest deadline 2024-03-12T05:36:27.687423+0000) 2024-03-12T05:36:36.223 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:35 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11[61802]: 2024-03-12T05:36:35.732+0000 7f27cb0fb700 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-03-12T05:36:11.088737+0000 front 2024-03-12T05:36:11.088779+0000 (oldest deadline 2024-03-12T05:36:34.588778+0000) 2024-03-12T05:36:36.845 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:36 smithi161 podman[76655]: 2024-03-12 05:36:36.547770879 +0000 UTC m=+5.610671558 container died fdc6ad5e949b4417202875e0e7289ea86d9d30a7e07cec77cd22d0ea7780d453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-12T05:36:36.845 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:36 smithi161 podman[76655]: 2024-03-12 05:36:36.85024044 +0000 UTC m=+5.913141109 container cleanup fdc6ad5e949b4417202875e0e7289ea86d9d30a7e07cec77cd22d0ea7780d453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-12T05:36:37.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:36 smithi161 bash[76655]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11 2024-03-12T05:36:37.221 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76672]: 2024-03-12 05:36:37.093056147 +0000 UTC m=+0.540537292 container remove fdc6ad5e949b4417202875e0e7289ea86d9d30a7e07cec77cd22d0ea7780d453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-12T05:36:37.682 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76765]: 2024-03-12 05:36:37.308232735 +0000 UTC m=+0.017512802 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-12T05:36:37.682 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76765]: 2024-03-12 05:36:37.410602734 +0000 UTC m=+0.119882793 container create e700c9572afe526f2734602f5419e700ec4bab943c428a7ed84c0198882604bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-12T05:36:37.683 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76765]: 2024-03-12 05:36:37.621244181 +0000 UTC m=+0.330524242 container init e700c9572afe526f2734602f5419e700ec4bab943c428a7ed84c0198882604bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-12T05:36:37.683 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76765]: 2024-03-12 05:36:37.624382117 +0000 UTC m=+0.333662167 container start e700c9572afe526f2734602f5419e700ec4bab943c428a7ed84c0198882604bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-12T05:36:37.683 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76765]: 2024-03-12 05:36:37.687617433 +0000 UTC m=+0.396897509 container attach e700c9572afe526f2734602f5419e700ec4bab943c428a7ed84c0198882604bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11-deactivate, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True) 2024-03-12T05:36:37.971 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:37 smithi161 podman[76765]: 2024-03-12 05:36:37.91280862 +0000 UTC m=+0.622088684 container died e700c9572afe526f2734602f5419e700ec4bab943c428a7ed84c0198882604bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True) 2024-03-12T05:36:38.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.11.service' 2024-03-12T05:36:38.387 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:38 smithi161 podman[76792]: 2024-03-12 05:36:38.348822708 +0000 UTC m=+0.426478299 container remove e700c9572afe526f2734602f5419e700ec4bab943c428a7ed84c0198882604bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-12T05:36:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:38 smithi161 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.11.service: Deactivated successfully. 2024-03-12T05:36:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:38 smithi161 systemd[1]: Stopped Ceph osd.11 for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:36:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 12 05:36:38 smithi161 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@osd.11.service: Consumed 4min 21.265s CPU time. 2024-03-12T05:36:38.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:38.402 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-12T05:36:38.402 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-12T05:36:38.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.a 2024-03-12T05:36:38.778 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:36:38 smithi100 systemd[1]: Stopping Ceph mds.a for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:39.066 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:36:38 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a[72314]: 2024-03-12T05:36:38.782+0000 7f5877cb8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:39.066 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:36:38 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a[72314]: 2024-03-12T05:36:38.782+0000 7f5877cb8700 -1 mds.a *** got signal Terminated *** 2024-03-12T05:36:43.053 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:36:42 smithi100 podman[109892]: 2024-03-12 05:36:42.791249942 +0000 UTC m=+4.186924325 container died 45a474f3c504279a1e4f9286edd0019739cd0861201fe14158074c878be1f4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:36:43.316 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:36:43 smithi100 podman[109892]: 2024-03-12 05:36:43.157730348 +0000 UTC m=+4.553404732 container cleanup 45a474f3c504279a1e4f9286edd0019739cd0861201fe14158074c878be1f4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:36:43.316 INFO:journalctl@ceph.mds.a.smithi100.stdout:Mar 12 05:36:43 smithi100 bash[109892]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-a 2024-03-12T05:36:43.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.a.service' 2024-03-12T05:36:43.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:43.625 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-12T05:36:43.625 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-12T05:36:43.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.d 2024-03-12T05:36:43.939 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:36:43 smithi100 systemd[1]: Stopping Ceph mds.d for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:44.316 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:36:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d[71362]: 2024-03-12T05:36:43.944+0000 7f02cd5a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:44.316 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:36:43 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d[71362]: 2024-03-12T05:36:43.944+0000 7f02cd5a7700 -1 mds.d *** got signal Terminated *** 2024-03-12T05:36:48.215 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:36:47 smithi100 podman[110013]: 2024-03-12 05:36:47.960398357 +0000 UTC m=+4.114401078 container died 929a7f2a1b639b7123cd788aca3ea14624f352c1da0fe09496d48cb57fc4545d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=) 2024-03-12T05:36:48.565 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:36:48 smithi100 podman[110013]: 2024-03-12 05:36:48.310990406 +0000 UTC m=+4.464993129 container cleanup 929a7f2a1b639b7123cd788aca3ea14624f352c1da0fe09496d48cb57fc4545d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-12T05:36:48.565 INFO:journalctl@ceph.mds.d.smithi100.stdout:Mar 12 05:36:48 smithi100 bash[110013]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-d 2024-03-12T05:36:48.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.d.service' 2024-03-12T05:36:48.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:48.712 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-12T05:36:48.712 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-12T05:36:48.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.g 2024-03-12T05:36:48.816 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:36:48 smithi100 systemd[1]: Stopping Ceph mds.g for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:49.316 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:36:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g[71920]: 2024-03-12T05:36:49.063+0000 7f21b4cdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:49.316 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:36:49 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g[71920]: 2024-03-12T05:36:49.063+0000 7f21b4cdd700 -1 mds.g *** got signal Terminated *** 2024-03-12T05:36:52.311 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:36:52 smithi100 podman[110134]: 2024-03-12 05:36:52.026799503 +0000 UTC m=+3.094188559 container died 5dc8a96ab3105d9bac8f7e1ebd94bdc3ea5bafd3d50649c32775381132af7fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:36:52.566 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:36:52 smithi100 podman[110134]: 2024-03-12 05:36:52.406880185 +0000 UTC m=+3.474269250 container cleanup 5dc8a96ab3105d9bac8f7e1ebd94bdc3ea5bafd3d50649c32775381132af7fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:36:52.566 INFO:journalctl@ceph.mds.g.smithi100.stdout:Mar 12 05:36:52 smithi100 bash[110134]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-g 2024-03-12T05:36:52.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.g.service' 2024-03-12T05:36:52.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:52.791 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-12T05:36:52.792 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-12T05:36:52.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.j 2024-03-12T05:36:53.066 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:52 smithi100 systemd[1]: Stopping Ceph mds.j for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:53.566 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j[70977]: 2024-03-12T05:36:53.118+0000 7f0ece7d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:53.566 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:53 smithi100 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j[70977]: 2024-03-12T05:36:53.118+0000 7f0ece7d5700 -1 mds.j *** got signal Terminated *** 2024-03-12T05:36:57.481 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 podman[110256]: 2024-03-12 05:36:57.12446687 +0000 UTC m=+4.106485553 container died d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-12T05:36:57.739 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 podman[110256]: 2024-03-12 05:36:57.560229281 +0000 UTC m=+4.542247968 container cleanup d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS) 2024-03-12T05:36:57.740 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 bash[110256]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j 2024-03-12T05:36:57.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.j.service' 2024-03-12T05:36:57.999 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 podman[110270]: 2024-03-12 05:36:57.744685885 +0000 UTC m=+0.613905149 container remove d5ecc4f550e537cc732d38c486df92324330446184c3fc4ab9296e8194806eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-j, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:36:58.000 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.j.service: Deactivated successfully. 2024-03-12T05:36:58.000 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 systemd[1]: Stopped Ceph mds.j for 8a1680ae-e02d-11ee-95c8-87774f69a715. 2024-03-12T05:36:58.000 INFO:journalctl@ceph.mds.j.smithi100.stdout:Mar 12 05:36:57 smithi100 systemd[1]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.j.service: Consumed 1.164s CPU time. 2024-03-12T05:36:58.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:36:58.010 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-12T05:36:58.010 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-12T05:36:58.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.b 2024-03-12T05:36:58.309 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:36:58 smithi156 systemd[1]: Stopping Ceph mds.b for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:36:58.703 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:36:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: 2024-03-12T05:36:58.313+0000 7fa388e01700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:36:58.703 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:36:58 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b[66877]: 2024-03-12T05:36:58.313+0000 7fa388e01700 -1 mds.b *** got signal Terminated *** 2024-03-12T05:37:00.884 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:37:00 smithi156 podman[76586]: 2024-03-12 05:37:00.548530943 +0000 UTC m=+2.334649379 container died 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-12T05:37:01.203 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:37:00 smithi156 podman[76586]: 2024-03-12 05:37:00.888459701 +0000 UTC m=+2.674578135 container cleanup 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-12T05:37:01.203 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:37:00 smithi156 bash[76586]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b 2024-03-12T05:37:01.203 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 12 05:37:01 smithi156 podman[76600]: 2024-03-12 05:37:01.123537766 +0000 UTC m=+0.575912355 container remove 8f30baa94e2b40264d721db03dd85149bfb80fc2b67971dd81abb12c607844c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-12T05:37:01.366 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.b.service' 2024-03-12T05:37:01.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:01.403 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-12T05:37:01.404 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-12T05:37:01.404 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.e 2024-03-12T05:37:01.703 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:37:01 smithi156 systemd[1]: Stopping Ceph mds.e for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:02.203 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:37:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e[66259]: 2024-03-12T05:37:01.733+0000 7f0eb5bfd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:02.203 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:37:01 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e[66259]: 2024-03-12T05:37:01.733+0000 7f0eb5bfd700 -1 mds.e *** got signal Terminated *** 2024-03-12T05:37:04.798 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:37:04 smithi156 podman[76707]: 2024-03-12 05:37:04.548351004 +0000 UTC m=+2.923083871 container died 06b18cfc8214a8c46ffb328b6388a929516e409bf3e40eb702475d1e0b800a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True) 2024-03-12T05:37:05.174 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:37:04 smithi156 podman[76707]: 2024-03-12 05:37:04.90219969 +0000 UTC m=+3.276932541 container cleanup 06b18cfc8214a8c46ffb328b6388a929516e409bf3e40eb702475d1e0b800a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-12T05:37:05.174 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 12 05:37:04 smithi156 bash[76707]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-e 2024-03-12T05:37:05.277 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.e.service' 2024-03-12T05:37:05.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:05.315 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-12T05:37:05.315 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-12T05:37:05.315 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.h 2024-03-12T05:37:05.453 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:05 smithi156 systemd[1]: Stopping Ceph mds.h for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:05.953 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h[66572]: 2024-03-12T05:37:05.640+0000 7fe86656a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:05.953 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:05 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h[66572]: 2024-03-12T05:37:05.640+0000 7fe86656a700 -1 mds.h *** got signal Terminated *** 2024-03-12T05:37:08.811 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:08 smithi156 podman[76839]: 2024-03-12 05:37:08.540657438 +0000 UTC m=+3.004298873 container died 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-12T05:37:09.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:08 smithi156 podman[76839]: 2024-03-12 05:37:08.88952403 +0000 UTC m=+3.353165456 container cleanup 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397) 2024-03-12T05:37:09.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:08 smithi156 bash[76839]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h 2024-03-12T05:37:09.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 12 05:37:09 smithi156 podman[76853]: 2024-03-12 05:37:09.040414156 +0000 UTC m=+0.493426896 container remove 9c66f71a566628773e40c00b95bfac9b234f745e66644474621eed6619792c5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-12T05:37:09.244 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.h.service' 2024-03-12T05:37:09.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:09.281 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-12T05:37:09.281 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-12T05:37:09.281 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.k 2024-03-12T05:37:09.596 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:09 smithi156 systemd[1]: Stopping Ceph mds.k for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:09.956 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k[65943]: 2024-03-12T05:37:09.601+0000 7fd61bc66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:09.956 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:09 smithi156 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k[65943]: 2024-03-12T05:37:09.601+0000 7fd61bc66700 -1 mds.k *** got signal Terminated *** 2024-03-12T05:37:10.937 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:10 smithi156 podman[76960]: 2024-03-12 05:37:10.655806941 +0000 UTC m=+1.155224909 container died 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-12T05:37:11.203 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:10 smithi156 podman[76960]: 2024-03-12 05:37:10.942579255 +0000 UTC m=+1.441997214 container cleanup 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397) 2024-03-12T05:37:11.204 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:10 smithi156 bash[76960]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k 2024-03-12T05:37:11.204 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 12 05:37:11 smithi156 podman[76974]: 2024-03-12 05:37:11.168388763 +0000 UTC m=+0.511733038 container remove 475daf3e5801d14be838fbaff497b1d55bbca433f0efaaacc95dcce79dd7012f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-12T05:37:11.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.k.service' 2024-03-12T05:37:11.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:11.301 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-12T05:37:11.301 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-12T05:37:11.301 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.c 2024-03-12T05:37:11.602 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:37:11 smithi161 systemd[1]: Stopping Ceph mds.c for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:11.971 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:37:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c[67019]: 2024-03-12T05:37:11.605+0000 7fbcf2e48700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:11.972 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:37:11 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c[67019]: 2024-03-12T05:37:11.605+0000 7fbcf2e48700 -1 mds.c *** got signal Terminated *** 2024-03-12T05:37:15.972 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:37:15 smithi161 podman[76871]: 2024-03-12 05:37:15.611357238 +0000 UTC m=+4.114541876 container died 7f4691dd82864f9a3801884091aca9c8283f109ae7fdeabca2da79f8fb4122e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-03-12T05:37:16.235 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:37:16 smithi161 podman[76871]: 2024-03-12 05:37:16.056007177 +0000 UTC m=+4.559191807 container cleanup 7f4691dd82864f9a3801884091aca9c8283f109ae7fdeabca2da79f8fb4122e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-12T05:37:16.236 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 12 05:37:16 smithi161 bash[76871]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-c 2024-03-12T05:37:16.374 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.c.service' 2024-03-12T05:37:16.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:16.413 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-12T05:37:16.413 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-12T05:37:16.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.f 2024-03-12T05:37:16.971 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:16 smithi161 systemd[1]: Stopping Ceph mds.f for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:16.972 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f[66705]: 2024-03-12T05:37:16.734+0000 7fcb10f8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:16.972 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:16 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f[66705]: 2024-03-12T05:37:16.734+0000 7fcb10f8f700 -1 mds.f *** got signal Terminated *** 2024-03-12T05:37:20.872 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:20 smithi161 podman[77004]: 2024-03-12 05:37:20.496246315 +0000 UTC m=+3.857100812 container died 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-12T05:37:21.222 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:20 smithi161 podman[77004]: 2024-03-12 05:37:20.942346993 +0000 UTC m=+4.303201497 container cleanup 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-12T05:37:21.222 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:20 smithi161 bash[77004]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f 2024-03-12T05:37:21.222 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 12 05:37:21 smithi161 podman[77018]: 2024-03-12 05:37:21.110118949 +0000 UTC m=+0.604469267 container remove 9434d62e6714ece19a2016a41136649fe221f0bc7a3722ada5c871df8e41cef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-12T05:37:21.359 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.f.service' 2024-03-12T05:37:21.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:21.396 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-12T05:37:21.396 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-12T05:37:21.396 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.i 2024-03-12T05:37:21.722 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:37:21 smithi161 systemd[1]: Stopping Ceph mds.i for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:22.221 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:37:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i[67336]: 2024-03-12T05:37:21.728+0000 7f27affb0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:22.222 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:37:21 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i[67336]: 2024-03-12T05:37:21.728+0000 7f27affb0700 -1 mds.i *** got signal Terminated *** 2024-03-12T05:37:24.766 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:37:24 smithi161 podman[77125]: 2024-03-12 05:37:24.500378002 +0000 UTC m=+2.878582255 container died f73d71f5907b029ae1349020bc8a31cf5c4a8255b36c6910e84316e0b7d602b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-03-12T05:37:25.109 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:37:24 smithi161 podman[77125]: 2024-03-12 05:37:24.862293561 +0000 UTC m=+3.240497812 container cleanup f73d71f5907b029ae1349020bc8a31cf5c4a8255b36c6910e84316e0b7d602b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-12T05:37:25.109 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 12 05:37:24 smithi161 bash[77125]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-i 2024-03-12T05:37:25.273 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.i.service' 2024-03-12T05:37:25.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:25.308 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-12T05:37:25.308 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-12T05:37:25.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.l 2024-03-12T05:37:25.628 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:25 smithi161 systemd[1]: Stopping Ceph mds.l for 8a1680ae-e02d-11ee-95c8-87774f69a715... 2024-03-12T05:37:25.971 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l[66390]: 2024-03-12T05:37:25.631+0000 7f7fb726e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-12T05:37:25.972 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:25 smithi161 ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l[66390]: 2024-03-12T05:37:25.631+0000 7f7fb726e700 -1 mds.l *** got signal Terminated *** 2024-03-12T05:37:28.061 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:27 smithi161 podman[77248]: 2024-03-12 05:37:27.815222663 +0000 UTC m=+2.287961187 container died 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-12T05:37:28.061 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:28 smithi161 podman[77248]: 2024-03-12 05:37:28.06543883 +0000 UTC m=+2.538177340 container cleanup 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:37:28.395 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:28 smithi161 bash[77248]: ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l 2024-03-12T05:37:28.395 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 12 05:37:28 smithi161 podman[77260]: 2024-03-12 05:37:28.216399944 +0000 UTC m=+0.395664059 container remove 7acebf3e2fe9a67b43bd7a2e2ec0310cc71887fbd9548f1b0baf43a37139042c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-8a1680ae-e02d-11ee-95c8-87774f69a715-mds-l, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-12T05:37:28.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8a1680ae-e02d-11ee-95c8-87774f69a715@mds.l.service' 2024-03-12T05:37:28.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T05:37:28.449 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-12T05:37:28.450 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T05:37:28.642 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:37:30.999 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T05:37:31.191 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:37:33.815 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T05:37:34.004 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:37:36.492 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:37:36.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:37:36.562 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T05:37:36.591 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-12T05:37:36.593 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi100/crash 2024-03-12T05:37:36.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash -- . 2024-03-12T05:37:36.635 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T05:37:36.635 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-03-12T05:37:36.637 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi156/crash 2024-03-12T05:37:36.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash -- . 2024-03-12T05:37:36.682 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T05:37:36.682 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2024-03-12T05:37:36.685 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi161/crash 2024-03-12T05:37:36.686 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash -- . 2024-03-12T05:37:36.720 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T05:37:36.720 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-03-12T05:37:36.721 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-12T05:37:36.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-12T05:37:36.756 INFO:tasks.cephadm:Compressing logs... 2024-03-12T05:37:36.756 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-12T05:37:36.798 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-12T05:37:36.801 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-12T05:37:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-12T05:37:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:: No such file or directory 2024-03-12T05:37:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.6.log 2024-03-12T05:37:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.a.log 2024-03-12T05:37:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-12T05:37:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.6.log: /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.g.log 2024-03-12T05:37:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.j.log 2024-03-12T05:37:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.3.log 2024-03-12T05:37:36.829 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.x.log 2024-03-12T05:37:36.829 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.0.log 2024-03-12T05:37:36.829 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-12T05:37:36.831 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-12T05:37:36.832 INFO:teuthology.orchestra.run.smithi100.stderr: 94.7% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.j.log.gz 2024-03-12T05:37:36.836 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-12T05:37:36.836 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-12T05:37:36.836 INFO:teuthology.orchestra.run.smithi100.stderr: 94.6%/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.0.log: -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.g.log.gz 2024-03-12T05:37:36.836 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log 2024-03-12T05:37:36.836 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-12T05:37:36.836 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log: 84.4% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-12T05:37:36.837 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.9.log 2024-03-12T05:37:36.837 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.d.log 2024-03-12T05:37:36.841 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-12T05:37:36.841 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.l.log 2024-03-12T05:37:36.841 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-12T05:37:36.841 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.i.log 2024-03-12T05:37:36.841 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.l.log: /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.11.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.z.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.8.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.5.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.f.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log 2024-03-12T05:37:36.842 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.2.log 2024-03-12T05:37:36.843 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.c.log 2024-03-12T05:37:36.843 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-12T05:37:36.843 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.c.log 2024-03-12T05:37:36.843 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.c.log: 94.5% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.l.log.gz 2024-03-12T05:37:36.844 INFO:teuthology.orchestra.run.smithi161.stderr: 92.5% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.z.log.gz 2024-03-12T05:37:36.845 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.9.log: /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log: 87.1% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-12T05:37:36.845 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.a.log 2024-03-12T05:37:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.d.log.gz 2024-03-12T05:37:36.848 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-12T05:37:36.848 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-12T05:37:36.849 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.10.log 2024-03-12T05:37:36.849 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-12T05:37:36.849 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.b.log 2024-03-12T05:37:36.850 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.4.log 2024-03-12T05:37:36.851 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.a.log: 91.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.1.log 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.k.log 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.h.log 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.7.log 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.e.log 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.y.log 2024-03-12T05:37:36.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-12T05:37:36.855 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log 2024-03-12T05:37:36.855 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-12T05:37:36.855 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.b.log 2024-03-12T05:37:36.855 INFO:teuthology.orchestra.run.smithi100.stderr: 94.7% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.a.log.gz 2024-03-12T05:37:36.855 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.b.log: 94.6% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.k.log.gz 2024-03-12T05:37:36.859 INFO:teuthology.orchestra.run.smithi156.stderr: 92.5% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.y.log.gz 2024-03-12T05:37:36.933 INFO:teuthology.orchestra.run.smithi156.stderr: 90.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-12T05:37:36.997 INFO:teuthology.orchestra.run.smithi161.stderr: 90.9% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-12T05:37:37.001 INFO:teuthology.orchestra.run.smithi161.stderr: 84.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-12T05:37:37.006 INFO:teuthology.orchestra.run.smithi161.stderr: 90.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-12T05:37:37.012 INFO:teuthology.orchestra.run.smithi161.stderr: 87.1% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-12T05:37:37.060 INFO:teuthology.orchestra.run.smithi156.stderr: 84.4% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-12T05:37:37.085 INFO:teuthology.orchestra.run.smithi156.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-12T05:37:37.094 INFO:teuthology.orchestra.run.smithi156.stderr: 87.1% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-12T05:37:37.104 INFO:teuthology.orchestra.run.smithi156.stderr: 91.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-12T05:37:37.307 INFO:teuthology.orchestra.run.smithi100.stderr: 89.9% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mgr.x.log.gz 2024-03-12T05:37:37.612 INFO:teuthology.orchestra.run.smithi161.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-12T05:37:38.750 INFO:teuthology.orchestra.run.smithi161.stderr: 94.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.c.log.gz 2024-03-12T05:37:39.652 INFO:teuthology.orchestra.run.smithi100.stderr: 92.5% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.a.log.gz 2024-03-12T05:37:41.384 INFO:teuthology.orchestra.run.smithi161.stderr: 92.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.c.log.gz 2024-03-12T05:37:51.089 INFO:teuthology.orchestra.run.smithi156.stderr: 92.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mon.b.log.gz 2024-03-12T05:37:52.269 INFO:teuthology.orchestra.run.smithi100.stderr: 94.2% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.9.log.gz 2024-03-12T05:37:52.593 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.0.log.gz 2024-03-12T05:37:54.479 INFO:teuthology.orchestra.run.smithi100.stderr: 94.1% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.6.log.gz 2024-03-12T05:37:56.849 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.3.log.gz 2024-03-12T05:37:56.851 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-03-12T05:37:56.851 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m20.039s 2024-03-12T05:37:56.851 INFO:teuthology.orchestra.run.smithi100.stderr:user 1m9.468s 2024-03-12T05:37:56.852 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m2.655s 2024-03-12T05:38:44.026 INFO:teuthology.orchestra.run.smithi161.stderr: 94.2% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.2.log.gz 2024-03-12T05:38:49.687 INFO:teuthology.orchestra.run.smithi161.stderr: 94.1% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.11.log.gz 2024-03-12T05:39:01.137 INFO:teuthology.orchestra.run.smithi161.stderr: 94.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.5.log.gz 2024-03-12T05:39:10.395 INFO:teuthology.orchestra.run.smithi161.stderr: 94.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.8.log.gz 2024-03-12T05:39:16.359 INFO:teuthology.orchestra.run.smithi161.stderr: 94.8% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.f.log.gz 2024-03-12T05:40:19.321 INFO:teuthology.orchestra.run.smithi156.stderr: 94.3% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.4.log.gz 2024-03-12T05:40:32.082 INFO:teuthology.orchestra.run.smithi156.stderr: 94.2% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.10.log.gz 2024-03-12T05:40:40.688 INFO:teuthology.orchestra.run.smithi156.stderr: 94.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.1.log.gz 2024-03-12T05:41:27.625 INFO:teuthology.orchestra.run.smithi156.stderr: 94.1% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-osd.7.log.gz 2024-03-12T05:42:03.965 INFO:teuthology.orchestra.run.smithi156.stderr: 94.9% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.b.log.gz 2024-03-12T05:42:58.055 INFO:teuthology.orchestra.run.smithi161.stderr: 92.9% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.i.log.gz 2024-03-12T05:42:58.057 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-03-12T05:42:58.057 INFO:teuthology.orchestra.run.smithi161.stderr:real 5m21.238s 2024-03-12T05:42:58.057 INFO:teuthology.orchestra.run.smithi161.stderr:user 5m1.085s 2024-03-12T05:42:58.057 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m11.655s 2024-03-12T05:43:00.306 INFO:teuthology.orchestra.run.smithi156.stderr: 94.8% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.e.log.gz 2024-03-12T05:52:45.141 INFO:teuthology.orchestra.run.smithi156.stderr: 93.0% -- replaced with /var/log/ceph/8a1680ae-e02d-11ee-95c8-87774f69a715/ceph-mds.h.log.gz 2024-03-12T05:52:45.199 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-03-12T05:52:45.199 INFO:teuthology.orchestra.run.smithi156.stderr:real 15m8.321s 2024-03-12T05:52:45.199 INFO:teuthology.orchestra.run.smithi156.stderr:user 10m56.944s 2024-03-12T05:52:45.200 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m26.799s 2024-03-12T05:52:45.200 INFO:tasks.cephadm:Archiving logs... 2024-03-12T05:52:45.201 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi100/log 2024-03-12T05:52:45.202 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T05:52:55.297 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi156/log 2024-03-12T05:52:55.298 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T05:54:40.772 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi161/log 2024-03-12T05:54:40.773 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T05:55:29.274 INFO:tasks.cephadm:Removing cluster... 2024-03-12T05:55:29.275 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --force 2024-03-12T05:55:29.468 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:55:29.814 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --force 2024-03-12T05:55:30.413 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:55:31.197 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8a1680ae-e02d-11ee-95c8-87774f69a715 --force 2024-03-12T05:55:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 8a1680ae-e02d-11ee-95c8-87774f69a715 2024-03-12T05:55:32.031 INFO:tasks.cephadm:Removing cephadm ... 2024-03-12T05:55:32.032 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T05:55:32.051 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T05:55:32.068 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T05:55:32.084 INFO:tasks.cephadm:Teardown complete 2024-03-12T05:55:32.084 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-12T05:55:32.096 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-12T05:55:32.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T05:55:32.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T05:55:32.111 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T05:55:32.234 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-12T05:55:32.244 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-12T05:55:32.245 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T05:55:32.248 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T05:55:32.250 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T05:55:32.262 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-12T05:55:32.263 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-03-12T05:55:32.264 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-03-12T05:55:32.266 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T05:55:32.267 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-12T05:55:32.267 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +1948us[+1948us] +/- 109ms 2024-03-12T05:55:32.267 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -1340us[-1344us] +/- 51ms 2024-03-12T05:55:32.267 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +647us[ +643us] +/- 81ms 2024-03-12T05:55:32.267 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T05:55:32.326 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T05:55:32.327 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-03-12T05:55:32.327 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 6 +1989us[+1994us] +/- 109ms 2024-03-12T05:55:32.327 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -1331us[-1326us] +/- 51ms 2024-03-12T05:55:32.327 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +633us[ +638us] +/- 81ms 2024-03-12T05:55:32.327 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T05:55:32.360 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T05:55:32.360 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-03-12T05:55:32.361 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +1916us[+1937us] +/- 109ms 2024-03-12T05:55:32.361 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -1355us[-1334us] +/- 51ms 2024-03-12T05:55:32.361 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +614us[ +614us] +/- 81ms 2024-03-12T05:55:32.361 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T05:55:32.362 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-12T05:55:32.374 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-12T05:55:32.375 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-12T05:55:32.385 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T05:55:32.479 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T05:55:32.600 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T05:55:32.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T05:55:32.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:55:32.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T05:55:32.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:55:32.762 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T05:55:32.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:55:32.793 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-12T05:55:32.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-12T05:55:32.817 INFO:teuthology.task.internal:Duration was 3741.301981 seconds 2024-03-12T05:55:32.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-12T05:55:32.829 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-12T05:55:32.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T05:55:32.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T05:55:32.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T05:55:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T05:55:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T05:55:32.900 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T05:55:33.306 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-12T05:55:33.306 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-03-12T05:55:33.307 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T05:55:33.327 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-03-12T05:55:33.327 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T05:55:33.359 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-03-12T05:55:33.359 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T05:55:33.382 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-12T05:55:33.382 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T05:55:33.385 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T05:55:33.402 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T05:55:33.452 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-12T05:55:33.452 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T05:55:33.455 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T05:55:33.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T05:55:33.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-12T05:55:33.863 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-12T05:55:33.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T05:55:33.891 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T05:55:33.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T05:55:33.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-12T05:55:33.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T05:55:34.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T05:55:34.004 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T05:55:34.024 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-03-12T05:55:34.036 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-03-12T05:55:34.042 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-03-12T05:55:34.069 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T05:55:34.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:55:34.091 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T05:55:34.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:55:34.123 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T05:55:34.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T05:55:34.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-12T05:55:34.150 INFO:teuthology.task.internal:Transferring archived files... 2024-03-12T05:55:34.151 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi100 2024-03-12T05:55:34.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T05:55:34.196 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi156 2024-03-12T05:55:34.196 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T05:55:34.235 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383/remote/smithi161 2024-03-12T05:55:34.235 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T05:55:34.274 INFO:teuthology.task.internal:Removing archive directory... 2024-03-12T05:55:34.274 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T05:55:34.277 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T05:55:34.279 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T05:55:34.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-12T05:55:34.336 INFO:teuthology.task.internal:Not uploading archives. 2024-03-12T05:55:34.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-12T05:55:34.348 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-12T05:55:34.348 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T05:55:34.350 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T05:55:34.353 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T05:55:34.362 INFO:teuthology.orchestra.run.smithi100.stdout: 265683 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 05:55 /home/ubuntu/cephtest 2024-03-12T05:55:34.364 INFO:teuthology.orchestra.run.smithi156.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 05:55 /home/ubuntu/cephtest 2024-03-12T05:55:34.377 INFO:teuthology.orchestra.run.smithi161.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 05:55 /home/ubuntu/cephtest 2024-03-12T05:55:34.378 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-12T05:55:34.390 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-12T05:55:34.473 INFO:teuthology.nuke:Checking targets against current locks 2024-03-12T05:55:34.500 DEBUG:teuthology.nuke:shortname: smithi100 2024-03-12T05:55:34.500 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T05:55:34.526 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 04:43:07.406793', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLsrrTkXrYAX4VTMAJ3DmX0gfV+s6kMUH4n/1d/EVHX0VIPgg0rH8USo1F0jtAVzpQLNsr66vtssHnozDRJ6oY4='} 2024-03-12T05:55:34.530 DEBUG:teuthology.nuke:shortname: smithi156 2024-03-12T05:55:34.531 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T05:55:34.551 DEBUG:teuthology.nuke:shortname: smithi161 2024-03-12T05:55:34.551 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T05:55:34.557 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 04:43:07.407838', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLs44ux6wfQ88fyomfMFzV/FoEmQZ9Sa62ENQ0Wu1a+vQ0Adi3v2FBtdMSjMZZJHjcFwrYwvcZ1wbRaTbtqnTdI='} 2024-03-12T05:55:34.573 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591383', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 04:43:07.408518', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAz6qeUcD/wLadGJnm4gDvX8rE2iA8OXFctEfTtlzyqT7KqOqCQKaNPrb71+0DjeCs2Not8Hy+UU2UDoxI1ndB0='} 2024-03-12T05:55:34.585 INFO:teuthology.orchestra.console.smithi100:Power off 2024-03-12T05:55:34.586 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T05:55:34.615 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-03-12T05:55:34.615 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:34.633 INFO:teuthology.orchestra.console.smithi156:Power off 2024-03-12T05:55:34.633 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T05:55:34.652 INFO:teuthology.orchestra.console.smithi161:Power off 2024-03-12T05:55:34.652 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T05:55:34.662 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-12T05:55:34.662 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2024-03-12T05:55:34.662 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:34.677 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-03-12T05:55:34.678 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:34.690 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-12T05:55:34.704 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-12T05:55:38.663 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:38.692 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:38.705 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:38.776 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-12T05:55:38.803 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-12T05:55:38.817 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-12T05:55:42.781 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:42.804 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:42.819 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:42.892 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-12T05:55:42.914 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-12T05:55:42.929 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-03-12T05:55:42.993 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-03-12T05:55:43.030 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-03-12T05:55:46.918 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T05:55:47.028 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2024-03-12T05:55:47.129 INFO:teuthology.orchestra.console.smithi156:Power off completed 2024-03-12T05:55:47.236 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 3741.3019807338715 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi100 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=69450056edff49a29b93dd18f48af00f status: fail success: false 2024-03-12T05:55:47.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T05:55:47.326 INFO:teuthology.run:FAIL